2024-05-17T08:55:44.646 INFO:root:teuthology version: 0.0.1.dev275+ga4b16c5 2024-05-17T08:55:44.646 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866 --verbose --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --name pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/orig.config.yaml 2024-05-17T08:55:44.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T08:55:44.788 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866 branch: wip-pdonnell-testing-20240517.012301-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7710866' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.012301-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 3790c0feaaca36c33bb1b31470733325d7d91e6a subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 3790c0feaaca36c33bb1b31470733325d7d91e6a kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.012301-debug sha1: 3790c0feaaca36c33bb1b31470733325d7d91e6a owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1072 sha1: 3790c0feaaca36c33bb1b31470733325d7d91e6a sleep_before_teardown: 0 subset: 322/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.012301-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3790c0feaaca36c33bb1b31470733325d7d91e6a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIhOaq4rZKfGooxW8/3rousAoL4Nx9brZTjPH3S8SeabhcwVQUyLSppm52Y9qCqWXc1jTECiGcHXN6A87ELWkFY= smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAD8qQdkjGC9r8PuczunJVRByWDvrlvBQ40xz2G9I0eJqzvVr1G3C2tYsAAtPywIrd70bvZR5rDYMkylIvE7cPc= smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfyPwM0Uj4udMx3HtbAudco+vYaRaVVWR08BcMYHu1zgRtKZm6XkCXqkkyhxZPjRBynkUyBxUe6L1jefh1yawo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: a4b16c58a5d7aa60dee491f6f096b7509e2f1269 timestamp: 2024-05-17_04:08:01 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-17T08:55:44.788 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa; will attempt to use it 2024-05-17T08:55:44.789 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks 2024-05-17T08:55:44.789 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-17T08:55:44.791 INFO:teuthology.task.internal:Checking packages... 2024-05-17T08:55:44.816 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3790c0feaaca36c33bb1b31470733325d7d91e6a' 2024-05-17T08:55:44.816 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T08:55:44.816 INFO:teuthology.packaging:ref: None 2024-05-17T08:55:44.816 INFO:teuthology.packaging:tag: None 2024-05-17T08:55:44.816 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.012301-debug 2024-05-17T08:55:44.816 INFO:teuthology.packaging:sha1: 3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T08:55:44.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.012301-debug 2024-05-17T08:55:45.047 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3778.g3790c0fe 2024-05-17T08:55:45.048 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-17T08:55:45.087 INFO:teuthology.task.internal:no buildpackages task found 2024-05-17T08:55:45.087 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-17T08:55:45.292 INFO:teuthology.task.internal:Saving configuration 2024-05-17T08:55:45.311 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-17T08:55:45.330 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T08:55:45.356 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 08:50:38.049332', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIhOaq4rZKfGooxW8/3rousAoL4Nx9brZTjPH3S8SeabhcwVQUyLSppm52Y9qCqWXc1jTECiGcHXN6A87ELWkFY='} 2024-05-17T08:55:45.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 08:50:38.051065', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAD8qQdkjGC9r8PuczunJVRByWDvrlvBQ40xz2G9I0eJqzvVr1G3C2tYsAAtPywIrd70bvZR5rDYMkylIvE7cPc='} 2024-05-17T08:55:45.399 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 08:50:38.047351', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfyPwM0Uj4udMx3HtbAudco+vYaRaVVWR08BcMYHu1zgRtKZm6XkCXqkkyhxZPjRBynkUyBxUe6L1jefh1yawo='} 2024-05-17T08:55:45.400 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-17T08:55:45.496 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-17T08:55:45.497 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-17T08:55:45.497 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-17T08:55:45.497 INFO:teuthology.run_tasks:Running task console_log... 2024-05-17T08:55:45.883 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f734109bac0>, signals=[15]) 2024-05-17T08:55:45.883 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-17T08:55:46.132 INFO:teuthology.task.internal:Opening connections... 2024-05-17T08:55:46.132 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T08:55:46.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:55:46.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-05-17T08:55:46.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:55:46.283 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-05-17T08:55:46.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:55:46.359 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-17T08:55:46.557 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-05-17T08:55:46.576 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-05-17T08:55:46.576 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-05-17T08:55:46.634 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-05-17T08:55:46.635 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T08:55:46.636 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T08:55:46.636 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T08:55:46.636 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-05-17T08:55:46.673 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-05-17T08:55:46.691 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-05-17T08:55:46.691 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-05-17T08:55:46.748 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T08:55:46.749 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T08:55:46.749 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-05-17T08:55:46.781 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-05-17T08:55:46.799 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-05-17T08:55:46.799 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T08:55:46.856 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-05-17T08:55:46.857 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T08:55:46.857 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T08:55:46.857 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T08:55:46.857 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-05-17T08:55:46.881 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-17T08:55:46.891 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-17T08:55:47.139 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-17T08:55:47.139 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-05-17T08:55:47.141 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-05-17T08:55:47.144 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-05-17T08:55:47.160 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-17T08:55:47.321 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-17T08:55:47.321 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-05-17T08:55:47.323 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-05-17T08:55:47.326 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-05-17T08:55:47.416 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-17T08:55:47.492 INFO:teuthology.run_tasks:Running task kernel... 2024-05-17T08:55:47.532 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-17T08:55:47.532 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-17T08:55:47.532 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T08:55:47.533 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T08:55:47.534 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T08:55:47.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T08:55:47.551 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-05-17T08:55:47.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T08:55:47.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T08:55:47.552 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-17T08:55:47.553 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-05-17T08:55:47.609 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-437.el9.x86_64 2024-05-17T08:55:47.609 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.14.0-437.el9.x86_64 2024-05-17T08:55:47.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2024-05-17T08:55:47.612 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-05-17T08:55:47.612 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-05-17T08:55:47.613 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.14.0-437.el9.x86_64 2024-05-17T08:55:47.613 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2024-05-17T08:55:47.613 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-05-17T08:55:47.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-05-17T08:55:49.842 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-17T08:55:49.895 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-17T08:55:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-17T08:55:51.975 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-17T08:55:53.154 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-05-17T08:55:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-05-17T08:55:56.310 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-05-17T08:55:58.935 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-05-17T08:55:59.677 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.2 MB 00:02 2024-05-17T08:55:59.922 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-05-17T08:56:00.338 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-05-17T08:56:01.092 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-05-17T08:56:02.072 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-05-17T08:56:03.028 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-17T08:56:03.423 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-17T08:56:05.453 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-05-17T08:56:07.485 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T08:56:07.485 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T08:56:07.590 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-17T08:56:07.599 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-05-17T08:56:07.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T08:56:07.601 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 78 M 2024-05-17T08:56:07.601 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 123 M 2024-05-17T08:56:07.601 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-17T08:56:07.838 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 2.8 MB/s | 550 kB 00:00 2024-05-17T08:56:08.176 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-17T08:56:08.355 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-17T08:56:08.506 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-05-17T08:56:09.047 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-05-17T08:56:09.305 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-17T08:56:09.305 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-17T08:56:09.305 INFO:teuthology.orchestra.run.smithi019.stdout:Total 46 MB/s | 78 MB 00:01 2024-05-17T08:56:10.018 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-17T08:56:10.133 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T08:56:10.133 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T08:56:10.225 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T08:56:10.236 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 78 M 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 123 M 2024-05-17T08:56:10.237 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-05-17T08:56:10.408 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-17T08:56:10.408 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-17T08:56:10.423 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 4.9 MB/s | 550 kB 00:00 2024-05-17T08:56:10.471 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T08:56:10.471 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T08:56:10.566 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T08:56:10.578 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-05-17T08:56:10.579 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-05-17T08:56:10.579 INFO:teuthology.orchestra.run.smithi069.stdout:Install 4 Packages 2024-05-17T08:56:10.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T08:56:10.579 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 78 M 2024-05-17T08:56:10.579 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 123 M 2024-05-17T08:56:10.579 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-05-17T08:56:10.912 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 26 MB/s | 15 MB 00:00 2024-05-17T08:56:10.974 INFO:teuthology.orchestra.run.smithi069.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 1.9 MB/s | 550 kB 00:00 2024-05-17T08:56:11.377 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-17T08:56:11.377 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-17T08:56:11.436 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-05-17T08:56:11.791 INFO:teuthology.orchestra.run.smithi069.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 14 MB/s | 15 MB 00:01 2024-05-17T08:56:11.862 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-17T08:56:11.862 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-05-17T08:56:11.862 INFO:teuthology.orchestra.run.smithi079.stdout:Total 48 MB/s | 78 MB 00:01 2024-05-17T08:56:12.333 INFO:teuthology.orchestra.run.smithi069.stdout:(3/4): kernel-modules-core-5.14.0-446.el9.x86_6 21 MB/s | 28 MB 00:01 2024-05-17T08:56:12.546 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-05-17T08:56:12.625 INFO:teuthology.orchestra.run.smithi069.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 18 MB/s | 34 MB 00:01 2024-05-17T08:56:12.625 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-05-17T08:56:12.625 INFO:teuthology.orchestra.run.smithi069.stdout:Total 38 MB/s | 78 MB 00:02 2024-05-17T08:56:12.877 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-05-17T08:56:12.877 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-05-17T08:56:13.310 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-05-17T08:56:13.646 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-05-17T08:56:13.646 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-05-17T08:56:13.796 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-05-17T08:56:13.797 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-05-17T08:56:13.889 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-17T08:56:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T08:56:14.569 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-05-17T08:56:14.570 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-05-17T08:56:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:16.175 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-05-17T08:56:16.874 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T08:56:16.952 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-05-17T08:56:16.961 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:17.555 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T08:56:17.589 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:17.658 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:19.634 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:19.780 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:20.316 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:20.452 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:22.669 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:22.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:24.541 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:25.123 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:25.179 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:27.799 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:29.251 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:29.876 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.115 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.115 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T08:56:59.115 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:59.115 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:59.135 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.489 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.530 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.567 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T08:56:59.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:59.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:59.789 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.789 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T08:56:59.789 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T08:56:59.789 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T08:56:59.941 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-05-17T08:57:00.306 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T08:57:00.354 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:57:00.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T08:57:00.354 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-17T08:57:00.354 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.354 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.354 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.355 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T08:57:00.355 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-17T08:57:00.433 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.433 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.433 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.434 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T08:57:00.434 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T08:57:00.434 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-17T08:57:00.434 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T08:57:00.434 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T08:57:00.484 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.484 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 68.el9 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8264254 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2024-05-17T08:57:00.515 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2024-05-17T08:57:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T08:57:00.517 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T08:57:00.517 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T08:57:00.517 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T08:57:00.630 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T08:57:00.631 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-05-17T08:57:00.659 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T08:57:00.735 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.735 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.736 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.736 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T08:57:00.736 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T08:57:00.736 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-17T08:57:00.736 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T08:57:00.736 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T08:57:00.787 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:00.788 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T08:57:00.819 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T08:57:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T08:57:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T08:57:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T08:57:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-05-17T08:57:00.821 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T08:57:00.821 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T08:57:00.822 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T08:57:00.822 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-446.el9.x86_64 2024-05-17T08:57:00.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T08:57:00.962 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T08:57:01.039 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:01.039 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T08:57:01.039 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T08:57:01.039 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T08:57:01.039 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T08:57:01.040 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-17T08:57:01.040 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T08:57:01.040 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T08:57:01.090 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T08:57:01.090 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T08:57:01.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-05-17T08:57:01.120 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-05-17T08:57:01.121 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T08:57:01.121 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T08:57:01.121 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T08:57:01.121 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T08:57:01.121 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-05-17T08:57:01.121 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T08:57:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T08:57:01.123 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T08:57:01.124 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 5.14.0-446.el9.x86_64 2024-05-17T08:57:01.124 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T08:57:01.141 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-05-17T08:57:01.426 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-05-17T08:57:01.765 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-05-17T08:57:02.512 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T08:57:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-05-17T08:57:02.540 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-05-17T08:57:02.559 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.GSF0CpCsI8 2024-05-17T08:57:02.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GSF0CpCsI8 2024-05-17T08:57:02.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.GSF0CpCsI8 2024-05-17T08:57:02.774 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.GSF0CpCsI8 is 6KB 2024-05-17T08:57:02.822 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.GSF0CpCsI8 2024-05-17T08:57:02.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T08:57:02.879 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T08:57:02.906 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T08:57:02.906 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T08:57:02.907 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T08:57:02.907 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T08:57:02.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T08:57:02.919 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-05-17T08:57:02.922 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-05-17T08:57:02.943 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.2d5ibdDChb 2024-05-17T08:57:02.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2d5ibdDChb 2024-05-17T08:57:03.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.2d5ibdDChb 2024-05-17T08:57:03.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-05-17T08:57:03.164 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.2d5ibdDChb is 6KB 2024-05-17T08:57:03.212 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.2d5ibdDChb 2024-05-17T08:57:03.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T08:57:03.229 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T08:57:03.269 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-05-17T08:57:03.271 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-05-17T08:57:03.286 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.V88fJchJdy 2024-05-17T08:57:03.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V88fJchJdy 2024-05-17T08:57:03.289 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T08:57:03.289 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T08:57:03.289 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T08:57:03.289 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T08:57:03.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T08:57:03.352 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.V88fJchJdy 2024-05-17T08:57:03.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-05-17T08:57:03.490 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.V88fJchJdy is 6KB 2024-05-17T08:57:03.538 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.V88fJchJdy 2024-05-17T08:57:03.553 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T08:57:03.624 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T08:57:03.624 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T08:57:03.624 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T08:57:03.624 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T08:57:03.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T08:57:03.765 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-05-17T08:57:33.098 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-17T08:57:33.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-17T08:57:33.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:57:33.439 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-17T08:57:33.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-17T08:57:33.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:57:33.768 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-17T08:57:33.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-17T08:57:33.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:58:33.151 DEBUG:teuthology.orchestra.remote:timed out 2024-05-17T08:58:33.443 DEBUG:teuthology.orchestra.remote:timed out 2024-05-17T08:58:33.770 DEBUG:teuthology.orchestra.remote:timed out 2024-05-17T08:58:42.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-17T08:58:42.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:58:42.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-17T08:58:42.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:58:42.754 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-17T08:58:42.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-17T08:58:42.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T08:58:43.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-17T08:58:43.146 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-446.el9.x86_64"... 2024-05-17T08:58:43.146 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-17T08:58:43.161 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-446.el9.x86_64 2024-05-17T08:58:43.162 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T08:58:43.162 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T08:58:43.162 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T08:58:43.765 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-05-17T08:58:44.163 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-17T08:58:44.163 DEBUG:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T08:58:44.210 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-05-17T08:58:44.229 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2024-05-17T08:58:44.250 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-17T08:58:44.251 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-446.el9.x86_64"... 2024-05-17T08:58:44.251 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-05-17T08:58:44.259 DEBUG:teuthology.parallel:result is None 2024-05-17T08:58:44.265 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-446.el9.x86_64 2024-05-17T08:58:44.266 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T08:58:44.266 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T08:58:44.266 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T08:58:44.743 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-17T08:58:44.743 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-446.el9.x86_64"... 2024-05-17T08:58:44.743 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-05-17T08:58:44.762 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-446.el9.x86_64 2024-05-17T08:58:44.762 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T08:58:44.762 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T08:58:44.762 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T08:58:45.267 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-17T08:58:45.267 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T08:58:45.345 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-05-17T08:58:45.388 DEBUG:teuthology.parallel:result is None 2024-05-17T08:58:45.763 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-17T08:58:45.763 DEBUG:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T08:58:45.836 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2024-05-17T08:58:45.880 DEBUG:teuthology.parallel:result is None 2024-05-17T08:58:45.881 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-17T08:58:45.979 INFO:teuthology.task.internal:Creating test directory... 2024-05-17T08:58:45.980 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T08:58:45.983 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T08:58:45.986 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T08:58:46.001 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-17T08:58:46.302 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-17T08:58:46.467 INFO:teuthology.task.internal:Creating archive directory... 2024-05-17T08:58:46.467 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T08:58:46.470 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T08:58:46.472 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T08:58:46.517 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-17T08:58:46.647 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-17T08:58:46.647 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T08:58:46.650 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T08:58:46.653 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T08:58:46.691 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T08:58:46.699 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T08:58:46.700 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T08:58:46.709 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T08:58:46.709 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T08:58:46.712 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T08:58:46.714 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-17T08:58:46.800 INFO:teuthology.task.internal:Configuring sudo... 2024-05-17T08:58:46.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T08:58:46.804 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T08:58:46.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T08:58:46.848 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-17T08:58:46.859 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-17T08:58:46.859 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T08:58:46.862 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T08:58:46.871 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T08:58:46.889 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T08:58:46.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T08:58:47.050 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T08:58:47.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T08:58:47.163 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T08:58:47.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T08:58:47.230 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T08:58:47.271 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T08:58:47.319 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T08:58:47.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T08:58:47.443 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T08:58:47.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T08:58:47.509 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T08:58:47.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T08:58:47.608 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T08:58:47.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T08:58:47.737 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T08:58:47.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T08:58:47.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-05-17T08:58:47.807 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-05-17T08:58:47.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-05-17T08:58:47.856 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T08:58:47.889 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T08:58:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T08:58:48.451 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-17T08:58:48.458 INFO:teuthology.task.internal:Starting timer... 2024-05-17T08:58:48.458 INFO:teuthology.run_tasks:Running task pcp... 2024-05-17T08:58:48.580 INFO:teuthology.run_tasks:Running task selinux... 2024-05-17T08:58:48.588 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-17T08:58:48.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-05-17T08:58:48.646 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-05-17T08:58:48.647 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-05-17T08:58:48.730 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-05-17T08:58:48.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-05-17T08:58:48.833 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-05-17T08:58:48.835 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-17T08:58:48.835 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-05-17T08:58:48.860 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-05-17T08:58:48.860 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-05-17T08:58:48.881 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-05-17T08:58:48.882 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-05-17T08:58:48.907 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-05-17T08:58:48.907 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive'} 2024-05-17T08:58:48.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T08:58:48.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T08:58:48.934 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T08:58:48.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T08:58:48.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T08:58:48.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T08:58:48.985 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-17T08:58:48.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-05-17T08:58:49.011 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-05-17T08:58:49.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-05-17T08:58:49.204 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-17T08:58:49.216 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-17T08:58:49.232 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-17T08:58:49.233 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi069.front.sepia.ceph.com,smithi079.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-17T09:03:16.777 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com')] 2024-05-17T09:03:16.778 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-17T09:03:16.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T09:03:16.857 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-17T09:03:16.946 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-17T09:03:16.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-17T09:03:16.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T09:03:17.024 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-05-17T09:03:17.103 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-05-17T09:03:17.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-17T09:03:17.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T09:03:17.178 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-05-17T09:03:17.263 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-17T09:03:17.263 INFO:teuthology.run_tasks:Running task clock... 2024-05-17T09:03:17.274 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-17T09:03:17.274 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T09:03:17.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T09:03:17.277 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T09:03:17.277 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T09:03:17.279 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T09:03:17.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T09:03:17.310 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T09:03:17.310 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T09:03:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T09:03:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T09:03:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T09:03:17.347 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T09:03:17.349 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-05-17T09:03:17.354 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-05-17T09:03:17.361 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-05-17T09:03:17.366 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-05-17T09:03:17.372 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-05-17T09:03:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T09:03:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T09:03:17.382 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-05-17T09:03:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T09:03:17.397 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T09:03:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T09:03:17.410 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T09:03:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-05-17T09:03:17.448 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T09:03:17.448 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-05-17T09:03:17.448 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.448 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.448 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.448 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-17T09:03:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T09:03:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-17T09:03:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.466 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-05-17T09:03:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T09:03:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-05-17T09:03:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T09:03:17.470 INFO:teuthology.run_tasks:Running task install... 2024-05-17T09:03:17.478 DEBUG:teuthology.task.install:project ceph 2024-05-17T09:03:17.479 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a'}} 2024-05-17T09:03:17.479 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a'} 2024-05-17T09:03:17.479 INFO:teuthology.task.install:Using flavor: default 2024-05-17T09:03:17.484 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-17T09:03:17.484 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-17T09:03:17.485 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a', 'tag': None, 'wait_for_package': False} 2024-05-17T09:03:17.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:03:17.487 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a', 'tag': None, 'wait_for_package': False} 2024-05-17T09:03:17.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:03:17.488 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a', 'tag': None, 'wait_for_package': False} 2024-05-17T09:03:17.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:03:17.634 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/ 2024-05-17T09:03:17.634 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3778.g3790c0fe 2024-05-17T09:03:17.642 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/ 2024-05-17T09:03:17.643 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3778.g3790c0fe 2024-05-17T09:03:17.646 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/ 2024-05-17T09:03:17.646 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3778.g3790c0fe 2024-05-17T09:03:17.755 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T09:03:17.755 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:03:17.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T09:03:17.781 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T09:03:17.782 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3790c0feaaca36c33bb1b31470733325d7d91e6a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T09:03:17.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T09:03:17.872 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T09:03:17.873 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:03:17.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T09:03:17.898 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T09:03:17.898 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:03:17.898 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T09:03:17.900 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T09:03:17.901 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3790c0feaaca36c33bb1b31470733325d7d91e6a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T09:03:17.922 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T09:03:17.924 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T09:03:17.924 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3790c0feaaca36c33bb1b31470733325d7d91e6a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T09:03:17.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T09:03:17.986 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-05-17T09:03:17.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-05-17T09:03:17.990 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T09:03:18.046 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T09:03:18.065 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T09:03:18.112 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-05-17T09:03:18.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-05-17T09:03:18.129 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-05-17T09:03:18.130 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-05-17T09:03:18.218 INFO:teuthology.orchestra.run.smithi079.stdout:57 files removed 2024-05-17T09:03:18.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T09:03:18.359 INFO:teuthology.orchestra.run.smithi019.stdout:57 files removed 2024-05-17T09:03:18.364 INFO:teuthology.orchestra.run.smithi069.stdout:57 files removed 2024-05-17T09:03:18.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T09:03:18.392 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T09:03:19.003 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 170 kB/s | 83 kB 00:00 2024-05-17T09:03:19.163 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 167 kB/s | 83 kB 00:00 2024-05-17T09:03:19.183 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 165 kB/s | 83 kB 00:00 2024-05-17T09:03:19.345 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-05-17T09:03:19.517 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-05-17T09:03:19.639 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-05-17T09:03:19.929 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-05-17T09:03:20.098 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-17T09:03:20.118 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-05-17T09:03:20.748 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-17T09:03:21.602 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.1 MB 00:01 2024-05-17T09:03:23.979 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.1 MB 00:04 2024-05-17T09:03:25.232 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-05-17T09:03:25.677 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 19 MB 00:02 2024-05-17T09:03:29.616 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-05-17T09:03:31.061 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-17T09:03:31.783 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-05-17T09:03:32.662 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-05-17T09:03:33.962 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-17T09:03:34.397 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-17T09:03:39.368 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 5.8 kB/s | 1.7 kB 00:00 2024-05-17T09:03:39.752 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-05-17T09:03:39.805 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-17T09:03:41.669 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T09:03:41.670 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T09:03:41.681 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T09:03:41.681 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T09:03:41.682 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T09:03:41.683 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T09:03:41.685 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T09:03:41.687 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T09:03:41.688 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T09:03:41.748 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-05-17T09:03:41.768 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 6.6 k 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 5.4 M 2024-05-17T09:03:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 863 k 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 147 k 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 1.5 M 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 143 k 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 3.5 M 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 7.4 M 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 49 k 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 10 M 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 43 M 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 273 k 2024-05-17T09:03:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 764 k 2024-05-17T09:03:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 224 k 2024-05-17T09:03:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 27 k 2024-05-17T09:03:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 18 k 2024-05-17T09:03:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T09:03:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T09:03:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 32 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 756 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 127 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 161 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 320 k 2024-05-17T09:03:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 301 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 99 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 86 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.0 M 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 169 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.3 M 2024-05-17T09:03:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.1 M 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 20 M 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 34 k 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 2.3 M 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 274 k 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 4.9 M 2024-05-17T09:03:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 18 M 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 16 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 25 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 158 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T09:03:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 497 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 5.2 M 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T09:03:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 45 k 2024-05-17T09:03:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 136 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T09:03:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T09:03:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T09:03:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T09:03:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T09:03:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T09:03:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T09:03:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T09:03:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-05-17T09:03:41.784 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 8 Packages 2024-05-17T09:03:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:03:41.785 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 219 M 2024-05-17T09:03:41.785 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-05-17T09:03:42.199 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T09:03:42.200 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T09:03:42.211 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T09:03:42.212 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T09:03:42.213 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T09:03:42.214 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T09:03:42.216 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T09:03:42.218 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T09:03:42.219 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T09:03:42.286 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 6.6 k 2024-05-17T09:03:42.301 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 5.4 M 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 863 k 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 147 k 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 1.5 M 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 143 k 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 3.5 M 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 7.4 M 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 49 k 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 10 M 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 43 M 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 273 k 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 764 k 2024-05-17T09:03:42.302 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 224 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 27 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 18 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 32 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 756 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 127 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T09:03:42.303 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 161 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 320 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 301 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 99 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 86 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.0 M 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 169 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T09:03:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.3 M 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.1 M 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 20 M 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 34 k 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 2.3 M 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 274 k 2024-05-17T09:03:42.305 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 4.9 M 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 18 M 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 16 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 25 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T09:03:42.306 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 158 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 497 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 5.2 M 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T09:03:42.307 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T09:03:42.308 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 45 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 136 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T09:03:42.309 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T09:03:42.310 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T09:03:42.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T09:03:42.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-05-17T09:03:42.313 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:03:42.314 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 219 M 2024-05-17T09:03:42.315 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-17T09:03:42.744 INFO:teuthology.orchestra.run.smithi069.stdout:(1/156): ceph-19.0.0-3778.g3790c0fe.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-17T09:03:43.448 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.0.0-3778.g3790c0fe.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-17T09:03:44.102 INFO:teuthology.orchestra.run.smithi069.stdout:(2/156): ceph-fuse-19.0.0-3778.g3790c0fe.el9.x8 635 kB/s | 863 kB 00:01 2024-05-17T09:03:44.369 INFO:teuthology.orchestra.run.smithi069.stdout:(3/156): ceph-base-19.0.0-3778.g3790c0fe.el9.x8 2.7 MB/s | 5.4 MB 00:01 2024-05-17T09:03:44.395 INFO:teuthology.orchestra.run.smithi069.stdout:(4/156): ceph-immutable-object-cache-19.0.0-377 504 kB/s | 147 kB 00:00 2024-05-17T09:03:44.837 INFO:teuthology.orchestra.run.smithi069.stdout:(5/156): ceph-mds-19.0.0-3778.g3790c0fe.el9.x86 4.8 MB/s | 2.3 MB 00:00 2024-05-17T09:03:45.198 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.0.0-3778.g3790c0fe.el9.x8 493 kB/s | 863 kB 00:01 2024-05-17T09:03:45.202 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-17T09:03:45.532 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-immutable-object-cache-19.0.0-377 442 kB/s | 147 kB 00:00 2024-05-17T09:03:45.912 INFO:teuthology.orchestra.run.smithi069.stdout:(6/156): ceph-mon-19.0.0-3778.g3790c0fe.el9.x86 4.5 MB/s | 4.9 MB 00:01 2024-05-17T09:03:46.062 INFO:teuthology.orchestra.run.smithi069.stdout:(7/156): ceph-mgr-19.0.0-3778.g3790c0fe.el9.x86 923 kB/s | 1.5 MB 00:01 2024-05-17T09:03:46.296 INFO:teuthology.orchestra.run.smithi069.stdout:(8/156): ceph-common-19.0.0-3778.g3790c0fe.el9. 5.0 MB/s | 20 MB 00:03 2024-05-17T09:03:46.362 INFO:teuthology.orchestra.run.smithi069.stdout:(9/156): ceph-selinux-19.0.0-3778.g3790c0fe.el9 380 kB/s | 25 kB 00:00 2024-05-17T09:03:47.344 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 16 kB 00:00 2024-05-17T09:03:47.640 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-base-19.0.0-3778.g3790c0fe.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-05-17T09:03:48.874 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mgr-19.0.0-3778.g3790c0fe.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-05-17T09:03:49.013 INFO:teuthology.orchestra.run.smithi069.stdout:(10/156): ceph-osd-19.0.0-3778.g3790c0fe.el9.x8 5.7 MB/s | 18 MB 00:03 2024-05-17T09:03:49.088 INFO:teuthology.orchestra.run.smithi069.stdout:(11/156): cephfs-mirror-19.0.0-3778.g3790c0fe.e 2.9 MB/s | 224 kB 00:00 2024-05-17T09:03:49.155 INFO:teuthology.orchestra.run.smithi069.stdout:(12/156): cephfs-shell-19.0.0-3778.g3790c0fe.el 407 kB/s | 27 kB 00:00 2024-05-17T09:03:49.190 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-mds-19.0.0-3778.g3790c0fe.el9.x86 633 kB/s | 2.3 MB 00:03 2024-05-17T09:03:49.222 INFO:teuthology.orchestra.run.smithi069.stdout:(13/156): libcephfs-devel-19.0.0-3778.g3790c0fe 476 kB/s | 32 kB 00:00 2024-05-17T09:03:49.372 INFO:teuthology.orchestra.run.smithi069.stdout:(14/156): libcephfs2-19.0.0-3778.g3790c0fe.el9. 4.9 MB/s | 756 kB 00:00 2024-05-17T09:03:49.447 INFO:teuthology.orchestra.run.smithi069.stdout:(15/156): libcephsqlite-19.0.0-3778.g3790c0fe.e 2.0 MB/s | 158 kB 00:00 2024-05-17T09:03:49.523 INFO:teuthology.orchestra.run.smithi069.stdout:(16/156): librados-devel-19.0.0-3778.g3790c0fe. 1.7 MB/s | 127 kB 00:00 2024-05-17T09:03:49.656 INFO:teuthology.orchestra.run.smithi069.stdout:(17/156): libradosstriper1-19.0.0-3778.g3790c0f 3.7 MB/s | 497 kB 00:00 2024-05-17T09:03:50.707 INFO:teuthology.orchestra.run.smithi069.stdout:(18/156): ceph-test-19.0.0-3778.g3790c0fe.el9.x 9.9 MB/s | 43 MB 00:04 2024-05-17T09:03:50.782 INFO:teuthology.orchestra.run.smithi069.stdout:(19/156): librgw2-19.0.0-3778.g3790c0fe.el9.x86 4.6 MB/s | 5.2 MB 00:01 2024-05-17T09:03:50.807 INFO:teuthology.orchestra.run.smithi069.stdout:(20/156): python3-ceph-argparse-19.0.0-3778.g37 452 kB/s | 45 kB 00:00 2024-05-17T09:03:50.832 INFO:teuthology.orchestra.run.smithi069.stdout:(21/156): python3-ceph-common-19.0.0-3778.g3790 2.6 MB/s | 136 kB 00:00 2024-05-17T09:03:50.849 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-05-17T09:03:50.857 INFO:teuthology.orchestra.run.smithi069.stdout:(22/156): python3-cephfs-19.0.0-3778.g3790c0fe. 3.1 MB/s | 161 kB 00:00 2024-05-17T09:03:50.908 INFO:teuthology.orchestra.run.smithi069.stdout:(23/156): python3-rados-19.0.0-3778.g3790c0fe.e 4.2 MB/s | 320 kB 00:00 2024-05-17T09:03:50.933 INFO:teuthology.orchestra.run.smithi069.stdout:(24/156): python3-rbd-19.0.0-3778.g3790c0fe.el9 3.9 MB/s | 301 kB 00:00 2024-05-17T09:03:50.958 INFO:teuthology.orchestra.run.smithi069.stdout:(25/156): python3-rgw-19.0.0-3778.g3790c0fe.el9 1.9 MB/s | 99 kB 00:00 2024-05-17T09:03:50.992 INFO:teuthology.orchestra.run.smithi069.stdout:(26/156): rbd-fuse-19.0.0-3778.g3790c0fe.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-17T09:03:51.051 INFO:teuthology.orchestra.run.smithi069.stdout:(27/156): rbd-nbd-19.0.0-3778.g3790c0fe.el9.x86 2.8 MB/s | 169 kB 00:00 2024-05-17T09:03:51.118 INFO:teuthology.orchestra.run.smithi069.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3778.g 514 kB/s | 34 kB 00:00 2024-05-17T09:03:51.201 INFO:teuthology.orchestra.run.smithi069.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3778.g3790c0f 1.7 MB/s | 143 kB 00:00 2024-05-17T09:03:51.418 INFO:teuthology.orchestra.run.smithi069.stdout:(30/156): rbd-mirror-19.0.0-3778.g3790c0fe.el9. 6.6 MB/s | 3.0 MB 00:00 2024-05-17T09:03:51.619 INFO:teuthology.orchestra.run.smithi069.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3778.g3790c 8.5 MB/s | 3.5 MB 00:00 2024-05-17T09:03:51.695 INFO:teuthology.orchestra.run.smithi069.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3778.g37 3.6 MB/s | 274 kB 00:00 2024-05-17T09:03:51.760 INFO:teuthology.orchestra.run.smithi069.stdout:(33/156): ceph-mgr-rook-19.0.0-3778.g3790c0fe.e 742 kB/s | 49 kB 00:00 2024-05-17T09:03:51.828 INFO:teuthology.orchestra.run.smithi069.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3778.g3 247 kB/s | 16 kB 00:00 2024-05-17T09:03:51.895 INFO:teuthology.orchestra.run.smithi069.stdout:(35/156): ceph-volume-19.0.0-3778.g3790c0fe.el9 4.0 MB/s | 273 kB 00:00 2024-05-17T09:03:52.020 INFO:teuthology.orchestra.run.smithi069.stdout:(36/156): cephadm-19.0.0-3778.g3790c0fe.el9.noa 6.0 MB/s | 764 kB 00:00 2024-05-17T09:03:52.079 INFO:teuthology.orchestra.run.smithi069.stdout:(37/156): cephfs-top-19.0.0-3778.g3790c0fe.el9. 313 kB/s | 18 kB 00:00 2024-05-17T09:03:52.224 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-mon-19.0.0-3778.g3790c0fe.el9.x86 1.5 MB/s | 4.9 MB 00:03 2024-05-17T09:03:52.229 INFO:teuthology.orchestra.run.smithi069.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 9.1 MB/s | 7.4 MB 00:00 2024-05-17T09:03:52.296 INFO:teuthology.orchestra.run.smithi069.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-05-17T09:03:52.446 INFO:teuthology.orchestra.run.smithi069.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-17T09:03:52.571 INFO:teuthology.orchestra.run.smithi069.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-05-17T09:03:52.597 INFO:teuthology.orchestra.run.smithi069.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-05-17T09:03:52.664 INFO:teuthology.orchestra.run.smithi069.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-17T09:03:52.739 INFO:teuthology.orchestra.run.smithi069.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-17T09:03:52.814 INFO:teuthology.orchestra.run.smithi069.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-17T09:03:52.890 INFO:teuthology.orchestra.run.smithi069.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-17T09:03:52.990 INFO:teuthology.orchestra.run.smithi069.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-05-17T09:03:53.199 INFO:teuthology.orchestra.run.smithi069.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-05-17T09:03:53.257 INFO:teuthology.orchestra.run.smithi069.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-17T09:03:53.316 INFO:teuthology.orchestra.run.smithi069.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-05-17T09:03:53.341 INFO:teuthology.orchestra.run.smithi069.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-05-17T09:03:53.375 INFO:teuthology.orchestra.run.smithi069.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-17T09:03:53.400 INFO:teuthology.orchestra.run.smithi069.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-05-17T09:03:53.459 INFO:teuthology.orchestra.run.smithi069.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-05-17T09:03:53.517 INFO:teuthology.orchestra.run.smithi069.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-05-17T09:03:53.568 INFO:teuthology.orchestra.run.smithi069.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-17T09:03:53.626 INFO:teuthology.orchestra.run.smithi069.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-05-17T09:03:53.718 INFO:teuthology.orchestra.run.smithi069.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-05-17T09:03:53.793 INFO:teuthology.orchestra.run.smithi069.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-05-17T09:03:53.819 INFO:teuthology.orchestra.run.smithi069.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-05-17T09:03:53.852 INFO:teuthology.orchestra.run.smithi069.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-05-17T09:03:53.877 INFO:teuthology.orchestra.run.smithi069.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-05-17T09:03:53.936 INFO:teuthology.orchestra.run.smithi069.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-05-17T09:03:53.961 INFO:teuthology.orchestra.run.smithi069.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-17T09:03:54.012 INFO:teuthology.orchestra.run.smithi069.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-17T09:03:54.379 INFO:teuthology.orchestra.run.smithi069.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-05-17T09:03:54.554 INFO:teuthology.orchestra.run.smithi069.stdout:(67/156): ceph-radosgw-19.0.0-3778.g3790c0fe.el 1.2 MB/s | 10 MB 00:08 2024-05-17T09:03:54.596 INFO:teuthology.orchestra.run.smithi069.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-05-17T09:03:54.663 INFO:teuthology.orchestra.run.smithi069.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 6.8 MB/s | 4.7 MB 00:00 2024-05-17T09:03:54.688 INFO:teuthology.orchestra.run.smithi069.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-17T09:03:54.797 INFO:teuthology.orchestra.run.smithi069.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-17T09:03:54.864 INFO:teuthology.orchestra.run.smithi069.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-17T09:03:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:(73/156): postgresql-contrib-13.14-1.el9.x86_64 1.9 MB/s | 911 kB 00:00 2024-05-17T09:03:55.148 INFO:teuthology.orchestra.run.smithi069.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-17T09:03:55.224 INFO:teuthology.orchestra.run.smithi069.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-05-17T09:03:55.299 INFO:teuthology.orchestra.run.smithi069.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-17T09:03:55.324 INFO:teuthology.orchestra.run.smithi069.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-17T09:03:55.349 INFO:teuthology.orchestra.run.smithi069.stdout:(78/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-05-17T09:03:55.375 INFO:teuthology.orchestra.run.smithi069.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-05-17T09:03:55.400 INFO:teuthology.orchestra.run.smithi069.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-17T09:03:55.425 INFO:teuthology.orchestra.run.smithi069.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-05-17T09:03:55.475 INFO:teuthology.orchestra.run.smithi069.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-17T09:03:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 927 kB/s | 77 kB 00:00 2024-05-17T09:03:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-05-17T09:03:55.702 INFO:teuthology.orchestra.run.smithi069.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 297 kB/s | 42 kB 00:00 2024-05-17T09:03:55.769 INFO:teuthology.orchestra.run.smithi069.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-17T09:03:55.794 INFO:teuthology.orchestra.run.smithi069.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-17T09:03:55.820 INFO:teuthology.orchestra.run.smithi069.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-05-17T09:03:55.845 INFO:teuthology.orchestra.run.smithi069.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-17T09:03:55.871 INFO:teuthology.orchestra.run.smithi069.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 670 kB/s | 51 kB 00:00 2024-05-17T09:03:55.896 INFO:teuthology.orchestra.run.smithi069.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 705 kB/s | 54 kB 00:00 2024-05-17T09:03:55.947 INFO:teuthology.orchestra.run.smithi069.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 549 kB/s | 42 kB 00:00 2024-05-17T09:03:55.972 INFO:teuthology.orchestra.run.smithi069.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 581 kB/s | 44 kB 00:00 2024-05-17T09:03:55.995 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-17T09:03:56.057 INFO:teuthology.orchestra.run.smithi069.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 428 kB/s | 36 kB 00:00 2024-05-17T09:03:56.132 INFO:teuthology.orchestra.run.smithi069.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-17T09:03:56.224 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-radosgw-19.0.0-3778.g3790c0fe.el9 2.5 MB/s | 10 MB 00:03 2024-05-17T09:03:56.249 INFO:teuthology.orchestra.run.smithi069.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-17T09:03:56.307 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-selinux-19.0.0-3778.g3790c0fe.el9 303 kB/s | 25 kB 00:00 2024-05-17T09:03:56.308 INFO:teuthology.orchestra.run.smithi069.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-05-17T09:03:56.350 INFO:teuthology.orchestra.run.smithi069.stdout:(98/156): qatlib-24.02.0-1.el9.x86_64.rpm 548 kB/s | 221 kB 00:00 2024-05-17T09:03:56.375 INFO:teuthology.orchestra.run.smithi069.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-17T09:03:56.409 INFO:teuthology.orchestra.run.smithi069.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-17T09:03:56.442 INFO:teuthology.orchestra.run.smithi069.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-05-17T09:03:56.501 INFO:teuthology.orchestra.run.smithi069.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-17T09:03:56.710 INFO:teuthology.orchestra.run.smithi069.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-17T09:03:56.802 INFO:teuthology.orchestra.run.smithi069.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-05-17T09:03:57.127 INFO:teuthology.orchestra.run.smithi069.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 60 kB/s | 19 kB 00:00 2024-05-17T09:03:57.277 INFO:teuthology.orchestra.run.smithi069.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 706 kB/s | 548 kB 00:00 2024-05-17T09:03:57.302 INFO:teuthology.orchestra.run.smithi069.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-05-17T09:03:57.361 INFO:teuthology.orchestra.run.smithi069.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-05-17T09:03:57.394 INFO:teuthology.orchestra.run.smithi069.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-05-17T09:03:57.462 INFO:teuthology.orchestra.run.smithi069.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-05-17T09:03:57.503 INFO:teuthology.orchestra.run.smithi069.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-17T09:03:57.654 INFO:teuthology.orchestra.run.smithi069.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-05-17T09:03:57.754 INFO:teuthology.orchestra.run.smithi069.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-17T09:03:57.779 INFO:teuthology.orchestra.run.smithi069.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-05-17T09:03:57.871 INFO:teuthology.orchestra.run.smithi069.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-05-17T09:03:57.897 INFO:teuthology.orchestra.run.smithi069.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-05-17T09:03:57.899 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-common-19.0.0-3778.g3790c0fe.el9 1.3 MB/s | 20 MB 00:14 2024-05-17T09:03:57.922 INFO:teuthology.orchestra.run.smithi069.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 192 kB/s | 32 kB 00:00 2024-05-17T09:03:57.973 INFO:teuthology.orchestra.run.smithi069.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-17T09:03:57.998 INFO:teuthology.orchestra.run.smithi069.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-17T09:03:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-17T09:03:58.049 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.0.0-3778.g3790c0fe.e 1.5 MB/s | 224 kB 00:00 2024-05-17T09:03:58.065 INFO:teuthology.orchestra.run.smithi069.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-05-17T09:03:58.090 INFO:teuthology.orchestra.run.smithi069.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-17T09:03:58.133 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.0.0-3778.g3790c0fe.el 324 kB/s | 27 kB 00:00 2024-05-17T09:03:58.174 INFO:teuthology.orchestra.run.smithi069.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-05-17T09:03:58.199 INFO:teuthology.orchestra.run.smithi069.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-05-17T09:03:58.224 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.0.0-3778.g3790c0fe 347 kB/s | 32 kB 00:00 2024-05-17T09:03:58.266 INFO:teuthology.orchestra.run.smithi069.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-17T09:03:58.287 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T09:03:58.288 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T09:03:58.291 INFO:teuthology.orchestra.run.smithi069.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-17T09:03:58.298 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T09:03:58.299 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T09:03:58.300 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T09:03:58.301 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T09:03:58.302 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T09:03:58.305 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T09:03:58.305 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T09:03:58.333 INFO:teuthology.orchestra.run.smithi069.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-05-17T09:03:58.365 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-05-17T09:03:58.378 INFO:teuthology.orchestra.run.smithi069.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 173 kB/s | 19 kB 00:00 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 6.6 k 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 5.4 M 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 863 k 2024-05-17T09:03:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 147 k 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 1.5 M 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 143 k 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 3.5 M 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 7.4 M 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 49 k 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 10 M 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 43 M 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 273 k 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 764 k 2024-05-17T09:03:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 224 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 27 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 18 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 32 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 756 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 127 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T09:03:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 161 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 320 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 301 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 99 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 86 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.0 M 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 169 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T09:03:58.390 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.3 M 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 3.1 M 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T09:03:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 20 M 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 34 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 2.3 M 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 274 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 4.9 M 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 18 M 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3778.g3790c0fe.el9 ceph-noarch 16 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 25 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T09:03:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 158 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T09:03:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 497 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 5.2 M 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T09:03:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 45 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.0.0-3778.g3790c0fe.el9 ceph 136 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T09:03:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T09:03:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T09:03:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T09:03:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T09:03:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-05-17T09:03:58.400 INFO:teuthology.orchestra.run.smithi079.stdout:Install 148 Packages 2024-05-17T09:03:58.401 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 8 Packages 2024-05-17T09:03:58.401 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:03:58.401 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 219 M 2024-05-17T09:03:58.401 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-05-17T09:03:58.409 INFO:teuthology.orchestra.run.smithi069.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 167 kB/s | 20 kB 00:00 2024-05-17T09:03:58.476 INFO:teuthology.orchestra.run.smithi069.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-17T09:03:58.501 INFO:teuthology.orchestra.run.smithi069.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 377 kB/s | 46 kB 00:00 2024-05-17T09:03:58.526 INFO:teuthology.orchestra.run.smithi069.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-05-17T09:03:58.567 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): libcephfs2-19.0.0-3778.g3790c0fe.el9. 2.2 MB/s | 756 kB 00:00 2024-05-17T09:03:58.568 INFO:teuthology.orchestra.run.smithi069.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 624 kB/s | 58 kB 00:00 2024-05-17T09:03:58.610 INFO:teuthology.orchestra.run.smithi069.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-17T09:03:58.635 INFO:teuthology.orchestra.run.smithi069.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-17T09:03:58.667 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): libcephsqlite-19.0.0-3778.g3790c0fe.e 1.5 MB/s | 158 kB 00:00 2024-05-17T09:03:58.677 INFO:teuthology.orchestra.run.smithi069.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-05-17T09:03:58.711 INFO:teuthology.orchestra.run.smithi069.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-05-17T09:03:58.736 INFO:teuthology.orchestra.run.smithi069.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-05-17T09:03:58.767 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): librados-devel-19.0.0-3778.g3790c0fe. 1.2 MB/s | 127 kB 00:00 2024-05-17T09:03:58.786 INFO:teuthology.orchestra.run.smithi069.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-17T09:03:58.812 INFO:teuthology.orchestra.run.smithi069.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-05-17T09:03:58.837 INFO:teuthology.orchestra.run.smithi069.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-05-17T09:03:58.887 INFO:teuthology.orchestra.run.smithi069.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-05-17T09:03:58.929 INFO:teuthology.orchestra.run.smithi069.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-17T09:03:58.946 INFO:teuthology.orchestra.run.smithi069.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-05-17T09:03:59.013 INFO:teuthology.orchestra.run.smithi069.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-17T09:03:59.034 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): libradosstriper1-19.0.0-3778.g3790c0f 1.8 MB/s | 497 kB 00:00 2024-05-17T09:03:59.047 INFO:teuthology.orchestra.run.smithi069.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-05-17T09:03:59.074 INFO:teuthology.orchestra.run.smithi069.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-17T09:03:59.313 INFO:teuthology.orchestra.run.smithi079.stdout:(1/156): ceph-19.0.0-3778.g3790c0fe.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-17T09:03:59.348 INFO:teuthology.orchestra.run.smithi069.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-05-17T09:03:59.481 INFO:teuthology.orchestra.run.smithi069.stdout:(149/156): librbd1-19.0.0-3778.g3790c0fe.el9.x8 7.6 MB/s | 3.1 MB 00:00 2024-05-17T09:04:00.165 INFO:teuthology.orchestra.run.smithi069.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 2.6 MB/s | 1.8 MB 00:00 2024-05-17T09:04:00.171 INFO:teuthology.orchestra.run.smithi079.stdout:(2/156): ceph-fuse-19.0.0-3778.g3790c0fe.el9.x8 1.0 MB/s | 863 kB 00:00 2024-05-17T09:04:00.248 INFO:teuthology.orchestra.run.smithi069.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 500 kB/s | 41 kB 00:00 2024-05-17T09:04:00.338 INFO:teuthology.orchestra.run.smithi079.stdout:(3/156): ceph-immutable-object-cache-19.0.0-377 883 kB/s | 147 kB 00:00 2024-05-17T09:04:00.357 INFO:teuthology.orchestra.run.smithi069.stdout:(152/156): NetworkManager-1.47.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:01 2024-05-17T09:04:00.382 INFO:teuthology.orchestra.run.smithi069.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 1.8 MB/s | 248 kB 00:00 2024-05-17T09:04:00.716 INFO:teuthology.orchestra.run.smithi069.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 1.3 MB 00:00 2024-05-17T09:04:00.718 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): librgw2-19.0.0-3778.g3790c0fe.el9.x86 3.1 MB/s | 5.2 MB 00:01 2024-05-17T09:04:00.810 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): python3-ceph-argparse-19.0.0-3778.g37 494 kB/s | 45 kB 00:00 2024-05-17T09:04:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-05-17T09:04:01.124 INFO:teuthology.orchestra.run.smithi069.stdout:(156/156): librados2-19.0.0-3778.g3790c0fe.el9. 1.6 MB/s | 3.3 MB 00:02 2024-05-17T09:04:01.130 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-05-17T09:04:01.131 INFO:teuthology.orchestra.run.smithi069.stdout:Total 11 MB/s | 219 MB 00:19 2024-05-17T09:04:01.327 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-common-19.0.0-3778.g3790 262 kB/s | 136 kB 00:00 2024-05-17T09:04:01.410 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-cephfs-19.0.0-3778.g3790c0fe. 1.9 MB/s | 161 kB 00:00 2024-05-17T09:04:01.633 INFO:teuthology.orchestra.run.smithi079.stdout:(4/156): ceph-base-19.0.0-3778.g3790c0fe.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-05-17T09:04:01.675 INFO:teuthology.orchestra.run.smithi079.stdout:(5/156): ceph-mds-19.0.0-3778.g3790c0fe.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-05-17T09:04:01.702 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): ceph-osd-19.0.0-3778.g3790c0fe.el9.x8 1.4 MB/s | 18 MB 00:12 2024-05-17T09:04:01.727 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rados-19.0.0-3778.g3790c0fe.e 1.0 MB/s | 320 kB 00:00 2024-05-17T09:04:01.803 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rgw-19.0.0-3778.g3790c0fe.el9 1.3 MB/s | 99 kB 00:00 2024-05-17T09:04:01.886 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): rbd-fuse-19.0.0-3778.g3790c0fe.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-17T09:04:01.962 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): python3-rbd-19.0.0-3778.g3790c0fe.el9 1.1 MB/s | 301 kB 00:00 2024-05-17T09:04:02.092 INFO:teuthology.orchestra.run.smithi079.stdout:(6/156): ceph-mgr-19.0.0-3778.g3790c0fe.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-05-17T09:04:02.152 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.0.0-3778.g3790c0fe.el9.x86 924 kB/s | 169 kB 00:00 2024-05-17T09:04:02.221 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3778.g 456 kB/s | 34 kB 00:00 2024-05-17T09:04:02.329 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3778.g3790c0f 1.3 MB/s | 143 kB 00:00 2024-05-17T09:04:02.353 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-05-17T09:04:02.503 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-05-17T09:04:02.503 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-05-17T09:04:02.663 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): rbd-mirror-19.0.0-3778.g3790c0fe.el9. 3.9 MB/s | 3.0 MB 00:00 2024-05-17T09:04:03.850 INFO:teuthology.orchestra.run.smithi079.stdout:(7/156): ceph-mon-19.0.0-3778.g3790c0fe.el9.x86 2.2 MB/s | 4.9 MB 00:02 2024-05-17T09:04:04.088 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3778.g3790c 2.0 MB/s | 3.5 MB 00:01 2024-05-17T09:04:04.197 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-05-17T09:04:04.263 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-rook-19.0.0-3778.g3790c0fe.e 742 kB/s | 49 kB 00:00 2024-05-17T09:04:04.289 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-mgr-modules-core-19.0.0-3778.g37 1.3 MB/s | 274 kB 00:00 2024-05-17T09:04:04.332 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3778.g3 245 kB/s | 16 kB 00:00 2024-05-17T09:04:04.464 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): ceph-volume-19.0.0-3778.g3790c0fe.el9 1.5 MB/s | 273 kB 00:00 2024-05-17T09:04:04.506 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): cephadm-19.0.0-3778.g3790c0fe.el9.noa 4.3 MB/s | 764 kB 00:00 2024-05-17T09:04:04.531 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): cephfs-top-19.0.0-3778.g3790c0fe.el9. 273 kB/s | 18 kB 00:00 2024-05-17T09:04:04.550 INFO:teuthology.orchestra.run.smithi079.stdout:(8/156): ceph-common-19.0.0-3778.g3790c0fe.el9. 3.5 MB/s | 20 MB 00:05 2024-05-17T09:04:04.587 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-05-17T09:04:04.588 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-05-17T09:04:04.625 INFO:teuthology.orchestra.run.smithi079.stdout:(9/156): ceph-selinux-19.0.0-3778.g3790c0fe.el9 336 kB/s | 25 kB 00:00 2024-05-17T09:04:04.982 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 169 kB/s | 80 kB 00:00 2024-05-17T09:04:05.274 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 669 kB/s | 195 kB 00:00 2024-05-17T09:04:05.341 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 993 kB/s | 803 kB 00:00 2024-05-17T09:04:05.434 INFO:teuthology.orchestra.run.smithi079.stdout:(10/156): ceph-osd-19.0.0-3778.g3790c0fe.el9.x8 5.3 MB/s | 18 MB 00:03 2024-05-17T09:04:05.516 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-05-17T09:04:05.526 INFO:teuthology.orchestra.run.smithi079.stdout:(11/156): cephfs-mirror-19.0.0-3778.g3790c0fe.e 2.4 MB/s | 224 kB 00:00 2024-05-17T09:04:05.593 INFO:teuthology.orchestra.run.smithi079.stdout:(12/156): cephfs-shell-19.0.0-3778.g3790c0fe.el 406 kB/s | 27 kB 00:00 2024-05-17T09:04:05.617 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-17T09:04:05.658 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-05-17T09:04:05.660 INFO:teuthology.orchestra.run.smithi079.stdout:(13/156): libcephfs-devel-19.0.0-3778.g3790c0fe 475 kB/s | 32 kB 00:00 2024-05-17T09:04:05.717 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-17T09:04:05.767 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-17T09:04:05.826 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-17T09:04:05.830 INFO:teuthology.orchestra.run.smithi079.stdout:(14/156): libcephfs2-19.0.0-3778.g3790c0fe.el9. 4.4 MB/s | 756 kB 00:00 2024-05-17T09:04:05.919 INFO:teuthology.orchestra.run.smithi079.stdout:(15/156): libcephsqlite-19.0.0-3778.g3790c0fe.e 1.7 MB/s | 158 kB 00:00 2024-05-17T09:04:05.934 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-05-17T09:04:05.985 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-17T09:04:05.994 INFO:teuthology.orchestra.run.smithi079.stdout:(16/156): librados-devel-19.0.0-3778.g3790c0fe. 1.7 MB/s | 127 kB 00:00 2024-05-17T09:04:06.035 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-17T09:04:06.069 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-05-17T09:04:06.094 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-17T09:04:06.111 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 783 kB/s | 33 kB 00:00 2024-05-17T09:04:06.111 INFO:teuthology.orchestra.run.smithi079.stdout:(17/156): libradosstriper1-19.0.0-3778.g3790c0f 4.2 MB/s | 497 kB 00:00 2024-05-17T09:04:06.161 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-17T09:04:06.211 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-17T09:04:06.262 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-17T09:04:06.295 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-05-17T09:04:06.354 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-17T09:04:06.404 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-05-17T09:04:06.455 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.3 MB/s | 3.0 MB 00:00 2024-05-17T09:04:06.471 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-05-17T09:04:06.489 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-17T09:04:06.522 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-17T09:04:06.547 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-17T09:04:06.564 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-05-17T09:04:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-17T09:04:06.923 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-17T09:04:06.965 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-17T09:04:06.978 INFO:teuthology.orchestra.run.smithi079.stdout:(18/156): librgw2-19.0.0-3778.g3790c0fe.el9.x86 6.0 MB/s | 5.2 MB 00:00 2024-05-17T09:04:06.999 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-17T09:04:07.037 INFO:teuthology.orchestra.run.smithi079.stdout:(19/156): python3-ceph-argparse-19.0.0-3778.g37 775 kB/s | 45 kB 00:00 2024-05-17T09:04:07.040 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-05-17T09:04:07.108 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-17T09:04:07.112 INFO:teuthology.orchestra.run.smithi079.stdout:(20/156): python3-ceph-common-19.0.0-3778.g3790 1.8 MB/s | 136 kB 00:00 2024-05-17T09:04:07.175 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-05-17T09:04:07.200 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 955 kB/s | 87 kB 00:00 2024-05-17T09:04:07.242 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-17T09:04:07.246 INFO:teuthology.orchestra.run.smithi079.stdout:(21/156): ceph-radosgw-19.0.0-3778.g3790c0fe.el 3.0 MB/s | 10 MB 00:03 2024-05-17T09:04:07.279 INFO:teuthology.orchestra.run.smithi079.stdout:(22/156): python3-cephfs-19.0.0-3778.g3790c0fe. 966 kB/s | 161 kB 00:00 2024-05-17T09:04:07.292 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-17T09:04:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-05-17T09:04:07.354 INFO:teuthology.orchestra.run.smithi079.stdout:(23/156): python3-rados-19.0.0-3778.g3790c0fe.e 2.9 MB/s | 320 kB 00:00 2024-05-17T09:04:07.380 INFO:teuthology.orchestra.run.smithi079.stdout:(24/156): python3-rbd-19.0.0-3778.g3790c0fe.el9 2.9 MB/s | 301 kB 00:00 2024-05-17T09:04:07.385 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-05-17T09:04:07.422 INFO:teuthology.orchestra.run.smithi079.stdout:(25/156): python3-rgw-19.0.0-3778.g3790c0fe.el9 1.4 MB/s | 99 kB 00:00 2024-05-17T09:04:07.435 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-17T09:04:07.447 INFO:teuthology.orchestra.run.smithi079.stdout:(26/156): rbd-fuse-19.0.0-3778.g3790c0fe.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-17T09:04:07.477 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-05-17T09:04:07.514 INFO:teuthology.orchestra.run.smithi079.stdout:(27/156): rbd-nbd-19.0.0-3778.g3790c0fe.el9.x86 2.5 MB/s | 169 kB 00:00 2024-05-17T09:04:07.572 INFO:teuthology.orchestra.run.smithi079.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3778.g 588 kB/s | 34 kB 00:00 2024-05-17T09:04:07.639 INFO:teuthology.orchestra.run.smithi079.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3778.g3790c0f 2.1 MB/s | 143 kB 00:00 2024-05-17T09:04:07.661 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-17T09:04:07.711 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-17T09:04:07.761 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-05-17T09:04:07.845 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-17T09:04:07.870 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-05-17T09:04:07.920 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 563 kB/s | 42 kB 00:00 2024-05-17T09:04:07.954 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-05-17T09:04:07.979 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-17T09:04:08.005 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-17T09:04:08.030 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-17T09:04:08.063 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-05-17T09:04:08.106 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-17T09:04:08.164 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-05-17T09:04:08.164 INFO:teuthology.orchestra.run.smithi079.stdout:(30/156): rbd-mirror-19.0.0-3778.g3790c0fe.el9. 4.1 MB/s | 3.0 MB 00:00 2024-05-17T09:04:08.214 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-17T09:04:08.223 INFO:teuthology.orchestra.run.smithi079.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3778.g3790c 6.1 MB/s | 3.5 MB 00:00 2024-05-17T09:04:08.273 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 619 kB/s | 36 kB 00:00 2024-05-17T09:04:08.307 INFO:teuthology.orchestra.run.smithi079.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3778.g37 3.2 MB/s | 274 kB 00:00 2024-05-17T09:04:08.315 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-17T09:04:08.365 INFO:teuthology.orchestra.run.smithi079.stdout:(33/156): ceph-mgr-rook-19.0.0-3778.g3790c0fe.e 847 kB/s | 49 kB 00:00 2024-05-17T09:04:08.416 INFO:teuthology.orchestra.run.smithi079.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3778.g3 330 kB/s | 16 kB 00:00 2024-05-17T09:04:08.483 INFO:teuthology.orchestra.run.smithi079.stdout:(35/156): ceph-volume-19.0.0-3778.g3790c0fe.el9 4.0 MB/s | 273 kB 00:00 2024-05-17T09:04:08.498 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-05-17T09:04:08.524 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-05-17T09:04:08.541 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-17T09:04:08.575 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-17T09:04:08.600 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-17T09:04:08.617 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-17T09:04:08.650 INFO:teuthology.orchestra.run.smithi079.stdout:(36/156): cephadm-19.0.0-3778.g3790c0fe.el9.noa 4.5 MB/s | 764 kB 00:00 2024-05-17T09:04:08.717 INFO:teuthology.orchestra.run.smithi079.stdout:(37/156): cephfs-top-19.0.0-3778.g3790c0fe.el9. 274 kB/s | 18 kB 00:00 2024-05-17T09:04:08.884 INFO:teuthology.orchestra.run.smithi079.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-05-17T09:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-05-17T09:04:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-05-17T09:04:09.101 INFO:teuthology.orchestra.run.smithi079.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-05-17T09:04:09.168 INFO:teuthology.orchestra.run.smithi079.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-17T09:04:09.344 INFO:teuthology.orchestra.run.smithi079.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-05-17T09:04:09.402 INFO:teuthology.orchestra.run.smithi079.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-17T09:04:09.460 INFO:teuthology.orchestra.run.smithi079.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-17T09:04:09.509 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 968 kB/s | 863 kB 00:00 2024-05-17T09:04:09.519 INFO:teuthology.orchestra.run.smithi079.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-17T09:04:09.586 INFO:teuthology.orchestra.run.smithi079.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-17T09:04:09.659 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 707 kB/s | 548 kB 00:00 2024-05-17T09:04:09.759 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-17T09:04:10.003 INFO:teuthology.orchestra.run.smithi079.stdout:(47/156): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-05-17T09:04:10.485 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): ceph-test-19.0.0-3778.g3790c0fe.el9. 3.0 MB/s | 43 MB 00:14 2024-05-17T09:04:10.510 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 309 kB/s | 309 kB 00:01 2024-05-17T09:04:10.577 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-05-17T09:04:10.603 INFO:teuthology.orchestra.run.smithi079.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 678 kB/s | 689 kB 00:01 2024-05-17T09:04:10.627 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-05-17T09:04:10.711 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 648 kB/s | 67 kB 00:00 2024-05-17T09:04:10.746 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-05-17T09:04:10.753 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-17T09:04:10.828 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-17T09:04:10.838 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T09:04:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 67 kB/s | 26 kB 00:00 2024-05-17T09:04:10.875 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T09:04:10.929 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-05-17T09:04:10.962 INFO:teuthology.orchestra.run.smithi079.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 985 kB/s | 943 kB 00:00 2024-05-17T09:04:10.970 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-05-17T09:04:11.012 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2024-05-17T09:04:11.038 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-17T09:04:11.080 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-17T09:04:11.098 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T09:04:11.114 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-17T09:04:11.147 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T09:04:11.172 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T09:04:11.197 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-05-17T09:04:11.222 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-05-17T09:04:11.223 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T09:04:11.322 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-17T09:04:11.356 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 953 kB/s | 303 kB 00:00 2024-05-17T09:04:11.423 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-05-17T09:04:11.465 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-05-17T09:04:11.479 INFO:teuthology.orchestra.run.smithi079.stdout:(50/156): ceph-test-19.0.0-3778.g3790c0fe.el9.x 6.3 MB/s | 43 MB 00:06 2024-05-17T09:04:11.490 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-05-17T09:04:11.524 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-17T09:04:11.557 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T09:04:11.566 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-05-17T09:04:11.600 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 180 kB/s | 20 kB 00:00 2024-05-17T09:04:11.604 INFO:teuthology.orchestra.run.smithi079.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 107 kB/s | 107 kB 00:00 2024-05-17T09:04:11.641 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-17T09:04:11.664 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:11.667 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 460 kB/s | 46 kB 00:00 2024-05-17T09:04:11.709 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 724 kB/s | 79 kB 00:00 2024-05-17T09:04:11.742 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-05-17T09:04:11.776 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-17T09:04:11.790 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:11.810 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-05-17T09:04:11.824 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:11.843 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-05-17T09:04:11.877 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-05-17T09:04:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:11.918 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-05-17T09:04:11.929 INFO:teuthology.orchestra.run.smithi079.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 22 kB/s | 21 kB 00:00 2024-05-17T09:04:11.944 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-17T09:04:11.962 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 10/164 2024-05-17T09:04:11.978 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-05-17T09:04:12.019 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-05-17T09:04:12.045 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-05-17T09:04:12.078 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-17T09:04:12.132 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T09:04:12.137 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2024-05-17T09:04:12.162 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-05-17T09:04:12.187 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-05-17T09:04:12.192 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T09:04:12.213 INFO:teuthology.orchestra.run.smithi079.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-05-17T09:04:12.220 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:12.263 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-17T09:04:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-17T09:04:12.317 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:12.346 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 14/164 2024-05-17T09:04:12.347 INFO:teuthology.orchestra.run.smithi079.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 362 kB/s | 313 kB 00:00 2024-05-17T09:04:12.401 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 14/164 2024-05-17T09:04:12.430 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T09:04:12.451 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T09:04:12.624 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T09:04:12.730 INFO:teuthology.orchestra.run.smithi079.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-05-17T09:04:12.794 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T09:04:12.855 INFO:teuthology.orchestra.run.smithi079.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-17T09:04:12.928 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T09:04:12.930 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librbd1-19.0.0-3778.g3790c0fe.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-05-17T09:04:12.988 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T09:04:13.055 INFO:teuthology.orchestra.run.smithi079.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-05-17T09:04:13.092 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:13.181 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:13.355 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): librados2-19.0.0-3778.g3790c0fe.el9. 2.8 MB/s | 3.3 MB 00:01 2024-05-17T09:04:13.388 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:13.406 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): NetworkManager-1.47.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-05-17T09:04:13.497 INFO:teuthology.orchestra.run.smithi079.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 209 kB/s | 160 kB 00:00 2024-05-17T09:04:13.502 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T09:04:13.514 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-05-17T09:04:13.535 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T09:04:13.539 INFO:teuthology.orchestra.run.smithi079.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-05-17T09:04:13.570 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T09:04:13.643 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T09:04:13.677 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T09:04:13.706 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-team-1.47.90-1.el9.x8 118 kB/s | 41 kB 00:00 2024-05-17T09:04:13.721 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T09:04:13.748 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-05-17T09:04:13.754 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T09:04:13.781 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T09:04:13.790 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): NetworkManager-libnm-1.47.90-1.el9.x 2.1 MB/s | 1.8 MB 00:00 2024-05-17T09:04:13.850 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T09:04:13.904 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T09:04:13.934 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T09:04:13.969 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T09:04:13.989 INFO:teuthology.orchestra.run.smithi079.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 710 kB/s | 662 kB 00:00 2024-05-17T09:04:14.045 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T09:04:14.081 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9. 32/164 2024-05-17T09:04:14.115 INFO:teuthology.orchestra.run.smithi079.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-05-17T09:04:14.124 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el 33/164 2024-05-17T09:04:14.152 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_6 34/164 2024-05-17T09:04:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T09:04:14.267 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T09:04:14.342 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T09:04:14.357 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 2.4 MB 00:00 2024-05-17T09:04:14.363 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-17T09:04:14.363 INFO:teuthology.orchestra.run.smithi019.stdout:Total 6.8 MB/s | 219 MB 00:32 2024-05-17T09:04:14.415 INFO:teuthology.orchestra.run.smithi079.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 635 kB/s | 582 kB 00:00 2024-05-17T09:04:14.479 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T09:04:14.535 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T09:04:14.665 INFO:teuthology.orchestra.run.smithi079.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-05-17T09:04:14.940 INFO:teuthology.orchestra.run.smithi079.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 354 kB/s | 292 kB 00:00 2024-05-17T09:04:15.050 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T09:04:15.132 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T09:04:15.161 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T09:04:15.204 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T09:04:15.241 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_6 44/164 2024-05-17T09:04:15.250 INFO:teuthology.orchestra.run.smithi079.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 226 kB/s | 188 kB 00:00 2024-05-17T09:04:15.283 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T09:04:15.341 INFO:teuthology.orchestra.run.smithi079.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 54 kB/s | 35 kB 00:00 2024-05-17T09:04:15.459 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T09:04:15.519 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T09:04:15.595 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-17T09:04:15.609 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T09:04:15.675 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T09:04:15.748 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T09:04:15.766 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-17T09:04:15.766 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-17T09:04:15.779 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T09:04:15.815 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T09:04:15.862 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T09:04:15.898 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T09:04:15.942 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T09:04:15.983 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T09:04:16.127 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T09:04:16.150 INFO:teuthology.orchestra.run.smithi079.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-05-17T09:04:16.400 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T09:04:16.444 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T09:04:16.475 INFO:teuthology.orchestra.run.smithi079.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-05-17T09:04:16.485 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T09:04:16.508 INFO:teuthology.orchestra.run.smithi079.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-17T09:04:16.579 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T09:04:16.619 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T09:04:16.668 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T09:04:16.693 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T09:04:16.718 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T09:04:16.800 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T09:04:16.925 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T09:04:16.975 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T09:04:17.003 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T09:04:17.043 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T09:04:17.084 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T09:04:17.116 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T09:04:17.146 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T09:04:17.180 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T09:04:17.208 INFO:teuthology.orchestra.run.smithi079.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 861 kB/s | 911 kB 00:01 2024-05-17T09:04:17.214 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T09:04:17.255 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T09:04:17.275 INFO:teuthology.orchestra.run.smithi079.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 166 kB/s | 133 kB 00:00 2024-05-17T09:04:17.306 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T09:04:17.392 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T09:04:17.437 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T09:04:17.468 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T09:04:17.515 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T09:04:17.621 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T09:04:17.678 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T09:04:17.743 INFO:teuthology.orchestra.run.smithi079.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-17T09:04:18.017 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-17T09:04:18.017 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-17T09:04:18.085 INFO:teuthology.orchestra.run.smithi079.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 108 kB/s | 87 kB 00:00 2024-05-17T09:04:18.218 INFO:teuthology.orchestra.run.smithi079.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-17T09:04:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T09:04:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T09:04:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T09:04:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T09:04:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T09:04:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T09:04:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T09:04:18.910 INFO:teuthology.orchestra.run.smithi079.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 297 kB/s | 244 kB 00:00 2024-05-17T09:04:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T09:04:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T09:04:19.002 INFO:teuthology.orchestra.run.smithi079.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-05-17T09:04:19.027 INFO:teuthology.orchestra.run.smithi079.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 307 kB/s | 248 kB 00:00 2024-05-17T09:04:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T09:04:19.552 INFO:teuthology.orchestra.run.smithi079.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-05-17T09:04:19.577 INFO:teuthology.orchestra.run.smithi079.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 38 kB/s | 26 kB 00:00 2024-05-17T09:04:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T09:04:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T09:04:19.861 INFO:teuthology.orchestra.run.smithi079.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 207 kB/s | 172 kB 00:00 2024-05-17T09:04:20.170 INFO:teuthology.orchestra.run.smithi079.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-05-17T09:04:20.678 INFO:teuthology.orchestra.run.smithi079.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 307 kB/s | 250 kB 00:00 2024-05-17T09:04:20.762 INFO:teuthology.orchestra.run.smithi079.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-05-17T09:04:20.987 INFO:teuthology.orchestra.run.smithi079.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 272 kB/s | 222 kB 00:00 2024-05-17T09:04:21.047 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T09:04:21.120 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T09:04:21.148 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T09:04:21.181 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T09:04:21.395 INFO:teuthology.orchestra.run.smithi079.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 108 kB/s | 77 kB 00:00 2024-05-17T09:04:21.421 INFO:teuthology.orchestra.run.smithi079.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-05-17T09:04:21.501 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T09:04:21.544 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 101/164 2024-05-17T09:04:21.617 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 101/164 2024-05-17T09:04:21.649 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 102/164 2024-05-17T09:04:21.685 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T09:04:21.813 INFO:teuthology.orchestra.run.smithi079.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 326 kB/s | 269 kB 00:00 2024-05-17T09:04:22.170 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T09:04:22.179 INFO:teuthology.orchestra.run.smithi079.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 203 kB/s | 159 kB 00:00 2024-05-17T09:04:22.199 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 105/164 2024-05-17T09:04:22.247 INFO:teuthology.orchestra.run.smithi079.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 338 kB/s | 279 kB 00:00 2024-05-17T09:04:22.270 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 105/164 2024-05-17T09:04:22.301 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 106/164 2024-05-17T09:04:22.489 INFO:teuthology.orchestra.run.smithi079.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 75 kB/s | 51 kB 00:00 2024-05-17T09:04:22.855 INFO:teuthology.orchestra.run.smithi079.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-05-17T09:04:23.164 INFO:teuthology.orchestra.run.smithi079.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-05-17T09:04:23.498 INFO:teuthology.orchestra.run.smithi079.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-05-17T09:04:23.873 INFO:teuthology.orchestra.run.smithi079.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-17T09:04:24.148 INFO:teuthology.orchestra.run.smithi079.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 224 kB/s | 221 kB 00:00 2024-05-17T09:04:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-17T09:04:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T09:04:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T09:04:24.356 INFO:teuthology.orchestra.run.smithi079.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 42 kB/s | 36 kB 00:00 2024-05-17T09:04:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T09:04:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T09:04:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T09:04:24.483 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:24.513 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:24.517 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T09:04:24.598 INFO:teuthology.orchestra.run.smithi079.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-05-17T09:04:24.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T09:04:25.005 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:25.057 INFO:teuthology.orchestra.run.smithi079.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 335 kB/s | 305 kB 00:00 2024-05-17T09:04:25.069 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:25.082 INFO:teuthology.orchestra.run.smithi079.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 80 kB/s | 58 kB 00:00 2024-05-17T09:04:25.117 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 108/164 2024-05-17T09:04:25.134 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:25.156 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:25.211 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 108/164 2024-05-17T09:04:25.211 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T09:04:25.211 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T09:04:25.211 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:25.243 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 109/164 2024-05-17T09:04:25.248 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:25.286 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 10/164 2024-05-17T09:04:25.374 INFO:teuthology.orchestra.run.smithi079.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 125 kB/s | 94 kB 00:00 2024-05-17T09:04:25.463 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T09:04:25.525 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T09:04:25.560 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:25.641 INFO:teuthology.orchestra.run.smithi079.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-05-17T09:04:25.667 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:25.686 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 14/164 2024-05-17T09:04:25.701 INFO:teuthology.orchestra.run.smithi079.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-05-17T09:04:25.716 INFO:teuthology.orchestra.run.smithi079.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-05-17T09:04:25.749 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 14/164 2024-05-17T09:04:25.762 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T09:04:25.786 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T09:04:25.942 INFO:teuthology.orchestra.run.smithi079.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-05-17T09:04:25.973 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T09:04:25.975 INFO:teuthology.orchestra.run.smithi079.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-17T09:04:26.001 INFO:teuthology.orchestra.run.smithi079.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-05-17T09:04:26.042 INFO:teuthology.orchestra.run.smithi079.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 623 kB/s | 26 kB 00:00 2024-05-17T09:04:26.093 INFO:teuthology.orchestra.run.smithi079.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-05-17T09:04:26.135 INFO:teuthology.orchestra.run.smithi079.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-17T09:04:26.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T09:04:26.177 INFO:teuthology.orchestra.run.smithi079.stdout:(110/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-05-17T09:04:26.261 INFO:teuthology.orchestra.run.smithi079.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-05-17T09:04:26.293 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T09:04:26.315 INFO:teuthology.orchestra.run.smithi079.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-05-17T09:04:26.353 INFO:teuthology.orchestra.run.smithi079.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-17T09:04:26.354 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T09:04:26.428 INFO:teuthology.orchestra.run.smithi079.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-05-17T09:04:26.456 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:26.503 INFO:teuthology.orchestra.run.smithi079.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-05-17T09:04:26.529 INFO:teuthology.orchestra.run.smithi079.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-17T09:04:26.537 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:26.571 INFO:teuthology.orchestra.run.smithi079.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-17T09:04:26.604 INFO:teuthology.orchestra.run.smithi079.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-17T09:04:26.638 INFO:teuthology.orchestra.run.smithi079.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-17T09:04:26.671 INFO:teuthology.orchestra.run.smithi079.stdout:(120/156): protobuf-compiler-3.14.0-13.el9.x86_ 838 kB/s | 863 kB 00:01 2024-05-17T09:04:26.688 INFO:teuthology.orchestra.run.smithi079.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-05-17T09:04:26.714 INFO:teuthology.orchestra.run.smithi079.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-17T09:04:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:26.764 INFO:teuthology.orchestra.run.smithi079.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-17T09:04:26.798 INFO:teuthology.orchestra.run.smithi079.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 187 kB/s | 11 kB 00:00 2024-05-17T09:04:26.831 INFO:teuthology.orchestra.run.smithi079.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 263 kB/s | 18 kB 00:00 2024-05-17T09:04:26.865 INFO:teuthology.orchestra.run.smithi079.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 351 kB/s | 23 kB 00:00 2024-05-17T09:04:26.891 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T09:04:26.898 INFO:teuthology.orchestra.run.smithi079.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-05-17T09:04:26.925 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T09:04:26.932 INFO:teuthology.orchestra.run.smithi079.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-05-17T09:04:26.961 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T09:04:26.999 INFO:teuthology.orchestra.run.smithi079.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-05-17T09:04:27.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T09:04:27.066 INFO:teuthology.orchestra.run.smithi079.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-05-17T09:04:27.088 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T09:04:27.100 INFO:teuthology.orchestra.run.smithi079.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-05-17T09:04:27.125 INFO:teuthology.orchestra.run.smithi079.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-05-17T09:04:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T09:04:27.164 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T09:04:27.175 INFO:teuthology.orchestra.run.smithi079.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-17T09:04:27.191 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T09:04:27.200 INFO:teuthology.orchestra.run.smithi079.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-17T09:04:27.226 INFO:teuthology.orchestra.run.smithi079.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-17T09:04:27.251 INFO:teuthology.orchestra.run.smithi079.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-05-17T09:04:27.264 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T09:04:27.276 INFO:teuthology.orchestra.run.smithi079.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-05-17T09:04:27.301 INFO:teuthology.orchestra.run.smithi079.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-17T09:04:27.315 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T09:04:27.327 INFO:teuthology.orchestra.run.smithi079.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-17T09:04:27.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T09:04:27.352 INFO:teuthology.orchestra.run.smithi079.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-17T09:04:27.377 INFO:teuthology.orchestra.run.smithi079.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-17T09:04:27.387 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T09:04:27.411 INFO:teuthology.orchestra.run.smithi079.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-17T09:04:27.445 INFO:teuthology.orchestra.run.smithi079.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-17T09:04:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T09:04:27.478 INFO:teuthology.orchestra.run.smithi079.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-17T09:04:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9. 32/164 2024-05-17T09:04:27.503 INFO:teuthology.orchestra.run.smithi079.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-17T09:04:27.529 INFO:teuthology.orchestra.run.smithi079.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-17T09:04:27.542 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el 33/164 2024-05-17T09:04:27.570 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_6 34/164 2024-05-17T09:04:27.621 INFO:teuthology.orchestra.run.smithi079.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-17T09:04:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T09:04:27.686 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T09:04:27.704 INFO:teuthology.orchestra.run.smithi079.stdout:(148/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:01 2024-05-17T09:04:27.764 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T09:04:27.909 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T09:04:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T09:04:28.130 INFO:teuthology.orchestra.run.smithi079.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 5.4 MB/s | 2.3 MB 00:00 2024-05-17T09:04:28.338 INFO:teuthology.orchestra.run.smithi079.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 8.7 MB/s | 1.8 MB 00:00 2024-05-17T09:04:28.397 INFO:teuthology.orchestra.run.smithi079.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 712 kB/s | 41 kB 00:00 2024-05-17T09:04:28.481 INFO:teuthology.orchestra.run.smithi079.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-05-17T09:04:28.523 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T09:04:28.611 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T09:04:28.631 INFO:teuthology.orchestra.run.smithi079.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-17T09:04:28.646 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T09:04:28.689 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T09:04:28.736 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_6 44/164 2024-05-17T09:04:28.786 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T09:04:28.839 INFO:teuthology.orchestra.run.smithi079.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-17T09:04:28.971 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T09:04:28.998 INFO:teuthology.orchestra.run.smithi079.stdout:(155/156): librados2-19.0.0-3778.g3790c0fe.el9. 2.3 MB/s | 3.3 MB 00:01 2024-05-17T09:04:29.030 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T09:04:29.125 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T09:04:29.196 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T09:04:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T09:04:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T09:04:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T09:04:29.356 INFO:teuthology.orchestra.run.smithi079.stdout:(156/156): librbd1-19.0.0-3778.g3790c0fe.el9.x8 1.8 MB/s | 3.1 MB 00:01 2024-05-17T09:04:29.363 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-05-17T09:04:29.363 INFO:teuthology.orchestra.run.smithi079.stdout:Total 7.1 MB/s | 219 MB 00:30 2024-05-17T09:04:29.400 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T09:04:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T09:04:29.478 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T09:04:29.521 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T09:04:29.665 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T09:04:29.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T09:04:29.739 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T09:04:29.781 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T09:04:29.879 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T09:04:29.914 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T09:04:29.955 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T09:04:29.987 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T09:04:30.029 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T09:04:30.120 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T09:04:30.255 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T09:04:30.295 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T09:04:30.327 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T09:04:30.365 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T09:04:30.405 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T09:04:30.445 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T09:04:30.474 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T09:04:30.509 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T09:04:30.533 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-05-17T09:04:30.542 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T09:04:30.584 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T09:04:30.619 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T09:04:30.683 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-05-17T09:04:30.683 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-05-17T09:04:30.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T09:04:30.773 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T09:04:30.805 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T09:04:30.854 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T09:04:30.963 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T09:04:31.016 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T09:04:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T09:04:31.829 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T09:04:31.862 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T09:04:31.958 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T09:04:32.013 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T09:04:32.104 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T09:04:32.139 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T09:04:32.325 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T09:04:32.362 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T09:04:32.451 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T09:04:32.885 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-05-17T09:04:32.885 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-05-17T09:04:33.146 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T09:04:33.256 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T09:04:34.593 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T09:04:34.667 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T09:04:34.694 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T09:04:34.728 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T09:04:35.072 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T09:04:35.124 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 101/164 2024-05-17T09:04:35.216 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 101/164 2024-05-17T09:04:35.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 102/164 2024-05-17T09:04:35.285 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T09:04:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T09:04:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 105/164 2024-05-17T09:04:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 105/164 2024-05-17T09:04:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 106/164 2024-05-17T09:04:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 109/164 2024-05-17T09:04:36.868 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-05-17T09:04:36.868 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-05-17T09:04:36.868 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-05-17T09:04:36.868 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-05-17T09:04:36.868 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-05-17T09:04:36.869 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-05-17T09:04:36.869 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-05-17T09:04:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:37.048 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 110/164 2024-05-17T09:04:37.064 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 110/164 2024-05-17T09:04:37.064 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:37.065 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T09:04:37.065 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:38.276 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:38.316 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:38.634 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 111/164 2024-05-17T09:04:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 111/164 2024-05-17T09:04:38.661 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:38.661 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T09:04:38.661 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T09:04:38.662 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T09:04:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:38.893 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 112/164 2024-05-17T09:04:38.908 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:38.917 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 112/164 2024-05-17T09:04:38.917 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:38.917 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T09:04:38.918 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T09:04:38.918 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T09:04:38.918 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:38.925 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-05-17T09:04:38.953 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 108/164 2024-05-17T09:04:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T09:04:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 108/164 2024-05-17T09:04:39.052 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T09:04:39.052 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T09:04:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T09:04:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 109/164 2024-05-17T09:04:39.141 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T09:04:39.177 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T09:04:39.202 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T09:04:39.260 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T09:04:39.377 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 113/164 2024-05-17T09:04:39.401 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 113/164 2024-05-17T09:04:39.401 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:39.401 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T09:04:39.401 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T09:04:39.401 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T09:04:39.401 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:39.507 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T09:04:39.572 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T09:04:39.601 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T09:04:39.631 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T09:04:39.720 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:39.809 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T09:04:39.846 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:39.861 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T09:04:39.871 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:39.938 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T09:04:39.963 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 10/164 2024-05-17T09:04:40.027 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T09:04:40.155 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T09:04:40.199 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T09:04:40.205 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T09:04:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T09:04:40.265 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T09:04:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T09:04:40.292 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:40.397 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:40.427 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 14/164 2024-05-17T09:04:40.481 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 14/164 2024-05-17T09:04:40.503 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T09:04:40.523 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T09:04:40.649 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 125/164 2024-05-17T09:04:40.677 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 125/164 2024-05-17T09:04:40.709 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.e 126/164 2024-05-17T09:04:40.715 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T09:04:40.771 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe. 127/164 2024-05-17T09:04:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 128/164 2024-05-17T09:04:40.902 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T09:04:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T09:04:41.112 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T09:04:41.218 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:41.306 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:41.532 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T09:04:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T09:04:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T09:04:41.711 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T09:04:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T09:04:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T09:04:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 128/164 2024-05-17T09:04:41.862 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T09:04:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 129/164 2024-05-17T09:04:41.895 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T09:04:41.922 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T09:04:41.992 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T09:04:42.045 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T09:04:42.075 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T09:04:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T09:04:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T09:04:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9. 32/164 2024-05-17T09:04:42.273 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el 33/164 2024-05-17T09:04:42.302 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_6 34/164 2024-05-17T09:04:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T09:04:42.425 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T09:04:42.500 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T09:04:42.631 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T09:04:42.693 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T09:04:42.830 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 129/164 2024-05-17T09:04:42.860 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 130/164 2024-05-17T09:04:42.974 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 130/164 2024-05-17T09:04:43.142 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el 131/164 2024-05-17T09:04:43.179 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 132/164 2024-05-17T09:04:43.204 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 132/164 2024-05-17T09:04:43.204 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:43.204 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T09:04:43.204 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T09:04:43.204 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T09:04:43.204 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:43.219 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T09:04:43.255 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 133/164 2024-05-17T09:04:43.267 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 133/164 2024-05-17T09:04:43.307 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T09:04:43.335 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T09:04:43.364 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 134/164 2024-05-17T09:04:43.378 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T09:04:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_6 44/164 2024-05-17T09:04:43.417 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 135/164 2024-05-17T09:04:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T09:04:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T09:04:43.694 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T09:04:43.786 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T09:04:43.850 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T09:04:43.923 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T09:04:43.954 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T09:04:43.991 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T09:04:44.037 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T09:04:44.082 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T09:04:44.125 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T09:04:44.167 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T09:04:44.303 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T09:04:44.347 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T09:04:44.365 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T09:04:44.377 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T09:04:44.391 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 137/164 2024-05-17T09:04:44.415 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 137/164 2024-05-17T09:04:44.415 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:44.415 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T09:04:44.415 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T09:04:44.415 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T09:04:44.415 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:44.418 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T09:04:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3778.g3790c 138/164 2024-05-17T09:04:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3778.g3790c 138/164 2024-05-17T09:04:44.482 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:44.482 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T09:04:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:44.513 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T09:04:44.534 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 139/164 2024-05-17T09:04:44.545 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T09:04:44.558 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 139/164 2024-05-17T09:04:44.558 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:44.558 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T09:04:44.558 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T09:04:44.558 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T09:04:44.558 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:44.586 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T09:04:44.618 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T09:04:44.643 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T09:04:44.726 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T09:04:44.852 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T09:04:44.876 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 140/164 2024-05-17T09:04:44.884 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T09:04:44.899 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 140/164 2024-05-17T09:04:44.900 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:44.900 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T09:04:44.900 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T09:04:44.900 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T09:04:44.900 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:04:44.912 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T09:04:44.956 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T09:04:44.993 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T09:04:45.025 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T09:04:45.055 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T09:04:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T09:04:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T09:04:45.156 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T09:04:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T09:04:45.277 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T09:04:45.321 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T09:04:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T09:04:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T09:04:45.530 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T09:04:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T09:04:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T09:04:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T09:04:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T09:04:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T09:04:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T09:04:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T09:04:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T09:04:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T09:04:46.876 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T09:04:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T09:04:47.609 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T09:04:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T09:04:48.965 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T09:04:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T09:04:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T09:04:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T09:04:49.146 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 141/164 2024-05-17T09:04:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 142/164 2024-05-17T09:04:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 143/164 2024-05-17T09:04:49.268 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T09:04:49.296 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T09:04:49.453 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T09:04:49.475 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 146/164 2024-05-17T09:04:49.537 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T09:04:49.559 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_ 148/164 2024-05-17T09:04:49.597 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T09:04:49.643 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T09:04:49.745 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 101/164 2024-05-17T09:04:49.836 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 101/164 2024-05-17T09:04:49.867 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 102/164 2024-05-17T09:04:49.904 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T09:04:50.255 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:04:50.299 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:04:50.320 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:04:50.395 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T09:04:50.423 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 105/164 2024-05-17T09:04:50.495 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 105/164 2024-05-17T09:04:50.525 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 106/164 2024-05-17T09:04:51.143 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 152/164 2024-05-17T09:04:51.245 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T09:04:51.254 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 109/164 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-05-17T09:04:51.255 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:51.297 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T09:04:51.377 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T09:04:51.415 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T09:04:51.437 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 110/164 2024-05-17T09:04:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T09:04:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T09:04:51.454 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 110/164 2024-05-17T09:04:51.454 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:51.455 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T09:04:51.455 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:51.517 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T09:04:51.550 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T09:04:51.550 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T09:04:51.626 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T09:04:51.651 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T09:04:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:04:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:04:51.844 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:04:51.845 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T09:04:51.938 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T09:04:52.011 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:04:52.800 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:52.838 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:53.099 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 111/164 2024-05-17T09:04:53.128 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 111/164 2024-05-17T09:04:53.128 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:53.128 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T09:04:53.128 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T09:04:53.128 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T09:04:53.128 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:53.366 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 112/164 2024-05-17T09:04:53.391 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 112/164 2024-05-17T09:04:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T09:04:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T09:04:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T09:04:53.391 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:53.400 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 107/164 2024-05-17T09:04:53.442 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 108/164 2024-05-17T09:04:53.532 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 108/164 2024-05-17T09:04:53.532 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T09:04:53.532 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T09:04:53.532 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:04:53.559 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 109/164 2024-05-17T09:04:53.875 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 113/164 2024-05-17T09:04:53.900 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 113/164 2024-05-17T09:04:53.900 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:53.900 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T09:04:53.900 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T09:04:53.901 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T09:04:53.901 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:54.014 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T09:04:54.073 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T09:04:54.144 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T09:04:54.339 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T09:04:54.511 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T09:04:54.591 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T09:04:54.690 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T09:04:54.955 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T09:04:55.009 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T09:04:55.043 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T09:04:55.070 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T09:04:55.500 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 125/164 2024-05-17T09:04:55.527 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 125/164 2024-05-17T09:04:55.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.e 126/164 2024-05-17T09:04:55.622 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe. 127/164 2024-05-17T09:04:55.649 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 128/164 2024-05-17T09:04:56.750 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 128/164 2024-05-17T09:04:56.785 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 129/164 2024-05-17T09:04:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:04:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 1/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 2/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 3/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 4/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3778.g3790c 5/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 6/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 7/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 10/164 2024-05-17T09:04:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 11/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 12/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 14/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_ 15/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 16/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 17/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_6 18/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 19/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 20/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el 21/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9. 22/164 2024-05-17T09:04:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_6 23/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 24/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 25/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 26/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 27/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 28/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 29/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe. 30/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 31/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 32/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 33/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el 34/164 2024-05-17T09:04:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 35/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.e 36/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 37/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 38/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 39/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T09:04:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T09:04:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T09:04:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T09:04:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T09:04:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T09:04:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T09:04:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T09:04:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T09:04:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 149/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 151/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T09:04:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T09:04:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T09:04:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T09:04:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T09:04:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T09:04:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T09:04:57.805 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 129/164 2024-05-17T09:04:57.827 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 130/164 2024-05-17T09:04:57.943 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 130/164 2024-05-17T09:04:58.119 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el 131/164 2024-05-17T09:04:58.146 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 132/164 2024-05-17T09:04:58.171 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 132/164 2024-05-17T09:04:58.171 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:58.171 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T09:04:58.171 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T09:04:58.171 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T09:04:58.171 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:58.222 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 133/164 2024-05-17T09:04:58.233 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 133/164 2024-05-17T09:04:58.336 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 134/164 2024-05-17T09:04:58.392 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 135/164 2024-05-17T09:04:59.391 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T09:04:59.415 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 137/164 2024-05-17T09:04:59.440 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 137/164 2024-05-17T09:04:59.440 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:59.440 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T09:04:59.440 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T09:04:59.440 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T09:04:59.441 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:59.491 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3778.g3790c 138/164 2024-05-17T09:04:59.515 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3778.g3790c 138/164 2024-05-17T09:04:59.515 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T09:04:59.516 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:59.567 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 139/164 2024-05-17T09:04:59.592 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 139/164 2024-05-17T09:04:59.592 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:59.592 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T09:04:59.592 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T09:04:59.592 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T09:04:59.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:04:59.926 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 140/164 2024-05-17T09:04:59.950 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 140/164 2024-05-17T09:04:59.950 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:04:59.950 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T09:04:59.950 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T09:04:59.951 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T09:04:59.951 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:00.490 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.491 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.492 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T09:05:00.493 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.494 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:00.495 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.496 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T09:05:00.497 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T09:05:00.498 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T09:05:00.499 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T09:05:00.500 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T09:05:00.501 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T09:05:00.502 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T09:05:00.502 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T09:05:00.502 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T09:05:00.502 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T09:05:00.502 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-05-17T09:05:00.502 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-05-17T09:05:00.901 DEBUG:teuthology.parallel:result is None 2024-05-17T09:05:04.472 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 141/164 2024-05-17T09:05:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 142/164 2024-05-17T09:05:04.555 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 143/164 2024-05-17T09:05:04.585 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T09:05:04.613 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T09:05:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 146/164 2024-05-17T09:05:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T09:05:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_ 148/164 2024-05-17T09:05:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T09:05:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 109/164 2024-05-17T09:05:04.938 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-05-17T09:05:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T09:05:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 110/164 2024-05-17T09:05:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 110/164 2024-05-17T09:05:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T09:05:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:05:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:05:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:05:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 152/164 2024-05-17T09:05:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T09:05:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T09:05:06.700 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 111/164 2024-05-17T09:05:06.727 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 111/164 2024-05-17T09:05:06.727 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:06.727 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T09:05:06.727 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T09:05:06.727 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T09:05:06.727 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T09:05:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T09:05:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T09:05:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T09:05:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T09:05:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 112/164 2024-05-17T09:05:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T09:05:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T09:05:06.984 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 112/164 2024-05-17T09:05:06.984 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:06.984 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T09:05:06.984 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T09:05:06.984 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T09:05:06.984 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T09:05:07.068 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T09:05:07.083 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:05:07.083 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:05:07.270 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:05:07.270 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T09:05:07.363 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T09:05:07.440 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:07.444 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 113/164 2024-05-17T09:05:07.467 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 113/164 2024-05-17T09:05:07.468 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:07.468 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T09:05:07.468 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T09:05:07.468 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T09:05:07.468 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T09:05:07.631 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T09:05:07.689 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T09:05:07.872 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T09:05:07.928 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T09:05:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T09:05:08.095 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T09:05:08.242 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T09:05:08.296 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T09:05:08.335 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T09:05:08.363 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T09:05:08.808 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 125/164 2024-05-17T09:05:08.836 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 125/164 2024-05-17T09:05:08.860 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.e 126/164 2024-05-17T09:05:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe. 127/164 2024-05-17T09:05:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 128/164 2024-05-17T09:05:09.988 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 128/164 2024-05-17T09:05:10.020 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 129/164 2024-05-17T09:05:10.984 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 129/164 2024-05-17T09:05:11.011 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 130/164 2024-05-17T09:05:11.123 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 130/164 2024-05-17T09:05:11.295 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el 131/164 2024-05-17T09:05:11.330 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 132/164 2024-05-17T09:05:11.354 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 132/164 2024-05-17T09:05:11.355 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:11.355 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T09:05:11.355 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T09:05:11.355 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T09:05:11.355 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:11.398 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 133/164 2024-05-17T09:05:11.409 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 133/164 2024-05-17T09:05:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 134/164 2024-05-17T09:05:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 135/164 2024-05-17T09:05:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T09:05:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 137/164 2024-05-17T09:05:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 137/164 2024-05-17T09:05:12.566 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:12.566 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T09:05:12.566 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T09:05:12.566 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T09:05:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3778.g3790c 138/164 2024-05-17T09:05:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3778.g3790c 138/164 2024-05-17T09:05:12.632 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:12.633 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T09:05:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:12.685 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 139/164 2024-05-17T09:05:12.708 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 139/164 2024-05-17T09:05:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T09:05:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T09:05:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T09:05:12.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 1/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 2/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 3/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 4/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3778.g3790c 5/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 6/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 7/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:05:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 10/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 11/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 12/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 14/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_ 15/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 16/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 17/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_6 18/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 19/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 20/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el 21/164 2024-05-17T09:05:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9. 22/164 2024-05-17T09:05:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_6 23/164 2024-05-17T09:05:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 24/164 2024-05-17T09:05:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 25/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 26/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 27/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 28/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 29/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe. 30/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 31/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 32/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 33/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el 34/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 35/164 2024-05-17T09:05:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.e 36/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 37/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 38/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 39/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T09:05:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T09:05:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T09:05:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T09:05:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T09:05:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T09:05:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T09:05:13.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T09:05:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 149/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 151/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T09:05:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T09:05:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T09:05:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 140/164 2024-05-17T09:05:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 140/164 2024-05-17T09:05:13.042 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T09:05:13.042 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T09:05:13.042 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T09:05:13.042 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T09:05:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T09:05:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T09:05:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T09:05:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T09:05:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T09:05:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T09:05:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T09:05:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T09:05:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T09:05:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T09:05:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T09:05:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T09:05:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T09:05:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T09:05:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T09:05:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:05:15.514 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-17T09:05:15.947 DEBUG:teuthology.parallel:result is None 2024-05-17T09:05:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 141/164 2024-05-17T09:05:17.365 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 142/164 2024-05-17T09:05:17.405 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 143/164 2024-05-17T09:05:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T09:05:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T09:05:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 146/164 2024-05-17T09:05:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T09:05:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_ 148/164 2024-05-17T09:05:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T09:05:17.799 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T09:05:18.414 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:05:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:05:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T09:05:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 152/164 2024-05-17T09:05:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T09:05:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T09:05:19.534 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T09:05:19.567 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T09:05:19.593 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T09:05:19.593 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T09:05:19.678 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T09:05:19.711 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T09:05:19.711 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T09:05:19.796 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T09:05:19.820 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T09:05:19.826 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:05:19.827 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:05:20.006 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T09:05:20.006 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T09:05:20.091 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T09:05:20.156 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:25.980 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:25.980 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 1/164 2024-05-17T09:05:25.980 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 2/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 3/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 4/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3778.g3790c 5/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 6/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 7/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 8/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 9/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 10/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 11/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 12/164 2024-05-17T09:05:25.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 13/164 2024-05-17T09:05:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 14/164 2024-05-17T09:05:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_ 15/164 2024-05-17T09:05:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 16/164 2024-05-17T09:05:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 17/164 2024-05-17T09:05:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_6 18/164 2024-05-17T09:05:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86 19/164 2024-05-17T09:05:25.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 20/164 2024-05-17T09:05:25.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el 21/164 2024-05-17T09:05:25.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9. 22/164 2024-05-17T09:05:25.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_6 23/164 2024-05-17T09:05:25.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 24/164 2024-05-17T09:05:25.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 25/164 2024-05-17T09:05:25.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 26/164 2024-05-17T09:05:25.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 27/164 2024-05-17T09:05:25.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 28/164 2024-05-17T09:05:25.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 29/164 2024-05-17T09:05:25.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe. 30/164 2024-05-17T09:05:25.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noa 31/164 2024-05-17T09:05:25.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.n 32/164 2024-05-17T09:05:25.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3778.g379 33/164 2024-05-17T09:05:25.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el 34/164 2024-05-17T09:05:25.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 35/164 2024-05-17T09:05:25.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.e 36/164 2024-05-17T09:05:25.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 37/164 2024-05-17T09:05:25.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 38/164 2024-05-17T09:05:25.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 39/164 2024-05-17T09:05:25.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T09:05:25.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T09:05:25.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T09:05:25.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T09:05:25.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T09:05:25.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T09:05:25.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T09:05:25.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T09:05:25.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T09:05:25.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T09:05:25.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T09:05:25.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T09:05:25.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T09:05:25.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T09:05:25.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T09:05:25.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T09:05:25.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T09:05:25.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T09:05:25.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T09:05:25.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T09:05:25.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T09:05:25.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T09:05:25.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T09:05:25.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T09:05:25.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T09:05:25.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T09:05:25.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T09:05:25.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T09:05:25.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T09:05:25.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T09:05:25.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T09:05:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T09:05:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T09:05:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T09:05:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T09:05:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T09:05:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T09:05:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T09:05:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T09:05:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T09:05:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T09:05:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T09:05:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T09:05:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T09:05:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T09:05:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T09:05:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T09:05:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T09:05:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T09:05:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T09:05:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T09:05:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T09:05:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T09:05:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T09:05:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T09:05:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T09:05:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T09:05:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T09:05:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T09:05:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T09:05:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T09:05:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T09:05:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T09:05:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T09:05:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 149/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T09:05:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 151/164 2024-05-17T09:05:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T09:05:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T09:05:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T09:05:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T09:05:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T09:05:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T09:05:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T09:05:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T09:05:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T09:05:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T09:05:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T09:05:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:19.0.0-3778.g3790c0fe.el9.noarch 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T09:05:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T09:05:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T09:05:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T09:05:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T09:05:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T09:05:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T09:05:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T09:05:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T09:05:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T09:05:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.0.0-3778.g3790c0fe.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T09:05:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T09:05:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T09:05:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:05:28.444 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-05-17T09:05:28.897 DEBUG:teuthology.parallel:result is None 2024-05-17T09:05:28.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:05:29.185 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T09:05:29.212 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3778.g3790c0fe.el9 2024-05-17T09:05:29.213 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3778.g3790c0fe.el9 2024-05-17T09:05:29.213 INFO:teuthology.task.install:The correct ceph version 19.0.0-3778.g3790c0fe is installed. 2024-05-17T09:05:29.215 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:05:29.467 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T09:05:29.494 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3778.g3790c0fe.el9 2024-05-17T09:05:29.494 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3778.g3790c0fe.el9 2024-05-17T09:05:29.494 INFO:teuthology.task.install:The correct ceph version 19.0.0-3778.g3790c0fe is installed. 2024-05-17T09:05:29.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:05:29.734 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T09:05:29.759 INFO:teuthology.orchestra.run.smithi079.stdout:19.0.0-3778.g3790c0fe.el9 2024-05-17T09:05:29.759 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3778.g3790c0fe.el9 2024-05-17T09:05:29.759 INFO:teuthology.task.install:The correct ceph version 19.0.0-3778.g3790c0fe is installed. 2024-05-17T09:05:29.761 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-17T09:05:29.761 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:05:29.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T09:05:29.795 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:05:29.795 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T09:05:29.827 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:05:29.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T09:05:29.857 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-17T09:05:29.857 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:05:29.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T09:05:29.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T09:05:29.958 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:05:29.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T09:05:29.988 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T09:05:30.057 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:05:30.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T09:05:30.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T09:05:30.154 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-17T09:05:30.154 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:05:30.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T09:05:30.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T09:05:30.247 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:05:30.247 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T09:05:30.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T09:05:30.339 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:05:30.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T09:05:30.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T09:05:30.436 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-17T09:05:30.436 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:05:30.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T09:05:30.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T09:05:30.528 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:05:30.528 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T09:05:30.553 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T09:05:30.619 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:05:30.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T09:05:30.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T09:05:30.717 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-17T09:05:30.814 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a', 'subvols': {'create': 2}} 2024-05-17T09:05:30.814 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:05:30.815 INFO:tasks.cephadm:Cluster fsid is 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:05:30.815 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-17T09:05:30.815 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.69', 'mon.c': '172.21.15.79'} 2024-05-17T09:05:30.815 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-05-17T09:05:30.815 INFO:tasks.cephadm:First mgr is x 2024-05-17T09:05:30.815 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-17T09:05:30.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-05-17T09:05:30.841 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-05-17T09:05:30.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-05-17T09:05:30.895 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-17T09:05:30.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:05:31.118 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3790c0feaaca36c33bb1b31470733325d7d91e6a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79482/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3778-g3790c0fe', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3778.g3790c0fe'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 02:57:02.276753', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.012301-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-17T09:05:31.261 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240517.012301-debug, sha1 3790c0feaaca36c33bb1b31470733325d7d91e6a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:05:31.263 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/x86_64/flavors/default/cephadm 2024-05-17T09:05:31.263 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/x86_64/flavors/default/cephadm 2024-05-17T09:05:31.264 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:31.917 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 17 09:05 /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:31.918 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:32.683 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 17 09:05 /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:32.683 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012301-debug/3790c0feaaca36c33bb1b31470733325d7d91e6a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:33.615 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 17 09:05 /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:33.615 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:33.636 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:33.656 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T09:05:33.691 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a on all hosts... 2024-05-17T09:05:33.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a pull 2024-05-17T09:05:33.694 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a pull 2024-05-17T09:05:33.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a pull 2024-05-17T09:05:34.527 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a... 2024-05-17T09:05:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a... 2024-05-17T09:05:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a... 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev)", 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9", 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6" 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-05-17T09:06:11.766 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev)", 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9", 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6" 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:06:31.970 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:06:33.891 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-17T09:06:33.892 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev)", 2024-05-17T09:06:33.892 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9", 2024-05-17T09:06:33.892 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-05-17T09:06:33.892 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6" 2024-05-17T09:06:33.892 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-05-17T09:06:33.892 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-17T09:06:33.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-05-17T09:06:33.949 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-05-17T09:06:33.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-05-17T09:06:34.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-05-17T09:06:34.038 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-05-17T09:06:34.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-05-17T09:06:34.091 INFO:tasks.cephadm:Writing seed config... 2024-05-17T09:06:34.092 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-17T09:06:34.092 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-17T09:06:34.092 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-17T09:06:34.093 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-17T09:06:34.094 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-17T09:06:34.095 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-17T09:06:34.096 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:06:34.096 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-17T09:06:34.114 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9c7a48b0-142c-11ef-bc9a-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-17T09:06:34.115 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service 2024-05-17T09:06:34.158 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-17T09:06:34.202 INFO:tasks.cephadm:Bootstrapping... 2024-05-17T09:06:34.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a -v bootstrap --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:06:34.404 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-17T09:06:34.404 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a', '-v', 'bootstrap', '--fsid', '9c7a48b0-142c-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-05-17T09:06:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-17T09:06:34.405 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-05-17T09:06:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 5.0.2 2024-05-17T09:06:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-05-17T09:06:34.431 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-05-17T09:06:34.439 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T09:06:34.439 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T09:06:34.445 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T09:06:34.445 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-17T09:06:34.452 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-17T09:06:34.458 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-17T09:06:34.459 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-17T09:06:34.459 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-05-17T09:06:34.482 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 5.0.2 2024-05-17T09:06:34.482 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-17T09:06:34.483 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-05-17T09:06:34.483 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-05-17T09:06:34.489 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T09:06:34.490 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T09:06:34.496 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T09:06:34.496 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-17T09:06:34.503 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-17T09:06:34.509 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-17T09:06:34.509 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-17T09:06:34.509 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-05-17T09:06:34.510 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:34.510 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140625524728016 on /run/cephadm/9c7a48b0-142c-11ef-bc9a-c7b262605968.lock 2024-05-17T09:06:34.510 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140625524728016 acquired on /run/cephadm/9c7a48b0-142c-11ef-bc9a-c7b262605968.lock 2024-05-17T09:06:34.510 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-05-17T09:06:34.510 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-05-17T09:06:34.510 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-17T09:06:34.514 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-05-17T09:06:34.514 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-05-17T09:06:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-17T09:06:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-17T09:06:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-17T09:06:34.519 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-17T09:06:34.519 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-17T09:06:34.520 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T09:06:34.520 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-17T09:06:34.520 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-17T09:06:34.520 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T09:06:34.521 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-17T09:06:34.521 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-17T09:06:34.521 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-17T09:06:34.521 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-17T09:06:34.521 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a... 2024-05-17T09:06:34.905 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 2024-05-17T09:06:34.906 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a... 2024-05-17T09:06:34.906 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-05-17T09:06:34.906 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:b12ea26486914772eb0b44d2bfe9967a349550bc942254a729b1335ce36a0f86 2024-05-17T09:06:34.906 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-17T09:06:34.906 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 2024-05-17T09:06:34.906 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-17T09:06:35.798 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev) 2024-05-17T09:06:35.798 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev) 2024-05-17T09:06:35.798 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-05-17T09:06:36.724 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-05-17T09:06:36.725 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-05-17T09:06:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAdHkdmuHvpAxAAmoqk0uOB3Ig4AyQUWzBdKA== 2024-05-17T09:06:38.602 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAeHkdmg7E6AhAAe0UzyRM1G6+7+oAEjL5t0w== 2024-05-17T09:06:39.524 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAeHkdmJtfWORAAOTlPk0XuPKm9/MSkwsaEJg== 2024-05-17T09:06:39.525 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-05-17T09:06:40.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:40.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T09:06:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:06:40.432 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-05-17T09:06:41.385 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-05-17T09:06:41.604 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-17T09:06:41.766 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-17T09:06:42.105 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968.target. 2024-05-17T09:06:42.105 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968.target. 2024-05-17T09:06:42.363 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a 2024-05-17T09:06:42.363 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-17T09:06:42.550 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968.target.wants/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@.service. 2024-05-17T09:06:43.069 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T09:06:43.069 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-17T09:06:43.076 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T09:06:43.076 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-17T09:06:43.076 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-17T09:06:43.076 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T09:06:43.076 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-05-17T09:06:43.077 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-05-17T09:06:43.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:43 smithi019 systemd[1]: Started Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.626682s) 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:44.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f58a4a8d700 1 Processor -- start 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f58a4a8d700 1 -- start start 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f58a4a8d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a010c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f58a4a8d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58a007d7f0 con 0x7f58a010a520 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f589e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a010c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f589e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a010c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39384/0 (socket says 172.21.15.19:39384) 2024-05-17T09:06:44.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.562+0000 7f589e59c700 1 -- 172.21.15.19:0/3785212317 learned_addr learned my addr 172.21.15.19:0/3785212317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.563+0000 7f589e59c700 1 -- 172.21.15.19:0/3785212317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a010ce80 con 0x7f58a010a520 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.563+0000 7f589e59c700 1 --2- 172.21.15.19:0/3785212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a010c940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58900098a0 tx=0x7f5890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c9f4b29a9b1b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.563+0000 7f589dd9b700 1 -- 172.21.15.19:0/3785212317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5890004160 con 0x7f58a010a520 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.563+0000 7f589dd9b700 1 -- 172.21.15.19:0/3785212317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5890004300 con 0x7f58a010a520 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.563+0000 7f589dd9b700 1 -- 172.21.15.19:0/3785212317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58900036a0 con 0x7f58a010a520 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 -- 172.21.15.19:0/3785212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 msgr2=0x7f58a010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 --2- 172.21.15.19:0/3785212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a010c940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58900098a0 tx=0x7f5890009bb0 comp rx=0 tx=0).stop 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 -- 172.21.15.19:0/3785212317 shutdown_connections 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 --2- 172.21.15.19:0/3785212317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a010c940 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 -- 172.21.15.19:0/3785212317 >> 172.21.15.19:0/3785212317 conn(0x7f58a0103f80 msgr2=0x7f58a0106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 -- 172.21.15.19:0/3785212317 shutdown_connections 2024-05-17T09:06:44.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.564+0000 7f58a4a8d700 1 -- 172.21.15.19:0/3785212317 wait complete. 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f58a4a8d700 1 Processor -- start 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f58a4a8d700 1 -- start start 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f58a4a8d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f58a4a8d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010d230 con 0x7f58a010a520 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f589e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f589e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a019d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39394/0 (socket says 172.21.15.19:39394) 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f589e59c700 1 -- 172.21.15.19:0/1733749509 learned_addr learned my addr 172.21.15.19:0/1733749509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.565+0000 7f589e59c700 1 -- 172.21.15.19:0/1733749509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a019da40 con 0x7f58a010a520 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f589e59c700 1 --2- 172.21.15.19:0/1733749509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a019d500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5890003db0 con 0x7f58a010a520 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5890011630 con 0x7f58a010a520 2024-05-17T09:06:44.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a019dcd0 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5890011950 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a01a08e0 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f589001a460 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.566+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f589001b780 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.567+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0064e10 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.570+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f589001b980 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.643+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f58a002e8b0 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.643+0000 7f58977fe700 1 -- 172.21.15.19:0/1733749509 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f589001bb60 con 0x7f58a010a520 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 msgr2=0x7f58a019d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 --2- 172.21.15.19:0/1733749509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a019d500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890004750 comp rx=0 tx=0).stop 2024-05-17T09:06:44.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 shutdown_connections 2024-05-17T09:06:44.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 --2- 172.21.15.19:0/1733749509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a010a520 0x7f58a019d500 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:44.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 >> 172.21.15.19:0/1733749509 conn(0x7f58a0103f80 msgr2=0x7f58a0106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:44.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 shutdown_connections 2024-05-17T09:06:44.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:43.644+0000 7f58a4a8d700 1 -- 172.21.15.19:0/1733749509 wait complete. 2024-05-17T09:06:44.278 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-05-17T09:06:44.278 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: monmap epoch 1 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: last_changed 2024-05-17T09:06:39.857078+0000 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: min_mon_release 19 (squid) 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: election_strategy: 1 2024-05-17T09:06:44.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:06:44.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: fsmap 2024-05-17T09:06:44.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T09:06:44.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: mgrmap e1: no daemons active 2024-05-17T09:06:44.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:44 smithi019 ceph-mon[31252]: from='client.? 172.21.15.19:0/1733749509' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-17T09:06:45.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:45 smithi019 ceph-mon[31252]: from='client.? 172.21.15.19:0/2678163455' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T09:06:45.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:45 smithi019 ceph-mon[31252]: from='client.? 172.21.15.19:0/2678163455' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-17T09:06:45.343 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T09:06:45.350 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb6960e700 1 Processor -- start 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb6960e700 1 -- start start 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb6960e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6410c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb6960e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6407d650 con 0x7fcb6410a510 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:45.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6410c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52448/0 (socket says 172.21.15.19:52448) 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb62ffd700 1 -- 172.21.15.19:0/3696199000 learned_addr learned my addr 172.21.15.19:0/3696199000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.704+0000 7fcb62ffd700 1 -- 172.21.15.19:0/3696199000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6410ce70 con 0x7fcb6410a510 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.705+0000 7fcb62ffd700 1 --2- 172.21.15.19:0/3696199000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6410c930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009e80 tx=0x7fcb5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc632530b0b3a198 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.705+0000 7fcb61ffb700 1 -- 172.21.15.19:0/3696199000 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb54017070 con 0x7fcb6410a510 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.705+0000 7fcb61ffb700 1 -- 172.21.15.19:0/3696199000 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54003a20 con 0x7fcb6410a510 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.705+0000 7fcb61ffb700 1 -- 172.21.15.19:0/3696199000 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb54003d20 con 0x7fcb6410a510 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.705+0000 7fcb6960e700 1 -- 172.21.15.19:0/3696199000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 msgr2=0x7fcb6410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.705+0000 7fcb6960e700 1 --2- 172.21.15.19:0/3696199000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6410c930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009e80 tx=0x7fcb5400b330 comp rx=0 tx=0).stop 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.706+0000 7fcb6960e700 1 -- 172.21.15.19:0/3696199000 shutdown_connections 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.706+0000 7fcb6960e700 1 --2- 172.21.15.19:0/3696199000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6410c930 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.706+0000 7fcb6960e700 1 -- 172.21.15.19:0/3696199000 >> 172.21.15.19:0/3696199000 conn(0x7fcb64104030 msgr2=0x7fcb64106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.706+0000 7fcb6960e700 1 -- 172.21.15.19:0/3696199000 shutdown_connections 2024-05-17T09:06:45.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.706+0000 7fcb6960e700 1 -- 172.21.15.19:0/3696199000 wait complete. 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.706+0000 7fcb6960e700 1 Processor -- start 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb6960e700 1 -- start start 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb6960e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb6960e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d220 con 0x7fcb6410a510 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6419d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52452/0 (socket says 172.21.15.19:52452) 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb62ffd700 1 -- 172.21.15.19:0/2678163455 learned_addr learned my addr 172.21.15.19:0/2678163455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb62ffd700 1 -- 172.21.15.19:0/2678163455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6419da70 con 0x7fcb6410a510 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb62ffd700 1 --2- 172.21.15.19:0/2678163455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6419d530 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb54004000 tx=0x7fcb54004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.707+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb54022020 con 0x7fcb6410a510 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.708+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54018040 con 0x7fcb6410a510 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.708+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6419dd00 con 0x7fcb6410a510 2024-05-17T09:06:45.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.708+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb54017070 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.708+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641a0910 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.708+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb54025020 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.708+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcb5400f050 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.709+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb64064e10 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.711+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcb5402c030 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.782+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fcb6402e850 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.785+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fcb54037440 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.786+0000 7fcb5bfff700 1 -- 172.21.15.19:0/2678163455 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcb54003a20 con 0x7fcb6410a510 2024-05-17T09:06:45.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 msgr2=0x7fcb6419d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 --2- 172.21.15.19:0/2678163455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6419d530 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb54004000 tx=0x7fcb54004ad0 comp rx=0 tx=0).stop 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 shutdown_connections 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 --2- 172.21.15.19:0/2678163455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb6410a510 0x7fcb6419d530 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 >> 172.21.15.19:0/2678163455 conn(0x7fcb64104030 msgr2=0x7fcb64105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 shutdown_connections 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:44.787+0000 7fcb6960e700 1 -- 172.21.15.19:0/2678163455 wait complete. 2024-05-17T09:06:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-05-17T09:06:46.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:46 smithi019 ceph-mon[31252]: from='client.? 172.21.15.19:0/1886027117' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:06:46.387 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.759+0000 7f96151ea700 1 Processor -- start 2024-05-17T09:06:46.387 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.759+0000 7f96151ea700 1 -- start start 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f96151ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f961010c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f96151ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f961010cbe0 con 0x7f961007d640 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f960ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f961010c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f960ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f961010c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52468/0 (socket says 172.21.15.19:52468) 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f960ed9d700 1 -- 172.21.15.19:0/1722682590 learned_addr learned my addr 172.21.15.19:0/1722682590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f960ed9d700 1 -- 172.21.15.19:0/1722682590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961010cd60 con 0x7f961007d640 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f960ed9d700 1 --2- 172.21.15.19:0/1722682590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f961010c610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95f80098a0 tx=0x7f95f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245217d381ecc631 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.760+0000 7f960dd9b700 1 -- 172.21.15.19:0/1722682590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f8004160 con 0x7f961007d640 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f960dd9b700 1 -- 172.21.15.19:0/1722682590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f95f8004300 con 0x7f961007d640 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f960dd9b700 1 -- 172.21.15.19:0/1722682590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f80119e0 con 0x7f961007d640 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 -- 172.21.15.19:0/1722682590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 msgr2=0x7f961010c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 --2- 172.21.15.19:0/1722682590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f961010c610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f95f80098a0 tx=0x7f95f8009bb0 comp rx=0 tx=0).stop 2024-05-17T09:06:46.388 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 -- 172.21.15.19:0/1722682590 shutdown_connections 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 --2- 172.21.15.19:0/1722682590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f961010c610 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 -- 172.21.15.19:0/1722682590 >> 172.21.15.19:0/1722682590 conn(0x7f9610103f80 msgr2=0x7f9610106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 -- 172.21.15.19:0/1722682590 shutdown_connections 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.761+0000 7f96151ea700 1 -- 172.21.15.19:0/1722682590 wait complete. 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.762+0000 7f96151ea700 1 Processor -- start 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.762+0000 7f96151ea700 1 -- start start 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f96151ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f9610109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f96151ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f961010dd20 con 0x7f961007d640 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f960ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f9610109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f960ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f9610109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52482/0 (socket says 172.21.15.19:52482) 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f960ed9d700 1 -- 172.21.15.19:0/1886027117 learned_addr learned my addr 172.21.15.19:0/1886027117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:46.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f960ed9d700 1 -- 172.21.15.19:0/1886027117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961010a2c0 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f960ed9d700 1 --2- 172.21.15.19:0/1886027117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f9610109d80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95f80116a0 tx=0x7f95f8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f80039f0 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f95f8003b90 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f801a600 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.763+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9610108510 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.764+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96101089f0 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.764+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f95f801ac40 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.764+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f95f8019700 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.764+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9610064e10 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.767+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f95f8024a50 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.838+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9610108c70 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.838+0000 7f9607fff700 1 -- 172.21.15.19:0/1886027117 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f95f8024430 con 0x7f961007d640 2024-05-17T09:06:46.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 msgr2=0x7f9610109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 --2- 172.21.15.19:0/1886027117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f9610109d80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95f80116a0 tx=0x7f95f8011a60 comp rx=0 tx=0).stop 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 shutdown_connections 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 --2- 172.21.15.19:0/1886027117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961007d640 0x7f9610109d80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 >> 172.21.15.19:0/1886027117 conn(0x7f9610103f80 msgr2=0x7f9610104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 shutdown_connections 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:45.839+0000 7f96151ea700 1 -- 172.21.15.19:0/1886027117 wait complete. 2024-05-17T09:06:46.391 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-05-17T09:06:46.660 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:46 smithi019 systemd[1]: Stopping Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:06:46.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:46 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31247]: 2024-05-17T09:06:46.658+0000 7f1be189d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:06:46.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:46 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31247]: 2024-05-17T09:06:46.658+0000 7f1be189d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-17T09:06:46.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:46 smithi019 podman[31497]: 2024-05-17 09:06:46.823434183 +0000 UTC m=+0.265455802 container died 3f97f2e6988743204a6dc5c27dbef88cd37679c7b77905cb9be749d48de9edaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-05-17T09:06:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 podman[31497]: 2024-05-17 09:06:47.159109902 +0000 UTC m=+0.601131509 container cleanup 3f97f2e6988743204a6dc5c27dbef88cd37679c7b77905cb9be749d48de9edaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:06:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 bash[31497]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a 2024-05-17T09:06:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 podman[31509]: 2024-05-17 09:06:47.321551615 +0000 UTC m=+0.497825733 container remove 3f97f2e6988743204a6dc5c27dbef88cd37679c7b77905cb9be749d48de9edaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:06:47.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-17T09:06:47.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 systemd[1]: Stopped Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:06:47.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 systemd[1]: Starting Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:06:47.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 podman[31601]: 2024-05-17 09:06:47.659807752 +0000 UTC m=+0.017451017 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:06:47.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 podman[31601]: 2024-05-17 09:06:47.790489585 +0000 UTC m=+0.148132821 container create 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 podman[31601]: 2024-05-17 09:06:47.979869226 +0000 UTC m=+0.337512480 container init 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:47 smithi019 podman[31601]: 2024-05-17 09:06:47.983320111 +0000 UTC m=+0.340963348 container start 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev), process ceph-mon, pid 2 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: pidfile_write: ignore empty --pid-file 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: load: jerasure load: lrc 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: RocksDB version: 7.9.2 2024-05-17T09:06:48.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Git sha 0 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Compile date 2024-05-17 02:02:33 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: DB SUMMARY 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: DB Session ID: 0RTGPFGKQ0AYGE9WC8XG 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: CURRENT file: CURRENT 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103522 ; 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.error_if_exists: 0 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.create_if_missing: 0 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.paranoid_checks: 1 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T09:06:48.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T09:06:48.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.env: 0x56142af2d500 2024-05-17T09:06:48.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T09:06:48.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.info_log: 0x56142c549fe0 2024-05-17T09:06:48.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T09:06:48.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.statistics: (nil) 2024-05-17T09:06:48.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.use_fsync: 0 2024-05-17T09:06:48.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_log_file_size: 0 2024-05-17T09:06:48.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T09:06:48.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T09:06:48.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.allow_fallocate: 1 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.use_direct_reads: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.db_log_dir: 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.wal_dir: 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T09:06:48.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.write_buffer_manager: 0x56142c51a8c0 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T09:06:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.unordered_write: 0 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T09:06:48.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.row_cache: None 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.wal_filter: None 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.two_write_queues: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.wal_compression: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.atomic_flush: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T09:06:48.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.log_readahead_size: 0 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_background_jobs: 2 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_background_compactions: -1 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_subcompactions: 1 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T09:06:48.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_open_files: -1 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_background_flushes: -1 2024-05-17T09:06:48.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Compression algorithms supported: 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kZSTD supported: 0 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kXpressCompression supported: 0 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kZlibCompression supported: 1 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kSnappyCompression supported: 1 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kLZ4Compression supported: 1 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: kBZip2Compression supported: 0 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T09:06:48.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.merge_operator: 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_filter: None 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T09:06:48.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56142c6501c0) 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-05-17T09:06:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x56142c54d1f0 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-05-17T09:06:48.076 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T09:06:48.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression: NoCompression 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.num_levels: 7 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T09:06:48.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T09:06:48.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T09:06:48.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T09:06:48.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.inplace_update_support: 0 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T09:06:48.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.bloom_locality: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.max_successive_merges: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.ttl: 2592000 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.enable_blob_files: false 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.min_blob_size: 0 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T09:06:48.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5218e8c6-e24b-444d-8a22-fde7f6a54456 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715936808023677, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-17T09:06:48.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715936808024803, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96763, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13782, "raw_average_key_size": 49, "raw_value_size": 89215, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715936808, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5218e8c6-e24b-444d-8a22-fde7f6a54456", "db_session_id": "0RTGPFGKQ0AYGE9WC8XG", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715936808024891, "job": 1, "event": "recovery_finished"} 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56142c531c00 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: DB pointer 0x561431324000 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T09:06:48.085 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 98.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 98.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-17T09:06:48.086 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T09:06:48.087 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 27.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 27.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x56142c54d1f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???) e1 preinit fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).mds e1 new map 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).mds e1 print_map 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-05-17T09:06:48.088 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 bash[31601]: 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 2024-05-17T09:06:48.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 systemd[1]: Started Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:06:48.090 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: monmap epoch 1 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: last_changed 2024-05-17T09:06:39.857078+0000 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: min_mon_release 19 (squid) 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: election_strategy: 1 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: fsmap 2024-05-17T09:06:48.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T09:06:48.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:48 smithi019 ceph-mon[31620]: mgrmap e1: no daemons active 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.549+0000 7f7d9508e700 1 Processor -- start 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.549+0000 7f7d9508e700 1 -- start start 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.549+0000 7f7d9508e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d90108d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.549+0000 7f7d9508e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90109340 con 0x7f7d90108940 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d90108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d90108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52496/0 (socket says 172.21.15.19:52496) 2024-05-17T09:06:49.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8ed9d700 1 -- 172.21.15.19:0/1352931566 learned_addr learned my addr 172.21.15.19:0/1352931566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8ed9d700 1 -- 172.21.15.19:0/1352931566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d90109a70 con 0x7f7d90108940 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8ed9d700 1 --2- 172.21.15.19:0/1352931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d90108d50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d800098a0 tx=0x7f7d80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5455f6f25e6ce56 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8dd9b700 1 -- 172.21.15.19:0/1352931566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d80004160 con 0x7f7d90108940 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8dd9b700 1 -- 172.21.15.19:0/1352931566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7d80004300 con 0x7f7d90108940 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.550+0000 7f7d8dd9b700 1 -- 172.21.15.19:0/1352931566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d800119e0 con 0x7f7d90108940 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 -- 172.21.15.19:0/1352931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 msgr2=0x7f7d90108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 --2- 172.21.15.19:0/1352931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d90108d50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d800098a0 tx=0x7f7d80009bb0 comp rx=0 tx=0).stop 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 -- 172.21.15.19:0/1352931566 shutdown_connections 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 --2- 172.21.15.19:0/1352931566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d90108d50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 -- 172.21.15.19:0/1352931566 >> 172.21.15.19:0/1352931566 conn(0x7f7d90103f80 msgr2=0x7f7d90106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 -- 172.21.15.19:0/1352931566 shutdown_connections 2024-05-17T09:06:49.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.551+0000 7f7d9508e700 1 -- 172.21.15.19:0/1352931566 wait complete. 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.552+0000 7f7d9508e700 1 Processor -- start 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.552+0000 7f7d9508e700 1 -- start start 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.552+0000 7f7d9508e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d9019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.552+0000 7f7d9508e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9010ce20 con 0x7f7d90108940 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d8ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d9019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d8ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d9019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52508/0 (socket says 172.21.15.19:52508) 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d8ed9d700 1 -- 172.21.15.19:0/3779582861 learned_addr learned my addr 172.21.15.19:0/3779582861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d8ed9d700 1 -- 172.21.15.19:0/3779582861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9019e2a0 con 0x7f7d90108940 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d8ed9d700 1 --2- 172.21.15.19:0/3779582861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d9019dd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d800119b0 tx=0x7f7d80011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d80003970 con 0x7f7d90108940 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9019e530 con 0x7f7d90108940 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7d80003b10 con 0x7f7d90108940 2024-05-17T09:06:49.203 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d80024610 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.553+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d901a1130 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.554+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d80024c60 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.554+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d80019700 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.554+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d90064e10 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.557+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7d8001aae0 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.628+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7d9002e8c0 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.630+0000 7f7d77fff700 1 -- 172.21.15.19:0/3779582861 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7d8001a430 con 0x7f7d90108940 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 msgr2=0x7f7d9019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 --2- 172.21.15.19:0/3779582861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d9019dd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d800119b0 tx=0x7f7d80011a90 comp rx=0 tx=0).stop 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 shutdown_connections 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 --2- 172.21.15.19:0/3779582861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d90108940 0x7f7d9019dd60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 >> 172.21.15.19:0/3779582861 conn(0x7f7d90103f80 msgr2=0x7f7d90194310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 shutdown_connections 2024-05-17T09:06:49.204 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:48.631+0000 7f7d9508e700 1 -- 172.21.15.19:0/3779582861 wait complete. 2024-05-17T09:06:49.205 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-17T09:06:49.227 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:06:49.227 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-05-17T09:06:49.227 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-17T09:06:49.228 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-17T09:06:49.475 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x 2024-05-17T09:06:49.475 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-17T09:06:49.628 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968.target.wants/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@.service. 2024-05-17T09:06:49.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:49 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3779582861' entity='client.admin' 2024-05-17T09:06:49.846 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:49 smithi019 systemd[1]: Starting Ceph mgr.x for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:06:50.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:49 smithi019 podman[31836]: 2024-05-17 09:06:49.84664109 +0000 UTC m=+0.092934811 container create 02efc6d99c29d95596618fa1b8efc5bf8c3e7cc973370a854ce6894cb96b9a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-17T09:06:50.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:49 smithi019 podman[31836]: 2024-05-17 09:06:49.771201391 +0000 UTC m=+0.017495133 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:06:50.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 podman[31836]: 2024-05-17 09:06:50.082103482 +0000 UTC m=+0.328397213 container init 02efc6d99c29d95596618fa1b8efc5bf8c3e7cc973370a854ce6894cb96b9a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-17T09:06:50.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 podman[31836]: 2024-05-17 09:06:50.084947956 +0000 UTC m=+0.331241694 container start 02efc6d99c29d95596618fa1b8efc5bf8c3e7cc973370a854ce6894cb96b9a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-17T09:06:50.157 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 bash[31836]: 02efc6d99c29d95596618fa1b8efc5bf8c3e7cc973370a854ce6894cb96b9a73 2024-05-17T09:06:50.173 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T09:06:50.173 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-17T09:06:50.182 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T09:06:50.182 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-17T09:06:50.182 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-17T09:06:50.182 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T09:06:50.193 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T09:06:50.193 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-17T09:06:50.202 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T09:06:50.202 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-17T09:06:50.202 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-17T09:06:50.202 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-17T09:06:50.202 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-05-17T09:06:50.203 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-05-17T09:06:50.432 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 systemd[1]: Started Ceph mgr.x for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:06:50.433 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:50.270+0000 7fbb5ed7c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T09:06:50.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:50.466+0000 7fbb5ed7c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:06:50.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:50.573+0000 7fbb5ed7c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T09:06:50.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:50 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1999533187' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T09:06:50.998 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:50.908+0000 7fbb5ed7c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T09:06:50.998 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:50.997+0000 7fbb5ed7c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T09:06:51.240 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:51.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:06:51.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:06:51.243 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T09:06:51.244 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T09:06:51.245 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T09:06:51.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T09:06:51.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T09:06:43.021808+0000", 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.657+0000 7fe5f103b700 1 Processor -- start 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.657+0000 7fe5f103b700 1 -- start start 2024-05-17T09:06:51.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.658+0000 7fe5f103b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec10c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.658+0000 7fe5f103b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec10cb80 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.658+0000 7fe5ead9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec10c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.658+0000 7fe5ead9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec10c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52532/0 (socket says 172.21.15.19:52532) 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.658+0000 7fe5ead9d700 1 -- 172.21.15.19:0/306479708 learned_addr learned my addr 172.21.15.19:0/306479708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.658+0000 7fe5ead9d700 1 -- 172.21.15.19:0/306479708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec10cd00 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.659+0000 7fe5ead9d700 1 --2- 172.21.15.19:0/306479708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec10c570 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0098a0 tx=0x7fe5dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe13fe85ad0c4a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.659+0000 7fe5e9d9b700 1 -- 172.21.15.19:0/306479708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5dc004160 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.659+0000 7fe5e9d9b700 1 -- 172.21.15.19:0/306479708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe5dc004300 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.659+0000 7fe5e9d9b700 1 -- 172.21.15.19:0/306479708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5dc0119e0 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.659+0000 7fe5f103b700 1 -- 172.21.15.19:0/306479708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 msgr2=0x7fe5ec10c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.659+0000 7fe5f103b700 1 --2- 172.21.15.19:0/306479708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec10c570 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0098a0 tx=0x7fe5dc009bb0 comp rx=0 tx=0).stop 2024-05-17T09:06:51.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.660+0000 7fe5f103b700 1 -- 172.21.15.19:0/306479708 shutdown_connections 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.660+0000 7fe5f103b700 1 --2- 172.21.15.19:0/306479708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec10c570 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.660+0000 7fe5f103b700 1 -- 172.21.15.19:0/306479708 >> 172.21.15.19:0/306479708 conn(0x7fe5ec103f80 msgr2=0x7fe5ec106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.660+0000 7fe5f103b700 1 -- 172.21.15.19:0/306479708 shutdown_connections 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.660+0000 7fe5f103b700 1 -- 172.21.15.19:0/306479708 wait complete. 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.660+0000 7fe5f103b700 1 Processor -- start 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5f103b700 1 -- start start 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5f103b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5f103b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec10dd20 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5ead9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5ead9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec19dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52546/0 (socket says 172.21.15.19:52546) 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5ead9d700 1 -- 172.21.15.19:0/1999533187 learned_addr learned my addr 172.21.15.19:0/1999533187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:51.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.661+0000 7fe5ead9d700 1 -- 172.21.15.19:0/1999533187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec19e220 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5ead9d700 1 --2- 172.21.15.19:0/1999533187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec19dce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0116a0 tx=0x7fe5dc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5dc0039f0 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ec19e4b0 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe5dc003b90 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ec1a10c0 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5dc01a600 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe5dc01acd0 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.662+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe5dc019700 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.663+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ec064e10 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.667+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe5dc024a50 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.745+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe5ec1a1290 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.745+0000 7fe5d3fff700 1 -- 172.21.15.19:0/1999533187 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe5dc024430 con 0x7fe5ec07d6e0 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.746+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 msgr2=0x7fe5ec19dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.746+0000 7fe5f103b700 1 --2- 172.21.15.19:0/1999533187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec19dce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0116a0 tx=0x7fe5dc011a60 comp rx=0 tx=0).stop 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.746+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 shutdown_connections 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.746+0000 7fe5f103b700 1 --2- 172.21.15.19:0/1999533187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5ec07d6e0 0x7fe5ec19dce0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.746+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 >> 172.21.15.19:0/1999533187 conn(0x7fe5ec103f80 msgr2=0x7fe5ec104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.747+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 shutdown_connections 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:50.747+0000 7fe5f103b700 1 -- 172.21.15.19:0/1999533187 wait complete. 2024-05-17T09:06:51.259 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-05-17T09:06:52.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:52 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:52.271+0000 7fbb5ed7c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T09:06:52.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:52 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:52.467+0000 7fbb5ed7c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T09:06:52.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:52 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:52.644+0000 7fbb5ed7c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T09:06:53.257 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:52 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:52.944+0000 7fbb5ed7c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T09:06:53.257 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:53 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:53.045+0000 7fbb5ed7c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T09:06:53.257 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:53 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:53.256+0000 7fbb5ed7c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T09:06:53.557 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:53 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:53.462+0000 7fbb5ed7c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:06:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:53 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2198249696' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T09:06:54.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T09:06:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T09:06:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T09:06:54.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:54.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:06:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T09:06:43.021808+0000", 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:06:54.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.670+0000 7fa1b4123700 1 Processor -- start 2024-05-17T09:06:54.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.670+0000 7fa1b4123700 1 -- start start 2024-05-17T09:06:54.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b4123700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:54.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b4123700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac10a450 con 0x7fa1ac109a50 2024-05-17T09:06:54.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b1ebf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:54.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b1ebf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52552/0 (socket says 172.21.15.19:52552) 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b1ebf700 1 -- 172.21.15.19:0/701663292 learned_addr learned my addr 172.21.15.19:0/701663292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b1ebf700 1 -- 172.21.15.19:0/701663292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac10ab80 con 0x7fa1ac109a50 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b1ebf700 1 --2- 172.21.15.19:0/701663292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac109e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098a0 tx=0x7fa1a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115fd6a7908c1664 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b0ebd700 1 -- 172.21.15.19:0/701663292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8004160 con 0x7fa1ac109a50 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.671+0000 7fa1b0ebd700 1 -- 172.21.15.19:0/701663292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa1a8004300 con 0x7fa1ac109a50 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b0ebd700 1 -- 172.21.15.19:0/701663292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a80119e0 con 0x7fa1ac109a50 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b4123700 1 -- 172.21.15.19:0/701663292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 msgr2=0x7fa1ac109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b4123700 1 --2- 172.21.15.19:0/701663292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac109e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098a0 tx=0x7fa1a8009bb0 comp rx=0 tx=0).stop 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b4123700 1 -- 172.21.15.19:0/701663292 shutdown_connections 2024-05-17T09:06:54.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b4123700 1 --2- 172.21.15.19:0/701663292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac109e60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b4123700 1 -- 172.21.15.19:0/701663292 >> 172.21.15.19:0/701663292 conn(0x7fa1ac105090 msgr2=0x7fa1ac107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.672+0000 7fa1b4123700 1 -- 172.21.15.19:0/701663292 shutdown_connections 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.673+0000 7fa1b4123700 1 -- 172.21.15.19:0/701663292 wait complete. 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.673+0000 7fa1b4123700 1 Processor -- start 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.673+0000 7fa1b4123700 1 -- start start 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.673+0000 7fa1b4123700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac07f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.673+0000 7fa1b4123700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac10df30 con 0x7fa1ac109a50 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b1ebf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac07f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b1ebf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac07f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52558/0 (socket says 172.21.15.19:52558) 2024-05-17T09:06:54.301 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b1ebf700 1 -- 172.21.15.19:0/2198249696 learned_addr learned my addr 172.21.15.19:0/2198249696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b1ebf700 1 -- 172.21.15.19:0/2198249696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac07f9c0 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b1ebf700 1 --2- 172.21.15.19:0/2198249696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac07f480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80119b0 tx=0x7fa1a8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8003970 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa1a8003b10 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac07dc10 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8024610 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.674+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac07e0f0 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.675+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa1a8024c10 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.675+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa1a8019700 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.675+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac064e10 con 0x7fa1ac109a50 2024-05-17T09:06:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.678+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa1a80094f0 con 0x7fa1ac109a50 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.751+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa1ac07e7e0 con 0x7fa1ac109a50 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.752+0000 7fa1a2ffd700 1 -- 172.21.15.19:0/2198249696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa1a801aae0 con 0x7fa1ac109a50 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 msgr2=0x7fa1ac07f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 --2- 172.21.15.19:0/2198249696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac07f480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80119b0 tx=0x7fa1a8011a90 comp rx=0 tx=0).stop 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 shutdown_connections 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 --2- 172.21.15.19:0/2198249696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ac109a50 0x7fa1ac07f480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 >> 172.21.15.19:0/2198249696 conn(0x7fa1ac105090 msgr2=0x7fa1ac105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 shutdown_connections 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:53.753+0000 7fa1b4123700 1 -- 172.21.15.19:0/2198249696 wait complete. 2024-05-17T09:06:54.303 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-05-17T09:06:54.367 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:54 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:54.110+0000 7fbb5ed7c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T09:06:54.367 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:54 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:54.343+0000 7fbb5ed7c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T09:06:55.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:54 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:54.783+0000 7fbb5ed7c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T09:06:55.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:54 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:54.884+0000 7fbb5ed7c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T09:06:55.642 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:55 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:55.268+0000 7fbb5ed7c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T09:06:55.642 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:55 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:55.641+0000 7fbb5ed7c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T09:06:55.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:55 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:55.738+0000 7fbb5ed7c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T09:06:56.410 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:55 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:55.991+0000 7fbb5ed7c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T09:06:56.411 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:56 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:56.130+0000 7fbb5ed7c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T09:06:56.411 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:56 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:56.230+0000 7fbb5ed7c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T09:06:56.672 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:56 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:56.575+0000 7fbb5ed7c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T09:06:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:56 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3473274195' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T09:06:56.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:56 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:56.675+0000 7fbb5ed7c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T09:06:57.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:57 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:57.319+0000 7fbb5ed7c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T09:06:57.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:06:57 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:06:57.546+0000 7fbb5ed7c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: Activating manager daemon x 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: mgrmap e2: x(active, starting, since 0.00380323s) 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T09:06:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: Manager daemon x is now available 2024-05-17T09:06:58.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T09:06:58.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T09:06:58.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' 2024-05-17T09:06:58.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' 2024-05-17T09:06:58.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:57 smithi019 ceph-mon[31620]: from='mgr.14100 172.21.15.19:0/4112643325' entity='mgr.x' 2024-05-17T09:06:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:06:59 smithi019 ceph-mon[31620]: mgrmap e3: x(active, since 1.0087s) 2024-05-17T09:07:00.843 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:00 smithi019 ceph-mon[31620]: mgrmap e4: x(active, since 2s) 2024-05-17T09:07:00.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-17T09:07:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T09:07:00.931 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T09:07:00.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T09:07:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T09:06:43.021808+0000", 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.710+0000 7f03a499f700 1 Processor -- start 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.710+0000 7f03a499f700 1 -- start start 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.710+0000 7f03a499f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.710+0000 7f03a499f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0108370 con 0x7f03a0109dc0 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a010a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60424/0 (socket says 172.21.15.19:60424) 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039f7fe700 1 -- 172.21.15.19:0/2436622807 learned_addr learned my addr 172.21.15.19:0/2436622807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039f7fe700 1 -- 172.21.15.19:0/2436622807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a01084f0 con 0x7f03a0109dc0 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039f7fe700 1 --2- 172.21.15.19:0/2436622807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a010a1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03880098a0 tx=0x7f0388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26729775c0dedd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039e7fc700 1 -- 172.21.15.19:0/2436622807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0388004160 con 0x7f03a0109dc0 2024-05-17T09:07:00.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039e7fc700 1 -- 172.21.15.19:0/2436622807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0388004300 con 0x7f03a0109dc0 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.711+0000 7f039e7fc700 1 -- 172.21.15.19:0/2436622807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03880119e0 con 0x7f03a0109dc0 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 -- 172.21.15.19:0/2436622807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 msgr2=0x7f03a010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 --2- 172.21.15.19:0/2436622807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a010a1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03880098a0 tx=0x7f0388009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 -- 172.21.15.19:0/2436622807 shutdown_connections 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 --2- 172.21.15.19:0/2436622807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a010a1d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 -- 172.21.15.19:0/2436622807 >> 172.21.15.19:0/2436622807 conn(0x7f03a0103e00 msgr2=0x7f03a0106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 -- 172.21.15.19:0/2436622807 shutdown_connections 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.712+0000 7f03a499f700 1 -- 172.21.15.19:0/2436622807 wait complete. 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.713+0000 7f03a499f700 1 Processor -- start 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.713+0000 7f03a499f700 1 -- start start 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.713+0000 7f03a499f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a007c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f03a499f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010cc60 con 0x7f03a0109dc0 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a007c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:00.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a007c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60440/0 (socket says 172.21.15.19:60440) 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039f7fe700 1 -- 172.21.15.19:0/3473274195 learned_addr learned my addr 172.21.15.19:0/3473274195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039f7fe700 1 -- 172.21.15.19:0/3473274195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a007ced0 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039f7fe700 1 --2- 172.21.15.19:0/3473274195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a007c990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0388000c00 tx=0x7f0388011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0388003970 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0388003b10 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.714+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a01a09a0 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.715+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f038801a690 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.715+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a01a0e80 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.715+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f038801abc0 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.715+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f038802d790 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.716+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a0064e10 con 0x7f03a0109dc0 2024-05-17T09:07:00.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.720+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f03880094f0 con 0x7f03a0109dc0 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.796+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f03a01a1110 con 0x7f03a0109dc0 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.796+0000 7f039cff9700 1 -- 172.21.15.19:0/3473274195 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0388024a60 con 0x7f03a0109dc0 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.797+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 msgr2=0x7f03a007c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.797+0000 7f03a499f700 1 --2- 172.21.15.19:0/3473274195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a007c990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0388000c00 tx=0x7f0388011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.797+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 shutdown_connections 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.798+0000 7f03a499f700 1 --2- 172.21.15.19:0/3473274195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a0109dc0 0x7f03a007c990 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.798+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 >> 172.21.15.19:0/3473274195 conn(0x7f03a0103e00 msgr2=0x7f03a0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.798+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 shutdown_connections 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:06:56.798+0000 7f03a499f700 1 -- 172.21.15.19:0/3473274195 wait complete. 2024-05-17T09:07:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (3/15)... 2024-05-17T09:07:06.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/237052037' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T09:07:09.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:07:09.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T09:07:09.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:09.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T09:07:09.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T09:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T09:06:43.021808+0000", 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.257+0000 7fb2e160b700 1 Processor -- start 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.257+0000 7fb2e160b700 1 -- start start 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.257+0000 7fb2e160b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc10c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.257+0000 7fb2e160b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc10cb80 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2daffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc10c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:09.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2daffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc10c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60368/0 (socket says 172.21.15.19:60368) 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2daffd700 1 -- 172.21.15.19:0/4197509766 learned_addr learned my addr 172.21.15.19:0/4197509766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2daffd700 1 -- 172.21.15.19:0/4197509766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc10cd00 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2daffd700 1 --2- 172.21.15.19:0/4197509766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc10c590 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0098a0 tx=0x7fb2cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3a3761bf2d4003 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2d9ffb700 1 -- 172.21.15.19:0/4197509766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2cc004160 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2d9ffb700 1 -- 172.21.15.19:0/4197509766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb2cc004300 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.258+0000 7fb2d9ffb700 1 -- 172.21.15.19:0/4197509766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2cc0119e0 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 -- 172.21.15.19:0/4197509766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 msgr2=0x7fb2dc10c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 --2- 172.21.15.19:0/4197509766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc10c590 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0098a0 tx=0x7fb2cc009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 -- 172.21.15.19:0/4197509766 shutdown_connections 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 --2- 172.21.15.19:0/4197509766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc10c590 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:09.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 -- 172.21.15.19:0/4197509766 >> 172.21.15.19:0/4197509766 conn(0x7fb2dc103f80 msgr2=0x7fb2dc106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 -- 172.21.15.19:0/4197509766 shutdown_connections 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.259+0000 7fb2e160b700 1 -- 172.21.15.19:0/4197509766 wait complete. 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.260+0000 7fb2e160b700 1 Processor -- start 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.260+0000 7fb2e160b700 1 -- start start 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.260+0000 7fb2e160b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.260+0000 7fb2e160b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc10dd20 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2daffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2daffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc19dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60380/0 (socket says 172.21.15.19:60380) 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2daffd700 1 -- 172.21.15.19:0/237052037 learned_addr learned my addr 172.21.15.19:0/237052037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2daffd700 1 -- 172.21.15.19:0/237052037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc19e220 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2daffd700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc19dce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0116a0 tx=0x7fb2cc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2cc0039f0 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc19e4b0 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb2cc003b90 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2cc01a600 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.261+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1a10c0 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.262+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc064e10 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.262+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fb2cc01a900 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.263+0000 7fb2c3fff700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fb2c403e3d0 0x7fb2c40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.263+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb2cc053ba0 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.263+0000 7fb2da7fc700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fb2c403e3d0 0x7fb2c40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.263+0000 7fb2da7fc700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fb2c403e3d0 0x7fb2c40408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.268+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2cc00e260 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.581+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb2dc1a1740 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.582+0000 7fb2c3fff700 1 -- 172.21.15.19:0/237052037 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb2cc0535c0 con 0x7fb2dc07d6c0 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fb2c403e3d0 msgr2=0x7fb2c40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fb2c403e3d0 0x7fb2c40408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 msgr2=0x7fb2dc19dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc19dce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0116a0 tx=0x7fb2cc011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 shutdown_connections 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fb2c403e3d0 0x7fb2c40408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 --2- 172.21.15.19:0/237052037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc07d6c0 0x7fb2dc19dce0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.584+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 >> 172.21.15.19:0/237052037 conn(0x7fb2dc103f80 msgr2=0x7fb2dc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.585+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 shutdown_connections 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:06.585+0000 7fb2e160b700 1 -- 172.21.15.19:0/237052037 wait complete. 2024-05-17T09:07:09.904 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-05-17T09:07:10.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:10 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1304866248' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:07:11.655 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.322+0000 7fe98e6dd700 1 Processor -- start 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.322+0000 7fe98e6dd700 1 -- start start 2024-05-17T09:07:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe98e6dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe98810ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe98e6dd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe98807cde0 con 0x7fe98810a5f0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe987fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe98810ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe987fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe98810ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60382/0 (socket says 172.21.15.19:60382) 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe987fff700 1 -- 172.21.15.19:0/1861916726 learned_addr learned my addr 172.21.15.19:0/1861916726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe987fff700 1 -- 172.21.15.19:0/1861916726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98810cf50 con 0x7fe98810a5f0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe987fff700 1 --2- 172.21.15.19:0/1861916726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe98810ca10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098a0 tx=0x7fe978009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61da10d8dfa954b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe9877fe700 1 -- 172.21.15.19:0/1861916726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe978004160 con 0x7fe98810a5f0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe9877fe700 1 -- 172.21.15.19:0/1861916726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe978004300 con 0x7fe98810a5f0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.323+0000 7fe9877fe700 1 -- 172.21.15.19:0/1861916726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9780119e0 con 0x7fe98810a5f0 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1861916726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 msgr2=0x7fe98810ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 --2- 172.21.15.19:0/1861916726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe98810ca10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098a0 tx=0x7fe978009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:11.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1861916726 shutdown_connections 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 --2- 172.21.15.19:0/1861916726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe98810ca10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1861916726 >> 172.21.15.19:0/1861916726 conn(0x7fe988103ff0 msgr2=0x7fe988106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1861916726 shutdown_connections 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.324+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1861916726 wait complete. 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe98e6dd700 1 Processor -- start 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe98e6dd700 1 -- start start 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe98e6dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe988109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe98e6dd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe98810de50 con 0x7fe98810a5f0 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe987fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe988109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe987fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe988109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60390/0 (socket says 172.21.15.19:60390) 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.325+0000 7fe987fff700 1 -- 172.21.15.19:0/1304866248 learned_addr learned my addr 172.21.15.19:0/1304866248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe987fff700 1 -- 172.21.15.19:0/1304866248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98810a2b0 con 0x7fe98810a5f0 2024-05-17T09:07:11.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe987fff700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe988109d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe978003f20 tx=0x7fe978011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9780038b0 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe978003a50 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe978019550 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe988108500 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.326+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881089e0 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.327+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fe978019800 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.327+0000 7fe985ffb700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fe97403e420 0x7fe974040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.327+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe978053b10 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.327+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe988064e10 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.328+0000 7fe97ffff700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fe97403e420 0x7fe974040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.328+0000 7fe97ffff700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fe97403e420 0x7fe974040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe970006fd0 tx=0x7fe970006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.333+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe978017e90 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.551+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe988108c60 con 0x7fe98810a5f0 2024-05-17T09:07:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.552+0000 7fe985ffb700 1 -- 172.21.15.19:0/1304866248 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fe978023390 con 0x7fe98810a5f0 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fe97403e420 msgr2=0x7fe974040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fe97403e420 0x7fe974040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe970006fd0 tx=0x7fe970006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 msgr2=0x7fe988109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe988109d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe978003f20 tx=0x7fe978011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 shutdown_connections 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fe97403e420 0x7fe974040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 --2- 172.21.15.19:0/1304866248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe98810a5f0 0x7fe988109d70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 >> 172.21.15.19:0/1304866248 conn(0x7fe988103ff0 msgr2=0x7fe988105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 shutdown_connections 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:10.556+0000 7fe98e6dd700 1 -- 172.21.15.19:0/1304866248 wait complete. 2024-05-17T09:07:11.660 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-05-17T09:07:12.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:12 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3845418826' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-17T09:07:12.815 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: ignoring --setuser ceph since I am not root 2024-05-17T09:07:12.815 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: ignoring --setgroup ceph since I am not root 2024-05-17T09:07:12.815 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:12.661+0000 7fab09170200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T09:07:13.068 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:12.860+0000 7fab09170200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:07:13.068 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:12.969+0000 7fab09170200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T09:07:13.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.116+0000 7f2da8e43700 1 Processor -- start 2024-05-17T09:07:13.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.116+0000 7f2da8e43700 1 -- start start 2024-05-17T09:07:13.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.116+0000 7f2da8e43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da410c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:13.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da8e43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2da410cb60 con 0x7f2da407d4e0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da410c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da410c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60400/0 (socket says 172.21.15.19:60400) 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da259c700 1 -- 172.21.15.19:0/3052177414 learned_addr learned my addr 172.21.15.19:0/3052177414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da259c700 1 -- 172.21.15.19:0/3052177414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da410cce0 con 0x7f2da407d4e0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da259c700 1 --2- 172.21.15.19:0/3052177414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da410c590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d940098a0 tx=0x7f2d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7141b16edacad381 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.117+0000 7f2da159a700 1 -- 172.21.15.19:0/3052177414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94004160 con 0x7f2da407d4e0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da159a700 1 -- 172.21.15.19:0/3052177414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2d94004300 con 0x7f2da407d4e0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da159a700 1 -- 172.21.15.19:0/3052177414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d940119e0 con 0x7f2da407d4e0 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da8e43700 1 -- 172.21.15.19:0/3052177414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 msgr2=0x7f2da410c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da8e43700 1 --2- 172.21.15.19:0/3052177414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da410c590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d940098a0 tx=0x7f2d94009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da8e43700 1 -- 172.21.15.19:0/3052177414 shutdown_connections 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da8e43700 1 --2- 172.21.15.19:0/3052177414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da410c590 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:13.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da8e43700 1 -- 172.21.15.19:0/3052177414 >> 172.21.15.19:0/3052177414 conn(0x7f2da4103f60 msgr2=0x7f2da41063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.118+0000 7f2da8e43700 1 -- 172.21.15.19:0/3052177414 shutdown_connections 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.119+0000 7f2da8e43700 1 -- 172.21.15.19:0/3052177414 wait complete. 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.119+0000 7f2da8e43700 1 Processor -- start 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.119+0000 7f2da8e43700 1 -- start start 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da8e43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da4109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da8e43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2da410dd00 con 0x7f2da407d4e0 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da4109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da4109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60408/0 (socket says 172.21.15.19:60408) 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da259c700 1 -- 172.21.15.19:0/3845418826 learned_addr learned my addr 172.21.15.19:0/3845418826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da259c700 1 -- 172.21.15.19:0/3845418826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da410a240 con 0x7f2da407d4e0 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.120+0000 7f2da259c700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da4109d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d940119b0 tx=0x7f2d94011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.121+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94003af0 con 0x7f2da407d4e0 2024-05-17T09:07:13.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.121+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2d9401a8a0 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.121+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da4108490 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.121+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94024830 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.121+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da4108970 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.122+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f2d94022020 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.122+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4064e10 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.122+0000 7f2d937fe700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f2d8c03e420 0x7f2d8c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.122+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d940535b0 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.122+0000 7f2da1d9b700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f2d8c03e420 0x7f2d8c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.123+0000 7f2da1d9b700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f2d8c03e420 0x7f2d8c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d98006fd0 tx=0x7f2d98006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.128+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d9400e260 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.401+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2da41090b0 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.453+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2d94053290 con 0x7f2da407d4e0 2024-05-17T09:07:13.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.455+0000 7f2d937fe700 1 -- 172.21.15.19:0/3845418826 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f2d9402dc90 con 0x7f2da407d4e0 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f2d8c03e420 msgr2=0x7f2d8c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f2d8c03e420 0x7f2d8c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d98006fd0 tx=0x7f2d98006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 msgr2=0x7f2da4109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da4109d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d940119b0 tx=0x7f2d94011a90 comp rx=0 tx=0).stop 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 shutdown_connections 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f2d8c03e420 0x7f2d8c040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 --2- 172.21.15.19:0/3845418826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da407d4e0 0x7f2da4109d00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 >> 172.21.15.19:0/3845418826 conn(0x7f2da4103f60 msgr2=0x7f2da4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 shutdown_connections 2024-05-17T09:07:13.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:12.456+0000 7f2da8e43700 1 -- 172.21.15.19:0/3845418826 wait complete. 2024-05-17T09:07:13.318 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:13.302+0000 7fab09170200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T09:07:13.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:13 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3845418826' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-17T09:07:13.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:13 smithi019 ceph-mon[31620]: mgrmap e5: x(active, since 14s) 2024-05-17T09:07:13.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:13.398+0000 7fab09170200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T09:07:14.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:07:14.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.526+0000 7fd752be3700 1 Processor -- start 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.526+0000 7fd752be3700 1 -- start start 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.526+0000 7fd752be3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c10b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.526+0000 7fd752be3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c10b9d0 con 0x7fd74c10aff0 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd75097f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c10b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd75097f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c10b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60436/0 (socket says 172.21.15.19:60436) 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd75097f700 1 -- 172.21.15.19:0/1825705560 learned_addr learned my addr 172.21.15.19:0/1825705560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:14.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd75097f700 1 -- 172.21.15.19:0/1825705560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c10bb50 con 0x7fd74c10aff0 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd75097f700 1 --2- 172.21.15.19:0/1825705560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c10b400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd7400098a0 tx=0x7fd740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f85d38c17f174b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd74b7fe700 1 -- 172.21.15.19:0/1825705560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd740004160 con 0x7fd74c10aff0 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd74b7fe700 1 -- 172.21.15.19:0/1825705560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd740004300 con 0x7fd74c10aff0 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.527+0000 7fd74b7fe700 1 -- 172.21.15.19:0/1825705560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7400119e0 con 0x7fd74c10aff0 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 -- 172.21.15.19:0/1825705560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 msgr2=0x7fd74c10b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 --2- 172.21.15.19:0/1825705560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c10b400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd7400098a0 tx=0x7fd740009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 -- 172.21.15.19:0/1825705560 shutdown_connections 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 --2- 172.21.15.19:0/1825705560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c10b400 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 -- 172.21.15.19:0/1825705560 >> 172.21.15.19:0/1825705560 conn(0x7fd74c1037d0 msgr2=0x7fd74c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 -- 172.21.15.19:0/1825705560 shutdown_connections 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.528+0000 7fd752be3700 1 -- 172.21.15.19:0/1825705560 wait complete. 2024-05-17T09:07:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.529+0000 7fd752be3700 1 Processor -- start 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.529+0000 7fd752be3700 1 -- start start 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd752be3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd752be3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c1087d0 con 0x7fd74c10aff0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd75097f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd75097f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60452/0 (socket says 172.21.15.19:60452) 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd75097f700 1 -- 172.21.15.19:0/4146652726 learned_addr learned my addr 172.21.15.19:0/4146652726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd75097f700 1 -- 172.21.15.19:0/4146652726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c19e1c0 con 0x7fd74c10aff0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd75097f700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c19dc80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd7400116a0 tx=0x7fd740011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd740003970 con 0x7fd74c10aff0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.530+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd740003b10 con 0x7fd74c10aff0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.531+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74c19e450 con 0x7fd74c10aff0 2024-05-17T09:07:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.531+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd740019550 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.531+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74c1a1060 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.532+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd740019830 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.532+0000 7fd749ffb700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 0x7fd73c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.532+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74c064e10 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.532+0000 7fd74bfff700 1 -- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 msgr2=0x7fd73c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.532+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd740054710 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.532+0000 7fd74bfff700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 0x7fd73c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.542+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd740016260 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.733+0000 7fd74bfff700 1 -- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 msgr2=0x7fd73c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.733+0000 7fd74bfff700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 0x7fd73c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.835+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd74c1a1700 con 0x7fd74c10aff0 2024-05-17T09:07:14.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.836+0000 7fd749ffb700 1 -- 172.21.15.19:0/4146652726 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd740010380 con 0x7fd74c10aff0 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 msgr2=0x7fd73c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 0x7fd73c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 msgr2=0x7fd74c19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c19dc80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd7400116a0 tx=0x7fd740011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 shutdown_connections 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7fd73c03e470 0x7fd73c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 --2- 172.21.15.19:0/4146652726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd74c10aff0 0x7fd74c19dc80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 >> 172.21.15.19:0/4146652726 conn(0x7fd74c1037d0 msgr2=0x7fd74c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.840+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 shutdown_connections 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:13.841+0000 7fd752be3700 1 -- 172.21.15.19:0/4146652726 wait complete. 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-17T09:07:14.479 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-05-17T09:07:14.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:14 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4146652726' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T09:07:14.735 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:14.703+0000 7fab09170200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T09:07:15.073 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:14.890+0000 7fab09170200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T09:07:15.073 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:15.072+0000 7fab09170200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T09:07:15.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:15.380+0000 7fab09170200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T09:07:15.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:15.479+0000 7fab09170200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T09:07:16.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:15.693+0000 7fab09170200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T09:07:16.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:15.899+0000 7fab09170200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:07:16.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:16.539+0000 7fab09170200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T09:07:17.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:16.774+0000 7fab09170200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T09:07:17.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:17.226+0000 7fab09170200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T09:07:17.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:17.329+0000 7fab09170200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T09:07:18.099 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:17.719+0000 7fab09170200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T09:07:18.099 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:18.098+0000 7fab09170200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T09:07:18.442 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:18.198+0000 7fab09170200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T09:07:18.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:18.458+0000 7fab09170200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T09:07:18.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:18.600+0000 7fab09170200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T09:07:18.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:18.703+0000 7fab09170200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T09:07:19.442 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:19.045+0000 7fab09170200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T09:07:19.442 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:19.142+0000 7fab09170200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T09:07:20.072 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:19.783+0000 7fab09170200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T09:07:20.072 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:20 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:20.015+0000 7fab09170200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: Active manager daemon x restarted 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: Activating manager daemon x 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: osdmap e2: 0 total, 0 up, 0 in 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: mgrmap e6: x(active, starting, since 0.00401334s) 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T09:07:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T09:07:20.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: Manager daemon x is now available 2024-05-17T09:07:20.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:20.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:20.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:20 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:21.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: Found migration_current of "None". Setting to last migration. 2024-05-17T09:07:21.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:21.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T09:07:21.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T09:07:21.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:21.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:21.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:21 smithi019 ceph-mon[31620]: mgrmap e7: x(active, since 1.00606s) 2024-05-17T09:07:21.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.942+0000 7f06fe6e2700 1 Processor -- start 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06fe6e2700 1 -- start start 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06fe6e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f8108d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06fe6e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06f8109330 con 0x7f06f8108930 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f8108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f8108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41334/0 (socket says 172.21.15.19:41334) 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06f7fff700 1 -- 172.21.15.19:0/2319038720 learned_addr learned my addr 172.21.15.19:0/2319038720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:21.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.943+0000 7f06f7fff700 1 -- 172.21.15.19:0/2319038720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f8109a60 con 0x7f06f8108930 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.944+0000 7f06f7fff700 1 --2- 172.21.15.19:0/2319038720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f8108d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06e800bb40 tx=0x7f06e800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68606314fe921661 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.944+0000 7f06f6ffd700 1 -- 172.21.15.19:0/2319038720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e800dbe0 con 0x7f06f8108930 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.944+0000 7f06f6ffd700 1 -- 172.21.15.19:0/2319038720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f06e80044a0 con 0x7f06f8108930 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.944+0000 7f06f6ffd700 1 -- 172.21.15.19:0/2319038720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e8003e30 con 0x7f06f8108930 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 -- 172.21.15.19:0/2319038720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 msgr2=0x7f06f8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/2319038720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f8108d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06e800bb40 tx=0x7f06e800be50 comp rx=0 tx=0).stop 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 -- 172.21.15.19:0/2319038720 shutdown_connections 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/2319038720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f8108d40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 -- 172.21.15.19:0/2319038720 >> 172.21.15.19:0/2319038720 conn(0x7f06f8104030 msgr2=0x7f06f8106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 -- 172.21.15.19:0/2319038720 shutdown_connections 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 -- 172.21.15.19:0/2319038720 wait complete. 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.945+0000 7f06fe6e2700 1 Processor -- start 2024-05-17T09:07:21.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06fe6e2700 1 -- start start 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06fe6e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06fe6e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06f807c840 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f819bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41348/0 (socket says 172.21.15.19:41348) 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06f7fff700 1 -- 172.21.15.19:0/1925646230 learned_addr learned my addr 172.21.15.19:0/1925646230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.946+0000 7f06f7fff700 1 -- 172.21.15.19:0/1925646230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f819c070 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.947+0000 7f06f7fff700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f819bb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f06e8003bc0 tx=0x7f06e8003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.947+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e8027070 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.947+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f06e80109f0 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.947+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f819c300 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.947+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e8022990 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.947+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f819ef10 con 0x7f06f8108930 2024-05-17T09:07:21.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.948+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f06e802a070 con 0x7f06f8108930 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.948+0000 7f06f57fa700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.948+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 --> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f06e40410e0 con 0x7f06e403e4c0 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.948+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06e801f070 con 0x7f06f8108930 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.948+0000 7f06f77fe700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 msgr2=0x7f06e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:14.948+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:15.149+0000 7f06f77fe700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 msgr2=0x7f06e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:15.149+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:15.550+0000 7f06f77fe700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 msgr2=0x7f06e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:15.550+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:16.351+0000 7f06f77fe700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 msgr2=0x7f06e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:16.351+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T09:07:21.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:17.952+0000 7f06f77fe700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 msgr2=0x7f06e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/747673746 2024-05-17T09:07:21.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:17.952+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T09:07:21.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:20.019+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f06e802c440 con 0x7f06f8108930 2024-05-17T09:07:21.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:20.020+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 msgr2=0x7f06e40409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:07:21.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:20.020+0000 7f06f57fa700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:21.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.020+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f06e80183a0 con 0x7f06f8108930 2024-05-17T09:07:21.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.021+0000 7f06f57fa700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f06e4041e50 0x7f06e4044270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.021+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f06e40410e0 con 0x7f06e4041e50 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.021+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f06e4041e50 0x7f06e4044270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.022+0000 7f06f77fe700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f06e4041e50 0x7f06e4044270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06e0003a10 tx=0x7f06e00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.023+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f06e40410e0 con 0x7f06e4041e50 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.031+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f06f8064e10 con 0x7f06e4041e50 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.031+0000 7f06f57fa700 1 -- 172.21.15.19:0/1925646230 <== mgr.14120 v2:172.21.15.19:6800/2577177111 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f06f8064e10 con 0x7f06e4041e50 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f06e4041e50 msgr2=0x7f06e4044270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f06e4041e50 0x7f06e4044270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06e0003a10 tx=0x7f06e00092b0 comp rx=0 tx=0).stop 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 msgr2=0x7f06f819bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f819bb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f06e8003bc0 tx=0x7f06e8003f80 comp rx=0 tx=0).stop 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 shutdown_connections 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f06e4041e50 0x7f06e4044270 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:21.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f8108930 0x7f06f819bb30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:21.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 --2- 172.21.15.19:0/1925646230 >> [v2:172.21.15.19:6800/747673746,v1:172.21.15.19:6801/747673746] conn(0x7f06e403e4c0 0x7f06e40409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:21.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 >> 172.21.15.19:0/1925646230 conn(0x7f06f8104030 msgr2=0x7f06f8191e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:21.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 shutdown_connections 2024-05-17T09:07:21.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:21.032+0000 7f06fe6e2700 1 -- 172.21.15.19:0/1925646230 wait complete. 2024-05-17T09:07:21.630 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-05-17T09:07:21.630 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-05-17T09:07:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:22 smithi019 ceph-mon[31620]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T09:07:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:22 smithi019 ceph-mon[31620]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T09:07:22.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.039+0000 7f41499e8700 1 Processor -- start 2024-05-17T09:07:22.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.039+0000 7f41499e8700 1 -- start start 2024-05-17T09:07:22.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.039+0000 7f41499e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414410c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.039+0000 7f41499e8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f414410cb70 con 0x7f414407d540 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.039+0000 7f4142ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414410c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4142ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414410c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41426/0 (socket says 172.21.15.19:41426) 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4142ffd700 1 -- 172.21.15.19:0/4175827979 learned_addr learned my addr 172.21.15.19:0/4175827979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4142ffd700 1 -- 172.21.15.19:0/4175827979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414410ccf0 con 0x7f414407d540 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4142ffd700 1 --2- 172.21.15.19:0/4175827979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414410c560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41340098a0 tx=0x7f4134009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc741953e2a3ced server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4141ffb700 1 -- 172.21.15.19:0/4175827979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4134004160 con 0x7f414407d540 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4141ffb700 1 -- 172.21.15.19:0/4175827979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4134004300 con 0x7f414407d540 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.040+0000 7f4141ffb700 1 -- 172.21.15.19:0/4175827979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41340119e0 con 0x7f414407d540 2024-05-17T09:07:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 -- 172.21.15.19:0/4175827979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 msgr2=0x7f414410c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 --2- 172.21.15.19:0/4175827979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414410c560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41340098a0 tx=0x7f4134009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 -- 172.21.15.19:0/4175827979 shutdown_connections 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 --2- 172.21.15.19:0/4175827979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414410c560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 -- 172.21.15.19:0/4175827979 >> 172.21.15.19:0/4175827979 conn(0x7f4144104030 msgr2=0x7f4144106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 -- 172.21.15.19:0/4175827979 shutdown_connections 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.041+0000 7f41499e8700 1 -- 172.21.15.19:0/4175827979 wait complete. 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f41499e8700 1 Processor -- start 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f41499e8700 1 -- start start 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f41499e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f41499e8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f414410dd10 con 0x7f414407d540 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f4142ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:22.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f4142ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41430/0 (socket says 172.21.15.19:41430) 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.042+0000 7f4142ffd700 1 -- 172.21.15.19:0/2607243133 learned_addr learned my addr 172.21.15.19:0/2607243133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f4142ffd700 1 -- 172.21.15.19:0/2607243133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414419e230 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f4142ffd700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414419dcf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4134003f20 tx=0x7f4134011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41340038b0 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4134003a50 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4134019550 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414419e4c0 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.043+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a10d0 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.044+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f4134019800 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.044+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4144064e10 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.044+0000 7f41489e6700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f412c03e340 0x7f412c040850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.044+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4134053990 con 0x7f414407d540 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.044+0000 7f41427fc700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f412c03e340 0x7f412c040850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:22.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.045+0000 7f41427fc700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f412c03e340 0x7f412c040850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4138006fd0 tx=0x7f4138006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.050+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4134017e90 con 0x7f414407d540 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.077+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f413402d960 con 0x7f414407d540 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.287+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f41441a12a0 con 0x7f412c03e340 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.292+0000 7f41489e6700 1 -- 172.21.15.19:0/2607243133 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41441a12a0 con 0x7f412c03e340 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f412c03e340 msgr2=0x7f412c040850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f412c03e340 0x7f412c040850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4138006fd0 tx=0x7f4138006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 msgr2=0x7f414419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414419dcf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4134003f20 tx=0x7f4134011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 shutdown_connections 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f412c03e340 0x7f412c040850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 --2- 172.21.15.19:0/2607243133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414407d540 0x7f414419dcf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 >> 172.21.15.19:0/2607243133 conn(0x7f4144104030 msgr2=0x7f4144105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 shutdown_connections 2024-05-17T09:07:22.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:22.295+0000 7f41499e8700 1 -- 172.21.15.19:0/2607243133 wait complete. 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: [17/May/2024:09:07:21] ENGINE Bus STARTING 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: [17/May/2024:09:07:22] ENGINE Serving on https://172.21.15.19:7150 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: mgrmap e8: x(active, since 2s) 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: [17/May/2024:09:07:22] ENGINE Serving on http://172.21.15.19:8765 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: [17/May/2024:09:07:22] ENGINE Bus STARTED 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:23.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:23.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:23.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:23 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-17T09:07:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.388+0000 7f280a6e3700 1 Processor -- start 2024-05-17T09:07:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.388+0000 7f280a6e3700 1 -- start start 2024-05-17T09:07:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.388+0000 7f280a6e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280407dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.388+0000 7f280a6e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f280407dff0 con 0x7f280407f590 2024-05-17T09:07:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f2803fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280407dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f2803fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280407dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41438/0 (socket says 172.21.15.19:41438) 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f2803fff700 1 -- 172.21.15.19:0/3491327632 learned_addr learned my addr 172.21.15.19:0/3491327632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f2803fff700 1 -- 172.21.15.19:0/3491327632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280407e170 con 0x7f280407f590 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f2803fff700 1 --2- 172.21.15.19:0/3491327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280407dab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098a0 tx=0x7f27f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cff4bb785b7862 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f28037fe700 1 -- 172.21.15.19:0/3491327632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4004160 con 0x7f280407f590 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f28037fe700 1 -- 172.21.15.19:0/3491327632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f27f4004300 con 0x7f280407f590 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.389+0000 7f28037fe700 1 -- 172.21.15.19:0/3491327632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f40119e0 con 0x7f280407f590 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 -- 172.21.15.19:0/3491327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 msgr2=0x7f280407dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 --2- 172.21.15.19:0/3491327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280407dab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098a0 tx=0x7f27f4009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 -- 172.21.15.19:0/3491327632 shutdown_connections 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 --2- 172.21.15.19:0/3491327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280407dab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 -- 172.21.15.19:0/3491327632 >> 172.21.15.19:0/3491327632 conn(0x7f28041050f0 msgr2=0x7f2804107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:24.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 -- 172.21.15.19:0/3491327632 shutdown_connections 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.390+0000 7f280a6e3700 1 -- 172.21.15.19:0/3491327632 wait complete. 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f280a6e3700 1 Processor -- start 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f280a6e3700 1 -- start start 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f280a6e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f280a6e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f280410fef0 con 0x7f280407f590 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f2803fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f2803fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280419fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41446/0 (socket says 172.21.15.19:41446) 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f2803fff700 1 -- 172.21.15.19:0/2000816035 learned_addr learned my addr 172.21.15.19:0/2000816035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.391+0000 7f2803fff700 1 -- 172.21.15.19:0/2000816035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28041a03e0 con 0x7f280407f590 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.392+0000 7f2803fff700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280419fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27f4003f20 tx=0x7f27f4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.392+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f40038b0 con 0x7f280407f590 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.392+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f27f4003a50 con 0x7f280407f590 2024-05-17T09:07:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.392+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28041a0670 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.392+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4019550 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.392+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28041a3280 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.393+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f27f401a430 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.393+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2804064e10 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.393+0000 7f2801ffb700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f27f003e470 0x7f27f0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.393+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27f4053ab0 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.393+0000 7f27fbfff700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f27f003e470 0x7f27f0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.394+0000 7f27fbfff700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f27f003e470 0x7f27f0040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006fd0 tx=0x7f27ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.399+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27f400b3e0 con 0x7f280407f590 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.619+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f28041070c0 con 0x7f27f003e470 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.619+0000 7f2801ffb700 1 -- 172.21.15.19:0/2000816035 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f28041070c0 con 0x7f27f003e470 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f27f003e470 msgr2=0x7f27f0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:24.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f27f003e470 0x7f27f0040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006fd0 tx=0x7f27ec006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 msgr2=0x7f280419fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280419fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27f4003f20 tx=0x7f27f4011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 shutdown_connections 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f27f003e470 0x7f27f0040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 --2- 172.21.15.19:0/2000816035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f280407f590 0x7f280419fea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 >> 172.21.15.19:0/2000816035 conn(0x7f28041050f0 msgr2=0x7f28041069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 shutdown_connections 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:23.622+0000 7f280a6e3700 1 -- 172.21.15.19:0/2000816035 wait complete. 2024-05-17T09:07:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-05-17T09:07:24.831 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:24 smithi019 ceph-mon[31620]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:25.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: Generating public/private rsa key pair. 2024-05-17T09:07:25.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: Your identification has been saved in /tmp/tmphe1zkktf/key. 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: Your public key has been saved in /tmp/tmphe1zkktf/key.pub. 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: The key fingerprint is: 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: SHA256:hMut/OXEO7xP/Yd2XQx5l7C8sMdlmXz0rGZJ1Zd4t6M ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: The key's randomart image is: 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: +---[RSA 3072]----+ 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . o| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . o o*| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . . . ==O| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . + . o+X*| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | o S +.=*+| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . . . . E= o| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | o .+ oo..o| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . +oo o.+| 2024-05-17T09:07:25.193 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: | . ++.. .o| 2024-05-17T09:07:25.194 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: +----[SHA256]-----+ 2024-05-17T09:07:25.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f113bfd6700 1 Processor -- start 2024-05-17T09:07:25.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f113bfd6700 1 -- start start 2024-05-17T09:07:25.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f113bfd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113410a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:25.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f113bfd6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f113410a890 con 0x7f1134109eb0 2024-05-17T09:07:25.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f1139d72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113410a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:25.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f1139d72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113410a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47382/0 (socket says 172.21.15.19:47382) 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.601+0000 7f1139d72700 1 -- 172.21.15.19:0/163103876 learned_addr learned my addr 172.21.15.19:0/163103876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.602+0000 7f1139d72700 1 -- 172.21.15.19:0/163103876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113410aa10 con 0x7f1134109eb0 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.602+0000 7f1139d72700 1 --2- 172.21.15.19:0/163103876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113410a2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11300098a0 tx=0x7f1130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6411575dccb9c86a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.602+0000 7f1138d70700 1 -- 172.21.15.19:0/163103876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1130004160 con 0x7f1134109eb0 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.602+0000 7f1138d70700 1 -- 172.21.15.19:0/163103876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1130004300 con 0x7f1134109eb0 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.602+0000 7f1138d70700 1 -- 172.21.15.19:0/163103876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11300119e0 con 0x7f1134109eb0 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 -- 172.21.15.19:0/163103876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 msgr2=0x7f113410a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 --2- 172.21.15.19:0/163103876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113410a2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11300098a0 tx=0x7f1130009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 -- 172.21.15.19:0/163103876 shutdown_connections 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 --2- 172.21.15.19:0/163103876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113410a2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:25.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 -- 172.21.15.19:0/163103876 >> 172.21.15.19:0/163103876 conn(0x7f11341037c0 msgr2=0x7f1134105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 -- 172.21.15.19:0/163103876 shutdown_connections 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.603+0000 7f113bfd6700 1 -- 172.21.15.19:0/163103876 wait complete. 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f113bfd6700 1 Processor -- start 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f113bfd6700 1 -- start start 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f113bfd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f113bfd6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f113410ff00 con 0x7f1134109eb0 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f1139d72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f1139d72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47384/0 (socket says 172.21.15.19:47384) 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f1139d72700 1 -- 172.21.15.19:0/2620328468 learned_addr learned my addr 172.21.15.19:0/2620328468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f1139d72700 1 -- 172.21.15.19:0/2620328468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11341a0420 con 0x7f1134109eb0 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.604+0000 7f1139d72700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113419fee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1130000c00 tx=0x7f1130011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.605+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1130003970 con 0x7f1134109eb0 2024-05-17T09:07:25.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.605+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1130003b10 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.605+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11341a06b0 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.605+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f113001a690 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.605+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11341a32c0 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.606+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1134064e10 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.606+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f113001a940 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.606+0000 7f112affd700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f112003e470 0x7f1120040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.606+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1130053a20 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.606+0000 7f1139571700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f112003e470 0x7f1120040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.607+0000 7f1139571700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f112003e470 0x7f1120040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1124006fd0 tx=0x7f1124006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.611+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1130017e90 con 0x7f1134109eb0 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.831+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1134105810 con 0x7f112003e470 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.916+0000 7f112affd700 1 -- 172.21.15.19:0/2620328468 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1134105810 con 0x7f112003e470 2024-05-17T09:07:25.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f112003e470 msgr2=0x7f1120040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f112003e470 0x7f1120040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1124006fd0 tx=0x7f1124006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 msgr2=0x7f113419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113419fee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1130000c00 tx=0x7f1130011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 shutdown_connections 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f112003e470 0x7f1120040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 --2- 172.21.15.19:0/2620328468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1134109eb0 0x7f113419fee0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 >> 172.21.15.19:0/2620328468 conn(0x7f11341037c0 msgr2=0x7f1134105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 shutdown_connections 2024-05-17T09:07:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:24.918+0000 7f113bfd6700 1 -- 172.21.15.19:0/2620328468 wait complete. 2024-05-17T09:07:26.091 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:25 smithi019 ceph-mon[31620]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:26.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:25 smithi019 ceph-mon[31620]: Generating ssh key... 2024-05-17T09:07:26.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:25 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:26.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:25 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba4a6d1700 1 Processor -- start 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba4a6d1700 1 -- start start 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba4a6d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4410c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba4a6d1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba4407d650 con 0x7fba4410a510 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4410c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47396/0 (socket says 172.21.15.19:47396) 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.857+0000 7fba43fff700 1 -- 172.21.15.19:0/584170746 learned_addr learned my addr 172.21.15.19:0/584170746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.858+0000 7fba43fff700 1 -- 172.21.15.19:0/584170746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4410ce70 con 0x7fba4410a510 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.858+0000 7fba43fff700 1 --2- 172.21.15.19:0/584170746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4410c930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba340098a0 tx=0x7fba34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e786e5bb74911a53 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:26.707 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.858+0000 7fba437fe700 1 -- 172.21.15.19:0/584170746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba34004160 con 0x7fba4410a510 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.858+0000 7fba437fe700 1 -- 172.21.15.19:0/584170746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fba34004300 con 0x7fba4410a510 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.858+0000 7fba437fe700 1 -- 172.21.15.19:0/584170746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba340119e0 con 0x7fba4410a510 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 -- 172.21.15.19:0/584170746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 msgr2=0x7fba4410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 --2- 172.21.15.19:0/584170746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4410c930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba340098a0 tx=0x7fba34009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 -- 172.21.15.19:0/584170746 shutdown_connections 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 --2- 172.21.15.19:0/584170746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4410c930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 -- 172.21.15.19:0/584170746 >> 172.21.15.19:0/584170746 conn(0x7fba44104030 msgr2=0x7fba44106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 -- 172.21.15.19:0/584170746 shutdown_connections 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.859+0000 7fba4a6d1700 1 -- 172.21.15.19:0/584170746 wait complete. 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba4a6d1700 1 Processor -- start 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba4a6d1700 1 -- start start 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba4a6d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:26.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba4a6d1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba4410dd10 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47398/0 (socket says 172.21.15.19:47398) 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba43fff700 1 -- 172.21.15.19:0/509022945 learned_addr learned my addr 172.21.15.19:0/509022945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.860+0000 7fba43fff700 1 -- 172.21.15.19:0/509022945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4419e230 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.861+0000 7fba43fff700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4419dcf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba34003f20 tx=0x7fba34011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.861+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba340038b0 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.861+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fba34003a50 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.861+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4419e4c0 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.861+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba34019550 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.861+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba441a10d0 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.862+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fba34019850 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.862+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba44064e10 con 0x7fba4410a510 2024-05-17T09:07:26.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.862+0000 7fba41ffb700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fba3003e420 0x7fba30040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.862+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba34053ba0 con 0x7fba4410a510 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.862+0000 7fba3bfff700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fba3003e420 0x7fba30040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.863+0000 7fba3bfff700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fba3003e420 0x7fba30040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fba2c006fd0 tx=0x7fba2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:25.867+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba34017e90 con 0x7fba4410a510 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.091+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fba441060a0 con 0x7fba3003e420 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.091+0000 7fba41ffb700 1 -- 172.21.15.19:0/509022945 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fba441060a0 con 0x7fba3003e420 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fba3003e420 msgr2=0x7fba30040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fba3003e420 0x7fba30040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fba2c006fd0 tx=0x7fba2c006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 msgr2=0x7fba4419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4419dcf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba34003f20 tx=0x7fba34011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 shutdown_connections 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fba3003e420 0x7fba30040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:26.710 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 --2- 172.21.15.19:0/509022945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba4410a510 0x7fba4419dcf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.094+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 >> 172.21.15.19:0/509022945 conn(0x7fba44104030 msgr2=0x7fba44105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.095+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 shutdown_connections 2024-05-17T09:07:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:26.095+0000 7fba4a6d1700 1 -- 172.21.15.19:0/509022945 wait complete. 2024-05-17T09:07:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-17T09:07:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-05-17T09:07:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-05-17T09:07:27.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:26 smithi019 ceph-mon[31620]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:27.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:27 smithi019 ceph-mon[31620]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:29.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:28 smithi019 ceph-mon[31620]: Deploying cephadm binary to smithi019 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.137+0000 7f48e38e7700 1 Processor -- start 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.137+0000 7f48e38e7700 1 -- start start 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.137+0000 7f48e38e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc108d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.137+0000 7f48e38e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc109330 con 0x7f48dc108930 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.137+0000 7f48e1683700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48e1683700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47402/0 (socket says 172.21.15.19:47402) 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48e1683700 1 -- 172.21.15.19:0/3441034501 learned_addr learned my addr 172.21.15.19:0/3441034501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48e1683700 1 -- 172.21.15.19:0/3441034501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc109a60 con 0x7f48dc108930 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48e1683700 1 --2- 172.21.15.19:0/3441034501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc108d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48d80098a0 tx=0x7f48d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a149a96b54700a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48d3fff700 1 -- 172.21.15.19:0/3441034501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d8004160 con 0x7f48dc108930 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48d3fff700 1 -- 172.21.15.19:0/3441034501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f48d8004300 con 0x7f48dc108930 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.138+0000 7f48d3fff700 1 -- 172.21.15.19:0/3441034501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d80119e0 con 0x7f48dc108930 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 -- 172.21.15.19:0/3441034501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 msgr2=0x7f48dc108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 --2- 172.21.15.19:0/3441034501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc108d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48d80098a0 tx=0x7f48d8009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 -- 172.21.15.19:0/3441034501 shutdown_connections 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 --2- 172.21.15.19:0/3441034501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc108d40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 -- 172.21.15.19:0/3441034501 >> 172.21.15.19:0/3441034501 conn(0x7f48dc104030 msgr2=0x7f48dc106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 -- 172.21.15.19:0/3441034501 shutdown_connections 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.139+0000 7f48e38e7700 1 -- 172.21.15.19:0/3441034501 wait complete. 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e38e7700 1 Processor -- start 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e38e7700 1 -- start start 2024-05-17T09:07:32.199 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e38e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e38e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc10ce10 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e1683700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e1683700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47412/0 (socket says 172.21.15.19:47412) 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e1683700 1 -- 172.21.15.19:0/3174518895 learned_addr learned my addr 172.21.15.19:0/3174518895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.140+0000 7f48e1683700 1 -- 172.21.15.19:0/3174518895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc19e2a0 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.141+0000 7f48e1683700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc19dd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48d80046e0 tx=0x7f48d8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.141+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d80039e0 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.141+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f48d8003b80 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.141+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d8024700 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.141+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc19e530 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.141+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1a1130 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.142+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc064e10 con 0x7f48dc108930 2024-05-17T09:07:32.200 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.142+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f48d8022020 con 0x7f48dc108930 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.142+0000 7f48d27fc700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f48c803e4c0 0x7f48c80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.142+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48d8053b90 con 0x7f48dc108930 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.142+0000 7f48e0e82700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f48c803e4c0 0x7f48c80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.143+0000 7f48e0e82700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f48c803e4c0 0x7f48c80409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48cc006fd0 tx=0x7f48cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.147+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48d802c350 con 0x7f48dc108930 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:27.365+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f48dc063bf0 con 0x7f48c803e4c0 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.627+0000 7f48d27fc700 1 -- 172.21.15.19:0/3174518895 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f48dc063bf0 con 0x7f48c803e4c0 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f48c803e4c0 msgr2=0x7f48c80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f48c803e4c0 0x7f48c80409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48cc006fd0 tx=0x7f48cc006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 msgr2=0x7f48dc19dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc19dd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48d80046e0 tx=0x7f48d8011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 shutdown_connections 2024-05-17T09:07:32.201 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f48c803e4c0 0x7f48c80409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 --2- 172.21.15.19:0/3174518895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48dc108930 0x7f48dc19dd60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.629+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 >> 172.21.15.19:0/3174518895 conn(0x7f48dc104030 msgr2=0x7f48dc194330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.630+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 shutdown_connections 2024-05-17T09:07:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:31.630+0000 7f48e38e7700 1 -- 172.21.15.19:0/3174518895 wait complete. 2024-05-17T09:07:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-05-17T09:07:32.908 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:32 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:32.908 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:32 smithi019 ceph-mon[31620]: Added host smithi019 2024-05-17T09:07:32.908 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:32 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:33.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.668+0000 7f3b4ba50700 1 Processor -- start 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b4ba50700 1 -- start start 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b4ba50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b4410c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b4ba50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410cb60 con 0x7f3b4407d540 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b497ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b4410c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b497ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b4410c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47422/0 (socket says 172.21.15.19:47422) 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b497ec700 1 -- 172.21.15.19:0/64755561 learned_addr learned my addr 172.21.15.19:0/64755561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.669+0000 7f3b497ec700 1 -- 172.21.15.19:0/64755561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4410cce0 con 0x7f3b4407d540 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.670+0000 7f3b497ec700 1 --2- 172.21.15.19:0/64755561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b4410c550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098a0 tx=0x7f3b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc35be21280eaf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:33.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.670+0000 7f3b3bfff700 1 -- 172.21.15.19:0/64755561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40004160 con 0x7f3b4407d540 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.670+0000 7f3b3bfff700 1 -- 172.21.15.19:0/64755561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3b40004300 con 0x7f3b4407d540 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.670+0000 7f3b3bfff700 1 -- 172.21.15.19:0/64755561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b400119e0 con 0x7f3b4407d540 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 -- 172.21.15.19:0/64755561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 msgr2=0x7f3b4410c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 --2- 172.21.15.19:0/64755561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b4410c550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098a0 tx=0x7f3b40009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 -- 172.21.15.19:0/64755561 shutdown_connections 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 --2- 172.21.15.19:0/64755561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b4410c550 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 -- 172.21.15.19:0/64755561 >> 172.21.15.19:0/64755561 conn(0x7f3b44103fe0 msgr2=0x7f3b44106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 -- 172.21.15.19:0/64755561 shutdown_connections 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 -- 172.21.15.19:0/64755561 wait complete. 2024-05-17T09:07:33.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 Processor -- start 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.671+0000 7f3b4ba50700 1 -- start start 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b4ba50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b44109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b4ba50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410dd00 con 0x7f3b4407d540 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b497ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b44109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b497ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b44109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47426/0 (socket says 172.21.15.19:47426) 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b497ec700 1 -- 172.21.15.19:0/4246159087 learned_addr learned my addr 172.21.15.19:0/4246159087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b497ec700 1 -- 172.21.15.19:0/4246159087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4410a200 con 0x7f3b4407d540 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b497ec700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b44109cc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b400116a0 tx=0x7f3b40011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40003970 con 0x7f3b4407d540 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3b40003b10 con 0x7f3b4407d540 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b44108450 con 0x7f3b4407d540 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40019550 con 0x7f3b4407d540 2024-05-17T09:07:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.672+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b44108930 con 0x7f3b4407d540 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.673+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b44064e10 con 0x7f3b4407d540 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.673+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3b400197c0 con 0x7f3b4407d540 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.674+0000 7f3b3a7fc700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f3b3003e4c0 0x7f3b300409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.674+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b40053880 con 0x7f3b4407d540 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.674+0000 7f3b48feb700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f3b3003e4c0 0x7f3b300409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.674+0000 7f3b48feb700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f3b3003e4c0 0x7f3b300409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.679+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b4002adc0 con 0x7f3b4407d540 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.907+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3b441090e0 con 0x7f3b3003e4c0 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.915+0000 7f3b3a7fc700 1 -- 172.21.15.19:0/4246159087 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3b441090e0 con 0x7f3b3003e4c0 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f3b3003e4c0 msgr2=0x7f3b300409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f3b3003e4c0 0x7f3b300409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 msgr2=0x7f3b44109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:33.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b44109cc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b400116a0 tx=0x7f3b40011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 shutdown_connections 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f3b3003e4c0 0x7f3b300409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 --2- 172.21.15.19:0/4246159087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4407d540 0x7f3b44109cc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 >> 172.21.15.19:0/4246159087 conn(0x7f3b44103fe0 msgr2=0x7f3b44105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 shutdown_connections 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:32.918+0000 7f3b4ba50700 1 -- 172.21.15.19:0/4246159087 wait complete. 2024-05-17T09:07:33.437 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-05-17T09:07:33.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:33 smithi019 ceph-mon[31620]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:33.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:33 smithi019 ceph-mon[31620]: Saving service mon spec with placement count:5 2024-05-17T09:07:33.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:33 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:34.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-17T09:07:34.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0fb1704700 1 Processor -- start 2024-05-17T09:07:34.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0fb1704700 1 -- start start 2024-05-17T09:07:34.657 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0fb1704700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac07dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0fb1704700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac07dff0 con 0x7f0fac07f590 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0faaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac07dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0faaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac07dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53694/0 (socket says 172.21.15.19:53694) 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.851+0000 7f0faaffd700 1 -- 172.21.15.19:0/3186775150 learned_addr learned my addr 172.21.15.19:0/3186775150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.852+0000 7f0faaffd700 1 -- 172.21.15.19:0/3186775150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac07e170 con 0x7f0fac07f590 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.852+0000 7f0faaffd700 1 --2- 172.21.15.19:0/3186775150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac07dab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0098a0 tx=0x7f0f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac28fe8cbe23174 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.852+0000 7f0fa9ffb700 1 -- 172.21.15.19:0/3186775150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f9c004160 con 0x7f0fac07f590 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.852+0000 7f0fa9ffb700 1 -- 172.21.15.19:0/3186775150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0f9c004300 con 0x7f0fac07f590 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.852+0000 7f0fa9ffb700 1 -- 172.21.15.19:0/3186775150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f9c0119e0 con 0x7f0fac07f590 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 -- 172.21.15.19:0/3186775150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 msgr2=0x7f0fac07dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 --2- 172.21.15.19:0/3186775150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac07dab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0098a0 tx=0x7f0f9c009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 -- 172.21.15.19:0/3186775150 shutdown_connections 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 --2- 172.21.15.19:0/3186775150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac07dab0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 -- 172.21.15.19:0/3186775150 >> 172.21.15.19:0/3186775150 conn(0x7f0fac1050f0 msgr2=0x7f0fac107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 -- 172.21.15.19:0/3186775150 shutdown_connections 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 -- 172.21.15.19:0/3186775150 wait complete. 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.853+0000 7f0fb1704700 1 Processor -- start 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0fb1704700 1 -- start start 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0fb1704700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0fb1704700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac10fef0 con 0x7f0fac07f590 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0faaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0faaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac19fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53710/0 (socket says 172.21.15.19:53710) 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0faaffd700 1 -- 172.21.15.19:0/126296894 learned_addr learned my addr 172.21.15.19:0/126296894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0faaffd700 1 -- 172.21.15.19:0/126296894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac1a03e0 con 0x7f0fac07f590 2024-05-17T09:07:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.854+0000 7f0faaffd700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac19fea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0116a0 tx=0x7f0f9c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.855+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f9c003970 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.855+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0f9c003b10 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.855+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac1a0670 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.855+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f9c019550 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.855+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac1a3280 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.856+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0f9c0197c0 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.856+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac064e10 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.856+0000 7f0f93fff700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f0f9403e4c0 0x7f0f940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.856+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0f9c053890 con 0x7f0fac07f590 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.856+0000 7f0faa7fc700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f0f9403e4c0 0x7f0f940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.856+0000 7f0faa7fc700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f0f9403e4c0 0x7f0f940409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006fd0 tx=0x7f0fa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:33.861+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f9c016260 con 0x7f0fac07f590 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.097+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0fac107140 con 0x7f0f9403e4c0 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.102+0000 7f0f93fff700 1 -- 172.21.15.19:0/126296894 <== mgr.14120 v2:172.21.15.19:6800/2577177111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0fac107140 con 0x7f0f9403e4c0 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f0f9403e4c0 msgr2=0x7f0f940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f0f9403e4c0 0x7f0f940409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006fd0 tx=0x7f0fa0006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 msgr2=0x7f0fac19fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac19fea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0116a0 tx=0x7f0f9c011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 shutdown_connections 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f0f9403e4c0 0x7f0f940409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 --2- 172.21.15.19:0/126296894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fac07f590 0x7f0fac19fea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.104+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 >> 172.21.15.19:0/126296894 conn(0x7f0fac1050f0 msgr2=0x7f0fac106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.105+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 shutdown_connections 2024-05-17T09:07:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:34.105+0000 7f0fb1704700 1 -- 172.21.15.19:0/126296894 wait complete. 2024-05-17T09:07:35.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:35 smithi019 ceph-mon[31620]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:35.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:35 smithi019 ceph-mon[31620]: Saving service mgr spec with placement count:2 2024-05-17T09:07:35.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:35 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:36.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.447+0000 7f7f48a20700 1 Processor -- start 2024-05-17T09:07:36.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.447+0000 7f7f48a20700 1 -- start start 2024-05-17T09:07:36.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.447+0000 7f7f48a20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f44108d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:36.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.447+0000 7f7f48a20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44109340 con 0x7f7f44108940 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f44108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f44108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53726/0 (socket says 172.21.15.19:53726) 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4259c700 1 -- 172.21.15.19:0/2641443573 learned_addr learned my addr 172.21.15.19:0/2641443573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4259c700 1 -- 172.21.15.19:0/2641443573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f44109a70 con 0x7f7f44108940 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4259c700 1 --2- 172.21.15.19:0/2641443573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f44108d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f340098a0 tx=0x7f7f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1d0b3b2726cc58 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4159a700 1 -- 172.21.15.19:0/2641443573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f34004160 con 0x7f7f44108940 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4159a700 1 -- 172.21.15.19:0/2641443573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7f34004300 con 0x7f7f44108940 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.448+0000 7f7f4159a700 1 -- 172.21.15.19:0/2641443573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f340119e0 con 0x7f7f44108940 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 -- 172.21.15.19:0/2641443573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 msgr2=0x7f7f44108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 --2- 172.21.15.19:0/2641443573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f44108d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f340098a0 tx=0x7f7f34009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 -- 172.21.15.19:0/2641443573 shutdown_connections 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 --2- 172.21.15.19:0/2641443573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f44108d50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:36.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 -- 172.21.15.19:0/2641443573 >> 172.21.15.19:0/2641443573 conn(0x7f7f44103f80 msgr2=0x7f7f44106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 -- 172.21.15.19:0/2641443573 shutdown_connections 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.449+0000 7f7f48a20700 1 -- 172.21.15.19:0/2641443573 wait complete. 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f48a20700 1 Processor -- start 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f48a20700 1 -- start start 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f48a20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f4419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f48a20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4410ce20 con 0x7f7f44108940 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f4259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f4419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f4259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f4419dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53728/0 (socket says 172.21.15.19:53728) 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.450+0000 7f7f4259c700 1 -- 172.21.15.19:0/1605088368 learned_addr learned my addr 172.21.15.19:0/1605088368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f4259c700 1 -- 172.21.15.19:0/1605088368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4419e2a0 con 0x7f7f44108940 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f4259c700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f4419dd60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f340119b0 tx=0x7f7f34011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f34003970 con 0x7f7f44108940 2024-05-17T09:07:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7f34003b10 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4419e530 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f34024610 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.451+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f441a1130 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.452+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7f3402d3f0 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.452+0000 7f7f337fe700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f7f2c03e4c0 0x7f7f2c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.452+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f340547f0 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.452+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f44064e10 con 0x7f7f44108940 2024-05-17T09:07:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.452+0000 7f7f41d9b700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f7f2c03e4c0 0x7f7f2c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.453+0000 7f7f41d9b700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f7f2c03e4c0 0x7f7f2c0409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.458+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f3400e260 con 0x7f7f44108940 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.673+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f7f4402e870 con 0x7f7f44108940 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.676+0000 7f7f337fe700 1 -- 172.21.15.19:0/1605088368 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f7f34010d90 con 0x7f7f44108940 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f7f2c03e4c0 msgr2=0x7f7f2c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f7f2c03e4c0 0x7f7f2c0409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f38006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 msgr2=0x7f7f4419dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f4419dd60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f340119b0 tx=0x7f7f34011a90 comp rx=0 tx=0).stop 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 shutdown_connections 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f7f2c03e4c0 0x7f7f2c0409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 --2- 172.21.15.19:0/1605088368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f44108940 0x7f7f4419dd60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 >> 172.21.15.19:0/1605088368 conn(0x7f7f44103f80 msgr2=0x7f7f44194310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 shutdown_connections 2024-05-17T09:07:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:35.678+0000 7f7f48a20700 1 -- 172.21.15.19:0/1605088368 wait complete. 2024-05-17T09:07:36.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:36 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1605088368' entity='client.admin' 2024-05-17T09:07:36.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:36 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:36.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:36 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:38.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:38 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2410756849' entity='client.admin' 2024-05-17T09:07:38.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:38 smithi019 ceph-mon[31620]: from='mgr.14120 172.21.15.19:0/3661627720' entity='mgr.x' 2024-05-17T09:07:38.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.914+0000 7fc41a956700 1 Processor -- start 2024-05-17T09:07:38.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.914+0000 7fc41a956700 1 -- start start 2024-05-17T09:07:38.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.914+0000 7fc41a956700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41410c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.914+0000 7fc41a956700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410cb60 con 0x7fc41407d520 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.915+0000 7fc413fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41410c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.915+0000 7fc413fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41410c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53732/0 (socket says 172.21.15.19:53732) 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.915+0000 7fc413fff700 1 -- 172.21.15.19:0/1171550362 learned_addr learned my addr 172.21.15.19:0/1171550362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.915+0000 7fc413fff700 1 -- 172.21.15.19:0/1171550362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41410cce0 con 0x7fc41407d520 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.915+0000 7fc413fff700 1 --2- 172.21.15.19:0/1171550362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41410c570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4040098a0 tx=0x7fc404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba25a0d910454fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.916+0000 7fc412ffd700 1 -- 172.21.15.19:0/1171550362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc404004160 con 0x7fc41407d520 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.916+0000 7fc412ffd700 1 -- 172.21.15.19:0/1171550362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc404004300 con 0x7fc41407d520 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.916+0000 7fc412ffd700 1 -- 172.21.15.19:0/1171550362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc404011960 con 0x7fc41407d520 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.916+0000 7fc41a956700 1 -- 172.21.15.19:0/1171550362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 msgr2=0x7fc41410c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 --2- 172.21.15.19:0/1171550362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41410c570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4040098a0 tx=0x7fc404009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:38.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 -- 172.21.15.19:0/1171550362 shutdown_connections 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 --2- 172.21.15.19:0/1171550362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41410c570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 -- 172.21.15.19:0/1171550362 >> 172.21.15.19:0/1171550362 conn(0x7fc414103fe0 msgr2=0x7fc414106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 -- 172.21.15.19:0/1171550362 shutdown_connections 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 -- 172.21.15.19:0/1171550362 wait complete. 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.917+0000 7fc41a956700 1 Processor -- start 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc41a956700 1 -- start start 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc41a956700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc41a956700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc41410dd40 con 0x7fc41407d520 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc413fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc413fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53744/0 (socket says 172.21.15.19:53744) 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc413fff700 1 -- 172.21.15.19:0/2410756849 learned_addr learned my addr 172.21.15.19:0/2410756849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.918+0000 7fc413fff700 1 -- 172.21.15.19:0/2410756849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41419e240 con 0x7fc41407d520 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.919+0000 7fc413fff700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41419dd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc404011670 tx=0x7fc404011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.919+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc404003aa0 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.919+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc40401b8d0 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.919+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4040197f0 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.919+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41419e4d0 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.919+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141a10e0 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.920+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc414064e10 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.920+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc404019990 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.920+0000 7fc4117fa700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fc3fc03e470 0x7fc3fc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.920+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc404054000 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.921+0000 7fc4137fe700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fc3fc03e470 0x7fc3fc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.921+0000 7fc4137fe700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fc3fc03e470 0x7fc3fc040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:36.926+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc40401f030 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.163+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc4141a12b0 con 0x7fc41407d520 2024-05-17T09:07:38.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.166+0000 7fc4117fa700 1 -- 172.21.15.19:0/2410756849 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc40402adc0 con 0x7fc41407d520 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fc3fc03e470 msgr2=0x7fc3fc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fc3fc03e470 0x7fc3fc040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 msgr2=0x7fc41419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41419dd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc404011670 tx=0x7fc404011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 shutdown_connections 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fc3fc03e470 0x7fc3fc040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 --2- 172.21.15.19:0/2410756849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc41407d520 0x7fc41419dd00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 >> 172.21.15.19:0/2410756849 conn(0x7fc414103fe0 msgr2=0x7fc4141058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 shutdown_connections 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:37.169+0000 7fc41a956700 1 -- 172.21.15.19:0/2410756849 wait complete. 2024-05-17T09:07:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-05-17T09:07:39.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:39 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3329249558' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-17T09:07:39.408 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: ignoring --setuser ceph since I am not root 2024-05-17T09:07:39.408 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: ignoring --setgroup ceph since I am not root 2024-05-17T09:07:39.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:39.406+0000 7f80da41f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T09:07:39.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:39.615+0000 7f80da41f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:07:39.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad2b2a700 1 Processor -- start 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad2b2a700 1 -- start start 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad2b2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc108720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad2b2a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc07d7f0 con 0x7f2acc108310 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad08c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad08c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53758/0 (socket says 172.21.15.19:53758) 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.811+0000 7f2ad08c6700 1 -- 172.21.15.19:0/1549634760 learned_addr learned my addr 172.21.15.19:0/1549634760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.812+0000 7f2ad08c6700 1 -- 172.21.15.19:0/1549634760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc108c60 con 0x7f2acc108310 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.812+0000 7f2ad08c6700 1 --2- 172.21.15.19:0/1549634760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc108720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00098a0 tx=0x7f2ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d107a4e355997b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.812+0000 7f2acb7fe700 1 -- 172.21.15.19:0/1549634760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0004160 con 0x7f2acc108310 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.812+0000 7f2acb7fe700 1 -- 172.21.15.19:0/1549634760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ac0004300 con 0x7f2acc108310 2024-05-17T09:07:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.812+0000 7f2acb7fe700 1 -- 172.21.15.19:0/1549634760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0011960 con 0x7f2acc108310 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/1549634760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 msgr2=0x7f2acc108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 --2- 172.21.15.19:0/1549634760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc108720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00098a0 tx=0x7f2ac0009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/1549634760 shutdown_connections 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 --2- 172.21.15.19:0/1549634760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc108720 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/1549634760 >> 172.21.15.19:0/1549634760 conn(0x7f2acc103f80 msgr2=0x7f2acc106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/1549634760 shutdown_connections 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.813+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/1549634760 wait complete. 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad2b2a700 1 Processor -- start 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad2b2a700 1 -- start start 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad2b2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc19ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad2b2a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc109b40 con 0x7f2acc108310 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad08c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc19ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:39.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad08c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc19ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53762/0 (socket says 172.21.15.19:53762) 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad08c6700 1 -- 172.21.15.19:0/3329249558 learned_addr learned my addr 172.21.15.19:0/3329249558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.814+0000 7f2ad08c6700 1 -- 172.21.15.19:0/3329249558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc1a0480 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.815+0000 7f2ad08c6700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc19ff40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0011670 tx=0x7f2ac0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.815+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0003a60 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.815+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ac001b8d0 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.815+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0023750 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.815+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc1a0710 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.815+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc1a3320 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.816+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc064e10 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.816+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2ac001b440 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.817+0000 7f2ac9ffb700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f2abc03e470 0x7f2abc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.817+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ac0053f90 con 0x7f2acc108310 2024-05-17T09:07:39.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.817+0000 7f2acbfff700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f2abc03e470 0x7f2abc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.817+0000 7f2acbfff700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f2abc03e470 0x7f2abc040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:38.822+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ac001f030 con 0x7f2acc108310 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.108+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f2acc02e850 con 0x7f2acc108310 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.170+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f2ac00202b0 con 0x7f2acc108310 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.172+0000 7f2ac9ffb700 1 -- 172.21.15.19:0/3329249558 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f2ac002b530 con 0x7f2acc108310 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f2abc03e470 msgr2=0x7f2abc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f2abc03e470 0x7f2abc040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 msgr2=0x7f2acc19ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc19ff40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0011670 tx=0x7f2ac0011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 shutdown_connections 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f2abc03e470 0x7f2abc040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 --2- 172.21.15.19:0/3329249558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2acc108310 0x7f2acc19ff40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 >> 172.21.15.19:0/3329249558 conn(0x7f2acc103f80 msgr2=0x7f2acc193ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:39.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 shutdown_connections 2024-05-17T09:07:39.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:39.174+0000 7f2ad2b2a700 1 -- 172.21.15.19:0/3329249558 wait complete. 2024-05-17T09:07:40.040 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:39.723+0000 7f80da41f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T09:07:40.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:40 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3329249558' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-17T09:07:40.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:40 smithi019 ceph-mon[31620]: mgrmap e9: x(active, since 19s) 2024-05-17T09:07:40.296 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:40.057+0000 7f80da41f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T09:07:40.296 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:40.148+0000 7f80da41f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T09:07:41.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:41 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/674736244' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T09:07:41.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:07:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-17T09:07:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T09:07:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-17T09:07:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.295+0000 7fa9a7aad700 1 Processor -- start 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.295+0000 7fa9a7aad700 1 -- start start 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.295+0000 7fa9a7aad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a0108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.295+0000 7fa9a7aad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a007cec0 con 0x7fa9a0108300 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a5849700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a0108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a5849700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a0108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53800/0 (socket says 172.21.15.19:53800) 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a5849700 1 -- 172.21.15.19:0/3423246864 learned_addr learned my addr 172.21.15.19:0/3423246864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a5849700 1 -- 172.21.15.19:0/3423246864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a0108c50 con 0x7fa9a0108300 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a5849700 1 --2- 172.21.15.19:0/3423246864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a0108710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0098a0 tx=0x7fa99c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eecc01f36f7c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a4847700 1 -- 172.21.15.19:0/3423246864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c004160 con 0x7fa9a0108300 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a4847700 1 -- 172.21.15.19:0/3423246864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa99c004300 con 0x7fa9a0108300 2024-05-17T09:07:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.296+0000 7fa9a4847700 1 -- 172.21.15.19:0/3423246864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c011960 con 0x7fa9a0108300 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 -- 172.21.15.19:0/3423246864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 msgr2=0x7fa9a0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 --2- 172.21.15.19:0/3423246864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a0108710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0098a0 tx=0x7fa99c009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 -- 172.21.15.19:0/3423246864 shutdown_connections 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 --2- 172.21.15.19:0/3423246864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a0108710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 -- 172.21.15.19:0/3423246864 >> 172.21.15.19:0/3423246864 conn(0x7fa9a0103ff0 msgr2=0x7fa9a0106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 -- 172.21.15.19:0/3423246864 shutdown_connections 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.297+0000 7fa9a7aad700 1 -- 172.21.15.19:0/3423246864 wait complete. 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.298+0000 7fa9a7aad700 1 Processor -- start 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.298+0000 7fa9a7aad700 1 -- start start 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.298+0000 7fa9a7aad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.298+0000 7fa9a7aad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0109cb0 con 0x7fa9a0108300 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a5849700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a5849700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53808/0 (socket says 172.21.15.19:53808) 2024-05-17T09:07:41.353 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a5849700 1 -- 172.21.15.19:0/674736244 learned_addr learned my addr 172.21.15.19:0/674736244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a5849700 1 -- 172.21.15.19:0/674736244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a01a0530 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a5849700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a019fff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa99c011670 tx=0x7fa99c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c003aa0 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa99c01b8d0 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01a07c0 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c0197f0 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.299+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01a33d0 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.300+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a0064e10 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.300+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fa99c01b440 con 0x7fa9a0108300 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.301+0000 7fa996ffd700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.301+0000 7fa9a5048700 1 -- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 msgr2=0x7fa98c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:41.354 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.301+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa99c053ed0 con 0x7fa9a0108300 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.301+0000 7fa9a5048700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.306+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa99c01f030 con 0x7fa9a0108300 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.502+0000 7fa9a5048700 1 -- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 msgr2=0x7fa98c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.502+0000 7fa9a5048700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.625+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa9a002e7b0 con 0x7fa9a0108300 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.626+0000 7fa996ffd700 1 -- 172.21.15.19:0/674736244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa99c019dc0 con 0x7fa9a0108300 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 msgr2=0x7fa98c0409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 msgr2=0x7fa9a019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a019fff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa99c011670 tx=0x7fa99c011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 shutdown_connections 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 --2- 172.21.15.19:0/674736244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108300 0x7fa9a019fff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:41.355 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 >> 172.21.15.19:0/674736244 conn(0x7fa9a0103ff0 msgr2=0x7fa9a0193f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:41.356 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.628+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 shutdown_connections 2024-05-17T09:07:41.356 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:40.629+0000 7fa9a7aad700 1 -- 172.21.15.19:0/674736244 wait complete. 2024-05-17T09:07:41.356 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-17T09:07:41.356 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-05-17T09:07:41.593 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:41 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:41.445+0000 7f80da41f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T09:07:41.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:41 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:41.631+0000 7f80da41f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T09:07:41.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:41 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:41.820+0000 7f80da41f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T09:07:42.382 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:42 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:42.127+0000 7f80da41f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T09:07:42.382 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:42 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:42.226+0000 7f80da41f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T09:07:42.650 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:42 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:42.443+0000 7f80da41f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T09:07:42.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:42 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:42.649+0000 7f80da41f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:07:43.523 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:43 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:43.287+0000 7f80da41f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T09:07:43.523 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:43 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:43.523+0000 7f80da41f200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T09:07:44.442 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:43 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:43.976+0000 7f80da41f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T09:07:44.442 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:44 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:44.080+0000 7f80da41f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T09:07:44.808 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:44 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:44.471+0000 7f80da41f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T09:07:45.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:44 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:44.850+0000 7f80da41f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T09:07:45.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:44 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:44.950+0000 7f80da41f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T09:07:45.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:45 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:45.211+0000 7f80da41f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T09:07:45.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:45 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:45.354+0000 7f80da41f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T09:07:45.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:45 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:45.455+0000 7f80da41f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T09:07:46.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:45 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:45.797+0000 7f80da41f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T09:07:46.192 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:45 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:45.896+0000 7f80da41f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T09:07:46.822 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:46 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:46.538+0000 7f80da41f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T09:07:46.823 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:07:46 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:07:46.769+0000 7f80da41f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: Active manager daemon x restarted 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: Activating manager daemon x 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: osdmap e3: 0 total, 0 up, 0 in 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: mgrmap e10: x(active, starting, since 0.00454966s) 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T09:07:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T09:07:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T09:07:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T09:07:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: Manager daemon x is now available 2024-05-17T09:07:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T09:07:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T09:07:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:47 smithi019 ceph-mon[31620]: [17/May/2024:09:07:47] ENGINE Bus STARTING 2024-05-17T09:07:48.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:47 smithi019 ceph-mon[31620]: mgrmap e11: x(active, since 1.0068s) 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.797+0000 7f29e038b700 1 Processor -- start 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.797+0000 7f29e038b700 1 -- start start 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.798+0000 7f29e038b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d807d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.798+0000 7f29e038b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29d807d760 con 0x7f29d807ed50 2024-05-17T09:07:48.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.798+0000 7f29de127700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d807d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.798+0000 7f29de127700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d807d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53824/0 (socket says 172.21.15.19:53824) 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.798+0000 7f29de127700 1 -- 172.21.15.19:0/726154696 learned_addr learned my addr 172.21.15.19:0/726154696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.798+0000 7f29de127700 1 -- 172.21.15.19:0/726154696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d807d8e0 con 0x7f29d807ed50 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.799+0000 7f29de127700 1 --2- 172.21.15.19:0/726154696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d807d220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29d40098a0 tx=0x7f29d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9399674e2a37297d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.799+0000 7f29dd125700 1 -- 172.21.15.19:0/726154696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4004160 con 0x7f29d807ed50 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.799+0000 7f29dd125700 1 -- 172.21.15.19:0/726154696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29d4004300 con 0x7f29d807ed50 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.799+0000 7f29dd125700 1 -- 172.21.15.19:0/726154696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4011960 con 0x7f29d807ed50 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 -- 172.21.15.19:0/726154696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 msgr2=0x7f29d807d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 --2- 172.21.15.19:0/726154696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d807d220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29d40098a0 tx=0x7f29d4009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 -- 172.21.15.19:0/726154696 shutdown_connections 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 --2- 172.21.15.19:0/726154696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d807d220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 -- 172.21.15.19:0/726154696 >> 172.21.15.19:0/726154696 conn(0x7f29d81050d0 msgr2=0x7f29d8107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:48.370 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 -- 172.21.15.19:0/726154696 shutdown_connections 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.800+0000 7f29e038b700 1 -- 172.21.15.19:0/726154696 wait complete. 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.801+0000 7f29e038b700 1 Processor -- start 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.801+0000 7f29e038b700 1 -- start start 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.801+0000 7f29e038b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.801+0000 7f29e038b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29d810ffb0 con 0x7f29d807ed50 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29de127700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29de127700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d819ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53836/0 (socket says 172.21.15.19:53836) 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29de127700 1 -- 172.21.15.19:0/1075641224 learned_addr learned my addr 172.21.15.19:0/1075641224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29de127700 1 -- 172.21.15.19:0/1075641224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d81a0450 con 0x7f29d807ed50 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29de127700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d819ff10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29d4011670 tx=0x7f29d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4003a60 con 0x7f29d807ed50 2024-05-17T09:07:48.371 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29d401b8d0 con 0x7f29d807ed50 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d81a06e0 con 0x7f29d807ed50 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4023750 con 0x7f29d807ed50 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.802+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d81a32f0 con 0x7f29d807ed50 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.803+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f29d4023a50 con 0x7f29d807ed50 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.804+0000 7f29cf7fe700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.804+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 --> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f29c4041090 con 0x7f29c403e470 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.804+0000 7f29dd926700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 msgr2=0x7f29c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.804+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f29d401ba80 con 0x7f29d807ed50 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:41.804+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:42.005+0000 7f29dd926700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 msgr2=0x7f29c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:42.005+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:42.406+0000 7f29dd926700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 msgr2=0x7f29c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:48.372 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:42.406+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:43.207+0000 7f29dd926700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 msgr2=0x7f29c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:43.207+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:44.809+0000 7f29dd926700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 msgr2=0x7f29c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2577177111 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:44.809+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:46.773+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f29d402c3f0 con 0x7f29d807ed50 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:46.774+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 msgr2=0x7f29c4040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:46.774+0000 7f29cf7fe700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.775+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f29d402b350 con 0x7f29d807ed50 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.775+0000 7f29cf7fe700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29d40548c0 0x7f29c4044000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.775+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f29c4041090 con 0x7f29d40548c0 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.776+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29d40548c0 0x7f29c4044000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:48.373 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.776+0000 7f29dd926700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29d40548c0 0x7f29c4044000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f29c8003a10 tx=0x7f29c80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.777+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f29c4041090 con 0x7f29d40548c0 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.785+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f29d8064e10 con 0x7f29d40548c0 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29cf7fe700 1 -- 172.21.15.19:0/1075641224 <== mgr.14152 v2:172.21.15.19:6800/3017687651 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f29d8064e10 con 0x7f29d40548c0 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29d40548c0 msgr2=0x7f29c4044000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29d40548c0 0x7f29c4044000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f29c8003a10 tx=0x7f29c80092b0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 msgr2=0x7f29d819ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d819ff10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29d4011670 tx=0x7f29d4011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 shutdown_connections 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29d40548c0 0x7f29c4044000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d807ed50 0x7f29d819ff10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 --2- 172.21.15.19:0/1075641224 >> [v2:172.21.15.19:6800/2577177111,v1:172.21.15.19:6801/2577177111] conn(0x7f29c403e470 0x7f29c4040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 >> 172.21.15.19:0/1075641224 conn(0x7f29d81050d0 msgr2=0x7f29d8105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:48.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 shutdown_connections 2024-05-17T09:07:48.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:47.786+0000 7f29e038b700 1 -- 172.21.15.19:0/1075641224 wait complete. 2024-05-17T09:07:48.375 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-05-17T09:07:48.375 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-05-17T09:07:49.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:48 smithi019 ceph-mon[31620]: [17/May/2024:09:07:47] ENGINE Serving on https://172.21.15.19:7150 2024-05-17T09:07:49.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:48 smithi019 ceph-mon[31620]: [17/May/2024:09:07:47] ENGINE Serving on http://172.21.15.19:8765 2024-05-17T09:07:49.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:48 smithi019 ceph-mon[31620]: [17/May/2024:09:07:47] ENGINE Bus STARTED 2024-05-17T09:07:49.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:48 smithi019 ceph-mon[31620]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T09:07:49.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:48 smithi019 ceph-mon[31620]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T09:07:50.008 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:49 smithi019 ceph-mon[31620]: mgrmap e12: x(active, since 2s) 2024-05-17T09:07:50.008 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:50.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:50.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce3ceb700 1 Processor -- start 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce3ceb700 1 -- start start 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce3ceb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc10a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce3ceb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10a890 con 0x7f0cdc109e70 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce1a87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc10a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce1a87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc10a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54600/0 (socket says 172.21.15.19:54600) 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.054+0000 7f0ce1a87700 1 -- 172.21.15.19:0/3109451054 learned_addr learned my addr 172.21.15.19:0/3109451054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:50.016 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.055+0000 7f0ce1a87700 1 -- 172.21.15.19:0/3109451054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc108b40 con 0x7f0cdc109e70 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.055+0000 7f0ce1a87700 1 --2- 172.21.15.19:0/3109451054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc10a280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80098a0 tx=0x7f0cd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa78ff26d18e8955 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.055+0000 7f0ce0a85700 1 -- 172.21.15.19:0/3109451054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd8004160 con 0x7f0cdc109e70 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.055+0000 7f0ce0a85700 1 -- 172.21.15.19:0/3109451054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cd8004300 con 0x7f0cdc109e70 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.055+0000 7f0ce0a85700 1 -- 172.21.15.19:0/3109451054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd8011960 con 0x7f0cdc109e70 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/3109451054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 msgr2=0x7f0cdc10a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 --2- 172.21.15.19:0/3109451054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc10a280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80098a0 tx=0x7f0cd8009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/3109451054 shutdown_connections 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 --2- 172.21.15.19:0/3109451054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc10a280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/3109451054 >> 172.21.15.19:0/3109451054 conn(0x7f0cdc103800 msgr2=0x7f0cdc105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/3109451054 shutdown_connections 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.056+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/3109451054 wait complete. 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce3ceb700 1 Processor -- start 2024-05-17T09:07:50.017 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce3ceb700 1 -- start start 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce3ceb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce3ceb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10cd80 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce1a87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce1a87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc19dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54614/0 (socket says 172.21.15.19:54614) 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce1a87700 1 -- 172.21.15.19:0/993239286 learned_addr learned my addr 172.21.15.19:0/993239286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.057+0000 7f0ce1a87700 1 -- 172.21.15.19:0/993239286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc19e250 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.058+0000 7f0ce1a87700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc19dd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8011980 tx=0x7f0cd8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.058+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd80039e0 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.058+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cd801b8d0 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.058+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cd80236b0 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.058+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc19e4e0 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.058+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc1a10e0 con 0x7f0cdc109e70 2024-05-17T09:07:50.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.059+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc064e10 con 0x7f0cdc109e70 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.059+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f0cd8023960 con 0x7f0cdc109e70 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.059+0000 7f0cceffd700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0cc803e4d0 0x7f0cc80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.059+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0cd8053fd0 con 0x7f0cdc109e70 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.060+0000 7f0ce1286700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0cc803e4d0 0x7f0cc80409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.060+0000 7f0ce1286700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0cc803e4d0 0x7f0cc80409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0006fd0 tx=0x7f0cd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.065+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cd801f030 con 0x7f0cdc109e70 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.294+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0cdc02e920 con 0x7f0cc803e4d0 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.381+0000 7f0cceffd700 1 -- 172.21.15.19:0/993239286 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0cdc02e920 con 0x7f0cc803e4d0 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0cc803e4d0 msgr2=0x7f0cc80409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0cc803e4d0 0x7f0cc80409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0006fd0 tx=0x7f0cd0006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 msgr2=0x7f0cdc19dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:50.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc19dd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8011980 tx=0x7f0cd8011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 shutdown_connections 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0cc803e4d0 0x7f0cc80409e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 --2- 172.21.15.19:0/993239286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cdc109e70 0x7f0cdc19dd10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 >> 172.21.15.19:0/993239286 conn(0x7f0cdc103800 msgr2=0x7f0cdc1942c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 shutdown_connections 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:49.384+0000 7f0ce3ceb700 1 -- 172.21.15.19:0/993239286 wait complete. 2024-05-17T09:07:50.020 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-05-17T09:07:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:50 smithi019 ceph-mon[31620]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:51.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9MgojP2QvLxzt/1FJJRUnuMSMIqpPeXQOWEps2N2W1zcW7ihnv3Uu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715936871, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-17T09:07:51.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdcab7d700 1 Processor -- start 2024-05-17T09:07:51.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdcab7d700 1 -- start start 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdcab7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc410a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdcab7d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc4108520 con 0x7fcdc4109f70 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdc8919700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc410a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdc8919700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc410a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54618/0 (socket says 172.21.15.19:54618) 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.507+0000 7fcdc8919700 1 -- 172.21.15.19:0/740961561 learned_addr learned my addr 172.21.15.19:0/740961561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.508+0000 7fcdc8919700 1 -- 172.21.15.19:0/740961561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc41086a0 con 0x7fcdc4109f70 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.508+0000 7fcdc8919700 1 --2- 172.21.15.19:0/740961561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc410a380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80098a0 tx=0x7fcdb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788b3feb99f7c077 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.508+0000 7fcdc37fe700 1 -- 172.21.15.19:0/740961561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdb8004160 con 0x7fcdc4109f70 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.508+0000 7fcdc37fe700 1 -- 172.21.15.19:0/740961561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb8004300 con 0x7fcdc4109f70 2024-05-17T09:07:51.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.508+0000 7fcdc37fe700 1 -- 172.21.15.19:0/740961561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdb8011960 con 0x7fcdc4109f70 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 -- 172.21.15.19:0/740961561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 msgr2=0x7fcdc410a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 --2- 172.21.15.19:0/740961561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc410a380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80098a0 tx=0x7fcdb8009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 -- 172.21.15.19:0/740961561 shutdown_connections 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 --2- 172.21.15.19:0/740961561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc410a380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 -- 172.21.15.19:0/740961561 >> 172.21.15.19:0/740961561 conn(0x7fcdc4103ff0 msgr2=0x7fcdc4106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 -- 172.21.15.19:0/740961561 shutdown_connections 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.509+0000 7fcdcab7d700 1 -- 172.21.15.19:0/740961561 wait complete. 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.510+0000 7fcdcab7d700 1 Processor -- start 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.510+0000 7fcdcab7d700 1 -- start start 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.510+0000 7fcdcab7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc407c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.510+0000 7fcdcab7d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410ce10 con 0x7fcdc4109f70 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.510+0000 7fcdc8919700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc407c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:51.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.510+0000 7fcdc8919700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc407c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54632/0 (socket says 172.21.15.19:54632) 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdc8919700 1 -- 172.21.15.19:0/977245201 learned_addr learned my addr 172.21.15.19:0/977245201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdc8919700 1 -- 172.21.15.19:0/977245201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc407ce70 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdc8919700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc407c930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8003f50 tx=0x7fcdb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdb801f070 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdb8003aa0 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc41a0bb0 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdb8019580 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.511+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc41a1030 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.513+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc4064e10 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.513+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fcdb8022070 con 0x7fcdc4109f70 2024-05-17T09:07:51.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.513+0000 7fcdc1ffb700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdb403e570 0x7fcdb4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.513+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcdb8053b30 con 0x7fcdc4109f70 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.513+0000 7fcdc3fff700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdb403e570 0x7fcdb4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.514+0000 7fcdc3fff700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdb403e570 0x7fcdb4040a80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0006fd0 tx=0x7fcdb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.523+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcdb802cd60 con 0x7fcdc4109f70 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:50.755+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fcdc4106000 con 0x7fcdb403e570 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.003+0000 7fcdc1ffb700 1 -- 172.21.15.19:0/977245201 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fcdc4106000 con 0x7fcdb403e570 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdb403e570 msgr2=0x7fcdb4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdb403e570 0x7fcdb4040a80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0006fd0 tx=0x7fcdb0006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 msgr2=0x7fcdc407c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc407c930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8003f50 tx=0x7fcdb8011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 shutdown_connections 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdb403e570 0x7fcdb4040a80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:51.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 --2- 172.21.15.19:0/977245201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc4109f70 0x7fcdc407c930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:51.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 >> 172.21.15.19:0/977245201 conn(0x7fcdc4103ff0 msgr2=0x7fcdc41058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:51.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 shutdown_connections 2024-05-17T09:07:51.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:51.006+0000 7fcdcab7d700 1 -- 172.21.15.19:0/977245201 wait complete. 2024-05-17T09:07:51.580 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-05-17T09:07:52.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:52 smithi019 ceph-mon[31620]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:07:52.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:53.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a25198700 1 Processor -- start 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a25198700 1 -- start start 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a25198700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2010c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a25198700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2010cb60 con 0x7f6a2007d500 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a1ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2010c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a1ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2010c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54644/0 (socket says 172.21.15.19:54644) 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.778+0000 7f6a1ed9d700 1 -- 172.21.15.19:0/1331785344 learned_addr learned my addr 172.21.15.19:0/1331785344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.779+0000 7f6a1ed9d700 1 -- 172.21.15.19:0/1331785344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2010cce0 con 0x7f6a2007d500 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.779+0000 7f6a1ed9d700 1 --2- 172.21.15.19:0/1331785344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2010c590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a100098a0 tx=0x7f6a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a941cecd99dbaab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.779+0000 7f6a1dd9b700 1 -- 172.21.15.19:0/1331785344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a10004160 con 0x7f6a2007d500 2024-05-17T09:07:53.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.779+0000 7f6a1dd9b700 1 -- 172.21.15.19:0/1331785344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a10004300 con 0x7f6a2007d500 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.779+0000 7f6a1dd9b700 1 -- 172.21.15.19:0/1331785344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a10011960 con 0x7f6a2007d500 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 -- 172.21.15.19:0/1331785344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 msgr2=0x7f6a2010c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 --2- 172.21.15.19:0/1331785344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2010c590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a100098a0 tx=0x7f6a10009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 -- 172.21.15.19:0/1331785344 shutdown_connections 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 --2- 172.21.15.19:0/1331785344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2010c590 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 -- 172.21.15.19:0/1331785344 >> 172.21.15.19:0/1331785344 conn(0x7f6a20103fe0 msgr2=0x7f6a20106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 -- 172.21.15.19:0/1331785344 shutdown_connections 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.780+0000 7f6a25198700 1 -- 172.21.15.19:0/1331785344 wait complete. 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a25198700 1 Processor -- start 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a25198700 1 -- start start 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a25198700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:53.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a25198700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2010dd40 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a1ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a1ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54654/0 (socket says 172.21.15.19:54654) 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.781+0000 7f6a1ed9d700 1 -- 172.21.15.19:0/1317831966 learned_addr learned my addr 172.21.15.19:0/1317831966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a1ed9d700 1 -- 172.21.15.19:0/1317831966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2019e240 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a1ed9d700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2019dd00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a10011670 tx=0x7f6a10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1001f070 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a10003b20 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2019e4d0 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a100195f0 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.782+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a10e0 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.783+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20064e10 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.783+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f6a10022070 con 0x7f6a2007d500 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.783+0000 7f6a07fff700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a0803e4c0 0x7f6a080409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:53.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.783+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a10053be0 con 0x7f6a2007d500 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.783+0000 7f6a1e59c700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a0803e4c0 0x7f6a080409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.784+0000 7f6a1e59c700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a0803e4c0 0x7f6a080409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a14006fd0 tx=0x7f6a14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:52.789+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a100172b0 con 0x7f6a2007d500 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.004+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6a1000bce0 con 0x7f6a2007d500 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.005+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6a201a12b0 con 0x7f6a2007d500 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.006+0000 7f6a07fff700 1 -- 172.21.15.19:0/1317831966 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6a1002acf0 con 0x7f6a2007d500 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a0803e4c0 msgr2=0x7f6a080409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a0803e4c0 0x7f6a080409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a14006fd0 tx=0x7f6a14006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 msgr2=0x7f6a2019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2019dd00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a10011670 tx=0x7f6a10011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 shutdown_connections 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a0803e4c0 0x7f6a080409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:53.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 --2- 172.21.15.19:0/1317831966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a2007d500 0x7f6a2019dd00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:53.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 >> 172.21.15.19:0/1317831966 conn(0x7f6a20103fe0 msgr2=0x7f6a20105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:53.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 shutdown_connections 2024-05-17T09:07:53.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:53.009+0000 7f6a25198700 1 -- 172.21.15.19:0/1317831966 wait complete. 2024-05-17T09:07:53.627 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T09:07:53.627 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-17T09:07:53.635 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T09:07:53.635 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-17T09:07:53.635 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-17T09:07:53.635 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-17T09:07:53.637 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout: Password: 5o9z61snn9 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config directory 2024-05-17T09:07:53.638 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-05-17T09:07:54.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:54 smithi019 ceph-mon[31620]: mgrmap e13: x(active, since 6s) 2024-05-17T09:07:54.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:54 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1317831966' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.086+0000 7f21569a3700 1 Processor -- start 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.086+0000 7f21569a3700 1 -- start start 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.086+0000 7f21569a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f2150109e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.086+0000 7f21569a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f215010a430 con 0x7f2150109a30 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f2150109e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f2150109e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56258/0 (socket says 172.21.15.19:56258) 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214ffff700 1 -- 172.21.15.19:0/3116210077 learned_addr learned my addr 172.21.15.19:0/3116210077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214ffff700 1 -- 172.21.15.19:0/3116210077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215010ab10 con 0x7f2150109a30 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214ffff700 1 --2- 172.21.15.19:0/3116210077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f2150109e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21400098a0 tx=0x7f2140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc6cfa917bf99f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214f7fe700 1 -- 172.21.15.19:0/3116210077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2140004160 con 0x7f2150109a30 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214f7fe700 1 -- 172.21.15.19:0/3116210077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2140004300 con 0x7f2150109a30 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.087+0000 7f214f7fe700 1 -- 172.21.15.19:0/3116210077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2140011960 con 0x7f2150109a30 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 -- 172.21.15.19:0/3116210077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 msgr2=0x7f2150109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 --2- 172.21.15.19:0/3116210077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f2150109e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21400098a0 tx=0x7f2140009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 -- 172.21.15.19:0/3116210077 shutdown_connections 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 --2- 172.21.15.19:0/3116210077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f2150109e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 -- 172.21.15.19:0/3116210077 >> 172.21.15.19:0/3116210077 conn(0x7f21501050f0 msgr2=0x7f2150107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 -- 172.21.15.19:0/3116210077 shutdown_connections 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.088+0000 7f21569a3700 1 -- 172.21.15.19:0/3116210077 wait complete. 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f21569a3700 1 Processor -- start 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f21569a3700 1 -- start start 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f21569a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f215019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f21569a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f215010dec0 con 0x7f2150109a30 2024-05-17T09:07:54.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f214ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f215019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f214ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f215019dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56260/0 (socket says 172.21.15.19:56260) 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.089+0000 7f214ffff700 1 -- 172.21.15.19:0/988941570 learned_addr learned my addr 172.21.15.19:0/988941570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f214ffff700 1 -- 172.21.15.19:0/988941570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215019e260 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f214ffff700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f215019dd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2140011980 tx=0x7f2140011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f214001f070 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2140003aa0 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2140019560 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215019e4f0 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.090+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21501a10f0 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.091+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2150064e10 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.091+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2140022070 con 0x7f2150109a30 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.091+0000 7f214dffb700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f213c03e5b0 0x7f213c040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:54.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.091+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2140053d20 con 0x7f2150109a30 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.092+0000 7f2147fff700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f213c03e5b0 0x7f213c040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.092+0000 7f2147fff700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f213c03e5b0 0x7f213c040ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2138006fd0 tx=0x7f2138006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.097+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f214001b810 con 0x7f2150109a30 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.307+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f21501a1400 con 0x7f2150109a30 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.308+0000 7f214dffb700 1 -- 172.21.15.19:0/988941570 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f2140019860 con 0x7f2150109a30 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f213c03e5b0 msgr2=0x7f213c040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f213c03e5b0 0x7f213c040ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2138006fd0 tx=0x7f2138006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 msgr2=0x7f215019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f215019dd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2140011980 tx=0x7f2140011a60 comp rx=0 tx=0).stop 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 shutdown_connections 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f213c03e5b0 0x7f213c040ac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 --2- 172.21.15.19:0/988941570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2150109a30 0x7f215019dd20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:54.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 >> 172.21.15.19:0/988941570 conn(0x7f21501050f0 msgr2=0x7f21501069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 shutdown_connections 2024-05-17T09:07:54.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:54.310+0000 7f21569a3700 1 -- 172.21.15.19:0/988941570 wait complete. 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f6745485700 1 Processor -- start 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f6745485700 1 -- start start 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f6745485700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674010c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f6745485700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f674007ce80 con 0x7f674010a550 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f673effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674010c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f673effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674010c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56274/0 (socket says 172.21.15.19:56274) 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.338+0000 7f673effd700 1 -- 172.21.15.19:0/2363887109 learned_addr learned my addr 172.21.15.19:0/2363887109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.339+0000 7f673effd700 1 -- 172.21.15.19:0/2363887109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674010ceb0 con 0x7f674010a550 2024-05-17T09:07:56.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.339+0000 7f673effd700 1 --2- 172.21.15.19:0/2363887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674010c970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67300098a0 tx=0x7f6730009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a259c12ae0367 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.339+0000 7f673dffb700 1 -- 172.21.15.19:0/2363887109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6730004160 con 0x7f674010a550 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.339+0000 7f673dffb700 1 -- 172.21.15.19:0/2363887109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6730004300 con 0x7f674010a550 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.339+0000 7f673dffb700 1 -- 172.21.15.19:0/2363887109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6730011960 con 0x7f674010a550 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 -- 172.21.15.19:0/2363887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 msgr2=0x7f674010c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 --2- 172.21.15.19:0/2363887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674010c970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67300098a0 tx=0x7f6730009bb0 comp rx=0 tx=0).stop 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 -- 172.21.15.19:0/2363887109 shutdown_connections 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 --2- 172.21.15.19:0/2363887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674010c970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 -- 172.21.15.19:0/2363887109 >> 172.21.15.19:0/2363887109 conn(0x7f6740103ff0 msgr2=0x7f6740106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 -- 172.21.15.19:0/2363887109 shutdown_connections 2024-05-17T09:07:56.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 -- 172.21.15.19:0/2363887109 wait complete. 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.340+0000 7f6745485700 1 Processor -- start 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f6745485700 1 -- start start 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f6745485700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f6745485700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f674010de50 con 0x7f674010a550 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f673effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f673effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56288/0 (socket says 172.21.15.19:56288) 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f673effd700 1 -- 172.21.15.19:0/3296405523 learned_addr learned my addr 172.21.15.19:0/3296405523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.341+0000 7f673effd700 1 -- 172.21.15.19:0/3296405523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674019e2e0 con 0x7f674010a550 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.342+0000 7f673effd700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674019dda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6730011670 tx=0x7f6730011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.342+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6730003aa0 con 0x7f674010a550 2024-05-17T09:07:56.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.342+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f673001b8d0 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.342+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674019e570 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.342+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67300197f0 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.342+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67401a1180 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.343+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6730019990 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.343+0000 7f6727fff700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f672803e510 0x7f6728040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.343+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6740064e10 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.343+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f673001bd80 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.343+0000 7f673e7fc700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f672803e510 0x7f6728040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.344+0000 7f673e7fc700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f672803e510 0x7f6728040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6734006fd0 tx=0x7f6734006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.349+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f673001f030 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.684+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f67401a1670 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.688+0000 7f6727fff700 1 -- 172.21.15.19:0/3296405523 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f673000b3e0 con 0x7f674010a550 2024-05-17T09:07:56.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f672803e510 msgr2=0x7f6728040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f672803e510 0x7f6728040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6734006fd0 tx=0x7f6734006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 msgr2=0x7f674019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674019dda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6730011670 tx=0x7f6730011a30 comp rx=0 tx=0).stop 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 shutdown_connections 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f672803e510 0x7f6728040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 --2- 172.21.15.19:0/3296405523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674010a550 0x7f674019dda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 >> 172.21.15.19:0/3296405523 conn(0x7f6740103ff0 msgr2=0x7f67401057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 shutdown_connections 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-17T09:07:55.690+0000 7f6745485700 1 -- 172.21.15.19:0/3296405523 wait complete. 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:07:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:07:56.262 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-05-17T09:07:56.296 INFO:tasks.cephadm:Fetching config... 2024-05-17T09:07:56.296 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:07:56.296 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-17T09:07:56.313 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-17T09:07:56.313 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:07:56.313 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-17T09:07:56.371 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-17T09:07:56.371 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:07:56.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-17T09:07:56.437 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-17T09:07:56.437 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:07:56.437 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-17T09:07:56.494 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-17T09:07:56.494 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T09:07:56.579 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:07:56.590 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T09:07:56.632 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:07:56.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T09:07:56.687 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdlJMsCObTMya9P1NJ+RmsSC9d4IDi6YYXrwYljXxo5ZfvWxxc8gi+qyL+sLO17buMAsFk4KqRA5YE/dvkMZo0VRhfGKpjJLpOhp0T5gtVuO0twK5tPfoNh0K/txKh7G/fAdxjbwhN5Uty3BuvPCaB06vqkh3FeEvEpema9v7Kqe3h1t+euU0YJI/db8gwX2L2T3z9Dj+xm3KGupWj8pec6wOwC6QylBZDYEqTky41Xhl6zMvzo3VIPoO84KmXsdx+9LkN1Oqtrdc39WD6VZEE+Mn9Fr5rlbxqGyiw8tenkdiTPtVzpQtO4BMw3hROb/F9hIWv+YlC7q858zELq0QyK11UzW6HzcCXamkekfitGdVI0O5n2CktU7X6lwhvgeSyCXt/jGqXy7UdWRMG45RLF0OZD46jpNSJqxOoX+j1V3+hpMJsMojpcbLNkO5l65KYxI6uaS1pwivjrDV8ELDkcgynv49dLQdhwD63YF9sA1FQ3W98DyNh+CjZH2t9jxM= ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:07:56.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-17T09:07:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:56 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3296405523' entity='client.admin' 2024-05-17T09:07:56.951 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 -- 172.21.15.19:0/568438101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 msgr2=0x7f4968106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 --2- 172.21.15.19:0/568438101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f4968106ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f495c009910 tx=0x7f495c009c20 comp rx=0 tx=0).stop 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 -- 172.21.15.19:0/568438101 shutdown_connections 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 --2- 172.21.15.19:0/568438101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f4968106ba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 -- 172.21.15.19:0/568438101 >> 172.21.15.19:0/568438101 conn(0x7f49680fe120 msgr2=0x7f49681005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 -- 172.21.15.19:0/568438101 shutdown_connections 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.626+0000 7f496f253700 1 -- 172.21.15.19:0/568438101 wait complete. 2024-05-17T09:07:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496f253700 1 Processor -- start 2024-05-17T09:07:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496f253700 1 -- start start 2024-05-17T09:07:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496f253700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f496819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496f253700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4968109a70 con 0x7f4968104780 2024-05-17T09:07:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496cfef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f496819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496cfef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f496819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56296/0 (socket says 172.21.15.19:56296) 2024-05-17T09:07:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.627+0000 7f496cfef700 1 -- 172.21.15.19:0/1964269989 learned_addr learned my addr 172.21.15.19:0/1964269989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:07:58.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f496cfef700 1 -- 172.21.15.19:0/1964269989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496819bd90 con 0x7f4968104780 2024-05-17T09:07:58.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f496cfef700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f496819b850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f495c005120 tx=0x7f495c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f495c027070 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f495c00bdf0 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496819c020 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f495c00fd70 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.628+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496819ec30 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.629+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496810a1f0 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.630+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f495c02a070 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.630+0000 7f4965ffb700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f495803e570 0x7f4958040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.630+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f495c0538a0 con 0x7f4968104780 2024-05-17T09:07:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.630+0000 7f4967fff700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f495803e570 0x7f4958040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:07:58.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.631+0000 7f4967fff700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f495803e570 0x7f4958040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:07:58.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.636+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f495c013070 con 0x7f4968104780 2024-05-17T09:07:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.852+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f496802e6f0 con 0x7f4968104780 2024-05-17T09:07:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.855+0000 7f4965ffb700 1 -- 172.21.15.19:0/1964269989 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f495c015d60 con 0x7f4968104780 2024-05-17T09:07:58.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f495803e570 msgr2=0x7f4958040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:58.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f495803e570 0x7f4958040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954006e40 comp rx=0 tx=0).stop 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 msgr2=0x7f496819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f496819b850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f495c005120 tx=0x7f495c005810 comp rx=0 tx=0).stop 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 shutdown_connections 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f495803e570 0x7f4958040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 --2- 172.21.15.19:0/1964269989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4968104780 0x7f496819b850 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 >> 172.21.15.19:0/1964269989 conn(0x7f49680fe120 msgr2=0x7f49681005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 shutdown_connections 2024-05-17T09:07:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:07:58.858+0000 7f496f253700 1 -- 172.21.15.19:0/1964269989 wait complete. 2024-05-17T09:07:59.718 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-17T09:07:59.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1964269989' entity='client.admin' 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:07:59.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:07:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:07:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.488+0000 7fbc6c341700 1 -- 172.21.15.19:0/1033295715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 msgr2=0x7fbc64106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.488+0000 7fbc6c341700 1 --2- 172.21.15.19:0/1033295715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64106b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00b0a0 tx=0x7fbc5c00b3b0 comp rx=0 tx=0).stop 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.488+0000 7fbc6c341700 1 -- 172.21.15.19:0/1033295715 shutdown_connections 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.488+0000 7fbc6c341700 1 --2- 172.21.15.19:0/1033295715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64106b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.488+0000 7fbc6c341700 1 -- 172.21.15.19:0/1033295715 >> 172.21.15.19:0/1033295715 conn(0x7fbc640fe120 msgr2=0x7fbc641005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.489+0000 7fbc6c341700 1 -- 172.21.15.19:0/1033295715 shutdown_connections 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.489+0000 7fbc6c341700 1 -- 172.21.15.19:0/1033295715 wait complete. 2024-05-17T09:08:01.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.489+0000 7fbc6c341700 1 Processor -- start 2024-05-17T09:08:01.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.489+0000 7fbc6c341700 1 -- start start 2024-05-17T09:08:01.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6c341700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:01.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6c341700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64109a70 con 0x7fbc64104760 2024-05-17T09:08:01.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6a0dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6a0dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56324/0 (socket says 172.21.15.19:56324) 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6a0dd700 1 -- 172.21.15.19:0/213178576 learned_addr learned my addr 172.21.15.19:0/213178576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6a0dd700 1 -- 172.21.15.19:0/213178576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc64199b60 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6a0dd700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64199620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00b830 tx=0x7fbc5c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc5c017040 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc5c00bb90 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc64199df0 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc5c012430 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.490+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6419ca00 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.491+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbc5c024020 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.492+0000 7fbc637fe700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc4c03e5b0 0x7fbc4c040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.492+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6410a1f0 con 0x7fbc64104760 2024-05-17T09:08:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.492+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc5c022020 con 0x7fbc64104760 2024-05-17T09:08:01.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.492+0000 7fbc698dc700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc4c03e5b0 0x7fbc4c040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:01.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.492+0000 7fbc698dc700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc4c03e5b0 0x7fbc4c040ac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc58006fd0 tx=0x7fbc58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:01.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.497+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc5c02b020 con 0x7fbc64104760 2024-05-17T09:08:01.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.719+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fbc6402e6f0 con 0x7fbc4c03e5b0 2024-05-17T09:08:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.723+0000 7fbc637fe700 1 -- 172.21.15.19:0/213178576 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc6402e6f0 con 0x7fbc4c03e5b0 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.725+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc4c03e5b0 msgr2=0x7fbc4c040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.725+0000 7fbc6c341700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc4c03e5b0 0x7fbc4c040ac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc58006fd0 tx=0x7fbc58006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.725+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 msgr2=0x7fbc64199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.725+0000 7fbc6c341700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64199620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00b830 tx=0x7fbc5c003ce0 comp rx=0 tx=0).stop 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.726+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 shutdown_connections 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.726+0000 7fbc6c341700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc4c03e5b0 0x7fbc4c040ac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.726+0000 7fbc6c341700 1 --2- 172.21.15.19:0/213178576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc64104760 0x7fbc64199620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.726+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 >> 172.21.15.19:0/213178576 conn(0x7fbc640fe120 msgr2=0x7fbc640feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.726+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 shutdown_connections 2024-05-17T09:08:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:01.726+0000 7fbc6c341700 1 -- 172.21.15.19:0/213178576 wait complete. 2024-05-17T09:08:02.465 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi069 2024-05-17T09:08:02.465 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:08:02.466 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.conf 2024-05-17T09:08:02.486 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:08:02.486 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:02.544 INFO:tasks.cephadm:Adding host smithi069 to orchestrator... 2024-05-17T09:08:02.544 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch host add smithi069 2024-05-17T09:08:02.795 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:03.031 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: Updating smithi019:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:03.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:02 smithi019 ceph-mon[31620]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:04.021 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:03 smithi019 ceph-mon[31620]: Updating smithi019:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T09:08:04.021 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:04.021 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:04.021 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 -- 172.21.15.19:0/4289265676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 msgr2=0x7f9a50102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 --2- 172.21.15.19:0/4289265676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a50102e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009910 tx=0x7f9a4c009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 -- 172.21.15.19:0/4289265676 shutdown_connections 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 --2- 172.21.15.19:0/4289265676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a50102e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 -- 172.21.15.19:0/4289265676 >> 172.21.15.19:0/4289265676 conn(0x7f9a500fe560 msgr2=0x7f9a501009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 -- 172.21.15.19:0/4289265676 shutdown_connections 2024-05-17T09:08:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.261+0000 7f9a578c9700 1 -- 172.21.15.19:0/4289265676 wait complete. 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a578c9700 1 Processor -- start 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a578c9700 1 -- start start 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a578c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a501a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a578c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501101b0 con 0x7f9a50102a40 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a55665700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a501a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a55665700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a501a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59100/0 (socket says 172.21.15.19:59100) 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.262+0000 7f9a55665700 1 -- 172.21.15.19:0/9431426 learned_addr learned my addr 172.21.15.19:0/9431426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a55665700 1 -- 172.21.15.19:0/9431426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501a1c10 con 0x7f9a50102a40 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a55665700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a501a16d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c005660 tx=0x7f9a4c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a4c026070 con 0x7f9a50102a40 2024-05-17T09:08:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a4c00b750 con 0x7f9a50102a40 2024-05-17T09:08:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a501a1ea0 con 0x7f9a50102a40 2024-05-17T09:08:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a4c0037b0 con 0x7f9a50102a40 2024-05-17T09:08:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.263+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5019b8f0 con 0x7f9a50102a40 2024-05-17T09:08:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.264+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5006c410 con 0x7f9a50102a40 2024-05-17T09:08:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.264+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9a4c029070 con 0x7f9a50102a40 2024-05-17T09:08:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.265+0000 7f9a427fc700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a3c03e510 0x7f9a3c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.265+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a4c01e070 con 0x7f9a50102a40 2024-05-17T09:08:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.265+0000 7f9a54e64700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a3c03e510 0x7f9a3c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.265+0000 7f9a54e64700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a3c03e510 0x7f9a3c040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a44006fd0 tx=0x7f9a44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:04.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.270+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a4c013070 con 0x7f9a50102a40 2024-05-17T09:08:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:04.490+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}) -- 0x7f9a5019ce20 con 0x7f9a3c03e510 2024-05-17T09:08:05.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:05 smithi019 ceph-mon[31620]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:05.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:05 smithi019 ceph-mon[31620]: Deploying cephadm binary to smithi069 2024-05-17T09:08:08.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:07 smithi019 ceph-mon[31620]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.759+0000 7f9a427fc700 1 -- 172.21.15.19:0/9431426 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9a5019ce20 con 0x7f9a3c03e510 2024-05-17T09:08:08.760 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi069' with addr '172.21.15.69' 2024-05-17T09:08:08.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a3c03e510 msgr2=0x7f9a3c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a3c03e510 0x7f9a3c040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a44006fd0 tx=0x7f9a44006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 msgr2=0x7f9a501a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a501a16d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c005660 tx=0x7f9a4c005740 comp rx=0 tx=0).stop 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 shutdown_connections 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a3c03e510 0x7f9a3c040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 --2- 172.21.15.19:0/9431426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a50102a40 0x7f9a501a16d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 >> 172.21.15.19:0/9431426 conn(0x7f9a500fe560 msgr2=0x7f9a5010c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 shutdown_connections 2024-05-17T09:08:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:08.763+0000 7f9a578c9700 1 -- 172.21.15.19:0/9431426 wait complete. 2024-05-17T09:08:09.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T09:08:09.611 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:10.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:10.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:09 smithi019 ceph-mon[31620]: Added host smithi069 2024-05-17T09:08:10.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:09 smithi019 ceph-mon[31620]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:10.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:10.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:11.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 -- 172.21.15.19:0/2527782466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 msgr2=0x7f476010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 --2- 172.21.15.19:0/2527782466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476010a040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f475c009910 tx=0x7f475c009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 -- 172.21.15.19:0/2527782466 shutdown_connections 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 --2- 172.21.15.19:0/2527782466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476010a040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 -- 172.21.15.19:0/2527782466 >> 172.21.15.19:0/2527782466 conn(0x7f47600ff650 msgr2=0x7f4760101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 -- 172.21.15.19:0/2527782466 shutdown_connections 2024-05-17T09:08:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.184+0000 7f47677e9700 1 -- 172.21.15.19:0/2527782466 wait complete. 2024-05-17T09:08:11.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f47677e9700 1 Processor -- start 2024-05-17T09:08:11.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f47677e9700 1 -- start start 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f47677e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f47677e9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f476010dfc0 con 0x7f4760109c30 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f4765585700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f4765585700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59120/0 (socket says 172.21.15.19:59120) 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f4765585700 1 -- 172.21.15.19:0/176976523 learned_addr learned my addr 172.21.15.19:0/176976523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.185+0000 7f4765585700 1 -- 172.21.15.19:0/176976523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476019fa20 con 0x7f4760109c30 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.186+0000 7f4765585700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476019f4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f475c0092e0 tx=0x7f475c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.186+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f475c026070 con 0x7f4760109c30 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.186+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f475c00b750 con 0x7f4760109c30 2024-05-17T09:08:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.186+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f475c003810 con 0x7f4760109c30 2024-05-17T09:08:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.186+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476019fcb0 con 0x7f4760109c30 2024-05-17T09:08:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.186+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4760199670 con 0x7f4760109c30 2024-05-17T09:08:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.187+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4760105200 con 0x7f4760109c30 2024-05-17T09:08:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.187+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f475c029070 con 0x7f4760109c30 2024-05-17T09:08:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.187+0000 7f47567fc700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f474c03e510 0x7f474c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.187+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f475c01e070 con 0x7f4760109c30 2024-05-17T09:08:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.187+0000 7f4764d84700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f474c03e510 0x7f474c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.188+0000 7f4764d84700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f474c03e510 0x7f474c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4750006fd0 tx=0x7f4750006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.192+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f475c013070 con 0x7f4760109c30 2024-05-17T09:08:11.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.414+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f476019ab80 con 0x7f474c03e510 2024-05-17T09:08:11.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.415+0000 7f47567fc700 1 -- 172.21.15.19:0/176976523 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f476019ab80 con 0x7f474c03e510 2024-05-17T09:08:11.415 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:08:11.415 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}] 2024-05-17T09:08:11.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f474c03e510 msgr2=0x7f474c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:11.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f474c03e510 0x7f474c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4750006fd0 tx=0x7f4750006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:11.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 msgr2=0x7f476019f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476019f4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f475c0092e0 tx=0x7f475c005710 comp rx=0 tx=0).stop 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 shutdown_connections 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f474c03e510 0x7f474c040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 --2- 172.21.15.19:0/176976523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4760109c30 0x7f476019f4e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 >> 172.21.15.19:0/176976523 conn(0x7f47600ff650 msgr2=0x7f47601005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 shutdown_connections 2024-05-17T09:08:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:11.418+0000 7f47677e9700 1 -- 172.21.15.19:0/176976523 wait complete. 2024-05-17T09:08:11.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:11 smithi019 ceph-mon[31620]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:11.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:12.007 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi079 2024-05-17T09:08:12.007 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:08:12.007 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.conf 2024-05-17T09:08:12.026 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:08:12.027 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:12.085 INFO:tasks.cephadm:Adding host smithi079 to orchestrator... 2024-05-17T09:08:12.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch host add smithi079 2024-05-17T09:08:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:13.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:12 smithi019 ceph-mon[31620]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:08:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3730666613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 msgr2=0x7f53a8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 --2- 172.21.15.19:0/3730666613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a8108e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5398009910 tx=0x7f5398009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3730666613 shutdown_connections 2024-05-17T09:08:13.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 --2- 172.21.15.19:0/3730666613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a8108e70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:13.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3730666613 >> 172.21.15.19:0/3730666613 conn(0x7f53a80fe5b0 msgr2=0x7f53a8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:13.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3730666613 shutdown_connections 2024-05-17T09:08:13.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.847+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3730666613 wait complete. 2024-05-17T09:08:13.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:13 smithi019 ceph-mon[31620]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.848+0000 7f53ae8a7700 1 Processor -- start 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.848+0000 7f53ae8a7700 1 -- start start 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.848+0000 7f53ae8a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.848+0000 7f53ae8a7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810ce60 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45368/0 (socket says 172.21.15.19:45368) 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a7fff700 1 -- 172.21.15.19:0/3153035231 learned_addr learned my addr 172.21.15.19:0/3153035231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a7fff700 1 -- 172.21.15.19:0/3153035231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a806ca60 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a7fff700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a819a670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53980092e0 tx=0x7f5398005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5398026070 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f539800b750 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a819ac70 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5398003810 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.849+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a819b130 con 0x7f53a8108a60 2024-05-17T09:08:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.850+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a8104170 con 0x7f53a8108a60 2024-05-17T09:08:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.850+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5398029070 con 0x7f53a8108a60 2024-05-17T09:08:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.851+0000 7f53a5ffb700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f539403e510 0x7f5394040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.851+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f539801e070 con 0x7f53a8108a60 2024-05-17T09:08:13.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.851+0000 7f539ffff700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f539403e510 0x7f5394040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:13.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.851+0000 7f539ffff700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f539403e510 0x7f5394040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5390006fd0 tx=0x7f5390006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:13.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:13.856+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5398013070 con 0x7f53a8108a60 2024-05-17T09:08:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:14.075+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}) -- 0x7f53a806c1b0 con 0x7f539403e510 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:15 smithi019 ceph-mon[31620]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-17T09:08:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:15 smithi019 ceph-mon[31620]: Updating smithi069:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:15 smithi019 ceph-mon[31620]: Deploying cephadm binary to smithi079 2024-05-17T09:08:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:15 smithi019 ceph-mon[31620]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:15 smithi019 ceph-mon[31620]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:15 smithi019 ceph-mon[31620]: Updating smithi069:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T09:08:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:17 smithi019 ceph-mon[31620]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.353+0000 7f53a5ffb700 1 -- 172.21.15.19:0/3153035231 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f53a806c1b0 con 0x7f539403e510 2024-05-17T09:08:18.354 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi079' with addr '172.21.15.79' 2024-05-17T09:08:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f539403e510 msgr2=0x7f5394040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f539403e510 0x7f5394040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5390006fd0 tx=0x7f5390006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 msgr2=0x7f53a819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a819a670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53980092e0 tx=0x7f5398005710 comp rx=0 tx=0).stop 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 shutdown_connections 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f539403e510 0x7f5394040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 --2- 172.21.15.19:0/3153035231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53a8108a60 0x7f53a819a670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 >> 172.21.15.19:0/3153035231 conn(0x7f53a80fe5b0 msgr2=0x7f53a80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 shutdown_connections 2024-05-17T09:08:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:18.356+0000 7f53ae8a7700 1 -- 172.21.15.19:0/3153035231 wait complete. 2024-05-17T09:08:18.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T09:08:19.198 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:19 smithi019 ceph-mon[31620]: Added host smithi079 2024-05-17T09:08:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:19 smithi019 ceph-mon[31620]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:19.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:20.496 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:20.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.761+0000 7faa88a6a700 1 -- 172.21.15.19:0/781539458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 msgr2=0x7faa84105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:20.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.761+0000 7faa88a6a700 1 --2- 172.21.15.19:0/781539458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84105120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b0a0 tx=0x7faa7400b3b0 comp rx=0 tx=0).stop 2024-05-17T09:08:20.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.761+0000 7faa88a6a700 1 -- 172.21.15.19:0/781539458 shutdown_connections 2024-05-17T09:08:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.761+0000 7faa88a6a700 1 --2- 172.21.15.19:0/781539458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84105120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.761+0000 7faa88a6a700 1 -- 172.21.15.19:0/781539458 >> 172.21.15.19:0/781539458 conn(0x7faa840fe5a0 msgr2=0x7faa841009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.761+0000 7faa88a6a700 1 -- 172.21.15.19:0/781539458 shutdown_connections 2024-05-17T09:08:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.762+0000 7faa88a6a700 1 -- 172.21.15.19:0/781539458 wait complete. 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.762+0000 7faa88a6a700 1 Processor -- start 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.762+0000 7faa88a6a700 1 -- start start 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa88a6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa88a6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa8410df60 con 0x7faa84104d10 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa8259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa8259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45376/0 (socket says 172.21.15.19:45376) 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa8259c700 1 -- 172.21.15.19:0/2939824247 learned_addr learned my addr 172.21.15.19:0/2939824247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa8259c700 1 -- 172.21.15.19:0/2939824247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa84104790 con 0x7faa84104d10 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.763+0000 7faa8259c700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84102940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b830 tx=0x7faa74007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.764+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa74017040 con 0x7faa84104d10 2024-05-17T09:08:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.764+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa84102e80 con 0x7faa84104d10 2024-05-17T09:08:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.764+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa74022070 con 0x7faa84104d10 2024-05-17T09:08:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.764+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa74004870 con 0x7faa84104d10 2024-05-17T09:08:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.764+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa841032e0 con 0x7faa84104d10 2024-05-17T09:08:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.765+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8406c0f0 con 0x7faa84104d10 2024-05-17T09:08:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.765+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7faa74027460 con 0x7faa84104d10 2024-05-17T09:08:20.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.765+0000 7faa737fe700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faa6c03e510 0x7faa6c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:20.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.765+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa7400f050 con 0x7faa84104d10 2024-05-17T09:08:20.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.765+0000 7faa81d9b700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faa6c03e510 0x7faa6c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:20.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.765+0000 7faa81d9b700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faa6c03e510 0x7faa6c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa78006fd0 tx=0x7faa78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.770+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa7401ce30 con 0x7faa84104d10 2024-05-17T09:08:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.991+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa84103f10 con 0x7faa6c03e510 2024-05-17T09:08:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.992+0000 7faa737fe700 1 -- 172.21.15.19:0/2939824247 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7faa84103f10 con 0x7faa6c03e510 2024-05-17T09:08:20.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:08:20.992 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}] 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faa6c03e510 msgr2=0x7faa6c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faa6c03e510 0x7faa6c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa78006fd0 tx=0x7faa78006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 msgr2=0x7faa84102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84102940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b830 tx=0x7faa74007940 comp rx=0 tx=0).stop 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 shutdown_connections 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faa6c03e510 0x7faa6c040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 --2- 172.21.15.19:0/2939824247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa84104d10 0x7faa84102940 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 >> 172.21.15.19:0/2939824247 conn(0x7faa840fe5a0 msgr2=0x7faa84105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 shutdown_connections 2024-05-17T09:08:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:20.994+0000 7faa88a6a700 1 -- 172.21.15.19:0/2939824247 wait complete. 2024-05-17T09:08:21.568 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-17T09:08:21.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-17T09:08:21.814 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:22 smithi019 ceph-mon[31620]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:22 smithi019 ceph-mon[31620]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:08:22.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.381+0000 7f712c957700 1 -- 172.21.15.19:0/2814280964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 msgr2=0x7f7128108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.381+0000 7f712c957700 1 --2- 172.21.15.19:0/2814280964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f7128108e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f711800b0a0 tx=0x7f711800b3b0 comp rx=0 tx=0).stop 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.381+0000 7f712c957700 1 -- 172.21.15.19:0/2814280964 shutdown_connections 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.381+0000 7f712c957700 1 --2- 172.21.15.19:0/2814280964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f7128108e90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.381+0000 7f712c957700 1 -- 172.21.15.19:0/2814280964 >> 172.21.15.19:0/2814280964 conn(0x7f71280785a0 msgr2=0x7f7128078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.381+0000 7f712c957700 1 -- 172.21.15.19:0/2814280964 shutdown_connections 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.382+0000 7f712c957700 1 -- 172.21.15.19:0/2814280964 wait complete. 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.382+0000 7f712c957700 1 Processor -- start 2024-05-17T09:08:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.382+0000 7f712c957700 1 -- start start 2024-05-17T09:08:23.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712c957700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f712819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:23.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712c957700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f712810ce80 con 0x7f7128108a80 2024-05-17T09:08:23.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f712819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f712819f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45388/0 (socket says 172.21.15.19:45388) 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712659c700 1 -- 172.21.15.19:0/3785081235 learned_addr learned my addr 172.21.15.19:0/3785081235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712659c700 1 -- 172.21.15.19:0/3785081235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712819fa30 con 0x7f7128108a80 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712659c700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f712819f4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7118015040 tx=0x7f711800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7118017040 con 0x7f7128108a80 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7118022070 con 0x7f7128108a80 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.383+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712819fcc0 con 0x7f7128108a80 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.384+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7118004830 con 0x7f7128108a80 2024-05-17T09:08:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.384+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7128199680 con 0x7f7128108a80 2024-05-17T09:08:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.385+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7128104190 con 0x7f7128108a80 2024-05-17T09:08:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.385+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7118027460 con 0x7f7128108a80 2024-05-17T09:08:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.385+0000 7f711f7fe700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f711403e5b0 0x7f7114040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.385+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f711800f050 con 0x7f7128108a80 2024-05-17T09:08:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.385+0000 7f7125d9b700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f711403e5b0 0x7f7114040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.386+0000 7f7125d9b700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f711403e5b0 0x7f7114040ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7110006fd0 tx=0x7f7110006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.390+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71180049d0 con 0x7f7128108a80 2024-05-17T09:08:23.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:23.610+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f712819ab90 con 0x7f7128108a80 2024-05-17T09:08:24.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.009+0000 7f711f7fe700 1 -- 172.21.15.19:0/3785081235 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7118025e40 con 0x7f7128108a80 2024-05-17T09:08:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f711403e5b0 msgr2=0x7f7114040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f711403e5b0 0x7f7114040ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7110006fd0 tx=0x7f7110006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 msgr2=0x7f712819f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f712819f4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7118015040 tx=0x7f711800bd40 comp rx=0 tx=0).stop 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 shutdown_connections 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f711403e5b0 0x7f7114040ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 --2- 172.21.15.19:0/3785081235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7128108a80 0x7f712819f4f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 >> 172.21.15.19:0/3785081235 conn(0x7f71280785a0 msgr2=0x7f7128106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 shutdown_connections 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:24.012+0000 7f712c957700 1 -- 172.21.15.19:0/3785081235 wait complete. 2024-05-17T09:08:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3785081235' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:24.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:24.661 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-05-17T09:08:24.661 INFO:tasks.cephadm:Adding mon.b on smithi069 2024-05-17T09:08:24.662 INFO:tasks.cephadm:Adding mon.c on smithi079 2024-05-17T09:08:24.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi069:172.21.15.69=b;smithi079:172.21.15.79=c' 2024-05-17T09:08:24.895 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3785081235' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: Updating smithi079:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 -- 172.21.15.79:0/1233790623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 msgr2=0x7f2a5c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 --2- 172.21.15.79:0/1233790623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c108e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009910 tx=0x7f2a4c009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 -- 172.21.15.79:0/1233790623 shutdown_connections 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 --2- 172.21.15.79:0/1233790623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c108e90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 -- 172.21.15.79:0/1233790623 >> 172.21.15.79:0/1233790623 conn(0x7f2a5c0785a0 msgr2=0x7f2a5c078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 -- 172.21.15.79:0/1233790623 shutdown_connections 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.186+0000 7f2a61a24700 1 -- 172.21.15.79:0/1233790623 wait complete. 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.187+0000 7f2a61a24700 1 Processor -- start 2024-05-17T09:08:26.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.187+0000 7f2a61a24700 1 -- start start 2024-05-17T09:08:26.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.187+0000 7f2a61a24700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:26.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.187+0000 7f2a61a24700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10ce80 con 0x7f2a5c108a80 2024-05-17T09:08:26.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a5affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:26.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a5affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.79:42298/0 (socket says 172.21.15.79:42298) 2024-05-17T09:08:26.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a5affd700 1 -- 172.21.15.79:0/2496336918 learned_addr learned my addr 172.21.15.79:0/2496336918 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a5affd700 1 -- 172.21.15.79:0/2496336918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c06bcf0 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a5affd700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c19a6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0092e0 tx=0x7f2a4c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a4c026070 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a4c00b750 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c19acb0 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a4c003810 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.188+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c19b170 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.189+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c104190 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.190+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2a4c029070 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.190+0000 7f2a60a22700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a4803e570 0x7f2a48040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.190+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2a4c01e070 con 0x7f2a5c108a80 2024-05-17T09:08:26.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.190+0000 7f2a527fc700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a4803e570 0x7f2a48040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:26.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.191+0000 7f2a527fc700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a4803e570 0x7f2a48040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2a44006fd0 tx=0x7f2a44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:26.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.196+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a4c013070 con 0x7f2a5c108a80 2024-05-17T09:08:26.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.409+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi069:172.21.15.69=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}) -- 0x7f2a5c19b450 con 0x7f2a4803e570 2024-05-17T09:08:26.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.416+0000 7f2a60a22700 1 -- 172.21.15.79:0/2496336918 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2a5c19b450 con 0x7f2a4803e570 2024-05-17T09:08:26.415 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mon update... 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a4803e570 msgr2=0x7f2a48040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a4803e570 0x7f2a48040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2a44006fd0 tx=0x7f2a44006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 msgr2=0x7f2a5c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c19a6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0092e0 tx=0x7f2a4c005710 comp rx=0 tx=0).stop 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 shutdown_connections 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a4803e570 0x7f2a48040a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 --2- 172.21.15.79:0/2496336918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a5c108a80 0x7f2a5c19a6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 >> 172.21.15.79:0/2496336918 conn(0x7f2a5c0785a0 msgr2=0x7f2a5c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 shutdown_connections 2024-05-17T09:08:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:26.418+0000 7f2a61a24700 1 -- 172.21.15.79:0/2496336918 wait complete. 2024-05-17T09:08:26.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:26 smithi019 ceph-mon[31620]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:26.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:26 smithi019 ceph-mon[31620]: Updating smithi079:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T09:08:26.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:26 smithi019 ceph-mon[31620]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:26.969 DEBUG:teuthology.orchestra.run.smithi069:mon.b> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.b.service 2024-05-17T09:08:26.972 DEBUG:teuthology.orchestra.run.smithi079:mon.c> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.c.service 2024-05-17T09:08:26.974 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T09:08:26.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T09:08:27.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi069:172.21.15.69=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:27.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi069:172.21.15.69=b;smithi079:172.21.15.79=c;count:3 2024-05-17T09:08:27.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:27.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:27.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:27.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:27.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:27.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:27.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:27.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: Deploying daemon mon.c on smithi079 2024-05-17T09:08:27.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:27 smithi019 ceph-mon[31620]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:28.535 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:30.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:29 smithi019 ceph-mon[31620]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 -- 172.21.15.79:0/1227897269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 msgr2=0x7fb8f0108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 --2- 172.21.15.79:0/1227897269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f0108e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009910 tx=0x7fb8e0009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 -- 172.21.15.79:0/1227897269 shutdown_connections 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 --2- 172.21.15.79:0/1227897269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f0108e70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 -- 172.21.15.79:0/1227897269 >> 172.21.15.79:0/1227897269 conn(0x7fb8f00fe5b0 msgr2=0x7fb8f0100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 -- 172.21.15.79:0/1227897269 shutdown_connections 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.615+0000 7fb8f8721700 1 -- 172.21.15.79:0/1227897269 wait complete. 2024-05-17T09:08:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f8721700 1 Processor -- start 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f8721700 1 -- start start 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f8721700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f8721700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010ce60 con 0x7fb8f0108a60 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f64bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f64bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.79:47644/0 (socket says 172.21.15.79:47644) 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.616+0000 7fb8f64bd700 1 -- 172.21.15.79:0/19527424 learned_addr learned my addr 172.21.15.79:0/19527424 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:08:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8f64bd700 1 -- 172.21.15.79:0/19527424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f006ca80 con 0x7fb8f0108a60 2024-05-17T09:08:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8f64bd700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f019a650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005660 tx=0x7fb8e0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0026070 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8e000b750 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f019ac50 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0003810 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.617+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f019b110 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.618+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f0104170 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.618+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb8e0029070 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.618+0000 7fb8e77fe700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8dc03e510 0x7fb8dc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.618+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8e001e070 con 0x7fb8f0108a60 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.619+0000 7fb8f5cbc700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8dc03e510 0x7fb8dc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.619+0000 7fb8f5cbc700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8dc03e510 0x7fb8dc040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.624+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb8f006ca80 con 0x7fb8f0108a60 2024-05-17T09:08:30.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.925+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb8f019b3f0 con 0x7fb8f0108a60 2024-05-17T09:08:30.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.925+0000 7fb8e77fe700 1 -- 172.21.15.79:0/19527424 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb8e0030300 con 0x7fb8f0108a60 2024-05-17T09:08:30.925 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:08:30.926 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":1,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","modified":"2024-05-17T09:06:39.857078Z","created":"2024-05-17T09:06:39.857078Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T09:08:30.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.927+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8dc03e510 msgr2=0x7fb8dc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:30.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.927+0000 7fb8f8721700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8dc03e510 0x7fb8dc040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec006e40 comp rx=0 tx=0).stop 2024-05-17T09:08:30.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.927+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 msgr2=0x7fb8f019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:30.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.927+0000 7fb8f8721700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f019a650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005660 tx=0x7fb8e0005740 comp rx=0 tx=0).stop 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.928+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 shutdown_connections 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.928+0000 7fb8f8721700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8dc03e510 0x7fb8dc040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.928+0000 7fb8f8721700 1 --2- 172.21.15.79:0/19527424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8f0108a60 0x7fb8f019a650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.928+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 >> 172.21.15.79:0/19527424 conn(0x7fb8f00fe5b0 msgr2=0x7fb8f00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.928+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 shutdown_connections 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:30.928+0000 7fb8f8721700 1 -- 172.21.15.79:0/19527424 wait complete. 2024-05-17T09:08:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 1 2024-05-17T09:08:31.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:31 smithi079 systemd[1]: Started Ceph mon.c for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:08:32.680 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T09:08:32.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T09:08:32.925 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:08:36.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: mon.a calling monitor election 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: mon.c calling monitor election 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: monmap epoch 2 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: last_changed 2024-05-17T09:08:31.044759+0000 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: min_mon_release 19 (squid) 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: election_strategy: 1 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: fsmap 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: mgrmap e13: x(active, since 49s) 2024-05-17T09:08:36.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: overall HEALTH_OK 2024-05-17T09:08:36.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:36.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:36.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:36.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:36.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: mon.a calling monitor election 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: mon.c calling monitor election 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:36.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: monmap epoch 2 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: last_changed 2024-05-17T09:08:31.044759+0000 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: min_mon_release 19 (squid) 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: election_strategy: 1 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: fsmap 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T09:08:36.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: mgrmap e13: x(active, since 49s) 2024-05-17T09:08:36.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: overall HEALTH_OK 2024-05-17T09:08:36.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:36.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:36.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:36.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:36.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:37.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 -- 172.21.15.79:0/1498950078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c0035c0 msgr2=0x7f8c0c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:37.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/1498950078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c0035c0 0x7f8c0c005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c24103b60 tx=0x7f8c14009e80 comp rx=0 tx=0).stop 2024-05-17T09:08:37.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 -- 172.21.15.79:0/1498950078 shutdown_connections 2024-05-17T09:08:37.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/1498950078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c0035c0 0x7f8c0c005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:37.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 -- 172.21.15.79:0/1498950078 >> 172.21.15.79:0/1498950078 conn(0x7f8c240ff790 msgr2=0x7f8c24101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:37.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 -- 172.21.15.79:0/1498950078 shutdown_connections 2024-05-17T09:08:37.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.382+0000 7f8c2a22d700 1 -- 172.21.15.79:0/1498950078 wait complete. 2024-05-17T09:08:37.384 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:37 smithi079 ceph-mon[35502]: Deploying daemon mon.b on smithi069 2024-05-17T09:08:37.384 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.383+0000 7f8c2a22d700 1 Processor -- start 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c2a22d700 1 -- start start 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c2a22d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c2a22d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c24075ff0 0x7f8c24076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c2a22d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410ba90 con 0x7f8c24075ff0 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c2a22d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410b910 con 0x7f8c0c0035c0 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:37.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:55904/0 (socket says 172.21.15.79:55904) 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 -- 172.21.15.79:0/2903876635 learned_addr learned my addr 172.21.15.79:0/2903876635 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 -- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 msgr2=0x7f8c24075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 -- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 msgr2=0x7f8c24075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c22ffd700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c24075ff0 0x7f8c24076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.384+0000 7f8c237fe700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c22ffd700 1 -- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 msgr2=0x7f8c24075ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c22ffd700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c22ffd700 1 -- 172.21.15.79:0/2903876635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240779e0 con 0x7f8c24075ff0 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c22ffd700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c24075ff0 0x7f8c24076400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800b4d0 tx=0x7f8c1800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8c18011630 con 0x7f8c24075ff0 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c18011c60 con 0x7f8c24075ff0 2024-05-17T09:08:37.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8c18010c90 con 0x7f8c24075ff0 2024-05-17T09:08:37.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c241a5ce0 con 0x7f8c24075ff0 2024-05-17T09:08:37.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.385+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c241a6240 con 0x7f8c24075ff0 2024-05-17T09:08:37.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.387+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24105370 con 0x7f8c24075ff0 2024-05-17T09:08:37.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.387+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8c18018020 con 0x7f8c24075ff0 2024-05-17T09:08:37.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.387+0000 7f8c20ff9700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c1003e320 0x7f8c10040830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:37.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.387+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8c18052020 con 0x7f8c24075ff0 2024-05-17T09:08:37.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.387+0000 7f8c237fe700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c1003e320 0x7f8c10040830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:37.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.388+0000 7f8c237fe700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c1003e320 0x7f8c10040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c1401c040 tx=0x7f8c140091d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:37.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.393+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c1801f020 con 0x7f8c24075ff0 2024-05-17T09:08:37.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:37 smithi019 ceph-mon[31620]: Deploying daemon mon.b on smithi069 2024-05-17T09:08:37.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:37.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.687+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8c2402e5e0 con 0x7f8c24075ff0 2024-05-17T09:08:37.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.688+0000 7f8c20ff9700 1 -- 172.21.15.79:0/2903876635 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f8c180187b0 con 0x7f8c24075ff0 2024-05-17T09:08:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:08:37.689 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":2,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","modified":"2024-05-17T09:08:31.044759Z","created":"2024-05-17T09:06:39.857078Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-17T09:08:37.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c1003e320 msgr2=0x7f8c10040830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:37.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c1003e320 0x7f8c10040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c1401c040 tx=0x7f8c140091d0 comp rx=0 tx=0).stop 2024-05-17T09:08:37.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c24075ff0 msgr2=0x7f8c24076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:37.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c24075ff0 0x7f8c24076400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800b4d0 tx=0x7f8c1800b890 comp rx=0 tx=0).stop 2024-05-17T09:08:37.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 shutdown_connections 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c1003e320 0x7f8c10040830 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c0c0035c0 0x7f8c24075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 --2- 172.21.15.79:0/2903876635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c24075ff0 0x7f8c24076400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 >> 172.21.15.79:0/2903876635 conn(0x7f8c240ff790 msgr2=0x7f8c24107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.691+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 shutdown_connections 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:37.692+0000 7f8c2a22d700 1 -- 172.21.15.79:0/2903876635 wait complete. 2024-05-17T09:08:37.693 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 2 2024-05-17T09:08:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:38 smithi019 ceph-mon[31620]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:38 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/2903876635' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T09:08:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:38 smithi079 ceph-mon[35502]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:38 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/2903876635' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T09:08:39.150 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T09:08:39.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T09:08:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:08:39.798 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:39 smithi069 ceph-mon[35081]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-17T09:08:39.799 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:39 smithi069 bash[35063]: c59ff213553a90f2b8a83b03979a5cf493021b0432333eba4d416931d608512a 2024-05-17T09:08:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:39 smithi069 systemd[1]: Started Ceph mon.b for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:08:45.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 -- 172.21.15.79:0/2208837670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc2c0035c0 msgr2=0x7efc2c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:45.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 --2- 172.21.15.79:0/2208837670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc2c0035c0 0x7efc2c005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc40102a60 tx=0x7efc3000fe60 comp rx=0 tx=0).stop 2024-05-17T09:08:45.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 -- 172.21.15.79:0/2208837670 shutdown_connections 2024-05-17T09:08:45.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 --2- 172.21.15.79:0/2208837670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc2c0035c0 0x7efc2c005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 -- 172.21.15.79:0/2208837670 >> 172.21.15.79:0/2208837670 conn(0x7efc400797d0 msgr2=0x7efc40079c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:45.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 -- 172.21.15.79:0/2208837670 shutdown_connections 2024-05-17T09:08:45.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.072+0000 7efc47269700 1 -- 172.21.15.79:0/2208837670 wait complete. 2024-05-17T09:08:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.073+0000 7efc47269700 1 Processor -- start 2024-05-17T09:08:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.073+0000 7efc47269700 1 -- start start 2024-05-17T09:08:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc47269700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc47269700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc4019ebc0 0x7efc401988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc47269700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 0x7efc401991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc47269700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc4010a990 con 0x7efc4019ebc0 2024-05-17T09:08:45.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc47269700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc4010a810 con 0x7efc2c0035c0 2024-05-17T09:08:45.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc47269700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efc4010ab10 con 0x7efc40198de0 2024-05-17T09:08:45.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:45.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.79:34628/0 (socket says 172.21.15.79:34628) 2024-05-17T09:08:45.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45806700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 0x7efc401991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:45.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 -- 172.21.15.79:0/1585706155 learned_addr learned my addr 172.21.15.79:0/1585706155 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:08:45.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 msgr2=0x7efc4019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 msgr2=0x7efc4019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc44804700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc4019ebc0 0x7efc401988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45005700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45806700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 msgr2=0x7efc401991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45806700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 msgr2=0x7efc401991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45806700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 0x7efc401991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.074+0000 7efc45806700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 0x7efc401991f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc44804700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 msgr2=0x7efc401991f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc44804700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 0x7efc401991f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc44804700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 msgr2=0x7efc4019e680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc44804700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc44804700 1 -- 172.21.15.79:0/1585706155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc401a5510 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc44804700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc4019ebc0 0x7efc401988a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b550 tx=0x7efc3c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c004230 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc3c0043d0 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc401a57a0 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c011a20 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.075+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401a5d30 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.076+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7efc3c011c00 con 0x7efc4019ebc0 2024-05-17T09:08:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.076+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc40064e10 con 0x7efc4019ebc0 2024-05-17T09:08:45.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.077+0000 7efc367fc700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc2403e9a0 0x7efc24040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:45.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.077+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efc3c052320 con 0x7efc4019ebc0 2024-05-17T09:08:45.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.077+0000 7efc45005700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc2403e9a0 0x7efc24040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:45.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.077+0000 7efc45005700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc2403e9a0 0x7efc24040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc3000f540 tx=0x7efc3000f9b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:45.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.082+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc3c003bc0 con 0x7efc4019ebc0 2024-05-17T09:08:45.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: mon.a calling monitor election 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: mon.c calling monitor election 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: mon.b calling monitor election 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: monmap epoch 3 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: last_changed 2024-05-17T09:08:39.777982+0000 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: min_mon_release 19 (squid) 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: election_strategy: 1 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: 2: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2024-05-17T09:08:45.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: fsmap 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: mgrmap e13: x(active, since 58s) 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: overall HEALTH_OK 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.183 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:08:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: mon.a calling monitor election 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: mon.c calling monitor election 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: mon.b calling monitor election 2024-05-17T09:08:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: monmap epoch 3 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: last_changed 2024-05-17T09:08:39.777982+0000 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: min_mon_release 19 (squid) 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: election_strategy: 1 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:08:45.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: 2: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: fsmap 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: mgrmap e13: x(active, since 58s) 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: overall HEALTH_OK 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: mon.a calling monitor election 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: mon.c calling monitor election 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: mon.b calling monitor election 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: monmap epoch 3 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: last_changed 2024-05-17T09:08:39.777982+0000 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: created 2024-05-17T09:06:39.857078+0000 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: min_mon_release 19 (squid) 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: election_strategy: 1 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: 2: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: fsmap 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T09:08:45.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: mgrmap e13: x(active, since 58s) 2024-05-17T09:08:45.275 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: overall HEALTH_OK 2024-05-17T09:08:45.275 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.275 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.275 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.275 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:45.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.374+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efc4002e5e0 con 0x7efc4019ebc0 2024-05-17T09:08:45.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.374+0000 7efc367fc700 1 -- 172.21.15.79:0/1585706155 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1277 (secure 0 0 0) 0x7efc3c0193a0 con 0x7efc4019ebc0 2024-05-17T09:08:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-17T09:08:45.375 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","modified":"2024-05-17T09:08:39.777982Z","created":"2024-05-17T09:06:39.857078Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-17T09:08:45.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc2403e9a0 msgr2=0x7efc24040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:45.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc2403e9a0 0x7efc24040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc3000f540 tx=0x7efc3000f9b0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc4019ebc0 msgr2=0x7efc401988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc4019ebc0 0x7efc401988a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b550 tx=0x7efc3c00b910 comp rx=0 tx=0).stop 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 shutdown_connections 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc2403e9a0 0x7efc24040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc40198de0 0x7efc401991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c0035c0 0x7efc4019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 --2- 172.21.15.79:0/1585706155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc4019ebc0 0x7efc401988a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 >> 172.21.15.79:0/1585706155 conn(0x7efc400797d0 msgr2=0x7efc40107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 shutdown_connections 2024-05-17T09:08:45.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:45.377+0000 7efc47269700 1 -- 172.21.15.79:0/1585706155 wait complete. 2024-05-17T09:08:45.379 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 3 2024-05-17T09:08:45.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:45 smithi079 ceph-mon[35502]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:45.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:45.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:45 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/1585706155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T09:08:45.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:45.974 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-17T09:08:45.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-17T09:08:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:45 smithi069 ceph-mon[35081]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:45 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/1585706155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T09:08:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:45 smithi019 ceph-mon[31620]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:45 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/1585706155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T09:08:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:08:46.235 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:47.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.858+0000 7f0b0a16d700 1 -- 172.21.15.19:0/899916214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 msgr2=0x7f0b041041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:47.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.858+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/899916214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b041041f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009910 tx=0x7f0af4009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:47.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.858+0000 7f0b0a16d700 1 -- 172.21.15.19:0/899916214 shutdown_connections 2024-05-17T09:08:47.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.858+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/899916214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b041041f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:47.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.858+0000 7f0b0a16d700 1 -- 172.21.15.19:0/899916214 >> 172.21.15.19:0/899916214 conn(0x7f0b040ff690 msgr2=0x7f0b04101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:47.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.859+0000 7f0b0a16d700 1 -- 172.21.15.19:0/899916214 shutdown_connections 2024-05-17T09:08:47.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.859+0000 7f0b0a16d700 1 -- 172.21.15.19:0/899916214 wait complete. 2024-05-17T09:08:47.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.859+0000 7f0b0a16d700 1 Processor -- start 2024-05-17T09:08:47.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 -- start start 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b0419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b0419fc70 0x7f0b04199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b04199e90 0x7f0b0419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410df20 con 0x7f0b04103de0 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410dda0 con 0x7f0b04199e90 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b0a16d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410e0a0 con 0x7f0b0419fc70 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b0419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b0419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37994/0 (socket says 172.21.15.19:37994) 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 -- 172.21.15.19:0/584010872 learned_addr learned my addr 172.21.15.19:0/584010872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b03fff700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b04199e90 0x7f0b0419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:47.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b02ffd700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b0419fc70 0x7f0b04199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 -- 172.21.15.19:0/584010872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b0419fc70 msgr2=0x7f0b04199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b0419fc70 0x7f0b04199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 -- 172.21.15.19:0/584010872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b04199e90 msgr2=0x7f0b0419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b04199e90 0x7f0b0419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.860+0000 7f0b037fe700 1 -- 172.21.15.19:0/584010872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b041a65c0 con 0x7f0b04103de0 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.861+0000 7f0b037fe700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b0419f730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0af400f690 tx=0x7f0af400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.861+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4027070 con 0x7f0b04103de0 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.861+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b041a6850 con 0x7f0b04103de0 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.861+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0af4005da0 con 0x7f0b04103de0 2024-05-17T09:08:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.861+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4017440 con 0x7f0b04103de0 2024-05-17T09:08:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.861+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b041a6db0 con 0x7f0b04103de0 2024-05-17T09:08:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.863+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b04064e10 con 0x7f0b04103de0 2024-05-17T09:08:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.863+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0af402a070 con 0x7f0b04103de0 2024-05-17T09:08:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.863+0000 7f0b00ff9700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0af003e9a0 0x7f0af0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.863+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0af40538e0 con 0x7f0b04103de0 2024-05-17T09:08:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.863+0000 7f0b02ffd700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0af003e9a0 0x7f0af0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.864+0000 7f0b02ffd700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0af003e9a0 0x7f0af0040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b0410b090 tx=0x7f0aec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:47.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:47.869+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0af4013070 con 0x7f0b04103de0 2024-05-17T09:08:48.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.090+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0b041a7040 con 0x7f0b04103de0 2024-05-17T09:08:48.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.091+0000 7f0b00ff9700 1 -- 172.21.15.19:0/584010872 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f0af400e3e0 con 0x7f0b04103de0 2024-05-17T09:08:48.091 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:48.092 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-05-17T09:08:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:08:48.092 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] 2024-05-17T09:08:48.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0af003e9a0 msgr2=0x7f0af0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0af003e9a0 0x7f0af0040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b0410b090 tx=0x7f0aec008040 comp rx=0 tx=0).stop 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 msgr2=0x7f0b0419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b0419f730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0af400f690 tx=0x7f0af400f770 comp rx=0 tx=0).stop 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 shutdown_connections 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b04199e90 0x7f0b0419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b04103de0 0x7f0b0419f730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0af003e9a0 0x7f0af0040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 --2- 172.21.15.19:0/584010872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b0419fc70 0x7f0b04199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 >> 172.21.15.19:0/584010872 conn(0x7f0b040ff690 msgr2=0x7f0b04109910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:48.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 shutdown_connections 2024-05-17T09:08:48.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:48.093+0000 7f0b0a16d700 1 -- 172.21.15.19:0/584010872 wait complete. 2024-05-17T09:08:48.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: Updating smithi019:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: Updating smithi069:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: Updating smithi079:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.400 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:48.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:48 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/584010872' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-17T09:08:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-17T09:08:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-17T09:08:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: Updating smithi019:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: Updating smithi069:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: Updating smithi079:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:48.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:48 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/584010872' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-17T09:08:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: Updating smithi069:/etc/ceph/ceph.conf 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: Updating smithi019:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: Updating smithi069:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: Updating smithi079:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:48.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:48.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:48.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:48.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:48 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/584010872' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:48.776 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-17T09:08:48.776 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:08:48.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T09:08:48.805 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:08:48.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:48.872 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:08:48.872 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T09:08:48.899 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:08:48.899 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:48.964 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:08:48.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T09:08:48.997 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:08:48.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T09:08:49.068 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-05-17T09:08:49.068 INFO:tasks.cephadm:Adding mgr.y on smithi069 2024-05-17T09:08:49.068 INFO:tasks.cephadm:Adding mgr.z on smithi079 2024-05-17T09:08:49.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi069=y;smithi079=z' 2024-05-17T09:08:49.347 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:08:49.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:49 smithi019 ceph-mon[31620]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T09:08:49.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:49 smithi019 ceph-mon[31620]: Reconfiguring daemon mon.a on smithi019 2024-05-17T09:08:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:49 smithi079 ceph-mon[35502]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T09:08:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:49 smithi079 ceph-mon[35502]: Reconfiguring daemon mon.a on smithi019 2024-05-17T09:08:49.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:49 smithi069 ceph-mon[35081]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T09:08:49.683 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:49 smithi069 ceph-mon[35081]: Reconfiguring daemon mon.a on smithi019 2024-05-17T09:08:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:50 smithi079 ceph-mon[35502]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:50.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:50.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:50.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:50.546 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:50 smithi069 ceph-mon[35081]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:50.546 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:50.546 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:50.546 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:50.546 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:50.546 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:50.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:50 smithi019 ceph-mon[31620]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:50.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:50.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:50.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:50.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:50.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:50.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.736+0000 7f7347509700 1 -- 172.21.15.79:0/3376763745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7340102be0 msgr2=0x7f7340102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:50.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.736+0000 7f7347509700 1 --2- 172.21.15.79:0/3376763745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7340102be0 0x7f7340102ff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f733c009910 tx=0x7f733c009c20 comp rx=0 tx=0).stop 2024-05-17T09:08:50.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 -- 172.21.15.79:0/3376763745 shutdown_connections 2024-05-17T09:08:50.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 --2- 172.21.15.79:0/3376763745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7340102be0 0x7f7340102ff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 -- 172.21.15.79:0/3376763745 >> 172.21.15.79:0/3376763745 conn(0x7f73400fe240 msgr2=0x7f73401006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:50.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 -- 172.21.15.79:0/3376763745 shutdown_connections 2024-05-17T09:08:50.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 -- 172.21.15.79:0/3376763745 wait complete. 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 Processor -- start 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.737+0000 7f7347509700 1 -- start start 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 0x7f73401987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340198d30 0x7f734019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f734019e200 0x7f73401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7340108a10 con 0x7f734019e200 2024-05-17T09:08:50.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7340108890 con 0x7f7340198d30 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7340108b90 con 0x7f7340102be0 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 0x7f73401987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 0x7f73401987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49494/0 (socket says 172.21.15.79:49494) 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 -- 172.21.15.79:0/188656414 learned_addr learned my addr 172.21.15.79:0/188656414 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7345aa6700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f734019e200 0x7f73401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7344aa4700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340198d30 0x7f734019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 -- 172.21.15.79:0/188656414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340198d30 msgr2=0x7f734019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340198d30 0x7f734019dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 -- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f734019e200 msgr2=0x7f73401a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f734019e200 0x7f73401a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 -- 172.21.15.79:0/188656414 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73401a0b90 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7344aa4700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340198d30 0x7f734019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73452a5700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 0x7f73401987f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f733c005920 tx=0x7f733c004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c027070 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f733c005da0 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c022cb0 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.738+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73401a0e20 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.739+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73401a1210 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.739+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73401a1470 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.739+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f733c02a070 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.740+0000 7f73367fc700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f732c03e9a0 0x7f732c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.740+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f733c053ba0 con 0x7f7340102be0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.740+0000 7f7344aa4700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f732c03e9a0 0x7f732c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.740+0000 7f7344aa4700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f732c03e9a0 0x7f732c040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73401998a0 tx=0x7f7330008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.745+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f733c013070 con 0x7f7340102be0 2024-05-17T09:08:50.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.959+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi069=y;smithi079=z", "target": ["mon-mgr", ""]}) -- 0x7f7340100130 con 0x7f732c03e9a0 2024-05-17T09:08:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.967+0000 7f73367fc700 1 -- 172.21.15.79:0/188656414 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7340100130 con 0x7f732c03e9a0 2024-05-17T09:08:50.967 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mgr update... 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f732c03e9a0 msgr2=0x7f732c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f732c03e9a0 0x7f732c040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73401998a0 tx=0x7f7330008040 comp rx=0 tx=0).stop 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 msgr2=0x7f73401987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 0x7f73401987f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f733c005920 tx=0x7f733c004f90 comp rx=0 tx=0).stop 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 shutdown_connections 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f732c03e9a0 0x7f732c040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7340102be0 0x7f73401987f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f734019e200 0x7f73401a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 --2- 172.21.15.79:0/188656414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340198d30 0x7f734019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:50.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 >> 172.21.15.79:0/188656414 conn(0x7f73400fe240 msgr2=0x7f73400ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:50.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 shutdown_connections 2024-05-17T09:08:50.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:08:50.969+0000 7f7347509700 1 -- 172.21.15.79:0/188656414 wait complete. 2024-05-17T09:08:51.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:51 smithi079 ceph-mon[35502]: Reconfiguring mon.b (monmap changed)... 2024-05-17T09:08:51.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:51 smithi079 ceph-mon[35502]: Reconfiguring daemon mon.b on smithi069 2024-05-17T09:08:51.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:51.555 DEBUG:teuthology.orchestra.run.smithi069:mgr.y> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-17T09:08:51.557 DEBUG:teuthology.orchestra.run.smithi079:mgr.z> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-17T09:08:51.559 INFO:tasks.cephadm:Deploying OSDs... 2024-05-17T09:08:51.560 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:08:51.560 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T09:08:51.578 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:51 smithi069 ceph-mon[35081]: Reconfiguring mon.b (monmap changed)... 2024-05-17T09:08:51.578 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:51 smithi069 ceph-mon[35081]: Reconfiguring daemon mon.b on smithi069 2024-05-17T09:08:51.579 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:51.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T09:08:51.580 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-05-17T09:08:51.639 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T09:08:51.639 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-17 09:07:57.252150491 +0000 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-17 09:02:33.743232028 +0000 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-17 09:02:33.743232028 +0000 2024-05-17T09:08:51.640 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-17 09:02:33.743232028 +0000 2024-05-17T09:08:51.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T09:08:51.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:51 smithi019 ceph-mon[31620]: Reconfiguring mon.b (monmap changed)... 2024-05-17T09:08:51.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:51 smithi019 ceph-mon[31620]: Reconfiguring daemon mon.b on smithi069 2024-05-17T09:08:51.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:51.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-17T09:08:51.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-17T09:08:51.711 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000288116 s, 1.8 MB/s 2024-05-17T09:08:51.712 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T09:08:51.772 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-05-17T09:08:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T09:08:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:51.830 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-17T09:08:51.831 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:51.831 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:51.831 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-17 09:07:57.281150227 +0000 2024-05-17T09:08:51.831 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-17 09:02:34.101228922 +0000 2024-05-17T09:08:51.831 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-17 09:02:34.101228922 +0000 2024-05-17T09:08:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-17 09:02:34.101228922 +0000 2024-05-17T09:08:51.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T09:08:51.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-17T09:08:51.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-17T09:08:51.897 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000193228 s, 2.6 MB/s 2024-05-17T09:08:51.899 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T09:08:51.957 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-05-17T09:08:52.015 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T09:08:52.015 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-17 09:07:57.307149990 +0000 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-17 09:02:34.448225912 +0000 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-17 09:02:34.448225912 +0000 2024-05-17T09:08:52.016 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-17 09:02:34.448225912 +0000 2024-05-17T09:08:52.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T09:08:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-17T09:08:52.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-17T09:08:52.082 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000178248 s, 2.9 MB/s 2024-05-17T09:08:52.083 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T09:08:52.141 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-17 09:07:57.334149744 +0000 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-17 09:02:34.779223040 +0000 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-17 09:02:34.779223040 +0000 2024-05-17T09:08:52.198 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-17 09:02:34.779223040 +0000 2024-05-17T09:08:52.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T09:08:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-17T09:08:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-17T09:08:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000325826 s, 1.6 MB/s 2024-05-17T09:08:52.265 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T09:08:52.323 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-05-17T09:08:52.323 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T09:08:52.343 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T09:08:52.343 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-17 09:08:39.257801409 +0000 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-17 09:02:27.713573467 +0000 2024-05-17T09:08:52.401 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-17 09:02:27.713573467 +0000 2024-05-17T09:08:52.402 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-17 09:02:27.713573467 +0000 2024-05-17T09:08:52.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T09:08:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi069=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: Saving service mgr spec with placement smithi019=x;smithi069=y;smithi079=z;count:3 2024-05-17T09:08:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:52.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:52.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: Reconfiguring mon.c (monmap changed)... 2024-05-17T09:08:52.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:52.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:52.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:52.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:52 smithi019 ceph-mon[31620]: Reconfiguring daemon mon.c on smithi079 2024-05-17T09:08:52.468 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-17T09:08:52.468 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-17T09:08:52.468 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000232881 s, 2.2 MB/s 2024-05-17T09:08:52.469 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi069=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: Saving service mgr spec with placement smithi019=x;smithi069=y;smithi079=z;count:3 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: Reconfiguring mon.c (monmap changed)... 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:52.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:52 smithi079 ceph-mon[35502]: Reconfiguring daemon mon.c on smithi079 2024-05-17T09:08:52.528 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-17 09:08:39.257801409 +0000 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-17 09:02:28.047565965 +0000 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-17 09:02:28.047565965 +0000 2024-05-17T09:08:52.586 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-17 09:02:28.047565965 +0000 2024-05-17T09:08:52.587 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi069=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: Saving service mgr spec with placement smithi019=x;smithi069=y;smithi079=z;count:3 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: Reconfiguring mon.c (monmap changed)... 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:52.649 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:52 smithi069 ceph-mon[35081]: Reconfiguring daemon mon.c on smithi079 2024-05-17T09:08:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-17T09:08:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-17T09:08:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000202249 s, 2.5 MB/s 2024-05-17T09:08:52.652 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T09:08:52.710 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-17 09:08:39.258801387 +0000 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-17 09:02:28.385558372 +0000 2024-05-17T09:08:52.766 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-17 09:02:28.385558372 +0000 2024-05-17T09:08:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-17 09:02:28.385558372 +0000 2024-05-17T09:08:52.767 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T09:08:52.831 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-17T09:08:52.831 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-17T09:08:52.831 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000181466 s, 2.8 MB/s 2024-05-17T09:08:52.832 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T09:08:52.890 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-05-17 09:08:39.258801387 +0000 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-05-17 09:02:28.710551071 +0000 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-05-17 09:02:28.710551071 +0000 2024-05-17T09:08:52.947 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-05-17 09:02:28.710551071 +0000 2024-05-17T09:08:52.948 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T09:08:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-05-17T09:08:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-05-17T09:08:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000197247 s, 2.6 MB/s 2024-05-17T09:08:53.012 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T09:08:53.069 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-17T09:08:53.069 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T09:08:53.086 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T09:08:53.086 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-05-17T09:08:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T09:08:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:53.142 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-17T09:08:53.142 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:53.142 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:53.142 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-17 09:08:29.892095444 +0000 2024-05-17T09:08:53.143 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-17 09:02:29.185803214 +0000 2024-05-17T09:08:53.143 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-17 09:02:29.185803214 +0000 2024-05-17T09:08:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-05-17 09:02:29.185803214 +0000 2024-05-17T09:08:53.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T09:08:53.206 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-17T09:08:53.207 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-17T09:08:53.207 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000274563 s, 1.9 MB/s 2024-05-17T09:08:53.208 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T09:08:53.265 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-17 09:08:29.893095422 +0000 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-17 09:02:29.515795922 +0000 2024-05-17T09:08:53.321 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-17 09:02:29.515795922 +0000 2024-05-17T09:08:53.322 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-05-17 09:02:29.515795922 +0000 2024-05-17T09:08:53.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T09:08:53.385 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-17T09:08:53.385 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-17T09:08:53.385 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000173985 s, 2.9 MB/s 2024-05-17T09:08:53.386 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T09:08:53.442 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-05-17T09:08:53.497 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T09:08:53.497 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:53.497 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-17T09:08:53.498 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:53.498 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:53.498 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-17 09:08:29.893095422 +0000 2024-05-17T09:08:53.498 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-17 09:02:29.837788806 +0000 2024-05-17T09:08:53.498 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-17 09:02:29.837788806 +0000 2024-05-17T09:08:53.498 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-05-17 09:02:29.837788806 +0000 2024-05-17T09:08:53.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T09:08:53.561 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-17T09:08:53.561 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-17T09:08:53.561 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000166902 s, 3.1 MB/s 2024-05-17T09:08:53.563 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T09:08:53.618 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-17 09:08:29.893095422 +0000 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-17 09:02:30.171781425 +0000 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-17 09:02:30.171781425 +0000 2024-05-17T09:08:53.675 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-05-17 09:02:30.171781425 +0000 2024-05-17T09:08:53.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T09:08:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-17T09:08:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-17T09:08:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000254665 s, 2.0 MB/s 2024-05-17T09:08:53.742 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T09:08:53.800 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-05-17T09:08:53.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T09:08:54.043 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:54.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:08:54.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:54.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:54.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:54.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:08:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T09:08:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:08:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:08:54.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:54.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:08:54.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T09:08:54.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:08:54.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:55.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:54 smithi079 ceph-mon[35502]: Deploying daemon mgr.z on smithi079 2024-05-17T09:08:55.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:54 smithi019 ceph-mon[31620]: Deploying daemon mgr.z on smithi079 2024-05-17T09:08:55.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:54 smithi069 ceph-mon[35081]: Deploying daemon mgr.z on smithi079 2024-05-17T09:08:55.502 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 systemd[1]: Starting Ceph mgr.z for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:08:55.772 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 podman[37328]: 2024-05-17 09:08:55.405189873 +0000 UTC m=+0.017585885 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:08:55.773 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 podman[37328]: 2024-05-17 09:08:55.50841907 +0000 UTC m=+0.120815110 container create 76c49bc365b0714021e241adb878f7bc5239511e9a776464acbd591d917830e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-17T09:08:55.773 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 podman[37328]: 2024-05-17 09:08:55.686132539 +0000 UTC m=+0.298528537 container init 76c49bc365b0714021e241adb878f7bc5239511e9a776464acbd591d917830e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-17T09:08:55.773 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 podman[37328]: 2024-05-17 09:08:55.689210857 +0000 UTC m=+0.301606864 container start 76c49bc365b0714021e241adb878f7bc5239511e9a776464acbd591d917830e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T09:08:55.773 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 ceph-mgr[37348]: -- 172.21.15.79:0/3467456878 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c398605a0 con 0x557c3e5ea480 2024-05-17T09:08:56.037 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:55 smithi079 ceph-mon[35502]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:56.038 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 bash[37328]: 76c49bc365b0714021e241adb878f7bc5239511e9a776464acbd591d917830e3 2024-05-17T09:08:56.038 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 systemd[1]: Started Ceph mgr.z for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:08:56.038 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:55.853+0000 7fa7e55d2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T09:08:56.038 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:56.036+0000 7fa7e55d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:08:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T09:08:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T09:08:56.054 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-17T09:08:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-17T09:08:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129983 s, 807 MB/s 2024-05-17T09:08:56.055 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-05-17T09:08:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-17T09:08:56.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:55 smithi069 ceph-mon[35081]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:56.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:55 smithi019 ceph-mon[31620]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:56.443 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:56.138+0000 7fa7e55d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T09:08:56.443 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:56.442+0000 7fa7e55d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T09:08:56.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-05-17T09:08:56.772 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:56.527+0000 7fa7e55d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T09:08:56.940 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:08:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:08:56.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T09:08:56.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:08:56.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:56.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:56 smithi019 ceph-mon[31620]: Deploying daemon mgr.y on smithi069 2024-05-17T09:08:57.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:57.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:57.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:57.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:08:57.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T09:08:57.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:08:57.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:57.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:56 smithi069 ceph-mon[35081]: Deploying daemon mgr.y on smithi069 2024-05-17T09:08:57.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:57.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:57.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:57.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:08:57.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T09:08:57.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:08:57.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:57.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:56 smithi079 ceph-mon[35502]: Deploying daemon mgr.y on smithi069 2024-05-17T09:08:58.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:57 smithi079 ceph-mon[35502]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:58.022 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:57 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:57.717+0000 7fa7e55d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T09:08:58.022 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:57 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:57.885+0000 7fa7e55d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T09:08:58.036 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:57 smithi019 ceph-mon[31620]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:58.099 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:57 smithi069 ceph-mon[35081]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:58.345 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:58.054+0000 7fa7e55d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T09:08:58.345 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:58.344+0000 7fa7e55d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T09:08:58.384 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:58 smithi069 ceph-mgr[36450]: -- 172.21.15.69:0/2590476649 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a457aca5a0 con 0x55a45c854900 2024-05-17T09:08:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 -- 172.21.15.19:0/2085114378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4004103b00 msgr2=0x7f4004103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 --2- 172.21.15.19:0/2085114378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4004103b00 0x7f4004103f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009990 tx=0x7f3ff8009ca0 comp rx=0 tx=0).stop 2024-05-17T09:08:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 -- 172.21.15.19:0/2085114378 shutdown_connections 2024-05-17T09:08:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 --2- 172.21.15.19:0/2085114378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40041044a0 0x7f400410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 --2- 172.21.15.19:0/2085114378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40041029d0 0x7f4004102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 --2- 172.21.15.19:0/2085114378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4004103b00 0x7f4004103f10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 -- 172.21.15.19:0/2085114378 >> 172.21.15.19:0/2085114378 conn(0x7f40040fe010 msgr2=0x7f40041004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:08:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 -- 172.21.15.19:0/2085114378 shutdown_connections 2024-05-17T09:08:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.561+0000 7f4009856700 1 -- 172.21.15.19:0/2085114378 wait complete. 2024-05-17T09:08:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.562+0000 7f4009856700 1 Processor -- start 2024-05-17T09:08:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.562+0000 7f4009856700 1 -- start start 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4009856700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 0x7f400410bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4009856700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4004103b00 0x7f400419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4009856700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40041044a0 0x7f40041a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4009856700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f400410eb80 con 0x7f40041029d0 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4009856700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f400410ea00 con 0x7f40041044a0 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4009856700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f400410ed00 con 0x7f4004103b00 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 0x7f400410bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 0x7f400410bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34100/0 (socket says 172.21.15.19:34100) 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 -- 172.21.15.19:0/4060839305 learned_addr learned my addr 172.21.15.19:0/4060839305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f40027fc700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4004103b00 0x7f400419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f40037fe700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40041044a0 0x7f40041a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 -- 172.21.15.19:0/4060839305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4004103b00 msgr2=0x7f400419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4004103b00 0x7f400419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 -- 172.21.15.19:0/4060839305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40041044a0 msgr2=0x7f40041a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40041044a0 0x7f40041a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 -- 172.21.15.19:0/4060839305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40041a2380 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.563+0000 7f4002ffd700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 0x7f400410bda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ef70 tx=0x7f3ff400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.564+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40092e0 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.564+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff400c9c0 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.564+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40041a2610 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.564+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4004440 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.564+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041a2b50 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.565+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40040512b0 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.565+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3ff401b030 con 0x7f40041029d0 2024-05-17T09:08:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.565+0000 7f4008854700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3fec03e750 0x7f3fec040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:08:58.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.565+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3ff40527d0 con 0x7f40041029d0 2024-05-17T09:08:58.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.565+0000 7f40027fc700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3fec03e750 0x7f3fec040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:08:58.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.566+0000 7f40027fc700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3fec03e750 0x7f3fec040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800b650 tx=0x7f3ff801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:08:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.570+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ff4022020 con 0x7f40041029d0 2024-05-17T09:08:58.653 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:58.440+0000 7fa7e55d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T09:08:58.653 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:58.653+0000 7fa7e55d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T09:08:58.659 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:58 smithi069 bash[36430]: 843efbb5ae808383f8d27c3da12c9bf64bd81956a04b308f4a706ce3817939f0 2024-05-17T09:08:58.659 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:58 smithi069 systemd[1]: Started Ceph mgr.y for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:08:58.659 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:08:58.475+0000 7fcd6eb6c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T09:08:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:08:58.798+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4004063bf0 con 0x7f3fec03e750 2024-05-17T09:08:58.930 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:08:58.673+0000 7fcd6eb6c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:08:58.930 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:08:58.799+0000 7fcd6eb6c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T09:08:59.022 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:58.872+0000 7fa7e55d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:08:59.430 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:08:59.127+0000 7fcd6eb6c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T09:08:59.430 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:08:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:08:59.237+0000 7fcd6eb6c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:59.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:08:59.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:08:59.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:08:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:59.531+0000 7fa7e55d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:08:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:08:59.762+0000 7fa7e55d2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:59.763 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:59.763 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:59.763 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:08:59.763 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:08:59.763 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:08:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:08:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:08:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:08:59.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:08:59.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:08:59.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:08:59.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:08:59.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:08:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:00.522 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:00.197+0000 7fa7e55d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T09:09:00.522 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:00.297+0000 7fa7e55d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T09:09:00.818 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:00.481+0000 7fcd6eb6c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T09:09:00.818 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:00.649+0000 7fcd6eb6c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T09:09:00.819 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:00.818+0000 7fcd6eb6c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T09:09:01.005 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:00.662+0000 7fa7e55d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T09:09:01.102 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:01.103 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:01 smithi069 ceph-mon[35081]: Reconfiguring daemon mgr.x on smithi019 2024-05-17T09:09:01.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:01.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:01 smithi019 ceph-mon[31620]: Reconfiguring daemon mgr.x on smithi019 2024-05-17T09:09:01.272 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:01.016+0000 7fa7e55d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T09:09:01.272 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:01.109+0000 7fa7e55d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T09:09:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:01.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:01 smithi079 ceph-mon[35502]: Reconfiguring daemon mgr.x on smithi019 2024-05-17T09:09:01.395 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:01.102+0000 7fcd6eb6c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T09:09:01.395 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:01.193+0000 7fcd6eb6c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T09:09:01.680 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:01.394+0000 7fcd6eb6c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T09:09:01.680 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:01.584+0000 7fcd6eb6c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T09:09:01.772 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:01.353+0000 7fa7e55d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T09:09:01.772 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:01.485+0000 7fa7e55d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T09:09:01.772 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:01.581+0000 7fa7e55d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T09:09:02.272 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:01.905+0000 7fa7e55d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T09:09:02.272 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:02.000+0000 7fa7e55d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T09:09:02.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:02 smithi079 ceph-mon[35502]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:02.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3168271286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de60f57-fe91-4722-8fac-00c122296aac"}]: dispatch 2024-05-17T09:09:02.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3168271286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7de60f57-fe91-4722-8fac-00c122296aac"}]': finished 2024-05-17T09:09:02.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:02 smithi079 ceph-mon[35502]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T09:09:02.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:02.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:02.401 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:02.183+0000 7fcd6eb6c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T09:09:02.401 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:02.400+0000 7fcd6eb6c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T09:09:02.401 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:02 smithi069 ceph-mon[35081]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:02.401 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3168271286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de60f57-fe91-4722-8fac-00c122296aac"}]: dispatch 2024-05-17T09:09:02.401 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3168271286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7de60f57-fe91-4722-8fac-00c122296aac"}]': finished 2024-05-17T09:09:02.401 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:02 smithi069 ceph-mon[35081]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T09:09:02.402 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:02.402 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:02 smithi019 ceph-mon[31620]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3168271286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de60f57-fe91-4722-8fac-00c122296aac"}]: dispatch 2024-05-17T09:09:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3168271286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7de60f57-fe91-4722-8fac-00c122296aac"}]': finished 2024-05-17T09:09:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:02 smithi019 ceph-mon[31620]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T09:09:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:03.022 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:02.603+0000 7fa7e55d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T09:09:03.022 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:09:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z[37344]: 2024-05-17T09:09:02.821+0000 7fa7e55d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T09:09:03.077 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:02.821+0000 7fcd6eb6c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T09:09:03.077 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:02.917+0000 7fcd6eb6c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T09:09:03.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:03.080+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f3ff4012050 con 0x7f40041029d0 2024-05-17T09:09:03.430 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:03.285+0000 7fcd6eb6c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T09:09:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:03 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4110092419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:03 smithi069 ceph-mon[35081]: Standby manager daemon z started 2024-05-17T09:09:03.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:03 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T09:09:03.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:03 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T09:09:03.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:03 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T09:09:03.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:03 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T09:09:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:03 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4110092419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:03 smithi019 ceph-mon[31620]: Standby manager daemon z started 2024-05-17T09:09:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:03 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T09:09:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:03 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T09:09:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:03 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T09:09:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:03 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T09:09:03.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:03 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4110092419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:03.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:03 smithi079 ceph-mon[35502]: Standby manager daemon z started 2024-05-17T09:09:03.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:03 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T09:09:03.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:03 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T09:09:03.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:03 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T09:09:03.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:03 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.79:0/2500732229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T09:09:03.930 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:03.633+0000 7fcd6eb6c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T09:09:03.930 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:03.723+0000 7fcd6eb6c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T09:09:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:04 smithi069 ceph-mon[35081]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:04 smithi069 ceph-mon[35081]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-17T09:09:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T09:09:04.430 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:03.965+0000 7fcd6eb6c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T09:09:04.430 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:04.097+0000 7fcd6eb6c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T09:09:04.431 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:04.192+0000 7fcd6eb6c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T09:09:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:04 smithi019 ceph-mon[31620]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:04 smithi019 ceph-mon[31620]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-17T09:09:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T09:09:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:04 smithi079 ceph-mon[35502]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:04 smithi079 ceph-mon[35502]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-17T09:09:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T09:09:04.930 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:04.525+0000 7fcd6eb6c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T09:09:04.930 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:04.618+0000 7fcd6eb6c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T09:09:05.680 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:05 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:05.216+0000 7fcd6eb6c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T09:09:05.680 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:09:05 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y[36445]: 2024-05-17T09:09:05.431+0000 7fcd6eb6c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T09:09:06.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:06.090+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f3ff4007b10 con 0x7f40041029d0 2024-05-17T09:09:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:06 smithi019 ceph-mon[31620]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:06 smithi019 ceph-mon[31620]: Standby manager daemon y started 2024-05-17T09:09:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:06 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T09:09:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:06 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T09:09:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:06 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T09:09:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:06 smithi019 ceph-mon[31620]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T09:09:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:06 smithi069 ceph-mon[35081]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:06 smithi069 ceph-mon[35081]: Standby manager daemon y started 2024-05-17T09:09:06.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:06 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T09:09:06.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:06 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T09:09:06.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:06 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T09:09:06.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:06 smithi069 ceph-mon[35081]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T09:09:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:06 smithi079 ceph-mon[35502]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:06 smithi079 ceph-mon[35502]: Standby manager daemon y started 2024-05-17T09:09:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:06 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T09:09:06.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:06 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T09:09:06.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:06 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T09:09:06.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:06 smithi079 ceph-mon[35502]: from='mgr.? 172.21.15.69:0/3639332926' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T09:09:07.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:07 smithi069 ceph-mon[35081]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-17T09:09:07.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T09:09:07.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:07 smithi019 ceph-mon[31620]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-17T09:09:07.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T09:09:07.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:07 smithi079 ceph-mon[35502]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-17T09:09:07.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T09:09:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:08.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:08.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:08.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:08.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:10 smithi069 ceph-mon[35081]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T09:09:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:10.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:10 smithi019 ceph-mon[31620]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:10.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T09:09:10.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:10 smithi079 ceph-mon[35502]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T09:09:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:11.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:11 smithi019 ceph-mon[31620]: Deploying daemon osd.0 on smithi019 2024-05-17T09:09:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:11 smithi069 ceph-mon[35081]: Deploying daemon osd.0 on smithi019 2024-05-17T09:09:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:11 smithi079 ceph-mon[35502]: Deploying daemon osd.0 on smithi019 2024-05-17T09:09:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:12 smithi069 ceph-mon[35081]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:12.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:12 smithi019 ceph-mon[31620]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:12 smithi079 ceph-mon[35502]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:14.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:14 smithi019 ceph-mon[31620]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:14 smithi069 ceph-mon[35081]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:14 smithi079 ceph-mon[35502]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:16.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:16 smithi019 ceph-mon[31620]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:16.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:16.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:16.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:16 smithi069 ceph-mon[35081]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:16 smithi079 ceph-mon[35502]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:16.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:18 smithi069 ceph-mon[35081]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:18.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:18 smithi019 ceph-mon[31620]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:18 smithi079 ceph-mon[35502]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:19.310 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-05-17T09:09:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.308+0000 7f4008854700 1 -- 172.21.15.19:0/4060839305 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4004063bf0 con 0x7f3fec03e750 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3fec03e750 msgr2=0x7f3fec040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3fec03e750 0x7f3fec040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800b650 tx=0x7f3ff801a040 comp rx=0 tx=0).stop 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 msgr2=0x7f400410bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 0x7f400410bda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ef70 tx=0x7f3ff400c3a0 comp rx=0 tx=0).stop 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 shutdown_connections 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40041044a0 0x7f40041a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:19.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40041029d0 0x7f400410bda0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3fec03e750 0x7f3fec040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 --2- 172.21.15.19:0/4060839305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4004103b00 0x7f400419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 >> 172.21.15.19:0/4060839305 conn(0x7f40040fe010 msgr2=0x7f4004106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:09:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 shutdown_connections 2024-05-17T09:09:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:09:19.311+0000 7f4009856700 1 -- 172.21.15.19:0/4060839305 wait complete. 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:19.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:19.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:19.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:19.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:19.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:19.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:19.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:19.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:19.944 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.0.service 2024-05-17T09:09:19.946 INFO:tasks.cephadm:Deploying osd.1 on smithi069 with /dev/vg_nvme/lv_4... 2024-05-17T09:09:19.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T09:09:20.194 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:09:20.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:20 smithi069 ceph-mon[35081]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T09:09:20.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:20 smithi019 ceph-mon[31620]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T09:09:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:20 smithi079 ceph-mon[35502]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T09:09:21.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:21 smithi069 ceph-mon[35081]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T09:09:21.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:21 smithi069 ceph-mon[35081]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T09:09:21.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:21 smithi069 ceph-mon[35081]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:09:21.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:21.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:21 smithi069 ceph-mon[35081]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:21.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:21 smithi019 ceph-mon[31620]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T09:09:21.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:21 smithi019 ceph-mon[31620]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T09:09:21.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:21 smithi019 ceph-mon[31620]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:09:21.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:21.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:21 smithi019 ceph-mon[31620]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:21.693 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:09:21 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0[43698]: 2024-05-17T09:09:21.385+0000 7f5533771700 -1 osd.0 0 waiting for initial osdmap 2024-05-17T09:09:21.693 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:09:21 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0[43698]: 2024-05-17T09:09:21.389+0000 7f552f526700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:09:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:21 smithi079 ceph-mon[35502]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T09:09:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:21 smithi079 ceph-mon[35502]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T09:09:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:21 smithi079 ceph-mon[35502]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:09:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:21 smithi079 ceph-mon[35502]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:21.902 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T09:09:21.902 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T09:09:21.902 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-17T09:09:21.902 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-17T09:09:21.902 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120812 s, 868 MB/s 2024-05-17T09:09:21.903 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-17T09:09:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:22 smithi019 ceph-mon[31620]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:09:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:22 smithi019 ceph-mon[31620]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T09:09:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:22.509 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-05-17T09:09:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:22 smithi069 ceph-mon[35081]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:09:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:22 smithi069 ceph-mon[35081]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T09:09:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:22.747 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:09:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:22 smithi079 ceph-mon[35502]: from='osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:09:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:22 smithi079 ceph-mon[35502]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T09:09:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:09:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:09:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] boot 2024-05-17T09:09:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T09:09:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi019 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:23.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] boot 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi019 2024-05-17T09:09:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:23.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:09:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:09:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: osd.0 [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] boot 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi019 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:23.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:24.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 -- 172.21.15.69:0/1555141111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484105490 msgr2=0x7f94841058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 --2- 172.21.15.69:0/1555141111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484105490 0x7f94841058a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9474009990 tx=0x7f9474009ca0 comp rx=0 tx=0).stop 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 -- 172.21.15.69:0/1555141111 shutdown_connections 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 --2- 172.21.15.69:0/1555141111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484104360 0x7f9484104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 --2- 172.21.15.69:0/1555141111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9484105e30 0x7f948410c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 --2- 172.21.15.69:0/1555141111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484105490 0x7f94841058a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 -- 172.21.15.69:0/1555141111 >> 172.21.15.69:0/1555141111 conn(0x7f94840ff9c0 msgr2=0x7f9484101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 -- 172.21.15.69:0/1555141111 shutdown_connections 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.172+0000 7f948b2f6700 1 -- 172.21.15.69:0/1555141111 wait complete. 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 Processor -- start 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 -- start start 2024-05-17T09:09:24.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 0x7f948419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9484105490 0x7f948419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484105e30 0x7f94841a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f948410fc70 con 0x7f9484105490 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f948410faf0 con 0x7f9484104360 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f948b2f6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f948410fdf0 con 0x7f9484105e30 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f9489092700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 0x7f948419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f9489092700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 0x7f948419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52646/0 (socket says 172.21.15.69:52646) 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f9489092700 1 -- 172.21.15.69:0/3034453986 learned_addr learned my addr 172.21.15.69:0/3034453986 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.173+0000 7f9488891700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9484105490 0x7f948419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489893700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484105e30 0x7f94841a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489092700 1 -- 172.21.15.69:0/3034453986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484105e30 msgr2=0x7f94841a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:24.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489092700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484105e30 0x7f94841a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489092700 1 -- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9484105490 msgr2=0x7f948419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489092700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9484105490 0x7f948419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489092700 1 -- 172.21.15.69:0/3034453986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94841a22f0 con 0x7f9484104360 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489893700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484105e30 0x7f94841a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f9489092700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 0x7f948419a820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9480007ee0 tx=0x7f948000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948000dbd0 con 0x7f9484104360 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f948000dd70 con 0x7f9484104360 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94841a25e0 con 0x7f9484104360 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800046f0 con 0x7f9484104360 2024-05-17T09:09:24.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.174+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94841a2b70 con 0x7f9484104360 2024-05-17T09:09:24.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.175+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94840512b0 con 0x7f9484104360 2024-05-17T09:09:24.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.176+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f948001a020 con 0x7f9484104360 2024-05-17T09:09:24.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.177+0000 7f947a7fc700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94700b2870 0x7f94700b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:24.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.177+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f94800e0490 con 0x7f9484104360 2024-05-17T09:09:24.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.177+0000 7f9488891700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94700b2870 0x7f94700b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:24.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.178+0000 7f9488891700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94700b2870 0x7f94700b4d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f947400b5c0 tx=0x7f947401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:09:24.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.181+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9480010070 con 0x7f9484104360 2024-05-17T09:09:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:24.389+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9484063bf0 con 0x7f94700b2870 2024-05-17T09:09:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:24 smithi069 ceph-mon[35081]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T09:09:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:09:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:24 smithi019 ceph-mon[31620]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T09:09:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:09:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:24 smithi079 ceph-mon[35502]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T09:09:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:09:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:25 smithi069 ceph-mon[35081]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:09:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:25 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:09:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:25 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:25.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:25 smithi069 ceph-mon[35081]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:25.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:25 smithi019 ceph-mon[31620]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:09:25.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:09:25.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:25.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:25 smithi019 ceph-mon[31620]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:25.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:25 smithi079 ceph-mon[35502]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:09:25.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:25 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:09:25.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:25 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:25.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:25 smithi079 ceph-mon[35502]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:28.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:28.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/160542988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]: dispatch 2024-05-17T09:09:28.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]: dispatch 2024-05-17T09:09:28.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]': finished 2024-05-17T09:09:28.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T09:09:28.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:28.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:27 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/1739449119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/160542988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]: dispatch 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]: dispatch 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]': finished 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:27 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/1739449119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:28.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:28.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/160542988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]: dispatch 2024-05-17T09:09:28.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]: dispatch 2024-05-17T09:09:28.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "840241e0-3ae2-4033-8bbb-01a3935c3123"}]': finished 2024-05-17T09:09:28.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T09:09:28.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:28.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:27 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/1739449119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:30.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:29 smithi069 ceph-mon[35081]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:30.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:29 smithi019 ceph-mon[31620]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:30.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:29 smithi079 ceph-mon[35502]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:32.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:31 smithi069 ceph-mon[35081]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:32.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:31 smithi019 ceph-mon[31620]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:32.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:31 smithi079 ceph-mon[35502]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:34.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:33 smithi069 ceph-mon[35081]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:33 smithi019 ceph-mon[31620]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:33 smithi079 ceph-mon[35502]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:35.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T09:09:35.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:35.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:34 smithi069 ceph-mon[35081]: Deploying daemon osd.1 on smithi069 2024-05-17T09:09:35.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T09:09:35.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:35.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:34 smithi019 ceph-mon[31620]: Deploying daemon osd.1 on smithi069 2024-05-17T09:09:35.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T09:09:35.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:35.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:34 smithi079 ceph-mon[35502]: Deploying daemon osd.1 on smithi069 2024-05-17T09:09:36.032 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:35 smithi069 ceph-mon[35081]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:36.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:35 smithi019 ceph-mon[31620]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:36.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:35 smithi079 ceph-mon[35502]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:37 smithi069 ceph-mon[35081]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:38.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:37 smithi019 ceph-mon[31620]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:38.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:37 smithi079 ceph-mon[35502]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:40 smithi069 ceph-mon[35081]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:40 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:40 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:40.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:40 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:40.543 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:40 smithi079 ceph-mon[35502]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:40.543 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:40 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:40.543 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:40 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:40.543 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:40 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:40.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:40 smithi019 ceph-mon[31620]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:40.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:40 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:40.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:40 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:40.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:40 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:42.572 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:42 smithi069 ceph-mon[35081]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:42.573 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.573 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.573 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:42.573 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:42.573 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:42 smithi019 ceph-mon[31620]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:42 smithi079 ceph-mon[35502]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:42.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:42.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:42.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.857+0000 7f947a7fc700 1 -- 172.21.15.69:0/3034453986 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9484063bf0 con 0x7f94700b2870 2024-05-17T09:09:42.857 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 1 on host 'smithi069' 2024-05-17T09:09:42.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94700b2870 msgr2=0x7f94700b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94700b2870 0x7f94700b4d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f947400b5c0 tx=0x7f947401a040 comp rx=0 tx=0).stop 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 msgr2=0x7f948419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 0x7f948419a820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9480007ee0 tx=0x7f948000d3b0 comp rx=0 tx=0).stop 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 shutdown_connections 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94700b2870 0x7f94700b4d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484105e30 0x7f94841a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9484104360 0x7f948419a820 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 --2- 172.21.15.69:0/3034453986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9484105490 0x7f948419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 >> 172.21.15.69:0/3034453986 conn(0x7f94840ff9c0 msgr2=0x7f9484108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 shutdown_connections 2024-05-17T09:09:42.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:09:42.859+0000 7f948b2f6700 1 -- 172.21.15.69:0/3034453986 wait complete. 2024-05-17T09:09:43.495 DEBUG:teuthology.orchestra.run.smithi069:osd.1> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.1.service 2024-05-17T09:09:43.498 INFO:tasks.cephadm:Deploying osd.2 on smithi079 with /dev/vg_nvme/lv_4... 2024-05-17T09:09:43.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T09:09:43.745 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:09:44.169 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:43 smithi079 ceph-mon[35502]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:44.169 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:44.169 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:44.169 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:44.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:43 smithi079 ceph-mon[35502]: from='osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T09:09:44.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:43 smithi079 ceph-mon[35502]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T09:09:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:43 smithi069 ceph-mon[35081]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:43 smithi069 ceph-mon[35081]: from='osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T09:09:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:43 smithi069 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T09:09:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:43 smithi019 ceph-mon[31620]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:09:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:43 smithi019 ceph-mon[31620]: from='osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T09:09:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:43 smithi019 ceph-mon[31620]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T09:09:44.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:44 smithi079 ceph-mon[35502]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T09:09:44.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:44 smithi079 ceph-mon[35502]: from='osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:09:44.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:44 smithi079 ceph-mon[35502]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T09:09:44.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:44.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:44 smithi079 ceph-mon[35502]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:09:45.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:44 smithi069 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T09:09:45.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:44 smithi069 ceph-mon[35081]: from='osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:09:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:44 smithi069 ceph-mon[35081]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T09:09:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:44 smithi069 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:09:45.181 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:09:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1[39654]: 2024-05-17T09:09:44.861+0000 7f0d5d853700 -1 osd.1 0 waiting for initial osdmap 2024-05-17T09:09:45.181 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:09:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1[39654]: 2024-05-17T09:09:44.865+0000 7f0d59608700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:09:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:44 smithi019 ceph-mon[31620]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T09:09:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:44 smithi019 ceph-mon[31620]: from='osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:09:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:44 smithi019 ceph-mon[31620]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T09:09:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:44 smithi019 ceph-mon[31620]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:09:45.616 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T09:09:45.616 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T09:09:45.616 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-17T09:09:45.616 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-17T09:09:45.616 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123575 s, 849 MB/s 2024-05-17T09:09:45.616 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-17T09:09:45.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:45 smithi069 ceph-mon[35081]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:45.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:45 smithi069 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:09:45.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:45 smithi069 ceph-mon[35081]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T09:09:45.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:45.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:46.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:45 smithi079 ceph-mon[35502]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:46.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:45 smithi079 ceph-mon[35502]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:09:46.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:45 smithi079 ceph-mon[35502]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T09:09:46.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:46.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:46.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-05-17T09:09:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:45 smithi019 ceph-mon[31620]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:45 smithi019 ceph-mon[31620]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:09:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:45 smithi019 ceph-mon[31620]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T09:09:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:46.440 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:09:47.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:09:47.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:09:47.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] boot 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi069 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 17088M 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:47.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:09:47.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] boot 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi069 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 17088M 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:47.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:09:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:09:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: osd.1 [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] boot 2024-05-17T09:09:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T09:09:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T09:09:47.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi069 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 17088M 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:09:47.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1670541157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341049d0 msgr2=0x7fc43410c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1670541157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341049d0 0x7fc43410c6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00b0a0 tx=0x7fc42c00b3b0 comp rx=0 tx=0).stop 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1670541157 shutdown_connections 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1670541157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341049d0 0x7fc43410c6e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1670541157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4341059a0 0x7fc434105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1670541157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc434103f50 0x7fc434104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1670541157 >> 172.21.15.79:0/1670541157 conn(0x7fc4340ff9c0 msgr2=0x7fc434101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:09:48.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1670541157 shutdown_connections 2024-05-17T09:09:48.004 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:47 smithi079 ceph-mon[35502]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:48.004 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:47 smithi079 ceph-mon[35502]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T09:09:48.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.003+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1670541157 wait complete. 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 Processor -- start 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 -- start start 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc434103f50 0x7fc434078560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4341049d0 0x7fc434076c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 0x7fc434077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410fde0 con 0x7fc434103f50 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410fc60 con 0x7fc4341049d0 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43c7f4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410ff60 con 0x7fc4341059a0 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43ad91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 0x7fc434077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43ad91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 0x7fc434077150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43908/0 (socket says 172.21.15.79:43908) 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc43ad91700 1 -- 172.21.15.79:0/1425581139 learned_addr learned my addr 172.21.15.79:0/1425581139 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:09:48.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.004+0000 7fc439d8f700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4341049d0 0x7fc434076c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43a590700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc434103f50 0x7fc434078560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43ad91700 1 -- 172.21.15.79:0/1425581139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4341049d0 msgr2=0x7fc434076c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43ad91700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4341049d0 0x7fc434076c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43ad91700 1 -- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc434103f50 msgr2=0x7fc434078560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43ad91700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc434103f50 0x7fc434078560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43ad91700 1 -- 172.21.15.79:0/1425581139 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4340778b0 con 0x7fc4341059a0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43ad91700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 0x7fc434077150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0062a0 tx=0x7fc42c004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c01f030 con 0x7fc4341059a0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42c007a40 con 0x7fc4341059a0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4341a49c0 con 0x7fc4341059a0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c004710 con 0x7fc4341059a0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.005+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4341a4db0 con 0x7fc4341059a0 2024-05-17T09:09:48.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.006+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4341105a0 con 0x7fc4341059a0 2024-05-17T09:09:48.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.008+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fc42c003a00 con 0x7fc4341059a0 2024-05-17T09:09:48.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.008+0000 7fc4277fe700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4200b27f0 0x7fc4200b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:09:48.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.008+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fc42c00f020 con 0x7fc4341059a0 2024-05-17T09:09:48.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.009+0000 7fc43a590700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4200b27f0 0x7fc4200b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:09:48.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.009+0000 7fc43a590700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4200b27f0 0x7fc4200b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc434105390 tx=0x7fc42800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:09:48.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.012+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc42c029020 con 0x7fc4341059a0 2024-05-17T09:09:48.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:47 smithi069 ceph-mon[35081]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:48.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:47 smithi069 ceph-mon[35081]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T09:09:48.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:47 smithi019 ceph-mon[31620]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-17T09:09:48.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:47 smithi019 ceph-mon[31620]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T09:09:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:09:48.221+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc43402e950 con 0x7fc4200b27f0 2024-05-17T09:09:49.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:48 smithi079 ceph-mon[35502]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:09:49.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:09:49.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:09:49.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:49.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:48 smithi069 ceph-mon[35081]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:09:49.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:09:49.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:09:49.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:49.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:48 smithi019 ceph-mon[31620]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:09:49.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:09:49.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:09:49.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:49 smithi069 ceph-mon[35081]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:50.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:49 smithi019 ceph-mon[31620]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:49 smithi079 ceph-mon[35502]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:51.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:50 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/156273110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]: dispatch 2024-05-17T09:09:51.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:50 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]: dispatch 2024-05-17T09:09:51.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:50 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]': finished 2024-05-17T09:09:51.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:50 smithi069 ceph-mon[35081]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T09:09:51.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:09:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:50 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/156273110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]: dispatch 2024-05-17T09:09:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:50 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]: dispatch 2024-05-17T09:09:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:50 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]': finished 2024-05-17T09:09:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:50 smithi019 ceph-mon[31620]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T09:09:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:09:51.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:50 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/156273110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]: dispatch 2024-05-17T09:09:51.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:50 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]: dispatch 2024-05-17T09:09:51.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:50 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c"}]': finished 2024-05-17T09:09:51.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:50 smithi079 ceph-mon[35502]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T09:09:51.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:09:52.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:51 smithi069 ceph-mon[35081]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:52.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:51 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/2829503688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:52.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:51 smithi019 ceph-mon[31620]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:52.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:51 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/2829503688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:52.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:51 smithi079 ceph-mon[35502]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:52.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:51 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/2829503688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:09:54.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:53 smithi069 ceph-mon[35081]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:54.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:53 smithi019 ceph-mon[31620]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:54.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:53 smithi079 ceph-mon[35502]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:56.113 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:55 smithi079 ceph-mon[35502]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:56.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:55 smithi069 ceph-mon[35081]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:56.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:55 smithi019 ceph-mon[31620]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:58.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:57 smithi069 ceph-mon[35081]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:57 smithi019 ceph-mon[31620]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:57 smithi079 ceph-mon[35502]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:09:58.960 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T09:09:58.960 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:58.960 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:58 smithi079 ceph-mon[35502]: Deploying daemon osd.2 on smithi079 2024-05-17T09:09:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T09:09:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:58 smithi069 ceph-mon[35081]: Deploying daemon osd.2 on smithi079 2024-05-17T09:09:59.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T09:09:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:09:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:58 smithi019 ceph-mon[31620]: Deploying daemon osd.2 on smithi079 2024-05-17T09:10:00.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:09:59 smithi079 ceph-mon[35502]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:00.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:09:59 smithi069 ceph-mon[35081]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:00.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:09:59 smithi019 ceph-mon[31620]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:01.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:00 smithi079 ceph-mon[35502]: overall HEALTH_OK 2024-05-17T09:10:01.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:00 smithi069 ceph-mon[35081]: overall HEALTH_OK 2024-05-17T09:10:01.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:00 smithi019 ceph-mon[31620]: overall HEALTH_OK 2024-05-17T09:10:02.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:01 smithi079 ceph-mon[35502]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:02.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:01 smithi069 ceph-mon[35081]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:02.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:01 smithi019 ceph-mon[31620]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:04.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:03 smithi069 ceph-mon[35081]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:04.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:04.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:04.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:03 smithi019 ceph-mon[31620]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:04.217 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:03 smithi079 ceph-mon[35502]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:04.217 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:04.217 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:04.217 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:06.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:05 smithi079 ceph-mon[35502]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:06.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:06.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:06.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:05 smithi069 ceph-mon[35081]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:06.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:05 smithi019 ceph-mon[31620]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:07.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.182+0000 7fc4277fe700 1 -- 172.21.15.79:0/1425581139 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc43402e950 con 0x7fc4200b27f0 2024-05-17T09:10:07.183 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 2 on host 'smithi079' 2024-05-17T09:10:07.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4200b27f0 msgr2=0x7fc4200b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:07.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4200b27f0 0x7fc4200b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc434105390 tx=0x7fc42800a380 comp rx=0 tx=0).stop 2024-05-17T09:10:07.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 msgr2=0x7fc434077150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:07.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 0x7fc434077150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0062a0 tx=0x7fc42c004f40 comp rx=0 tx=0).stop 2024-05-17T09:10:07.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 shutdown_connections 2024-05-17T09:10:07.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4200b27f0 0x7fc4200b4d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:07.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4341059a0 0x7fc434077150 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:07.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc434103f50 0x7fc434078560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:07.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 --2- 172.21.15.79:0/1425581139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4341049d0 0x7fc434076c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:07.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.184+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 >> 172.21.15.79:0/1425581139 conn(0x7fc4340ff9c0 msgr2=0x7fc4341086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:10:07.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.185+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 shutdown_connections 2024-05-17T09:10:07.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:10:07.185+0000 7fc43c7f4700 1 -- 172.21.15.79:0/1425581139 wait complete. 2024-05-17T09:10:07.801 DEBUG:teuthology.orchestra.run.smithi079:osd.2> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.2.service 2024-05-17T09:10:07.803 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-05-17T09:10:07.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T09:10:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:10:08.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:07 smithi079 ceph-mon[35502]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:08.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:08.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:08.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:08.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:07 smithi079 ceph-mon[35502]: from='osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T09:10:08.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:07 smithi079 ceph-mon[35502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T09:10:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:07 smithi069 ceph-mon[35081]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:07 smithi069 ceph-mon[35081]: from='osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T09:10:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:07 smithi069 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T09:10:08.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:07 smithi019 ceph-mon[31620]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:08.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:08.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:08.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:08.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:07 smithi019 ceph-mon[31620]: from='osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T09:10:08.182 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:07 smithi019 ceph-mon[31620]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T09:10:09.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:09 smithi079 ceph-mon[35502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T09:10:09.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:09 smithi079 ceph-mon[35502]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T09:10:09.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:09 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:09.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:09 smithi079 ceph-mon[35502]: from='osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:10:09.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:09 smithi079 ceph-mon[35502]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:10:09.353 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2[40637]: 2024-05-17T09:10:09.150+0000 7f8bc98b3700 -1 osd.2 0 waiting for initial osdmap 2024-05-17T09:10:09.354 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2[40637]: 2024-05-17T09:10:09.154+0000 7f8bc5668700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:10:09.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:09 smithi069 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T09:10:09.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:09 smithi069 ceph-mon[35081]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T09:10:09.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:09 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:09.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:09 smithi069 ceph-mon[35081]: from='osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:10:09.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:09 smithi069 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:10:09.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:09 smithi019 ceph-mon[31620]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T09:10:09.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:09 smithi019 ceph-mon[31620]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T09:10:09.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:09.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:09 smithi019 ceph-mon[31620]: from='osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:10:09.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:09 smithi019 ceph-mon[31620]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:10:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T09:10:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T09:10:10.054 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-17T09:10:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-17T09:10:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125776 s, 834 MB/s 2024-05-17T09:10:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-17T09:10:10.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:10 smithi019 ceph-mon[31620]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:10.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:10 smithi019 ceph-mon[31620]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:10:10.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:10 smithi019 ceph-mon[31620]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T09:10:10.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:10.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:10 smithi069 ceph-mon[35081]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:10 smithi069 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:10:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:10 smithi069 ceph-mon[35081]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T09:10:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:10 smithi079 ceph-mon[35502]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:10 smithi079 ceph-mon[35502]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:10:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:10 smithi079 ceph-mon[35502]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T09:10:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:10.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-05-17T09:10:10.997 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:10:11.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:10:11.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:10:11.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] boot 2024-05-17T09:10:11.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T09:10:11.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:11.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:11.380 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:10:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:10:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] boot 2024-05-17T09:10:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:11.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: osd.2 [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] boot 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:11.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T09:10:11.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:11.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:11.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:11.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 ceph-mon[35081]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi079 2024-05-17T09:10:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 17088M 2024-05-17T09:10:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T09:10:12.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 ceph-mon[35081]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T09:10:12.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43308]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43308]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.431 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43311]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 ceph-mon[31620]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:12.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi079 2024-05-17T09:10:12.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 17088M 2024-05-17T09:10:12.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T09:10:12.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 ceph-mon[31620]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T09:10:12.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T09:10:12.434 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T09:10:12.434 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.435 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.435 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47891]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.435 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47894]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T09:10:12.435 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47894]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.435 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47894]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.435 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47894]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44373]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44373]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44373]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44373]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44376]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44376]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44376]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.484 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44376]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.484 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 ceph-mon[35502]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-17T09:10:12.484 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi079 2024-05-17T09:10:12.485 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 17088M 2024-05-17T09:10:12.485 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T09:10:12.485 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 ceph-mon[35502]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T09:10:12.485 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47901]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47901]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47905]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47905]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47905]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:12 smithi019 sudo[47905]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 -- 172.21.15.19:0/352979183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 msgr2=0x7f16b4104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 --2- 172.21.15.19:0/352979183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b4104740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009990 tx=0x7f16a4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 -- 172.21.15.19:0/352979183 shutdown_connections 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 --2- 172.21.15.19:0/352979183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16b4105e00 0x7f16b410c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 --2- 172.21.15.19:0/352979183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b4104740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 --2- 172.21.15.19:0/352979183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4105460 0x7f16b4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 -- 172.21.15.19:0/352979183 >> 172.21.15.19:0/352979183 conn(0x7f16b40ff9b0 msgr2=0x7f16b4101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 -- 172.21.15.19:0/352979183 shutdown_connections 2024-05-17T09:10:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.713+0000 7f16b8c69700 1 -- 172.21.15.19:0/352979183 wait complete. 2024-05-17T09:10:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 Processor -- start 2024-05-17T09:10:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 -- start start 2024-05-17T09:10:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4105460 0x7f16b419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16b4105e00 0x7f16b41a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16b410fc30 con 0x7f16b4104330 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16b410fab0 con 0x7f16b4105e00 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b8c69700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16b410fdb0 con 0x7f16b4105460 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59760/0 (socket says 172.21.15.19:59760) 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 -- 172.21.15.19:0/1691452510 learned_addr learned my addr 172.21.15.19:0/1691452510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b1d9b700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4105460 0x7f16b419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b2d9d700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16b4105e00 0x7f16b41a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 -- 172.21.15.19:0/1691452510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4105460 msgr2=0x7f16b419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4105460 0x7f16b419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 -- 172.21.15.19:0/1691452510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16b4105e00 msgr2=0x7f16b41a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16b4105e00 0x7f16b41a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:12.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.714+0000 7f16b259c700 1 -- 172.21.15.19:0/1691452510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41a2370 con 0x7f16b4104330 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.715+0000 7f16b259c700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b419a810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16a40098e0 tx=0x7f16a400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.715+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4026070 con 0x7f16b4104330 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.715+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b41a2600 con 0x7f16b4104330 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.715+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16a4003d20 con 0x7f16b4104330 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.715+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400fcf0 con 0x7f16b4104330 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.715+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41a2b30 con 0x7f16b4104330 2024-05-17T09:10:12.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.716+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b40512b0 con 0x7f16b4104330 2024-05-17T09:10:12.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.718+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f16a4029070 con 0x7f16b4104330 2024-05-17T09:10:12.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.719+0000 7f16a37fe700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f169c0b27f0 0x7f169c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:12.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.719+0000 7f16b1d9b700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f169c0b27f0 0x7f169c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:12.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.719+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f16a40e1e60 con 0x7f16b4104330 2024-05-17T09:10:12.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.720+0000 7f16b1d9b700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f169c0b27f0 0x7f169c0b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f16b419b8c0 tx=0x7f16a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:10:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.722+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16a405edf0 con 0x7f16b4104330 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44379]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44379]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44379]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44379]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44382]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44382]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:12 smithi079 sudo[44382]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T09:10:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43314]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43317]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T09:10:12.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T09:10:12.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43317]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T09:10:12.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:12 smithi069 sudo[43317]: pam_unix(sudo:session): session closed for user root 2024-05-17T09:10:12.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:12.948+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f16b4063bf0 con 0x7f169c0b27f0 2024-05-17T09:10:13.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T09:10:13.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:10:13.432 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:13.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T09:10:13.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T09:10:13.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:10:13.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:10:13.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:13.445+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f16a4009d10 con 0x7f16b4104330 2024-05-17T09:10:13.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T09:10:13.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T09:10:13.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:10:13.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:14 smithi069 ceph-mon[35081]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:14 smithi069 ceph-mon[35081]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:10:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:14 smithi069 ceph-mon[35081]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T09:10:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:14 smithi069 ceph-mon[35081]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T09:10:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:14 smithi019 ceph-mon[31620]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:14 smithi019 ceph-mon[31620]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:10:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:14 smithi019 ceph-mon[31620]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T09:10:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:14 smithi019 ceph-mon[31620]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T09:10:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:14 smithi079 ceph-mon[35502]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:14 smithi079 ceph-mon[35502]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:10:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:14 smithi079 ceph-mon[35502]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T09:10:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:14 smithi079 ceph-mon[35502]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T09:10:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:16 smithi019 ceph-mon[31620]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:16 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/952557025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "774aea07-241e-4f94-aa94-5e8823cb5b48"}]: dispatch 2024-05-17T09:10:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:16 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/952557025' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "774aea07-241e-4f94-aa94-5e8823cb5b48"}]': finished 2024-05-17T09:10:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:16 smithi019 ceph-mon[31620]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T09:10:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:16 smithi079 ceph-mon[35502]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:16 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/952557025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "774aea07-241e-4f94-aa94-5e8823cb5b48"}]: dispatch 2024-05-17T09:10:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:16 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/952557025' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "774aea07-241e-4f94-aa94-5e8823cb5b48"}]': finished 2024-05-17T09:10:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:16 smithi079 ceph-mon[35502]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T09:10:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:16 smithi069 ceph-mon[35081]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:16 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/952557025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "774aea07-241e-4f94-aa94-5e8823cb5b48"}]: dispatch 2024-05-17T09:10:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:16 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/952557025' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "774aea07-241e-4f94-aa94-5e8823cb5b48"}]': finished 2024-05-17T09:10:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:16 smithi069 ceph-mon[35081]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T09:10:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:17.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:17 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4151550126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:10:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:17 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4151550126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:10:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:17 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4151550126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:10:18.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:18 smithi019 ceph-mon[31620]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:18 smithi079 ceph-mon[35502]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:18 smithi069 ceph-mon[35081]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:20.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:20 smithi019 ceph-mon[31620]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:20 smithi079 ceph-mon[35502]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:20 smithi069 ceph-mon[35081]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:22 smithi079 ceph-mon[35502]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:22.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:22 smithi019 ceph-mon[31620]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:22 smithi069 ceph-mon[35081]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:24.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:24 smithi019 ceph-mon[31620]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:24.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T09:10:24.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:24 smithi079 ceph-mon[35502]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T09:10:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:24 smithi069 ceph-mon[35081]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T09:10:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:25.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:25 smithi019 ceph-mon[31620]: Deploying daemon osd.3 on smithi019 2024-05-17T09:10:25.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:25 smithi079 ceph-mon[35502]: Deploying daemon osd.3 on smithi019 2024-05-17T09:10:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:25 smithi069 ceph-mon[35081]: Deploying daemon osd.3 on smithi019 2024-05-17T09:10:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:26 smithi079 ceph-mon[35502]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:26.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:26 smithi019 ceph-mon[31620]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:26 smithi069 ceph-mon[35081]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:28.482 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:28 smithi069 ceph-mon[35081]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:28 smithi079 ceph-mon[35502]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:28.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:28 smithi019 ceph-mon[31620]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:30 smithi069 ceph-mon[35081]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:30.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:30 smithi019 ceph-mon[31620]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:30.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:30.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:30.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:30 smithi079 ceph-mon[35502]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:32 smithi069 ceph-mon[35081]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:32.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:32 smithi019 ceph-mon[31620]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:32 smithi079 ceph-mon[35502]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:32.611 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-05-17T09:10:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.610+0000 7f16a37fe700 1 -- 172.21.15.19:0/1691452510 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f16b4063bf0 con 0x7f169c0b27f0 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f169c0b27f0 msgr2=0x7f169c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f169c0b27f0 0x7f169c0b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f16b419b8c0 tx=0x7f16a8009260 comp rx=0 tx=0).stop 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 msgr2=0x7f16b419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b419a810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16a40098e0 tx=0x7f16a400b9e0 comp rx=0 tx=0).stop 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 shutdown_connections 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16b4105e00 0x7f16b41a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b4104330 0x7f16b419a810 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f169c0b27f0 0x7f169c0b4d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:32.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 --2- 172.21.15.19:0/1691452510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16b4105460 0x7f16b419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:32.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.613+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 >> 172.21.15.19:0/1691452510 conn(0x7f16b40ff9b0 msgr2=0x7f16b4108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:10:32.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.614+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 shutdown_connections 2024-05-17T09:10:32.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:10:32.614+0000 7f16b8c69700 1 -- 172.21.15.19:0/1691452510 wait complete. 2024-05-17T09:10:33.332 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.3.service 2024-05-17T09:10:33.334 INFO:tasks.cephadm:Deploying osd.4 on smithi069 with /dev/vg_nvme/lv_3... 2024-05-17T09:10:33.335 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T09:10:33.584 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:10:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:33 smithi069 ceph-mon[35081]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:33 smithi069 ceph-mon[35081]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T09:10:33.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:33 smithi019 ceph-mon[31620]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:33.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:33 smithi019 ceph-mon[31620]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T09:10:33.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:33.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:34.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:34.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:34.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:33 smithi079 ceph-mon[35502]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:34.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:33 smithi079 ceph-mon[35502]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T09:10:34.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:34.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:34.645 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:34 smithi069 ceph-mon[35081]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T09:10:34.645 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:34 smithi069 ceph-mon[35081]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T09:10:34.645 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:34 smithi069 ceph-mon[35081]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:10:34.645 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:34.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:34 smithi019 ceph-mon[31620]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T09:10:34.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:34 smithi019 ceph-mon[31620]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T09:10:34.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:34 smithi019 ceph-mon[31620]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:10:34.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:34.943 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:10:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3[50171]: 2024-05-17T09:10:34.586+0000 7fac78aed700 -1 osd.3 0 waiting for initial osdmap 2024-05-17T09:10:34.943 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:10:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3[50171]: 2024-05-17T09:10:34.592+0000 7fac748a2700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:10:35.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:34 smithi079 ceph-mon[35502]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T09:10:35.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:34 smithi079 ceph-mon[35502]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T09:10:35.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:34 smithi079 ceph-mon[35502]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:10:35.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:35.484 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T09:10:35.484 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T09:10:35.484 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-17T09:10:35.484 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-17T09:10:35.484 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0162567 s, 645 MB/s 2024-05-17T09:10:35.484 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-17T09:10:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:35 smithi019 ceph-mon[31620]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:10:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:35 smithi019 ceph-mon[31620]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T09:10:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:35 smithi019 ceph-mon[31620]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:35.893 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:35 smithi069 ceph-mon[35081]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:10:35.893 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:35 smithi069 ceph-mon[35081]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T09:10:35.893 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:35.893 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:35.893 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:35 smithi069 ceph-mon[35081]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:36.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:35 smithi079 ceph-mon[35502]: from='osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:10:36.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:35 smithi079 ceph-mon[35502]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T09:10:36.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:36.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:36.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:35 smithi079 ceph-mon[35502]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-17T09:10:36.179 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-05-17T09:10:36.417 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:10:36.712 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:36 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:10:36.712 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:36 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:10:36.712 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:36 smithi069 ceph-mon[35081]: osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] boot 2024-05-17T09:10:36.712 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:36 smithi069 ceph-mon[35081]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T09:10:36.713 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:36 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:36.713 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:36 smithi069 ceph-mon[35081]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T09:10:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:36 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:10:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:36 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:10:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:36 smithi019 ceph-mon[31620]: osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] boot 2024-05-17T09:10:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:36 smithi019 ceph-mon[31620]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T09:10:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:36.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:36 smithi019 ceph-mon[31620]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T09:10:37.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:36 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:10:37.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:36 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:10:37.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:36 smithi079 ceph-mon[35502]: osd.3 [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] boot 2024-05-17T09:10:37.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:36 smithi079 ceph-mon[35502]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T09:10:37.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T09:10:37.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:36 smithi079 ceph-mon[35502]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T09:10:38.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 -- 172.21.15.69:0/2746346192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c1059a0 msgr2=0x7efc0c105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 --2- 172.21.15.69:0/2746346192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c1059a0 0x7efc0c105db0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efc08009990 tx=0x7efc08009ca0 comp rx=0 tx=0).stop 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 -- 172.21.15.69:0/2746346192 shutdown_connections 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 --2- 172.21.15.69:0/2746346192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c1049d0 0x7efc0c10c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 --2- 172.21.15.69:0/2746346192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c1059a0 0x7efc0c105db0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 --2- 172.21.15.69:0/2746346192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc0c103f50 0x7efc0c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 -- 172.21.15.69:0/2746346192 >> 172.21.15.69:0/2746346192 conn(0x7efc0c0ff9c0 msgr2=0x7efc0c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:10:38.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 -- 172.21.15.69:0/2746346192 shutdown_connections 2024-05-17T09:10:38.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.018+0000 7efc13575700 1 -- 172.21.15.69:0/2746346192 wait complete. 2024-05-17T09:10:38.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 Processor -- start 2024-05-17T09:10:38.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 -- start start 2024-05-17T09:10:38.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 0x7efc0c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:38.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c1049d0 0x7efc0c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc0c1059a0 0x7efc0c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c10fd70 con 0x7efc0c1049d0 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c10fbf0 con 0x7efc0c103f50 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc13575700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c10fef0 con 0x7efc0c1059a0 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 0x7efc0c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 0x7efc0c19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38912/0 (socket says 172.21.15.69:38912) 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 -- 172.21.15.69:0/495058439 learned_addr learned my addr 172.21.15.69:0/495058439 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc10b10700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c1049d0 0x7efc0c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11b12700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc0c1059a0 0x7efc0c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 -- 172.21.15.69:0/495058439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc0c1059a0 msgr2=0x7efc0c1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc0c1059a0 0x7efc0c1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 -- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c1049d0 msgr2=0x7efc0c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c1049d0 0x7efc0c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.019+0000 7efc11311700 1 -- 172.21.15.69:0/495058439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c1a2480 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.020+0000 7efc11311700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 0x7efc0c19a920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc080047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.020+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08027070 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.020+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc08003f60 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.020+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c1a2710 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.020+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08022cb0 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.020+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c1a2bc0 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.021+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c0512b0 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.023+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efc0802a070 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.024+0000 7efc027fc700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efbf80b2900 0x7efbf80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.024+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7efc080e2510 con 0x7efc0c103f50 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.024+0000 7efc10b10700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efbf80b2900 0x7efbf80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:10:38.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.025+0000 7efc10b10700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efbf80b2900 0x7efbf80b4e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005950 tx=0x7efbfc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:10:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.028+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc08031020 con 0x7efc0c103f50 2024-05-17T09:10:38.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:38.237+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efc0c02e910 con 0x7efbf80b2900 2024-05-17T09:10:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi019 2024-05-17T09:10:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:38 smithi069 ceph-mon[35081]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi019 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-17T09:10:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:38 smithi019 ceph-mon[31620]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T09:10:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi019 2024-05-17T09:10:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:10:38.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:38.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:38 smithi079 ceph-mon[35502]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T09:10:39.122 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:39 smithi069 ceph-mon[35081]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:10:39.122 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:10:39.122 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:10:39.122 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:39.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:39 smithi019 ceph-mon[31620]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:10:39.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:10:39.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:10:39.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:39.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:39 smithi079 ceph-mon[35502]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:10:39.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:10:39.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:10:39.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:40 smithi069 ceph-mon[35081]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:40.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:40 smithi019 ceph-mon[31620]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:40 smithi079 ceph-mon[35502]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:41.337 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:41 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/1106978094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]: dispatch 2024-05-17T09:10:41.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:41 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]: dispatch 2024-05-17T09:10:41.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:41 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]': finished 2024-05-17T09:10:41.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:41 smithi069 ceph-mon[35081]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T09:10:41.338 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:10:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:41 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/1106978094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]: dispatch 2024-05-17T09:10:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:41 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]: dispatch 2024-05-17T09:10:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:41 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]': finished 2024-05-17T09:10:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:41 smithi019 ceph-mon[31620]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T09:10:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:10:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:41 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/1106978094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]: dispatch 2024-05-17T09:10:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:41 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]: dispatch 2024-05-17T09:10:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:41 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8760fee7-c08a-4a4e-bc91-754119cb9d3c"}]': finished 2024-05-17T09:10:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:41 smithi079 ceph-mon[35502]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T09:10:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:10:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:42 smithi069 ceph-mon[35081]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:42 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/584447017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:10:42.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:42 smithi019 ceph-mon[31620]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:42.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:42 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/584447017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:10:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:42 smithi079 ceph-mon[35502]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:42 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/584447017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:10:44.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:44 smithi069 ceph-mon[35081]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-17T09:10:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:44 smithi019 ceph-mon[31620]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-17T09:10:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:44 smithi079 ceph-mon[35502]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-17T09:10:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:46 smithi069 ceph-mon[35081]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:10:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:46 smithi019 ceph-mon[31620]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:10:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:46 smithi079 ceph-mon[35502]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:10:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:48 smithi069 ceph-mon[35081]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-17T09:10:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:48 smithi019 ceph-mon[31620]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-17T09:10:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:48 smithi079 ceph-mon[35502]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-17T09:10:49.275 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T09:10:49.275 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:49.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T09:10:49.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T09:10:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:10:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:50 smithi069 ceph-mon[35081]: Deploying daemon osd.4 on smithi069 2024-05-17T09:10:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:50 smithi069 ceph-mon[35081]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:10:50.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:50 smithi019 ceph-mon[31620]: Deploying daemon osd.4 on smithi069 2024-05-17T09:10:50.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:50 smithi019 ceph-mon[31620]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:10:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:50 smithi079 ceph-mon[35502]: Deploying daemon osd.4 on smithi069 2024-05-17T09:10:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:50 smithi079 ceph-mon[35502]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:10:52.247 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:52 smithi069 ceph-mon[35081]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:10:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:52 smithi019 ceph-mon[31620]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:10:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:52 smithi079 ceph-mon[35502]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:10:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:54 smithi069 ceph-mon[35081]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-17T09:10:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:54 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:54 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:54 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:54 smithi019 ceph-mon[31620]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-17T09:10:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:54 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:54 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:54 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:54 smithi079 ceph-mon[35502]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-17T09:10:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:54 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:54 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:10:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:54 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:56.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:56 smithi069 ceph-mon[35081]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:56.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:56 smithi019 ceph-mon[31620]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:56 smithi079 ceph-mon[35502]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.362+0000 7efc027fc700 1 -- 172.21.15.69:0/495058439 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc0c02e910 con 0x7efbf80b2900 2024-05-17T09:10:57.362 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 4 on host 'smithi069' 2024-05-17T09:10:57.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efbf80b2900 msgr2=0x7efbf80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efbf80b2900 0x7efbf80b4e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005950 tx=0x7efbfc00a300 comp rx=0 tx=0).stop 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 msgr2=0x7efc0c19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 0x7efc0c19a920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc080047b0 comp rx=0 tx=0).stop 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 shutdown_connections 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efbf80b2900 0x7efbf80b4e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc0c1059a0 0x7efc0c1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc0c103f50 0x7efc0c19a920 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 --2- 172.21.15.69:0/495058439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c1049d0 0x7efc0c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 >> 172.21.15.69:0/495058439 conn(0x7efc0c0ff9c0 msgr2=0x7efc0c108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 shutdown_connections 2024-05-17T09:10:57.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:10:57.364+0000 7efc13575700 1 -- 172.21.15.69:0/495058439 wait complete. 2024-05-17T09:10:57.975 DEBUG:teuthology.orchestra.run.smithi069:osd.4> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.4.service 2024-05-17T09:10:57.978 INFO:tasks.cephadm:Deploying osd.5 on smithi079 with /dev/vg_nvme/lv_3... 2024-05-17T09:10:57.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T09:10:58.225 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:10:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:58 smithi079 ceph-mon[35502]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:58 smithi079 ceph-mon[35502]: from='osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T09:10:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:58 smithi079 ceph-mon[35502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T09:10:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:58 smithi069 ceph-mon[35081]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:58 smithi069 ceph-mon[35081]: from='osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T09:10:58.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:58 smithi069 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T09:10:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:58 smithi019 ceph-mon[31620]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:10:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:58 smithi019 ceph-mon[31620]: from='osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T09:10:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:58 smithi019 ceph-mon[31620]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T09:10:59.377 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:59.377 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:59.377 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T09:10:59.378 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T09:10:59.378 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:10:59.378 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: from='osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:10:59.378 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:10:59 smithi069 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: from='osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:10:59.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:10:59 smithi079 ceph-mon[35502]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:10:59.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:59.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:10:59.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T09:10:59.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T09:10:59.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:10:59.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: from='osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:10:59.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:10:59 smithi019 ceph-mon[31620]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:10:59.680 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:10:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:10:59.373+0000 7f9417885700 -1 osd.4 0 waiting for initial osdmap 2024-05-17T09:10:59.681 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:10:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:10:59.379+0000 7f941363a700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:11:00.129 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T09:11:00.129 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T09:11:00.130 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-17T09:11:00.130 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-17T09:11:00.130 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121898 s, 860 MB/s 2024-05-17T09:11:00.130 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-17T09:11:00.244 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:00 smithi069 ceph-mon[35081]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:11:00.244 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:00 smithi069 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:11:00.244 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:00 smithi069 ceph-mon[35081]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T09:11:00.245 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:00 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:00.245 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:00 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:00 smithi019 ceph-mon[31620]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:11:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:00 smithi019 ceph-mon[31620]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:11:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:00 smithi019 ceph-mon[31620]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T09:11:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:00 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:00 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:00.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:00 smithi079 ceph-mon[35502]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:11:00.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:00 smithi079 ceph-mon[35502]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:11:00.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:00 smithi079 ceph-mon[35502]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T09:11:00.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:00 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:00.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:00 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:00.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-05-17T09:11:00.957 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:11:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:01 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:11:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:01 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:11:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:01 smithi069 ceph-mon[35081]: osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] boot 2024-05-17T09:11:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:01 smithi069 ceph-mon[35081]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T09:11:01.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:01.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:01 smithi069 ceph-mon[35081]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:11:01.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:01 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:11:01.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:01 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:11:01.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:01 smithi019 ceph-mon[31620]: osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] boot 2024-05-17T09:11:01.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:01 smithi019 ceph-mon[31620]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T09:11:01.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:01.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:01 smithi019 ceph-mon[31620]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:11:01.754 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:01 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:11:01.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:01 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:11:01.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:01 smithi079 ceph-mon[35502]: osd.4 [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] boot 2024-05-17T09:11:01.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:01 smithi079 ceph-mon[35502]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T09:11:01.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T09:11:01.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:01 smithi079 ceph-mon[35502]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-17T09:11:02.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 -- 172.21.15.79:0/718512855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181059a0 msgr2=0x7fd418105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 --2- 172.21.15.79:0/718512855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181059a0 0x7fd418105db0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd414009990 tx=0x7fd414009ca0 comp rx=0 tx=0).stop 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 -- 172.21.15.79:0/718512855 shutdown_connections 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 --2- 172.21.15.79:0/718512855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181059a0 0x7fd418105db0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 --2- 172.21.15.79:0/718512855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4181049d0 0x7fd41810c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 --2- 172.21.15.79:0/718512855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd418103f50 0x7fd418104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 -- 172.21.15.79:0/718512855 >> 172.21.15.79:0/718512855 conn(0x7fd4180ff9c0 msgr2=0x7fd418101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 -- 172.21.15.79:0/718512855 shutdown_connections 2024-05-17T09:11:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.487+0000 7fd41f49c700 1 -- 172.21.15.79:0/718512855 wait complete. 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 Processor -- start 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 -- start start 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd418103f50 0x7fd41819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 0x7fd41819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4181059a0 0x7fd4181a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810fd90 con 0x7fd418103f50 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810fc10 con 0x7fd4181059a0 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41f49c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810ff10 con 0x7fd4181049d0 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 0x7fd41819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 0x7fd41819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51522/0 (socket says 172.21.15.79:51522) 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 -- 172.21.15.79:0/3169273863 learned_addr learned my addr 172.21.15.79:0/3169273863 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41d238700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd418103f50 0x7fd41819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41da39700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4181059a0 0x7fd4181a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 -- 172.21.15.79:0/3169273863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4181059a0 msgr2=0x7fd4181a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4181059a0 0x7fd4181a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 -- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd418103f50 msgr2=0x7fd41819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd418103f50 0x7fd41819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.488+0000 7fd41ca37700 1 -- 172.21.15.79:0/3169273863 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181a2460 con 0x7fd4181049d0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.489+0000 7fd41ca37700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 0x7fd41819aed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d980 tx=0x7fd40800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.489+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408017070 con 0x7fd4181049d0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.489+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd408005020 con 0x7fd4181049d0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.489+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4181a2750 con 0x7fd4181049d0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.489+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40800f980 con 0x7fd4181049d0 2024-05-17T09:11:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.489+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a2c40 con 0x7fd4181049d0 2024-05-17T09:11:02.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.490+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4180512b0 con 0x7fd4181049d0 2024-05-17T09:11:02.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.491+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd408004a30 con 0x7fd4181049d0 2024-05-17T09:11:02.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.492+0000 7fd40e7fc700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4040b2880 0x7fd4040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:02.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.492+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fd4080e13d0 con 0x7fd4181049d0 2024-05-17T09:11:02.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.492+0000 7fd41d238700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4040b2880 0x7fd4040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:02.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.493+0000 7fd41d238700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4040b2880 0x7fd4040b4d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd414006010 tx=0x7fd41401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:11:02.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.496+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd408020020 con 0x7fd4181049d0 2024-05-17T09:11:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T09:11:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi069 2024-05-17T09:11:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T09:11:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi069 2024-05-17T09:11:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:02.704+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd4180008d0 con 0x7fd4040b2880 2024-05-17T09:11:02.705 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi069 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 8544M 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:02.706 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:02.707 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:02.707 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:03.602 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:03 smithi079 ceph-mon[35502]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:03.602 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:03.602 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:03.602 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:03.602 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:03 smithi079 ceph-mon[35502]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:03.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:03 smithi069 ceph-mon[35081]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:03.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:03.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:03.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:03.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:03 smithi069 ceph-mon[35081]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:03.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:03 smithi019 ceph-mon[31620]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:03.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:03.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:03.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:03.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:03 smithi019 ceph-mon[31620]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:05 smithi069 ceph-mon[35081]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:05 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/1748455969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]: dispatch 2024-05-17T09:11:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:05 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]: dispatch 2024-05-17T09:11:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:05 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]': finished 2024-05-17T09:11:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:05 smithi069 ceph-mon[35081]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T09:11:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:05 smithi019 ceph-mon[31620]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:05 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/1748455969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]: dispatch 2024-05-17T09:11:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:05 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]: dispatch 2024-05-17T09:11:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:05 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]': finished 2024-05-17T09:11:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:05 smithi019 ceph-mon[31620]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T09:11:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:05 smithi079 ceph-mon[35502]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:05 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/1748455969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]: dispatch 2024-05-17T09:11:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:05 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]: dispatch 2024-05-17T09:11:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:05 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7609df91-0a4b-4f8f-9a41-9d644e9a3ee3"}]': finished 2024-05-17T09:11:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:05 smithi079 ceph-mon[35502]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T09:11:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:07.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:06 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/1465763025' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:11:07.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/1465763025' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:11:07.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:06 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/1465763025' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:11:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:07 smithi069 ceph-mon[35081]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:08.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:07 smithi019 ceph-mon[31620]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:07 smithi079 ceph-mon[35502]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:10.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:09 smithi069 ceph-mon[35081]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:10.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:09 smithi019 ceph-mon[31620]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:10.203 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:09 smithi079 ceph-mon[35502]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:11.984 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:11 smithi079 ceph-mon[35502]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:11 smithi069 ceph-mon[35081]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:11 smithi019 ceph-mon[31620]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:13 smithi079 ceph-mon[35502]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T09:11:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:13 smithi079 ceph-mon[35502]: Deploying daemon osd.5 on smithi079 2024-05-17T09:11:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:13 smithi069 ceph-mon[35081]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T09:11:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:13 smithi069 ceph-mon[35081]: Deploying daemon osd.5 on smithi079 2024-05-17T09:11:14.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:13 smithi019 ceph-mon[31620]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:14.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T09:11:14.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:14.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:13 smithi019 ceph-mon[31620]: Deploying daemon osd.5 on smithi079 2024-05-17T09:11:16.108 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:15 smithi079 ceph-mon[35502]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:16.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:15 smithi069 ceph-mon[35081]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:16.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:15 smithi019 ceph-mon[31620]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:18.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:17 smithi079 ceph-mon[35502]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:18.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:17 smithi069 ceph-mon[35081]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:17 smithi019 ceph-mon[31620]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:20.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:19 smithi079 ceph-mon[35502]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:20.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:20.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:20.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:19 smithi069 ceph-mon[35081]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:19 smithi019 ceph-mon[31620]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:22.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:21 smithi069 ceph-mon[35081]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:22.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:22.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:22.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:21 smithi019 ceph-mon[31620]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:21 smithi079 ceph-mon[35502]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:22.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.732+0000 7fd40e7fc700 1 -- 172.21.15.79:0/3169273863 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4180008d0 con 0x7fd4040b2880 2024-05-17T09:11:22.732 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 5 on host 'smithi079' 2024-05-17T09:11:22.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4040b2880 msgr2=0x7fd4040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4040b2880 0x7fd4040b4d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd414006010 tx=0x7fd41401a040 comp rx=0 tx=0).stop 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 msgr2=0x7fd41819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 0x7fd41819aed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d980 tx=0x7fd40800dc90 comp rx=0 tx=0).stop 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 shutdown_connections 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4181059a0 0x7fd4181a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd418103f50 0x7fd41819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4040b2880 0x7fd4040b4d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 --2- 172.21.15.79:0/3169273863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4181049d0 0x7fd41819aed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 >> 172.21.15.79:0/3169273863 conn(0x7fd4180ff9c0 msgr2=0x7fd4181086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 shutdown_connections 2024-05-17T09:11:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:11:22.735+0000 7fd41f49c700 1 -- 172.21.15.79:0/3169273863 wait complete. 2024-05-17T09:11:23.413 DEBUG:teuthology.orchestra.run.smithi079:osd.5> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.5.service 2024-05-17T09:11:23.415 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-05-17T09:11:23.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T09:11:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:11:23.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:23.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:23.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:23.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:23 smithi019 ceph-mon[31620]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:23.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:23 smithi019 ceph-mon[31620]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T09:11:24.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:24.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:24.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:24.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:23 smithi079 ceph-mon[35502]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:24.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:23 smithi079 ceph-mon[35502]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T09:11:24.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:24.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:24.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:24.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:23 smithi069 ceph-mon[35081]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:24.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:23 smithi069 ceph-mon[35081]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T09:11:24.907 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:24 smithi079 ceph-mon[35502]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T09:11:24.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:24 smithi079 ceph-mon[35502]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T09:11:24.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:24 smithi079 ceph-mon[35502]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:11:24.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:24.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:24 smithi019 ceph-mon[31620]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T09:11:24.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:24 smithi019 ceph-mon[31620]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T09:11:24.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:24 smithi019 ceph-mon[31620]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:11:24.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:25.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:24 smithi069 ceph-mon[35081]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T09:11:25.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:24 smithi069 ceph-mon[35081]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T09:11:25.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:24 smithi069 ceph-mon[35081]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:11:25.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:25.522 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:11:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:11:25.250+0000 7f818d5f3700 -1 osd.5 0 waiting for initial osdmap 2024-05-17T09:11:25.522 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:11:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:11:25.257+0000 7f81853a0700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:11:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T09:11:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T09:11:25.672 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-17T09:11:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-17T09:11:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123554 s, 849 MB/s 2024-05-17T09:11:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-17T09:11:25.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:25 smithi019 ceph-mon[31620]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:11:25.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:25 smithi019 ceph-mon[31620]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T09:11:25.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:25.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:25.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:25 smithi019 ceph-mon[31620]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:25.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:25 smithi019 ceph-mon[31620]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' 2024-05-17T09:11:26.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:25 smithi079 ceph-mon[35502]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:11:26.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:25 smithi079 ceph-mon[35502]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T09:11:26.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:25 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:26.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:25 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:26.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:25 smithi079 ceph-mon[35502]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:26.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:25 smithi079 ceph-mon[35502]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' 2024-05-17T09:11:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:25 smithi069 ceph-mon[35081]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:11:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:25 smithi069 ceph-mon[35081]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T09:11:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:25 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:25 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:25 smithi069 ceph-mon[35081]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-17T09:11:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:25 smithi069 ceph-mon[35081]: from='osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663]' entity='osd.5' 2024-05-17T09:11:26.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-05-17T09:11:26.558 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:11:26.903 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:11:26.903 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:11:26.903 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] boot 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:11:27.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:11:27.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:27.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] boot 2024-05-17T09:11:27.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T09:11:27.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:27.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: osd.5 [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] boot 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:27.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:27.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:27.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:27.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:27.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:27 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi079 2024-05-17T09:11:27.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:27 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 8544M 2024-05-17T09:11:27.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:27 smithi019 ceph-mon[31620]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T09:11:27.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:27 smithi019 ceph-mon[31620]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:28.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:27 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi079 2024-05-17T09:11:28.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:27 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 8544M 2024-05-17T09:11:28.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:27 smithi079 ceph-mon[35502]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T09:11:28.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:27 smithi079 ceph-mon[35502]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:28.082 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:27 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi079 2024-05-17T09:11:28.082 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:27 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 8544M 2024-05-17T09:11:28.082 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:27 smithi069 ceph-mon[35081]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T09:11:28.082 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:27 smithi069 ceph-mon[35081]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:28.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 -- 172.21.15.19:0/24092804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf4102690 msgr2=0x7fabf4102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:28.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 --2- 172.21.15.19:0/24092804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf4102690 0x7fabf4102aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009990 tx=0x7fabe4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 -- 172.21.15.19:0/24092804 shutdown_connections 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 --2- 172.21.15.19:0/24092804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf410a170 0x7fabf410c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 --2- 172.21.15.19:0/24092804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf4102690 0x7fabf4102aa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 --2- 172.21.15.19:0/24092804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabf4107810 0x7fabf4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 -- 172.21.15.19:0/24092804 >> 172.21.15.19:0/24092804 conn(0x7fabf40fdff0 msgr2=0x7fabf41004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 -- 172.21.15.19:0/24092804 shutdown_connections 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.205+0000 7fabf9332700 1 -- 172.21.15.19:0/24092804 wait complete. 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 Processor -- start 2024-05-17T09:11:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 -- start start 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabf4102690 0x7fabf419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf4107810 0x7fabf419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 0x7fabf41a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabf410fc40 con 0x7fabf410a170 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabf410fac0 con 0x7fabf4107810 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.206+0000 7fabf9332700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fabf410fdc0 con 0x7fabf4102690 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 0x7fabf41a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 0x7fabf41a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45764/0 (socket says 172.21.15.19:45764) 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 -- 172.21.15.19:0/372366070 learned_addr learned my addr 172.21.15.19:0/372366070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf27fc700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf4107810 0x7fabf419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf2ffd700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabf4102690 0x7fabf419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 -- 172.21.15.19:0/372366070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabf4102690 msgr2=0x7fabf419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabf4102690 0x7fabf419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 -- 172.21.15.19:0/372366070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf4107810 msgr2=0x7fabf419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf4107810 0x7fabf419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 -- 172.21.15.19:0/372366070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41a23a0 con 0x7fabf410a170 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf37fe700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 0x7fabf41a1d20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabec007970 tx=0x7fabec007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec00d5e0 con 0x7fabf410a170 2024-05-17T09:11:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41a2690 con 0x7fabf410a170 2024-05-17T09:11:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabec00dc10 con 0x7fabf410a170 2024-05-17T09:11:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.207+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec013cf0 con 0x7fabf410a170 2024-05-17T09:11:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.208+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41a2bd0 con 0x7fabf410a170 2024-05-17T09:11:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.209+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf40512b0 con 0x7fabf410a170 2024-05-17T09:11:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.210+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fabec01a020 con 0x7fabf410a170 2024-05-17T09:11:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.211+0000 7fabdbfff700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabdc0b2880 0x7fabdc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.211+0000 7fabf2ffd700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabdc0b2880 0x7fabdc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.211+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fabec0e1240 con 0x7fabf410a170 2024-05-17T09:11:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.211+0000 7fabf2ffd700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabdc0b2880 0x7fabdc0b4d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b5c0 tx=0x7fabe401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:11:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.215+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabec010070 con 0x7fabf410a170 2024-05-17T09:11:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:28.440+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fabf4063bf0 con 0x7fabdc0b2880 2024-05-17T09:11:29.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:28 smithi079 ceph-mon[35502]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T09:11:29.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:29.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:29.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:29.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:28 smithi019 ceph-mon[31620]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T09:11:29.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:29.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:29.079 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:29.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:28 smithi069 ceph-mon[35081]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T09:11:29.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:29.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:29.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:30.017 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:29 smithi019 ceph-mon[31620]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:30.018 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:29 smithi019 ceph-mon[31620]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:30.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:29 smithi079 ceph-mon[35502]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:30.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:29 smithi079 ceph-mon[35502]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:30.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:29 smithi069 ceph-mon[35081]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:30.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:29 smithi069 ceph-mon[35081]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:31.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:31 smithi019 ceph-mon[31620]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:31.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/367702017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37a6a6d7-3e20-4503-afce-96f03af0e774"}]: dispatch 2024-05-17T09:11:31.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/367702017' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37a6a6d7-3e20-4503-afce-96f03af0e774"}]': finished 2024-05-17T09:11:31.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:31 smithi019 ceph-mon[31620]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T09:11:31.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:32.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:31 smithi069 ceph-mon[35081]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:32.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/367702017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37a6a6d7-3e20-4503-afce-96f03af0e774"}]: dispatch 2024-05-17T09:11:32.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/367702017' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37a6a6d7-3e20-4503-afce-96f03af0e774"}]': finished 2024-05-17T09:11:32.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:31 smithi069 ceph-mon[35081]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T09:11:32.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:32.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:31 smithi079 ceph-mon[35502]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:32.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/367702017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37a6a6d7-3e20-4503-afce-96f03af0e774"}]: dispatch 2024-05-17T09:11:32.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/367702017' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37a6a6d7-3e20-4503-afce-96f03af0e774"}]': finished 2024-05-17T09:11:32.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:31 smithi079 ceph-mon[35502]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T09:11:32.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:33.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:32 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/406284793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:11:33.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:32 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/406284793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:11:33.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:32 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/406284793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:11:34.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:33 smithi069 ceph-mon[35081]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-17T09:11:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:33 smithi019 ceph-mon[31620]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-17T09:11:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:33 smithi079 ceph-mon[35502]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-17T09:11:36.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:35 smithi069 ceph-mon[35081]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:11:36.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:35 smithi019 ceph-mon[31620]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:11:36.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:35 smithi079 ceph-mon[35502]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:11:38.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:37 smithi019 ceph-mon[31620]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-17T09:11:38.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:37 smithi069 ceph-mon[35081]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-17T09:11:38.275 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:37 smithi079 ceph-mon[35502]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-17T09:11:39.883 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:39 smithi079 ceph-mon[35502]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:11:39.883 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T09:11:39.883 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:39 smithi069 ceph-mon[35081]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:11:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T09:11:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:40.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:39 smithi069 ceph-mon[35081]: Deploying daemon osd.6 on smithi019 2024-05-17T09:11:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:39 smithi019 ceph-mon[31620]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:11:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T09:11:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:39 smithi019 ceph-mon[31620]: Deploying daemon osd.6 on smithi019 2024-05-17T09:11:40.198 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:39 smithi079 ceph-mon[35502]: Deploying daemon osd.6 on smithi019 2024-05-17T09:11:41.908 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:41 smithi019 ceph-mon[31620]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:11:42.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:41 smithi069 ceph-mon[35081]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:11:42.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:41 smithi079 ceph-mon[35502]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:11:44.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:43 smithi019 ceph-mon[31620]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-17T09:11:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:43 smithi069 ceph-mon[35081]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-17T09:11:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:43 smithi079 ceph-mon[35502]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-17T09:11:46.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:45 smithi019 ceph-mon[31620]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:46.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:46.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:46.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:45 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:45 smithi069 ceph-mon[35081]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:46.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:45 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:45 smithi079 ceph-mon[35502]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:45 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:48.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:47 smithi019 ceph-mon[31620]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:48.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:47 smithi069 ceph-mon[35081]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:48.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:47 smithi079 ceph-mon[35502]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:49.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.659+0000 7fabdbfff700 1 -- 172.21.15.19:0/372366070 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fabf4063bf0 con 0x7fabdc0b2880 2024-05-17T09:11:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-05-17T09:11:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabdc0b2880 msgr2=0x7fabdc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabdc0b2880 0x7fabdc0b4d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b5c0 tx=0x7fabe401a040 comp rx=0 tx=0).stop 2024-05-17T09:11:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 msgr2=0x7fabf41a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 0x7fabf41a1d20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabec007970 tx=0x7fabec007c80 comp rx=0 tx=0).stop 2024-05-17T09:11:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 shutdown_connections 2024-05-17T09:11:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabdc0b2880 0x7fabdc0b4d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:49.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabf4102690 0x7fabf419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:49.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf410a170 0x7fabf41a1d20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:49.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.661+0000 7fabf9332700 1 --2- 172.21.15.19:0/372366070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf4107810 0x7fabf419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:49.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.662+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 >> 172.21.15.19:0/372366070 conn(0x7fabf40fdff0 msgr2=0x7fabf41087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:11:49.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.662+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 shutdown_connections 2024-05-17T09:11:49.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:11:49.662+0000 7fabf9332700 1 -- 172.21.15.19:0/372366070 wait complete. 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T09:11:49.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:49.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:49.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:50.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:50.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:50.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T09:11:50.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:50.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:50.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:50.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:50.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T09:11:50.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:11:50.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:50.416 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.6.service 2024-05-17T09:11:50.419 INFO:tasks.cephadm:Deploying osd.7 on smithi069 with /dev/vg_nvme/lv_2... 2024-05-17T09:11:50.420 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T09:11:50.667 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:11:50.996 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:50 smithi069 ceph-mon[35081]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T09:11:50.996 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:50 smithi069 ceph-mon[35081]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T09:11:50.997 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:50 smithi069 ceph-mon[35081]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:11:50.997 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:51.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:50 smithi079 ceph-mon[35502]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T09:11:51.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:50 smithi079 ceph-mon[35502]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T09:11:51.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:50 smithi079 ceph-mon[35502]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:11:51.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:50 smithi019 ceph-mon[31620]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T09:11:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:50 smithi019 ceph-mon[31620]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T09:11:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:50 smithi019 ceph-mon[31620]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:11:51.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:51.193 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:11:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:11:50.738+0000 7f179dbb5700 -1 osd.6 0 waiting for initial osdmap 2024-05-17T09:11:51.193 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:11:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:11:50.752+0000 7f179996a700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:11:51.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:51 smithi019 ceph-mon[31620]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:11:51.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:51 smithi019 ceph-mon[31620]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T09:11:51.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:51.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:51.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:51 smithi019 ceph-mon[31620]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:52.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:51 smithi079 ceph-mon[35502]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:11:52.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:51 smithi079 ceph-mon[35502]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T09:11:52.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:52.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:52.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:51 smithi079 ceph-mon[35502]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:52.043 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:51 smithi069 ceph-mon[35081]: from='osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:11:52.043 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:51 smithi069 ceph-mon[35081]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T09:11:52.043 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:52.043 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:52.043 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:51 smithi069 ceph-mon[35081]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-05-17T09:11:52.707 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T09:11:52.707 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T09:11:52.707 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-17T09:11:52.707 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-17T09:11:52.707 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139315 s, 753 MB/s 2024-05-17T09:11:52.707 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-17T09:11:53.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:52 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:11:53.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:52 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:11:53.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:53.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:52 smithi079 ceph-mon[35502]: osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] boot 2024-05-17T09:11:53.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:52 smithi079 ceph-mon[35502]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T09:11:53.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:52 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:53.049 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:52 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:11:53.049 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:52 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:11:53.049 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:53.049 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:52 smithi069 ceph-mon[35081]: osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] boot 2024-05-17T09:11:53.049 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:52 smithi069 ceph-mon[35081]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T09:11:53.049 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:52 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:53.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:52 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:11:53.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:52 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:11:53.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:53.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:52 smithi019 ceph-mon[31620]: osd.6 [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] boot 2024-05-17T09:11:53.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:52 smithi019 ceph-mon[31620]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T09:11:53.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:52 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T09:11:53.376 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-05-17T09:11:53.624 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:11:53.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T09:11:53.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:53.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:53.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:53.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:53.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:53.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:53.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:53.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:53.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:53.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:54.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T09:11:54.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:54.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:54.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:54.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:54.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:54.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:11:54.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:54.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:54.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:11:54.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:11:54.907 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:54 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi019 2024-05-17T09:11:54.907 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:54 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-17T09:11:55.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:54 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi019 2024-05-17T09:11:55.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:54 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-17T09:11:55.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:54 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi019 2024-05-17T09:11:55.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:54 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-17T09:11:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 -- 172.21.15.69:0/1095321899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0104330 msgr2=0x7f06c0104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 --2- 172.21.15.69:0/1095321899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0104330 0x7f06c0104740 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009990 tx=0x7f06b0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:11:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 -- 172.21.15.69:0/1095321899 shutdown_connections 2024-05-17T09:11:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 --2- 172.21.15.69:0/1095321899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06c0105e00 0x7f06c010c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 --2- 172.21.15.69:0/1095321899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0104330 0x7f06c0104740 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 --2- 172.21.15.69:0/1095321899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c0105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 -- 172.21.15.69:0/1095321899 >> 172.21.15.69:0/1095321899 conn(0x7f06c00ff9b0 msgr2=0x7f06c0101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:11:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 -- 172.21.15.69:0/1095321899 shutdown_connections 2024-05-17T09:11:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.450+0000 7f06c4aef700 1 -- 172.21.15.69:0/1095321899 wait complete. 2024-05-17T09:11:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 Processor -- start 2024-05-17T09:11:55.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 -- start start 2024-05-17T09:11:55.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06c0104330 0x7f06c019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:55.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:55.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 0x7f06c01a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010fc30 con 0x7f06c0105460 2024-05-17T09:11:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010fab0 con 0x7f06c0105e00 2024-05-17T09:11:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.451+0000 7f06c4aef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010fdb0 con 0x7f06c0104330 2024-05-17T09:11:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 0x7f06c01a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06b5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 0x7f06c01a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48150/0 (socket says 172.21.15.69:48150) 2024-05-17T09:11:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 -- 172.21.15.69:0/3914210153 learned_addr learned my addr 172.21.15.69:0/3914210153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-17T09:11:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06b5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.69:55436/0 (socket says 172.21.15.69:55436) 2024-05-17T09:11:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06be59c700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06c0104330 0x7f06c019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 -- 172.21.15.69:0/3914210153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06c0104330 msgr2=0x7f06c019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06c0104330 0x7f06c019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 -- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 msgr2=0x7f06c019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:11:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:11:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 -- 172.21.15.69:0/3914210153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c01a23a0 con 0x7f06c0105e00 2024-05-17T09:11:55.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06b5d9b700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:11:55.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06bed9d700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 0x7f06c01a1d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ada0 tx=0x7f06b800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:11:55.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8017020 con 0x7f06c0105e00 2024-05-17T09:11:55.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c01a2690 con 0x7f06c0105e00 2024-05-17T09:11:55.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06b800cd60 con 0x7f06c0105e00 2024-05-17T09:11:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8005280 con 0x7f06c0105e00 2024-05-17T09:11:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.452+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01a2bd0 con 0x7f06c0105e00 2024-05-17T09:11:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.453+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c00512b0 con 0x7f06c0105e00 2024-05-17T09:11:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.455+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f06b800c8d0 con 0x7f06c0105e00 2024-05-17T09:11:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.456+0000 7f06b7fff700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06ac0b2900 0x7f06ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:11:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.456+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f06b80e11f0 con 0x7f06c0105e00 2024-05-17T09:11:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.456+0000 7f06be59c700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06ac0b2900 0x7f06ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:11:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.456+0000 7f06be59c700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06ac0b2900 0x7f06ac0b4e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f06b0006010 tx=0x7f06b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:11:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.459+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06b8021020 con 0x7f06c0105e00 2024-05-17T09:11:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:11:55.670+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f06c0063bf0 con 0x7f06ac0b2900 2024-05-17T09:11:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:55 smithi069 ceph-mon[35081]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:55 smithi079 ceph-mon[35502]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:56.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:55 smithi019 ceph-mon[31620]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:56.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:11:56.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:11:56.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:11:56.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:56 smithi069 ceph-mon[35081]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:57.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:56 smithi079 ceph-mon[35502]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:57.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:56 smithi019 ceph-mon[31620]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:11:57.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:57 smithi069 ceph-mon[35081]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:57 smithi019 ceph-mon[31620]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:57 smithi079 ceph-mon[35502]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:11:59.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:58 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/2599895110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]: dispatch 2024-05-17T09:11:59.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:58 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]: dispatch 2024-05-17T09:11:59.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:58 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]': finished 2024-05-17T09:11:59.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:58 smithi079 ceph-mon[35502]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T09:11:59.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:11:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:58 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/2599895110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]: dispatch 2024-05-17T09:11:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:58 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]: dispatch 2024-05-17T09:11:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:58 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]': finished 2024-05-17T09:11:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:58 smithi069 ceph-mon[35081]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T09:11:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:11:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:58 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/2599895110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]: dispatch 2024-05-17T09:11:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:58 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]: dispatch 2024-05-17T09:11:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:58 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "829c24b4-70f4-4d39-8126-1766fe7920b4"}]': finished 2024-05-17T09:11:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:58 smithi019 ceph-mon[31620]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T09:11:59.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:00.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:59 smithi019 ceph-mon[31620]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:00.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:11:59 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/4157796819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:00.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:59 smithi069 ceph-mon[35081]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:00.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:11:59 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/4157796819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:00.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:59 smithi079 ceph-mon[35502]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:00.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:11:59 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/4157796819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:02.161 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:01 smithi069 ceph-mon[35081]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:02.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:01 smithi019 ceph-mon[31620]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:02.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:01 smithi079 ceph-mon[35502]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:04.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:03 smithi069 ceph-mon[35081]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:03 smithi019 ceph-mon[31620]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:04.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:03 smithi079 ceph-mon[35502]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:05 smithi069 ceph-mon[35081]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T09:12:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:05 smithi019 ceph-mon[31620]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T09:12:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:05 smithi079 ceph-mon[35502]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T09:12:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:06.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:06 smithi069 ceph-mon[35081]: Deploying daemon osd.7 on smithi069 2024-05-17T09:12:07.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:06 smithi019 ceph-mon[31620]: Deploying daemon osd.7 on smithi069 2024-05-17T09:12:07.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:06 smithi079 ceph-mon[35502]: Deploying daemon osd.7 on smithi069 2024-05-17T09:12:07.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:07 smithi069 ceph-mon[35081]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:08.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:07 smithi019 ceph-mon[31620]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:07 smithi079 ceph-mon[35502]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:09.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:09 smithi069 ceph-mon[35081]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:09 smithi019 ceph-mon[31620]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:09 smithi079 ceph-mon[35502]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:11 smithi069 ceph-mon[35081]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:11 smithi019 ceph-mon[31620]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:11 smithi079 ceph-mon[35502]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:12.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:12.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:14.174 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:13 smithi069 ceph-mon[35081]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:14.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:13 smithi019 ceph-mon[31620]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:13 smithi079 ceph-mon[35502]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:15.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.290+0000 7f06b7fff700 1 -- 172.21.15.69:0/3914210153 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f06c0063bf0 con 0x7f06ac0b2900 2024-05-17T09:12:15.290 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 7 on host 'smithi069' 2024-05-17T09:12:15.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06ac0b2900 msgr2=0x7f06ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:15.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06ac0b2900 0x7f06ac0b4e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f06b0006010 tx=0x7f06b001a040 comp rx=0 tx=0).stop 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 msgr2=0x7f06c01a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 0x7f06c01a1d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ada0 tx=0x7f06b800c330 comp rx=0 tx=0).stop 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 shutdown_connections 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06ac0b2900 0x7f06ac0b4e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06c0104330 0x7f06c019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06c0105e00 0x7f06c01a1d20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 --2- 172.21.15.69:0/3914210153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0105460 0x7f06c019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 >> 172.21.15.69:0/3914210153 conn(0x7f06c00ff9b0 msgr2=0x7f06c0108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.292+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 shutdown_connections 2024-05-17T09:12:15.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:12:15.293+0000 7f06c4aef700 1 -- 172.21.15.69:0/3914210153 wait complete. 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.608 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:15.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:15.936 DEBUG:teuthology.orchestra.run.smithi069:osd.7> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.7.service 2024-05-17T09:12:15.939 INFO:tasks.cephadm:Deploying osd.8 on smithi079 with /dev/vg_nvme/lv_2... 2024-05-17T09:12:15.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T09:12:16.187 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:12:16.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:16 smithi079 ceph-mon[35502]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:16.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:16 smithi079 ceph-mon[35502]: from='osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T09:12:16.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:16 smithi079 ceph-mon[35502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T09:12:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:16 smithi069 ceph-mon[35081]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:16 smithi069 ceph-mon[35081]: from='osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T09:12:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:16 smithi069 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T09:12:16.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:16 smithi019 ceph-mon[31620]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:16.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:16 smithi019 ceph-mon[31620]: from='osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T09:12:16.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:16 smithi019 ceph-mon[31620]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T09:12:17.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:17 smithi069 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T09:12:17.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:17 smithi069 ceph-mon[35081]: from='osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:12:17.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:17 smithi069 ceph-mon[35081]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T09:12:17.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:17.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:17 smithi069 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:12:17.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:17 smithi069 ceph-mon[35081]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:17.603 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:12:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:12:17.351+0000 7f829bc05700 -1 osd.7 0 waiting for initial osdmap 2024-05-17T09:12:17.603 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:12:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:12:17.360+0000 7f82979ba700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:12:17.609 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:17 smithi079 ceph-mon[35502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T09:12:17.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:17 smithi079 ceph-mon[35502]: from='osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:12:17.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:17 smithi079 ceph-mon[35502]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T09:12:17.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:17.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:17 smithi079 ceph-mon[35502]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:12:17.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:17 smithi079 ceph-mon[35502]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:17.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:17 smithi019 ceph-mon[31620]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T09:12:17.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:17 smithi019 ceph-mon[31620]: from='osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:12:17.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:17 smithi019 ceph-mon[31620]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T09:12:17.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:17.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:17 smithi019 ceph-mon[31620]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:12:17.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:17 smithi019 ceph-mon[31620]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-17T09:12:18.180 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T09:12:18.181 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T09:12:18.181 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-17T09:12:18.181 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-17T09:12:18.181 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123717 s, 848 MB/s 2024-05-17T09:12:18.181 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-17T09:12:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:12:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:12:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:12:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T09:12:18.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] boot 2024-05-17T09:12:18.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T09:12:18.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:18 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:12:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:12:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:12:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T09:12:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] boot 2024-05-17T09:12:18.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T09:12:18.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:18 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: osd.7 [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] boot 2024-05-17T09:12:18.730 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T09:12:18.731 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:18 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T09:12:18.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-05-17T09:12:18.994 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi069 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-17T09:12:20.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:20.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:20.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:19 smithi079 ceph-mon[35502]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T09:12:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi069 2024-05-17T09:12:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:19 smithi069 ceph-mon[35081]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi069 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 5696M 2024-05-17T09:12:20.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:20.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:20.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:20.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:19 smithi019 ceph-mon[31620]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T09:12:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 -- 172.21.15.79:0/1643088008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac010a100 msgr2=0x7fcac010c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/1643088008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac010a100 0x7fcac010c520 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b0a0 tx=0x7fcab800b3b0 comp rx=0 tx=0).stop 2024-05-17T09:12:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 -- 172.21.15.79:0/1643088008 shutdown_connections 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/1643088008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac010a100 0x7fcac010c520 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/1643088008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac0104e40 0x7fcac0107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/1643088008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac01077a0 0x7fcac0109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 -- 172.21.15.79:0/1643088008 >> 172.21.15.79:0/1643088008 conn(0x7fcac00fe880 msgr2=0x7fcac0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 -- 172.21.15.79:0/1643088008 shutdown_connections 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 -- 172.21.15.79:0/1643088008 wait complete. 2024-05-17T09:12:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.614+0000 7fcac5fdc700 1 Processor -- start 2024-05-17T09:12:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 -- start start 2024-05-17T09:12:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac0104e40 0x7fcac019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 0x7fcac019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac010a100 0x7fcac01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010fcc0 con 0x7fcac0104e40 2024-05-17T09:12:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010fb40 con 0x7fcac010a100 2024-05-17T09:12:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcac5fdc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010fe40 con 0x7fcac01077a0 2024-05-17T09:12:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 0x7fcac019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 0x7fcac019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33618/0 (socket says 172.21.15.79:33618) 2024-05-17T09:12:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 -- 172.21.15.79:0/3034006499 learned_addr learned my addr 172.21.15.79:0/3034006499 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:12:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabf7fe700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac0104e40 0x7fcac019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabffff700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac010a100 0x7fcac01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 -- 172.21.15.79:0/3034006499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac010a100 msgr2=0x7fcac01a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac010a100 0x7fcac01a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 -- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac0104e40 msgr2=0x7fcac019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac0104e40 0x7fcac019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.615+0000 7fcabeffd700 1 -- 172.21.15.79:0/3034006499 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac01a2360 con 0x7fcac01077a0 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.616+0000 7fcabeffd700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 0x7fcac019ad40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcab400b530 tx=0x7fcab400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:12:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.616+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4015020 con 0x7fcac01077a0 2024-05-17T09:12:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.616+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab4011cc0 con 0x7fcac01077a0 2024-05-17T09:12:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.616+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac01a2650 con 0x7fcac01077a0 2024-05-17T09:12:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.616+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4010c80 con 0x7fcac01077a0 2024-05-17T09:12:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.616+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a2c10 con 0x7fcac01077a0 2024-05-17T09:12:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.617+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac00512b0 con 0x7fcac01077a0 2024-05-17T09:12:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.618+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcab4018020 con 0x7fcac01077a0 2024-05-17T09:12:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.619+0000 7fcabcff9700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa80b2880 0x7fcaa80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.619+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7fcab40e12e0 con 0x7fcac01077a0 2024-05-17T09:12:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.619+0000 7fcabf7fe700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa80b2880 0x7fcaa80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:20.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.620+0000 7fcabf7fe700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa80b2880 0x7fcaa80b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005950 tx=0x7fcab000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:12:20.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.623+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcab401f020 con 0x7fcac01077a0 2024-05-17T09:12:20.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:20.835+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcac0063bf0 con 0x7fcaa80b2880 2024-05-17T09:12:21.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:21 smithi069 ceph-mon[35081]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T09:12:21.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:12:21.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:12:21.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:21.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:21 smithi019 ceph-mon[31620]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T09:12:21.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:12:21.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:12:21.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:21.443 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:21 smithi079 ceph-mon[35502]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T09:12:21.443 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:12:21.443 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:12:21.443 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:22.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:22 smithi069 ceph-mon[35081]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:22.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:22 smithi069 ceph-mon[35081]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:12:22.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:22 smithi079 ceph-mon[35502]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:22.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:22 smithi079 ceph-mon[35502]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:12:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:22 smithi019 ceph-mon[31620]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:22 smithi019 ceph-mon[31620]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:12:24.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:24 smithi079 ceph-mon[35502]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:24.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:24 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/2894192541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]: dispatch 2024-05-17T09:12:24.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:24 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]: dispatch 2024-05-17T09:12:24.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:24 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]': finished 2024-05-17T09:12:24.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:24 smithi079 ceph-mon[35502]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T09:12:24.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:24 smithi069 ceph-mon[35081]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:24 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/2894192541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]: dispatch 2024-05-17T09:12:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:24 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]: dispatch 2024-05-17T09:12:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:24 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]': finished 2024-05-17T09:12:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:24 smithi069 ceph-mon[35081]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T09:12:24.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:24 smithi019 ceph-mon[31620]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:24 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/2894192541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]: dispatch 2024-05-17T09:12:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:24 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]: dispatch 2024-05-17T09:12:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:24 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "106603bf-b7f4-438c-abc4-b660029e1128"}]': finished 2024-05-17T09:12:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:24 smithi019 ceph-mon[31620]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T09:12:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:25.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:25 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/1112364386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:25.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:25 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/1112364386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:25.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:25 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/1112364386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:26.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:26 smithi069 ceph-mon[35081]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T09:12:26.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:26 smithi019 ceph-mon[31620]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T09:12:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:26 smithi079 ceph-mon[35502]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T09:12:28.224 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:28 smithi079 ceph-mon[35502]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-17T09:12:28.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:28 smithi069 ceph-mon[35081]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-17T09:12:28.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:28 smithi019 ceph-mon[31620]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-17T09:12:30.205 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:30 smithi079 ceph-mon[35502]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-17T09:12:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:30 smithi069 ceph-mon[35081]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-17T09:12:30.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:30 smithi019 ceph-mon[31620]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-17T09:12:32.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:32 smithi079 ceph-mon[35502]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:32.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T09:12:32.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:32.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:32 smithi079 ceph-mon[35502]: Deploying daemon osd.8 on smithi079 2024-05-17T09:12:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:32 smithi069 ceph-mon[35081]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T09:12:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:32.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:32 smithi069 ceph-mon[35081]: Deploying daemon osd.8 on smithi079 2024-05-17T09:12:32.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:32 smithi019 ceph-mon[31620]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:32.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T09:12:32.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:32.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:32 smithi019 ceph-mon[31620]: Deploying daemon osd.8 on smithi079 2024-05-17T09:12:34.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:34 smithi069 ceph-mon[35081]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:34.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:34 smithi019 ceph-mon[31620]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:34.450 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:34 smithi079 ceph-mon[35502]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:36.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:36 smithi069 ceph-mon[35081]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:36.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:36 smithi019 ceph-mon[31620]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:36.476 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:36 smithi079 ceph-mon[35502]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:38.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:38 smithi079 ceph-mon[35502]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:38.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:38.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:38.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:38 smithi069 ceph-mon[35081]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:38 smithi019 ceph-mon[31620]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:38.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:40.285 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:40 smithi079 ceph-mon[35502]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:40 smithi069 ceph-mon[35081]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:40.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:40 smithi019 ceph-mon[31620]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:40.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.906+0000 7fcabcff9700 1 -- 172.21.15.79:0/3034006499 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcac0063bf0 con 0x7fcaa80b2880 2024-05-17T09:12:40.907 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 8 on host 'smithi079' 2024-05-17T09:12:40.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa80b2880 msgr2=0x7fcaa80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:40.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa80b2880 0x7fcaa80b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005950 tx=0x7fcab000b410 comp rx=0 tx=0).stop 2024-05-17T09:12:40.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 msgr2=0x7fcac019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:40.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 0x7fcac019ad40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcab400b530 tx=0x7fcab400b840 comp rx=0 tx=0).stop 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 shutdown_connections 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac010a100 0x7fcac01a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac0104e40 0x7fcac019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa80b2880 0x7fcaa80b4d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 --2- 172.21.15.79:0/3034006499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcac01077a0 0x7fcac019ad40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.908+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 >> 172.21.15.79:0/3034006499 conn(0x7fcac00fe880 msgr2=0x7fcac0105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.909+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 shutdown_connections 2024-05-17T09:12:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:12:40.909+0000 7fcac5fdc700 1 -- 172.21.15.79:0/3034006499 wait complete. 2024-05-17T09:12:41.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:41.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:41.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:41.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T09:12:41.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:41 smithi079 ceph-mon[35502]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T09:12:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T09:12:41.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:41 smithi069 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:12:41.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T09:12:41.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:41 smithi019 ceph-mon[31620]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T09:12:41.671 DEBUG:teuthology.orchestra.run.smithi079:osd.8> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.8.service 2024-05-17T09:12:41.674 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-05-17T09:12:41.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T09:12:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:12:42.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:42 smithi079 ceph-mon[35502]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:42.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:42 smithi079 ceph-mon[35502]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T09:12:42.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:42 smithi079 ceph-mon[35502]: from='osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:12:42.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:42 smithi079 ceph-mon[35502]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T09:12:42.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:42.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:42 smithi079 ceph-mon[35502]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:12:42.305 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:42 smithi019 ceph-mon[31620]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:42.305 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:42 smithi019 ceph-mon[31620]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T09:12:42.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:42 smithi019 ceph-mon[31620]: from='osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:12:42.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:42 smithi019 ceph-mon[31620]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T09:12:42.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:42.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:42 smithi019 ceph-mon[31620]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:12:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:42 smithi069 ceph-mon[35081]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:42 smithi069 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T09:12:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:42 smithi069 ceph-mon[35081]: from='osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:12:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:42 smithi069 ceph-mon[35081]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T09:12:42.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:42.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:42 smithi069 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:12:43.187 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:12:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:12:42.906+0000 7f8ea1f49700 -1 osd.8 0 waiting for initial osdmap 2024-05-17T09:12:43.187 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:12:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:12:42.917+0000 7f8e9dcfe700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:12:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:12:44.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T09:12:44.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] boot 2024-05-17T09:12:44.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T09:12:44.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:44.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:12:44.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T09:12:44.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] boot 2024-05-17T09:12:44.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T09:12:44.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: osd.8 [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] boot 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T09:12:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T09:12:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T09:12:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T09:12:44.326 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-17T09:12:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-17T09:12:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0140448 s, 747 MB/s 2024-05-17T09:12:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-17T09:12:45.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-05-17T09:12:45.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:12:45.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:12:45.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:45.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:45.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:12:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:12:45.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:12:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:12:46.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:46 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi079 2024-05-17T09:12:46.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:46 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 5696M 2024-05-17T09:12:46.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:46 smithi019 ceph-mon[31620]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:12:46.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:46 smithi019 ceph-mon[31620]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T09:12:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:46 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi079 2024-05-17T09:12:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:46 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 5696M 2024-05-17T09:12:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:46 smithi069 ceph-mon[35081]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:12:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:46 smithi069 ceph-mon[35081]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T09:12:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:46 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi079 2024-05-17T09:12:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:46 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 5696M 2024-05-17T09:12:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:46 smithi079 ceph-mon[35502]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:12:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:46 smithi079 ceph-mon[35502]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T09:12:46.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.927+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3364981021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 msgr2=0x7f0d4c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.927+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3364981021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c105890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009990 tx=0x7f0d38009ca0 comp rx=0 tx=0).stop 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3364981021 shutdown_connections 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3364981021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d4c104350 0x7f0d4c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3364981021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d4c105e20 0x7f0d4c10c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3364981021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c105890 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3364981021 >> 172.21.15.19:0/3364981021 conn(0x7f0d4c0ff9b0 msgr2=0x7f0d4c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3364981021 shutdown_connections 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3364981021 wait complete. 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 Processor -- start 2024-05-17T09:12:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.928+0000 7f0d52a9d700 1 -- start start 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d52a9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d4c104350 0x7f0d4c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d52a9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d52a9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d4c105e20 0x7f0d4c1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d52a9d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c10fc70 con 0x7f0d4c105480 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d52a9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c10faf0 con 0x7f0d4c105e20 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d52a9d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c10fdf0 con 0x7f0d4c104350 2024-05-17T09:12:46.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36688/0 (socket says 172.21.15.19:36688) 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 -- 172.21.15.19:0/3190329646 learned_addr learned my addr 172.21.15.19:0/3190329646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d50839700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d4c104350 0x7f0d4c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d5103a700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d4c105e20 0x7f0d4c1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 -- 172.21.15.19:0/3190329646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d4c104350 msgr2=0x7f0d4c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d4c104350 0x7f0d4c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 -- 172.21.15.19:0/3190329646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d4c105e20 msgr2=0x7f0d4c1a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d4c105e20 0x7f0d4c1a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 -- 172.21.15.19:0/3190329646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c1a2360 con 0x7f0d4c105480 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.929+0000 7f0d4bfff700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c19ae20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0d380057c0 tx=0x7f0d38009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.930+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38027070 con 0x7f0d4c105480 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.930+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c1a25f0 con 0x7f0d4c105480 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.930+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d38003f60 con 0x7f0d4c105480 2024-05-17T09:12:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.930+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38022bb0 con 0x7f0d4c105480 2024-05-17T09:12:46.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.930+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a2a40 con 0x7f0d4c105480 2024-05-17T09:12:46.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.931+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c0512b0 con 0x7f0d4c105480 2024-05-17T09:12:46.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.932+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0d3802a070 con 0x7f0d4c105480 2024-05-17T09:12:46.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.933+0000 7f0d49ffb700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:12:46.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.933+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f0d380e30b0 con 0x7f0d4c105480 2024-05-17T09:12:46.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.933+0000 7f0d50839700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:12:46.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.934+0000 7f0d50839700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c105380 tx=0x7f0d4000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:12:46.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:46.937+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d38031080 con 0x7f0d4c105480 2024-05-17T09:12:47.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:12:47.159+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0d4c063bf0 con 0x7f0d3c0b2880 2024-05-17T09:12:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:48 smithi069 ceph-mon[35081]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:12:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:48 smithi069 ceph-mon[35081]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:12:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:12:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:12:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:48 smithi019 ceph-mon[31620]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:12:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:48 smithi019 ceph-mon[31620]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:12:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:12:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:12:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:48 smithi079 ceph-mon[35502]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:12:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:48 smithi079 ceph-mon[35502]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:12:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:12:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:12:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:50 smithi069 ceph-mon[35081]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-17T09:12:50.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:50 smithi019 ceph-mon[31620]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-17T09:12:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:50 smithi079 ceph-mon[35502]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-17T09:12:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:51 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1363447147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e132748b-b8b9-4b25-b233-d786e59d9330"}]: dispatch 2024-05-17T09:12:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:51 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1363447147' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e132748b-b8b9-4b25-b233-d786e59d9330"}]': finished 2024-05-17T09:12:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:51 smithi069 ceph-mon[35081]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T09:12:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:12:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:51 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/518609213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:51 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1363447147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e132748b-b8b9-4b25-b233-d786e59d9330"}]: dispatch 2024-05-17T09:12:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:51 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1363447147' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e132748b-b8b9-4b25-b233-d786e59d9330"}]': finished 2024-05-17T09:12:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:51 smithi019 ceph-mon[31620]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T09:12:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:12:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:51 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/518609213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:51.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:51 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1363447147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e132748b-b8b9-4b25-b233-d786e59d9330"}]: dispatch 2024-05-17T09:12:51.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:51 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1363447147' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e132748b-b8b9-4b25-b233-d786e59d9330"}]': finished 2024-05-17T09:12:51.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:51 smithi079 ceph-mon[35502]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T09:12:51.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:12:51.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:51 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/518609213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:12:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:52 smithi019 ceph-mon[31620]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T09:12:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:52 smithi079 ceph-mon[35502]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T09:12:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:52 smithi069 ceph-mon[35081]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T09:12:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:54 smithi019 ceph-mon[31620]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:12:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:54 smithi079 ceph-mon[35502]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:12:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:54 smithi069 ceph-mon[35081]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:12:56.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:56 smithi019 ceph-mon[31620]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T09:12:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:56 smithi079 ceph-mon[35502]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T09:12:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:56 smithi069 ceph-mon[35081]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T09:12:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:58 smithi019 ceph-mon[31620]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T09:12:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:58.482 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:58 smithi069 ceph-mon[35081]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:58.483 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T09:12:58.483 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:58 smithi079 ceph-mon[35502]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:12:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T09:12:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:12:59.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:12:59 smithi019 ceph-mon[31620]: Deploying daemon osd.9 on smithi019 2024-05-17T09:12:59.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:12:59 smithi079 ceph-mon[35502]: Deploying daemon osd.9 on smithi019 2024-05-17T09:12:59.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:12:59 smithi069 ceph-mon[35081]: Deploying daemon osd.9 on smithi019 2024-05-17T09:13:00.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:00 smithi019 ceph-mon[31620]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:00 smithi079 ceph-mon[35502]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:00 smithi069 ceph-mon[35081]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:02 smithi019 ceph-mon[31620]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:02 smithi079 ceph-mon[35502]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:02 smithi069 ceph-mon[35081]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:04.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:04 smithi019 ceph-mon[31620]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:04 smithi079 ceph-mon[35502]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:04 smithi069 ceph-mon[35081]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:05.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:05.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:05.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:05.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:05 smithi019 ceph-mon[31620]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:05.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:05.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:05.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:05.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:05 smithi079 ceph-mon[35502]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:05.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:05.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:05.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:05.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:05 smithi069 ceph-mon[35081]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:08.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:07 smithi019 ceph-mon[31620]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:08.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:08.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:08.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:07 smithi069 ceph-mon[35081]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:08.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:07 smithi079 ceph-mon[35502]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:08.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.570+0000 7f0d49ffb700 1 -- 172.21.15.19:0/3190329646 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d4c063bf0 con 0x7f0d3c0b2880 2024-05-17T09:13:08.571 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-05-17T09:13:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d3c0b2880 msgr2=0x7f0d3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c105380 tx=0x7f0d4000a3b0 comp rx=0 tx=0).stop 2024-05-17T09:13:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 msgr2=0x7f0d4c19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c19ae20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0d380057c0 tx=0x7f0d38009580 comp rx=0 tx=0).stop 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 shutdown_connections 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d4c104350 0x7f0d4c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d4c105e20 0x7f0d4c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 --2- 172.21.15.19:0/3190329646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d4c105480 0x7f0d4c19ae20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 >> 172.21.15.19:0/3190329646 conn(0x7f0d4c0ff9b0 msgr2=0x7f0d4c108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 shutdown_connections 2024-05-17T09:13:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:13:08.573+0000 7f0d52a9d700 1 -- 172.21.15.19:0/3190329646 wait complete. 2024-05-17T09:13:09.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:08 smithi069 ceph-mon[35081]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T09:13:09.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:09.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:09.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:08 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:09.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:08 smithi019 ceph-mon[31620]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T09:13:09.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:09.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:09.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:08 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:09.227 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.9.service 2024-05-17T09:13:09.230 INFO:tasks.cephadm:Deploying osd.10 on smithi069 with /dev/vg_nvme/lv_1... 2024-05-17T09:13:09.230 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T09:13:09.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:08 smithi079 ceph-mon[35502]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T09:13:09.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:09.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:09.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:08 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:09.477 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:13:10.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:09 smithi069 ceph-mon[35081]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:10.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:09 smithi069 ceph-mon[35081]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T09:13:10.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:09 smithi069 ceph-mon[35081]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T09:13:10.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:09 smithi069 ceph-mon[35081]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:13:10.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:09 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:09 smithi019 ceph-mon[31620]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:09 smithi019 ceph-mon[31620]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T09:13:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:09 smithi019 ceph-mon[31620]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T09:13:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:09 smithi019 ceph-mon[31620]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:13:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:10.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:09 smithi079 ceph-mon[35502]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:10.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:09 smithi079 ceph-mon[35502]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T09:13:10.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:09 smithi079 ceph-mon[35502]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T09:13:10.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:09 smithi079 ceph-mon[35502]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-17T09:13:10.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:09 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:10.565 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:13:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:13:10.391+0000 7fc9e3807700 -1 osd.9 0 waiting for initial osdmap 2024-05-17T09:13:10.565 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:13:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:13:10.407+0000 7fc9db5b4700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:13:11.004 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:10 smithi069 ceph-mon[35081]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:13:11.004 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:10 smithi069 ceph-mon[35081]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T09:13:11.004 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:11.004 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:11.004 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:10 smithi069 ceph-mon[35081]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' 2024-05-17T09:13:11.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:10 smithi019 ceph-mon[31620]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:13:11.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:10 smithi019 ceph-mon[31620]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T09:13:11.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:11.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:11.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:10 smithi019 ceph-mon[31620]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' 2024-05-17T09:13:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:10 smithi079 ceph-mon[35502]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-17T09:13:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:10 smithi079 ceph-mon[35502]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T09:13:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:10 smithi079 ceph-mon[35502]: from='osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087]' entity='osd.9' 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0145205 s, 722 MB/s 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 2024-05-17T09:13:11.596 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] boot 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T09:13:12.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:12.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:13:12.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:13:12.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:12.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:12.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] boot 2024-05-17T09:13:12.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T09:13:12.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:13:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:13:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-05-17T09:13:12.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:12.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: osd.9 [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] boot 2024-05-17T09:13:12.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T09:13:12.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T09:13:12.366 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-05-17T09:13:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:13:13.808 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi019 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:13.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:13.810 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.810 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:13.810 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:13 smithi069 ceph-mon[35081]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T09:13:13.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi019 2024-05-17T09:13:13.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:13.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:13 smithi019 ceph-mon[31620]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T09:13:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi019 2024-05-17T09:13:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:14.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:14.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:13 smithi079 ceph-mon[35502]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T09:13:14.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 -- 172.21.15.69:0/56497009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8103230 msgr2=0x7f7da8103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:14.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 --2- 172.21.15.69:0/56497009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8103230 0x7f7da8103640 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009990 tx=0x7f7d98009ca0 comp rx=0 tx=0).stop 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 -- 172.21.15.69:0/56497009 shutdown_connections 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 --2- 172.21.15.69:0/56497009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8104d00 0x7f7da810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 --2- 172.21.15.69:0/56497009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8103230 0x7f7da8103640 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 --2- 172.21.15.69:0/56497009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da8104360 0x7f7da8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 -- 172.21.15.69:0/56497009 >> 172.21.15.69:0/56497009 conn(0x7f7da80fe870 msgr2=0x7f7da8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 -- 172.21.15.69:0/56497009 shutdown_connections 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 -- 172.21.15.69:0/56497009 wait complete. 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 Processor -- start 2024-05-17T09:13:14.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.352+0000 7f7dad553700 1 -- start start 2024-05-17T09:13:14.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7dad553700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8103230 0x7f7da819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:14.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7dad553700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da8104360 0x7f7da819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:14.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7dad553700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 0x7f7da81a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:14.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7dad553700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7da810eb80 con 0x7f7da8103230 2024-05-17T09:13:14.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7dad553700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7da810ea00 con 0x7f7da8104d00 2024-05-17T09:13:14.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7dad553700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7da810ed00 con 0x7f7da8104360 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 0x7f7da81a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 0x7f7da81a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38686/0 (socket says 172.21.15.69:38686) 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 -- 172.21.15.69:0/380727480 learned_addr learned my addr 172.21.15.69:0/380727480 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da67fc700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da8104360 0x7f7da819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da6ffd700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8103230 0x7f7da819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 -- 172.21.15.69:0/380727480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da8104360 msgr2=0x7f7da819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da8104360 0x7f7da819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:14.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 -- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8103230 msgr2=0x7f7da819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8103230 0x7f7da819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 -- 172.21.15.69:0/380727480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da81a2440 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da6ffd700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8103230 0x7f7da819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7da77fe700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 0x7f7da81a1dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7da0007ee0 tx=0x7f7da000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.353+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da000dbd0 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.354+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da81a2730 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.354+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7da000dd70 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.354+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da00058d0 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.354+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da81a2c20 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.354+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da80512b0 con 0x7f7da8104d00 2024-05-17T09:13:14.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.356+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7da001a020 con 0x7f7da8104d00 2024-05-17T09:13:14.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.357+0000 7f7d8ffff700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d900b2880 0x7f7d900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:14.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.357+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f7da00e1860 con 0x7f7da8104d00 2024-05-17T09:13:14.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.357+0000 7f7da6ffd700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d900b2880 0x7f7d900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:14.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.358+0000 7f7da6ffd700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d900b2880 0x7f7d900b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d98006010 tx=0x7f7d9801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:13:14.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.360+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7da0010070 con 0x7f7da8104d00 2024-05-17T09:13:14.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:14.573+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7da8100990 con 0x7f7d900b2880 2024-05-17T09:13:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:13:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:13:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:14.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:13:14.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:13:14.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:15.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:13:15.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:13:15.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:15.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:15 smithi069 ceph-mon[35081]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:13:15.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:15 smithi069 ceph-mon[35081]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:15.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:15 smithi019 ceph-mon[31620]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:13:15.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:15 smithi019 ceph-mon[31620]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:16.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:15 smithi079 ceph-mon[35502]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:13:16.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:15 smithi079 ceph-mon[35502]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:18.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:17 smithi069 ceph-mon[35081]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:18.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:17 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/1036295440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]: dispatch 2024-05-17T09:13:18.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:17 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]: dispatch 2024-05-17T09:13:18.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:17 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]': finished 2024-05-17T09:13:18.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:17 smithi069 ceph-mon[35081]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T09:13:18.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:17 smithi019 ceph-mon[31620]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:17 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/1036295440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]: dispatch 2024-05-17T09:13:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:17 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]: dispatch 2024-05-17T09:13:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:17 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]': finished 2024-05-17T09:13:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:17 smithi019 ceph-mon[31620]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T09:13:18.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:18.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:17 smithi079 ceph-mon[35502]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:18.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:17 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/1036295440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]: dispatch 2024-05-17T09:13:18.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:17 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]: dispatch 2024-05-17T09:13:18.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:17 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a14a50a-2686-4e06-9473-0cf82a95e252"}]': finished 2024-05-17T09:13:18.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:17 smithi079 ceph-mon[35502]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T09:13:18.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:19.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:18 smithi069 ceph-mon[35081]: from='client.? 172.21.15.69:0/2518139419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:13:19.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:18 smithi019 ceph-mon[31620]: from='client.? 172.21.15.69:0/2518139419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:13:19.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:18 smithi079 ceph-mon[35502]: from='client.? 172.21.15.69:0/2518139419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:13:20.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:19 smithi069 ceph-mon[35081]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:19 smithi019 ceph-mon[31620]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:20.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:19 smithi079 ceph-mon[35502]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:22.165 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:21 smithi069 ceph-mon[35081]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:21 smithi019 ceph-mon[31620]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:22.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:21 smithi079 ceph-mon[35502]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:24.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:23 smithi069 ceph-mon[35081]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:24.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:23 smithi019 ceph-mon[31620]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:24.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:23 smithi079 ceph-mon[35502]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:25.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T09:13:25.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:25.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T09:13:25.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:25.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T09:13:25.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:25 smithi069 ceph-mon[35081]: Deploying daemon osd.10 on smithi069 2024-05-17T09:13:26.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:25 smithi069 ceph-mon[35081]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:26.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:25 smithi019 ceph-mon[31620]: Deploying daemon osd.10 on smithi069 2024-05-17T09:13:26.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:25 smithi019 ceph-mon[31620]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:26.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:25 smithi079 ceph-mon[35502]: Deploying daemon osd.10 on smithi069 2024-05-17T09:13:26.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:25 smithi079 ceph-mon[35502]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:28.131 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:27 smithi069 ceph-mon[35081]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:27 smithi019 ceph-mon[31620]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:28.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:27 smithi079 ceph-mon[35502]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:30.027 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:29 smithi069 ceph-mon[35081]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:30.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:29 smithi019 ceph-mon[31620]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:30.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:29 smithi079 ceph-mon[35502]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:31 smithi069 ceph-mon[35081]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:31.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:31.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:31.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:31.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:31 smithi019 ceph-mon[31620]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:32.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:32.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:32.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:32.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:31 smithi079 ceph-mon[35502]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:34.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:33 smithi069 ceph-mon[35081]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:34.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:34.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:34.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:33 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:33 smithi019 ceph-mon[31620]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:33 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:33 smithi079 ceph-mon[35502]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:34.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:33 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:34.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.549+0000 7f7d8ffff700 1 -- 172.21.15.69:0/380727480 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7da8100990 con 0x7f7d900b2880 2024-05-17T09:13:34.549 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 10 on host 'smithi069' 2024-05-17T09:13:34.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d900b2880 msgr2=0x7f7d900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:34.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d900b2880 0x7f7d900b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d98006010 tx=0x7f7d9801a040 comp rx=0 tx=0).stop 2024-05-17T09:13:34.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 msgr2=0x7f7da81a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 0x7f7da81a1dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7da0007ee0 tx=0x7f7da000d3b0 comp rx=0 tx=0).stop 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 shutdown_connections 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da8104d00 0x7f7da81a1dc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7da8103230 0x7f7da819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d900b2880 0x7f7d900b4d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 --2- 172.21.15.69:0/380727480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da8104360 0x7f7da819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 >> 172.21.15.69:0/380727480 conn(0x7f7da80fe870 msgr2=0x7f7da8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 shutdown_connections 2024-05-17T09:13:34.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-05-17T09:13:34.551+0000 7f7dad553700 1 -- 172.21.15.69:0/380727480 wait complete. 2024-05-17T09:13:35.199 DEBUG:teuthology.orchestra.run.smithi069:osd.10> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.10.service 2024-05-17T09:13:35.201 INFO:tasks.cephadm:Deploying osd.11 on smithi079 with /dev/vg_nvme/lv_1... 2024-05-17T09:13:35.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T09:13:35.451 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:13:35.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:35.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:35.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:35 smithi079 ceph-mon[35502]: from='osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T09:13:35.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:35 smithi079 ceph-mon[35502]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T09:13:35.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:35.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:35 smithi079 ceph-mon[35502]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:35 smithi069 ceph-mon[35081]: from='osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T09:13:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:35 smithi069 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T09:13:35.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:35.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:35 smithi069 ceph-mon[35081]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:35 smithi019 ceph-mon[31620]: from='osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T09:13:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:35 smithi019 ceph-mon[31620]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T09:13:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:35 smithi019 ceph-mon[31620]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:36.745 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:36 smithi069 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T09:13:36.745 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:36 smithi069 ceph-mon[35081]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T09:13:36.745 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:36 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:36.745 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:36 smithi069 ceph-mon[35081]: from='osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:13:36.746 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:36 smithi069 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:13:36.746 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:13:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:13:36.555+0000 7fa2b2c78700 -1 osd.10 0 waiting for initial osdmap 2024-05-17T09:13:36.746 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:13:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:13:36.566+0000 7fa2aea2d700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:13:36.844 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:36 smithi079 ceph-mon[35502]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T09:13:36.844 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:36 smithi079 ceph-mon[35502]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T09:13:36.844 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:36.844 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:36 smithi079 ceph-mon[35502]: from='osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:13:36.844 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:36 smithi079 ceph-mon[35502]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:13:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:36 smithi019 ceph-mon[31620]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T09:13:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:36 smithi019 ceph-mon[31620]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T09:13:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:36 smithi019 ceph-mon[31620]: from='osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:13:36.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:36 smithi019 ceph-mon[31620]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-05-17T09:13:37.505 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T09:13:37.505 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T09:13:37.505 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-17T09:13:37.505 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-17T09:13:37.505 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012243 s, 856 MB/s 2024-05-17T09:13:37.505 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:37.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.911 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:13:37.911 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:13:37.911 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:13:37.911 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T09:13:37.911 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.911 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.912 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:37.912 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-05-17T09:13:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:38.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-05-17T09:13:38.513 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] boot 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:38.877 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] boot 2024-05-17T09:13:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:38.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: osd.10 [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] boot 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:13:38.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:38.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:38.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:13:38.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 -- 172.21.15.79:0/1292547093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3410a0e0 msgr2=0x7fdd3410c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/1292547093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3410a0e0 0x7fdd3410c500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009990 tx=0x7fdd2c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 -- 172.21.15.79:0/1292547093 shutdown_connections 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/1292547093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3410a0e0 0x7fdd3410c500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/1292547093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd34104e20 0x7fdd34107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/1292547093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34107780 0x7fdd34109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 -- 172.21.15.79:0/1292547093 >> 172.21.15.79:0/1292547093 conn(0x7fdd340fe860 msgr2=0x7fdd34100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 -- 172.21.15.79:0/1292547093 shutdown_connections 2024-05-17T09:13:40.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.292+0000 7fdd3bffe700 1 -- 172.21.15.79:0/1292547093 wait complete. 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 Processor -- start 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 -- start start 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 0x7fdd3419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd34107780 0x7fdd3419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3410a0e0 0x7fdd341a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410fc30 con 0x7fdd34107780 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410fab0 con 0x7fdd3410a0e0 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3bffe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410fdb0 con 0x7fdd34104e20 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd39d9a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 0x7fdd3419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd39d9a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 0x7fdd3419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47122/0 (socket says 172.21.15.79:47122) 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd39d9a700 1 -- 172.21.15.79:0/268192035 learned_addr learned my addr 172.21.15.79:0/268192035 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.293+0000 7fdd3a59b700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3410a0e0 0x7fdd341a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:40.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39599700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd34107780 0x7fdd3419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39d9a700 1 -- 172.21.15.79:0/268192035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3410a0e0 msgr2=0x7fdd341a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39d9a700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3410a0e0 0x7fdd341a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39d9a700 1 -- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd34107780 msgr2=0x7fdd3419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39d9a700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd34107780 0x7fdd3419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39d9a700 1 -- 172.21.15.79:0/268192035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd341a22b0 con 0x7fdd34104e20 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd39d9a700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 0x7fdd3419a750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007f10 tx=0x7fdd3000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3000ddd0 con 0x7fdd34104e20 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd30018070 con 0x7fdd34104e20 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341a25a0 con 0x7fdd34104e20 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300125b0 con 0x7fdd34104e20 2024-05-17T09:13:40.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.294+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341a2ae0 con 0x7fdd34104e20 2024-05-17T09:13:40.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.295+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd340512b0 con 0x7fdd34104e20 2024-05-17T09:13:40.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.297+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdd300139a0 con 0x7fdd34104e20 2024-05-17T09:13:40.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.297+0000 7fdd2affd700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdd200b2880 0x7fdd200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:13:40.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.298+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7fdd300e1cc0 con 0x7fdd34104e20 2024-05-17T09:13:40.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.298+0000 7fdd39599700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdd200b2880 0x7fdd200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:13:40.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.298+0000 7fdd39599700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdd200b2880 0x7fdd200b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd24005950 tx=0x7fdd24005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:13:40.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.301+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd30010070 con 0x7fdd34104e20 2024-05-17T09:13:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:40 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi069 2024-05-17T09:13:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:40 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-17T09:13:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:40 smithi069 ceph-mon[35081]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:40 smithi069 ceph-mon[35081]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T09:13:40.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:40 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi069 2024-05-17T09:13:40.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:40 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-17T09:13:40.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:40 smithi019 ceph-mon[31620]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:40.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:40 smithi019 ceph-mon[31620]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T09:13:40.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:13:40.511+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdd34063bf0 con 0x7fdd200b2880 2024-05-17T09:13:40.511 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:40 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi069 2024-05-17T09:13:40.511 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:40 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 4272M 2024-05-17T09:13:40.511 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:40 smithi079 ceph-mon[35502]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:40.511 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:40 smithi079 ceph-mon[35502]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T09:13:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:13:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:13:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:41 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:13:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:13:41.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:41 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:41.454 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T09:13:41.454 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T09:13:41.454 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:41 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:42.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:42 smithi079 ceph-mon[35502]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:13:42.418 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:42 smithi079 ceph-mon[35502]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:42 smithi069 ceph-mon[35081]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:13:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:42 smithi069 ceph-mon[35081]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:42.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:42 smithi019 ceph-mon[31620]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:13:42.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:42 smithi019 ceph-mon[31620]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:44.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:44.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/3748779394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]: dispatch 2024-05-17T09:13:44.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]: dispatch 2024-05-17T09:13:44.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]': finished 2024-05-17T09:13:44.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T09:13:44.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:13:44.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:44 smithi069 ceph-mon[35081]: from='client.? 172.21.15.79:0/1760535028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/3748779394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]: dispatch 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]: dispatch 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]': finished 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:13:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:44 smithi019 ceph-mon[31620]: from='client.? 172.21.15.79:0/1760535028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:13:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/3748779394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]: dispatch 2024-05-17T09:13:44.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]: dispatch 2024-05-17T09:13:44.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68"}]': finished 2024-05-17T09:13:44.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T09:13:44.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:13:44.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:44 smithi079 ceph-mon[35502]: from='client.? 172.21.15.79:0/1760535028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T09:13:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:46 smithi069 ceph-mon[35081]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T09:13:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:46 smithi019 ceph-mon[31620]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T09:13:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:46 smithi079 ceph-mon[35502]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-17T09:13:48.410 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:48 smithi079 ceph-mon[35502]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:13:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:48 smithi069 ceph-mon[35081]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:13:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:48 smithi019 ceph-mon[31620]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T09:13:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:50 smithi079 ceph-mon[35502]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T09:13:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:50 smithi069 ceph-mon[35081]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T09:13:50.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:50 smithi019 ceph-mon[31620]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T09:13:51.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T09:13:51.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T09:13:51.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T09:13:51.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:13:52.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:52 smithi079 ceph-mon[35502]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:13:52.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:52 smithi079 ceph-mon[35502]: Deploying daemon osd.11 on smithi079 2024-05-17T09:13:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:52 smithi069 ceph-mon[35081]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:13:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:52 smithi069 ceph-mon[35081]: Deploying daemon osd.11 on smithi079 2024-05-17T09:13:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:52 smithi019 ceph-mon[31620]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:13:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:52 smithi019 ceph-mon[31620]: Deploying daemon osd.11 on smithi079 2024-05-17T09:13:54.322 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:54 smithi079 ceph-mon[35502]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:13:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:54 smithi069 ceph-mon[35081]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:13:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:54 smithi019 ceph-mon[31620]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T09:13:56.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:56 smithi069 ceph-mon[35081]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-17T09:13:56.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:56 smithi079 ceph-mon[35502]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-17T09:13:56.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:56 smithi019 ceph-mon[31620]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-17T09:13:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:58 smithi079 ceph-mon[35502]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:13:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:58 smithi069 ceph-mon[35081]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:13:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:58.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:58 smithi019 ceph-mon[31620]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:13:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:13:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:13:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:13:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:00.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:00 smithi079 ceph-mon[35502]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:00.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:00 smithi069 ceph-mon[35081]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:00 smithi019 ceph-mon[31620]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:01.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.136+0000 7fdd2affd700 1 -- 172.21.15.79:0/268192035 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdd34063bf0 con 0x7fdd200b2880 2024-05-17T09:14:01.137 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 11 on host 'smithi079' 2024-05-17T09:14:01.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdd200b2880 msgr2=0x7fdd200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:01.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdd200b2880 0x7fdd200b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd24005950 tx=0x7fdd24005ed0 comp rx=0 tx=0).stop 2024-05-17T09:14:01.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 msgr2=0x7fdd3419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:01.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 0x7fdd3419a750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007f10 tx=0x7fdd3000d3b0 comp rx=0 tx=0).stop 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 shutdown_connections 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdd200b2880 0x7fdd200b4d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd34104e20 0x7fdd3419a750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3410a0e0 0x7fdd341a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 --2- 172.21.15.79:0/268192035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd34107780 0x7fdd3419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 >> 172.21.15.79:0/268192035 conn(0x7fdd340fe860 msgr2=0x7fdd34105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 shutdown_connections 2024-05-17T09:14:01.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:01.139+0000 7fdd3bffe700 1 -- 172.21.15.79:0/268192035 wait complete. 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T09:14:01.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:01 smithi079 ceph-mon[35502]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T09:14:01.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:01.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:01.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T09:14:01.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:01 smithi019 ceph-mon[31620]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T09:14:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:01 smithi069 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T09:14:01.801 DEBUG:teuthology.orchestra.run.smithi079:osd.11> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.11.service 2024-05-17T09:14:01.803 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-17T09:14:01.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T09:14:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:02.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T09:14:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T09:14:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:02.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:14:02.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:14:02.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:02.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:02.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:02.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:02.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:02.772 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:14:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:14:02.547+0000 7f8e93269700 -1 osd.11 0 waiting for initial osdmap 2024-05-17T09:14:02.773 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:14:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:14:02.561+0000 7f8e8b016700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T09:14:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:03 smithi069 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:14:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:03 smithi069 ceph-mon[35081]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T09:14:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:03 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:03.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:03 smithi069 ceph-mon[35081]: from='osd.11 ' entity='osd.11' 2024-05-17T09:14:03.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:03 smithi079 ceph-mon[35502]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:14:03.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:03 smithi079 ceph-mon[35502]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T09:14:03.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:03.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:03 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:03.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:03 smithi079 ceph-mon[35502]: from='osd.11 ' entity='osd.11' 2024-05-17T09:14:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:03 smithi019 ceph-mon[31620]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-17T09:14:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:03 smithi019 ceph-mon[31620]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T09:14:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:03 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:03.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:03 smithi019 ceph-mon[31620]: from='osd.11 ' entity='osd.11' 2024-05-17T09:14:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 -- 172.21.15.19:0/286604696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8001059a0 msgr2=0x7fc800105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 --2- 172.21.15.19:0/286604696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8001059a0 0x7fc800105db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000b0a0 tx=0x7fc7f000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:14:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 -- 172.21.15.19:0/286604696 shutdown_connections 2024-05-17T09:14:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 --2- 172.21.15.19:0/286604696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8001049d0 0x7fc80010c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 --2- 172.21.15.19:0/286604696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8001059a0 0x7fc800105db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 --2- 172.21.15.19:0/286604696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc800103f50 0x7fc800104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 -- 172.21.15.19:0/286604696 >> 172.21.15.19:0/286604696 conn(0x7fc8000ff9c0 msgr2=0x7fc800101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 -- 172.21.15.19:0/286604696 shutdown_connections 2024-05-17T09:14:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.904+0000 7fc804d6a700 1 -- 172.21.15.19:0/286604696 wait complete. 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.905+0000 7fc804d6a700 1 Processor -- start 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.905+0000 7fc804d6a700 1 -- start start 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc804d6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 0x7fc80019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc804d6a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8001049d0 0x7fc80019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc804d6a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8001059a0 0x7fc8001a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc804d6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc80010fdf0 con 0x7fc800103f50 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc804d6a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc80010fc70 con 0x7fc8001059a0 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc804d6a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc80010ff70 con 0x7fc8001049d0 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 0x7fc80019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 0x7fc80019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38528/0 (socket says 172.21.15.19:38528) 2024-05-17T09:14:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 -- 172.21.15.19:0/182997911 learned_addr learned my addr 172.21.15.19:0/182997911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fdd9b700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8001049d0 0x7fc80019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fed9d700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8001059a0 0x7fc8001a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 -- 172.21.15.19:0/182997911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8001049d0 msgr2=0x7fc80019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8001049d0 0x7fc80019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 -- 172.21.15.19:0/182997911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8001059a0 msgr2=0x7fc8001a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8001059a0 0x7fc8001a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 -- 172.21.15.19:0/182997911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8001a2500 con 0x7fc800103f50 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7fe59c700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 0x7fc80019a9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000f80 tx=0x7fc7f0004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.906+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0021070 con 0x7fc800103f50 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.907+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7f0007b80 con 0x7fc800103f50 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.907+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8001a2790 con 0x7fc800103f50 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.907+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00052a0 con 0x7fc800103f50 2024-05-17T09:14:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.907+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8001a2d40 con 0x7fc800103f50 2024-05-17T09:14:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.908+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8000512b0 con 0x7fc800103f50 2024-05-17T09:14:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.909+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc7f0024070 con 0x7fc800103f50 2024-05-17T09:14:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.910+0000 7fc7f77fe700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc7ec0b2900 0x7fc7ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.910+0000 7fc7fdd9b700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc7ec0b2900 0x7fc7ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.910+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc7f000f020 con 0x7fc800103f50 2024-05-17T09:14:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.911+0000 7fc7fdd9b700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc7ec0b2900 0x7fc7ec0b4e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc80019ba50 tx=0x7fc7e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:03.914+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7f002b020 con 0x7fc800103f50 2024-05-17T09:14:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.129+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc8000643c0 con 0x7fc800103f50 2024-05-17T09:14:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.129+0000 7fc7f77fe700 1 -- 172.21.15.19:0/182997911 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fc7f00ab9b0 con 0x7fc800103f50 2024-05-17T09:14:04.130 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc7ec0b2900 msgr2=0x7fc7ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc7ec0b2900 0x7fc7ec0b4e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc80019ba50 tx=0x7fc7e8009260 comp rx=0 tx=0).stop 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 msgr2=0x7fc80019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 0x7fc80019a9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000f80 tx=0x7fc7f0004d80 comp rx=0 tx=0).stop 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 shutdown_connections 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8001059a0 0x7fc8001a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc800103f50 0x7fc80019a9a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc7ec0b2900 0x7fc7ec0b4e10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 --2- 172.21.15.19:0/182997911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8001049d0 0x7fc80019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:04.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 >> 172.21.15.19:0/182997911 conn(0x7fc8000ff9c0 msgr2=0x7fc800108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 shutdown_connections 2024-05-17T09:14:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:04.131+0000 7fc804d6a700 1 -- 172.21.15.19:0/182997911 wait complete. 2024-05-17T09:14:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: purged_snaps scrub starts 2024-05-17T09:14:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: purged_snaps scrub ok 2024-05-17T09:14:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] boot 2024-05-17T09:14:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T09:14:04.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: purged_snaps scrub starts 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: purged_snaps scrub ok 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] boot 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T09:14:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: purged_snaps scrub starts 2024-05-17T09:14:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: purged_snaps scrub ok 2024-05-17T09:14:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-17T09:14:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: osd.11 [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] boot 2024-05-17T09:14:04.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T09:14:04.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T09:14:04.830 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715937243,"num_in_osds":12,"osd_in_since":1715937223,"num_remapped_pgs":0} 2024-05-17T09:14:04.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:14:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/182997911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:05.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:05.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/182997911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:05.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/182997911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T09:14:05.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T09:14:05.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:05.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:06 smithi019 ceph-mon[31620]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:06 smithi019 ceph-mon[31620]: Detected new or changed devices on smithi079 2024-05-17T09:14:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:06 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 4272M 2024-05-17T09:14:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:06 smithi069 ceph-mon[35081]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:06 smithi069 ceph-mon[35081]: Detected new or changed devices on smithi079 2024-05-17T09:14:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:06 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 4272M 2024-05-17T09:14:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:06 smithi079 ceph-mon[35502]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:06 smithi079 ceph-mon[35502]: Detected new or changed devices on smithi079 2024-05-17T09:14:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:06 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 4272M 2024-05-17T09:14:06.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 -- 172.21.15.19:0/205442001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70109850 msgr2=0x7f0e701111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:06.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 --2- 172.21.15.19:0/205442001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70109850 0x7f0e701111b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0e680099c0 tx=0x7f0e68009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:06.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 -- 172.21.15.19:0/205442001 shutdown_connections 2024-05-17T09:14:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 --2- 172.21.15.19:0/205442001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e701045d0 0x7f0e701049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 --2- 172.21.15.19:0/205442001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70109850 0x7f0e701111b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 --2- 172.21.15.19:0/205442001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e70108f00 0x7f0e70109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 -- 172.21.15.19:0/205442001 >> 172.21.15.19:0/205442001 conn(0x7f0e700ffec0 msgr2=0x7f0e70102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.840+0000 7f0e77edb700 1 -- 172.21.15.19:0/205442001 shutdown_connections 2024-05-17T09:14:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.841+0000 7f0e77edb700 1 -- 172.21.15.19:0/205442001 wait complete. 2024-05-17T09:14:06.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.841+0000 7f0e77edb700 1 Processor -- start 2024-05-17T09:14:06.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.841+0000 7f0e77edb700 1 -- start start 2024-05-17T09:14:06.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e77edb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e701045d0 0x7f0e70076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:06.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e77edb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 0x7f0e70077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e77edb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e70109850 0x7f0e70077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e77edb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e70114330 con 0x7f0e70108f00 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e77edb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701141b0 con 0x7f0e70109850 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e77edb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701144b0 con 0x7f0e701045d0 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 0x7f0e70077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 0x7f0e70077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38536/0 (socket says 172.21.15.19:38536) 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 -- 172.21.15.19:0/3344811959 learned_addr learned my addr 172.21.15.19:0/3344811959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75c77700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e701045d0 0x7f0e70076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e76478700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e70109850 0x7f0e70077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 -- 172.21.15.19:0/3344811959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e701045d0 msgr2=0x7f0e70076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e701045d0 0x7f0e70076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 -- 172.21.15.19:0/3344811959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e70109850 msgr2=0x7f0e70077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e70109850 0x7f0e70077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 -- 172.21.15.19:0/3344811959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e701a9080 con 0x7f0e70108f00 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.842+0000 7f0e75476700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 0x7f0e70077100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400d980 tx=0x7f0e6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.843+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64017070 con 0x7f0e70108f00 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.843+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e701a9370 con 0x7f0e70108f00 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.843+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e64009dc0 con 0x7f0e70108f00 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.843+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400f3c0 con 0x7f0e70108f00 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.843+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e701a98b0 con 0x7f0e70108f00 2024-05-17T09:14:06.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.844+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e700512b0 con 0x7f0e70108f00 2024-05-17T09:14:06.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.845+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e64009930 con 0x7f0e70108f00 2024-05-17T09:14:06.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.846+0000 7f0e6effd700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0e580b2880 0x7f0e580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:06.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.846+0000 7f0e75c77700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0e580b2880 0x7f0e580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:06.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.846+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0e640e1800 con 0x7f0e70108f00 2024-05-17T09:14:06.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.847+0000 7f0e75c77700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0e580b2880 0x7f0e580b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e600060b0 tx=0x7f0e6000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:06.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:06.850+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e640a99c0 con 0x7f0e70108f00 2024-05-17T09:14:07.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.071+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e7002e5e0 con 0x7f0e70108f00 2024-05-17T09:14:07.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.072+0000 7f0e6effd700 1 -- 172.21.15.19:0/3344811959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18382 (secure 0 0 0) 0x7f0e640a92c0 con 0x7f0e70108f00 2024-05-17T09:14:07.075 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:07.075 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":71,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:14:04.563477+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:14:07.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0e580b2880 msgr2=0x7f0e580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:07.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0e580b2880 0x7f0e580b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e600060b0 tx=0x7f0e6000c040 comp rx=0 tx=0).stop 2024-05-17T09:14:07.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 msgr2=0x7f0e70077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 0x7f0e70077100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400d980 tx=0x7f0e6400dc90 comp rx=0 tx=0).stop 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 shutdown_connections 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0e580b2880 0x7f0e580b4d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e701045d0 0x7f0e70076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e70109850 0x7f0e70077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 --2- 172.21.15.19:0/3344811959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70108f00 0x7f0e70077100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.074+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 >> 172.21.15.19:0/3344811959 conn(0x7f0e700ffec0 msgr2=0x7f0e701086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.075+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 shutdown_connections 2024-05-17T09:14:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:07.075+0000 7f0e77edb700 1 -- 172.21.15.19:0/3344811959 wait complete. 2024-05-17T09:14:07.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:07 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3344811959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:14:07.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:07 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3344811959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:14:07.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:07 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3344811959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:14:07.924 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T09:10:10.865915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T09:14:07.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-17T09:14:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:08.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:08 smithi019 ceph-mon[31620]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:08 smithi069 ceph-mon[35081]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:08 smithi079 ceph-mon[35502]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.992+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/4049304758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc1045d0 msgr2=0x7f7cdc1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.992+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/4049304758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc1045d0 0x7f7cdc1049e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0099c0 tx=0x7f7ccc009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/4049304758 shutdown_connections 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/4049304758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc109850 0x7f7cdc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/4049304758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc1045d0 0x7f7cdc1049e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/4049304758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc108f00 0x7f7cdc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/4049304758 >> 172.21.15.19:0/4049304758 conn(0x7f7cdc0ffe60 msgr2=0x7f7cdc1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:09.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/4049304758 shutdown_connections 2024-05-17T09:14:09.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/4049304758 wait complete. 2024-05-17T09:14:09.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.993+0000 7f7ce0c7a700 1 Processor -- start 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 -- start start 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1045d0 0x7f7cdc075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 0x7f7cdc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc109850 0x7f7cdc076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1143a0 con 0x7f7cdc108f00 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc114220 con 0x7f7cdc1045d0 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7ce0c7a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc114520 con 0x7f7cdc109850 2024-05-17T09:14:09.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7cd1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 0x7f7cdc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7cd1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 0x7f7cdc075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38556/0 (socket says 172.21.15.19:38556) 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7cd1d9b700 1 -- 172.21.15.19:0/48989490 learned_addr learned my addr 172.21.15.19:0/48989490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7cda59c700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1045d0 0x7f7cdc075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.994+0000 7f7cdad9d700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc109850 0x7f7cdc076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd1d9b700 1 -- 172.21.15.19:0/48989490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc109850 msgr2=0x7f7cdc076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd1d9b700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc109850 0x7f7cdc076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd1d9b700 1 -- 172.21.15.19:0/48989490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1045d0 msgr2=0x7f7cdc075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd1d9b700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1045d0 0x7f7cdc075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd1d9b700 1 -- 172.21.15.19:0/48989490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc1a9120 con 0x7f7cdc108f00 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cdad9d700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc109850 0x7f7cdc076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd1d9b700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 0x7f7cdc075ff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400b500 tx=0x7f7cc400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4015020 con 0x7f7cdc108f00 2024-05-17T09:14:09.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc1a9410 con 0x7f7cdc108f00 2024-05-17T09:14:09.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cc4011cc0 con 0x7f7cdc108f00 2024-05-17T09:14:09.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4010c80 con 0x7f7cdc108f00 2024-05-17T09:14:09.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.995+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1a99a0 con 0x7f7cdc108f00 2024-05-17T09:14:09.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.996+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cdc0512b0 con 0x7f7cdc108f00 2024-05-17T09:14:10.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.998+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7cc4018020 con 0x7f7cdc108f00 2024-05-17T09:14:10.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.998+0000 7f7cd3fff700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7cc80b2880 0x7f7cc80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:10.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.998+0000 7f7cda59c700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7cc80b2880 0x7f7cc80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:10.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.998+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7cc40e1550 con 0x7f7cdc108f00 2024-05-17T09:14:10.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:09.999+0000 7f7cda59c700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7cc80b2880 0x7f7cc80b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009990 tx=0x7f7ccc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.003+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cc401f020 con 0x7f7cdc108f00 2024-05-17T09:14:10.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.219+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7cdc076f40 con 0x7f7cdc108f00 2024-05-17T09:14:10.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.220+0000 7f7cd3fff700 1 -- 172.21.15.19:0/48989490 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f7cc40a9710 con 0x7f7cdc108f00 2024-05-17T09:14:10.220 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-17T09:14:10.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7cc80b2880 msgr2=0x7f7cc80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:10.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7cc80b2880 0x7f7cc80b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009990 tx=0x7f7ccc005a90 comp rx=0 tx=0).stop 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 msgr2=0x7f7cdc075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 0x7f7cdc075ff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400b500 tx=0x7f7cc400b810 comp rx=0 tx=0).stop 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 shutdown_connections 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7cc80b2880 0x7f7cc80b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cdc109850 0x7f7cdc076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cdc1045d0 0x7f7cdc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 --2- 172.21.15.19:0/48989490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cdc108f00 0x7f7cdc075ff0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 >> 172.21.15.19:0/48989490 conn(0x7f7cdc0ffe60 msgr2=0x7f7cdc1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 shutdown_connections 2024-05-17T09:14:10.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:10.222+0000 7f7ce0c7a700 1 -- 172.21.15.19:0/48989490 wait complete. 2024-05-17T09:14:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:10 smithi069 ceph-mon[35081]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:10.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:10 smithi019 ceph-mon[31620]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:10 smithi079 ceph-mon[35502]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:10.948 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-05-17T09:14:10.948 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.b on smithi069 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.e on smithi069 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.h on smithi069 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.k on smithi069 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.c on smithi079 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.f on smithi079 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.i on smithi079 2024-05-17T09:14:10.949 INFO:tasks.cephadm:Adding mds.l on smithi079 2024-05-17T09:14:10.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l' 2024-05-17T09:14:11.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:11 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/48989490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:14:11.207 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:14:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:11 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/48989490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:14:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:11 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/48989490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:14:12.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:12 smithi079 ceph-mon[35502]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:12 smithi069 ceph-mon[35081]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:12.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:12 smithi019 ceph-mon[31620]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 -- 172.21.15.79:0/3874682432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f324410a100 msgr2=0x7f324410c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 --2- 172.21.15.79:0/3874682432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f324410a100 0x7f324410c520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b0a0 tx=0x7f323c00b3b0 comp rx=0 tx=0).stop 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 -- 172.21.15.79:0/3874682432 shutdown_connections 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 --2- 172.21.15.79:0/3874682432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f324410a100 0x7f324410c520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 --2- 172.21.15.79:0/3874682432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3244104e40 0x7f3244107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 --2- 172.21.15.79:0/3874682432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32441077a0 0x7f3244109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:12.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 -- 172.21.15.79:0/3874682432 >> 172.21.15.79:0/3874682432 conn(0x7f32440fe880 msgr2=0x7f3244100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:12.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 -- 172.21.15.79:0/3874682432 shutdown_connections 2024-05-17T09:14:12.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.793+0000 7f3248a0d700 1 -- 172.21.15.79:0/3874682432 wait complete. 2024-05-17T09:14:12.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.794+0000 7f3248a0d700 1 Processor -- start 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.794+0000 7f3248a0d700 1 -- start start 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.794+0000 7f3248a0d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244104e40 0x7f32441046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.794+0000 7f3248a0d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 0x7f3244102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.794+0000 7f3248a0d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324410a100 0x7f3244103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.794+0000 7f3248a0d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f324410fc70 con 0x7f324410a100 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3248a0d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f324410faf0 con 0x7f3244104e40 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3248a0d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f324410fdf0 con 0x7f32441077a0 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 0x7f3244102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 0x7f3244102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47852/0 (socket says 172.21.15.79:47852) 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 -- 172.21.15.79:0/885565638 learned_addr learned my addr 172.21.15.79:0/885565638 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f324259c700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244104e40 0x7f32441046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:12.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3242d9d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324410a100 0x7f3244103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 -- 172.21.15.79:0/885565638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244104e40 msgr2=0x7f32441046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244104e40 0x7f32441046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 -- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324410a100 msgr2=0x7f3244103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324410a100 0x7f3244103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 -- 172.21.15.79:0/885565638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32441039f0 con 0x7f32441077a0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3242d9d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324410a100 0x7f3244103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3239d9b700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 0x7f3244102d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f322c00d980 tx=0x7f322c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c017070 con 0x7f32441077a0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32441a4850 con 0x7f32441077a0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f322c005020 con 0x7f32441077a0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c009ab0 con 0x7f32441077a0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.795+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32441a4d30 con 0x7f32441077a0 2024-05-17T09:14:12.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.796+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32440512b0 con 0x7f32441077a0 2024-05-17T09:14:12.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.798+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f322c0051d0 con 0x7f32441077a0 2024-05-17T09:14:12.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.798+0000 7f323bfff700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f32300b2880 0x7f32300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:12.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.799+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f322c0e2060 con 0x7f32441077a0 2024-05-17T09:14:12.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.799+0000 7f324259c700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f32300b2880 0x7f32300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:12.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.799+0000 7f324259c700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f32300b2880 0x7f32300b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3234005950 tx=0x7f323400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:12.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:12.802+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f322c020020 con 0x7f32441077a0 2024-05-17T09:14:13.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.016+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}) -- 0x7f3244063cf0 con 0x7f32300b2880 2024-05-17T09:14:13.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.026+0000 7f323bfff700 1 -- 172.21.15.79:0/885565638 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3244063cf0 con 0x7f32300b2880 2024-05-17T09:14:13.027 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mds.all update... 2024-05-17T09:14:13.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f32300b2880 msgr2=0x7f32300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:13.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f32300b2880 0x7f32300b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3234005950 tx=0x7f323400a300 comp rx=0 tx=0).stop 2024-05-17T09:14:13.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 msgr2=0x7f3244102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:13.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 0x7f3244102d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f322c00d980 tx=0x7f322c00dc90 comp rx=0 tx=0).stop 2024-05-17T09:14:13.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 shutdown_connections 2024-05-17T09:14:13.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324410a100 0x7f3244103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3244104e40 0x7f32441046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f32300b2880 0x7f32300b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 --2- 172.21.15.79:0/885565638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32441077a0 0x7f3244102d50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 >> 172.21.15.79:0/885565638 conn(0x7f32440fe880 msgr2=0x7f3244105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.028+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 shutdown_connections 2024-05-17T09:14:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-17T09:14:13.029+0000 7f3248a0d700 1 -- 172.21.15.79:0/885565638 wait complete. 2024-05-17T09:14:13.723 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.a.service 2024-05-17T09:14:13.726 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.d.service 2024-05-17T09:14:13.769 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.g.service 2024-05-17T09:14:13.812 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.j.service 2024-05-17T09:14:13.855 DEBUG:teuthology.orchestra.run.smithi069:mds.b> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.b.service 2024-05-17T09:14:13.858 DEBUG:teuthology.orchestra.run.smithi069:mds.e> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.e.service 2024-05-17T09:14:13.900 DEBUG:teuthology.orchestra.run.smithi069:mds.h> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.h.service 2024-05-17T09:14:13.943 DEBUG:teuthology.orchestra.run.smithi069:mds.k> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.k.service 2024-05-17T09:14:13.986 DEBUG:teuthology.orchestra.run.smithi079:mds.c> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.c.service 2024-05-17T09:14:13.989 DEBUG:teuthology.orchestra.run.smithi079:mds.f> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.f.service 2024-05-17T09:14:14.032 DEBUG:teuthology.orchestra.run.smithi079:mds.i> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.i.service 2024-05-17T09:14:14.075 DEBUG:teuthology.orchestra.run.smithi079:mds.l> sudo journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.l.service 2024-05-17T09:14:14.118 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-17T09:14:14.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:14:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-17T09:14:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:14.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:14.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:14.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:14.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:14.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:14.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:14.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:14:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-17T09:14:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:14.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:14.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:14.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:14.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:14.273 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:14.389 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:14.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:14.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:14.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:14.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:15.384 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:15 smithi079 ceph-mon[35502]: Deploying daemon mds.l on smithi079 2024-05-17T09:14:15.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:15 smithi069 ceph-mon[35081]: Deploying daemon mds.l on smithi079 2024-05-17T09:14:15.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:15 smithi019 ceph-mon[31620]: Deploying daemon mds.l on smithi079 2024-05-17T09:14:16.022 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:15 smithi079 systemd[1]: Starting Ceph mds.l for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 -- 172.21.15.19:0/2917178077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060107e20 msgr2=0x7fe060108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 --2- 172.21.15.19:0/2917178077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060107e20 0x7fe060108230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0099c0 tx=0x7fe04c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 -- 172.21.15.19:0/2917178077 shutdown_connections 2024-05-17T09:14:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 --2- 172.21.15.19:0/2917178077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe060103540 0x7fe060103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 --2- 172.21.15.19:0/2917178077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe060108770 0x7fe0601100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 --2- 172.21.15.19:0/2917178077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060107e20 0x7fe060108230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 -- 172.21.15.19:0/2917178077 >> 172.21.15.19:0/2917178077 conn(0x7fe060078df0 msgr2=0x7fe060079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 -- 172.21.15.19:0/2917178077 shutdown_connections 2024-05-17T09:14:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.164+0000 7fe067129700 1 -- 172.21.15.19:0/2917178077 wait complete. 2024-05-17T09:14:16.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:16 smithi019 ceph-mon[31620]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 Processor -- start 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 -- start start 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe060103540 0x7fe06019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe060107e20 0x7fe06019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 0x7fe06019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe060113230 con 0x7fe060108770 2024-05-17T09:14:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601130b0 con 0x7fe060107e20 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe067129700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601133b0 con 0x7fe060103540 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.165+0000 7fe0656c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 0x7fe06019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 0x7fe06019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50336/0 (socket says 172.21.15.19:50336) 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 -- 172.21.15.19:0/3183934835 learned_addr learned my addr 172.21.15.19:0/3183934835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe064ec5700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe060103540 0x7fe06019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe05ffff700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe060107e20 0x7fe06019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 -- 172.21.15.19:0/3183934835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe060103540 msgr2=0x7fe06019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe060103540 0x7fe06019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 -- 172.21.15.19:0/3183934835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe060107e20 msgr2=0x7fe06019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe060107e20 0x7fe06019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 -- 172.21.15.19:0/3183934835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0601a6fc0 con 0x7fe060108770 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe0656c6700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 0x7fe06019c710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe058007ee0 tx=0x7fe05800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058004bb0 con 0x7fe060108770 2024-05-17T09:14:16.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0601a72b0 con 0x7fe060108770 2024-05-17T09:14:16.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe058004d50 con 0x7fe060108770 2024-05-17T09:14:16.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.166+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580058b0 con 0x7fe060108770 2024-05-17T09:14:16.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.167+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601a77c0 con 0x7fe060108770 2024-05-17T09:14:16.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.168+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0600512b0 con 0x7fe060108770 2024-05-17T09:14:16.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.169+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe05801a020 con 0x7fe060108770 2024-05-17T09:14:16.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.170+0000 7fe05dffb700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe0500b2880 0x7fe0500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:16.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.170+0000 7fe064ec5700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe0500b2880 0x7fe0500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:16.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.170+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe0580e1440 con 0x7fe060108770 2024-05-17T09:14:16.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.170+0000 7fe064ec5700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe0500b2880 0x7fe0500b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe054009c60 tx=0x7fe054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:16.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.175+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe058010070 con 0x7fe060108770 2024-05-17T09:14:16.356 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:16 smithi079 ceph-mon[35502]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:16.357 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 podman[65134]: 2024-05-17 09:14:15.986600429 +0000 UTC m=+0.016954817 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:16.357 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 podman[65134]: 2024-05-17 09:14:16.112942638 +0000 UTC m=+0.143296999 container create 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-17T09:14:16.357 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 podman[65134]: 2024-05-17 09:14:16.357692268 +0000 UTC m=+0.388046634 container init 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:14:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:16 smithi069 ceph-mon[35081]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:16.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.441+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe06002e5e0 con 0x7fe060108770 2024-05-17T09:14:16.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.441+0000 7fe05dffb700 1 -- 172.21.15.19:0/3183934835 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fe0580a9560 con 0x7fe060108770 2024-05-17T09:14:16.442 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:16.442 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe0500b2880 msgr2=0x7fe0500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe0500b2880 0x7fe0500b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe054009c60 tx=0x7fe054009260 comp rx=0 tx=0).stop 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 msgr2=0x7fe06019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 0x7fe06019c710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe058007ee0 tx=0x7fe05800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 shutdown_connections 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe0500b2880 0x7fe0500b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe060103540 0x7fe06019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe060108770 0x7fe06019c710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 --2- 172.21.15.19:0/3183934835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe060107e20 0x7fe06019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 >> 172.21.15.19:0/3183934835 conn(0x7fe060078df0 msgr2=0x7fe060101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 shutdown_connections 2024-05-17T09:14:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:16.444+0000 7fe067129700 1 -- 172.21.15.19:0/3183934835 wait complete. 2024-05-17T09:14:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-05-17T09:14:16.772 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 podman[65134]: 2024-05-17 09:14:16.36055427 +0000 UTC m=+0.390908633 container start 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-17T09:14:16.772 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 ceph-mds[65153]: -- 172.21.15.79:0/3935618246 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ac273ba40 con 0x558ac2783180 2024-05-17T09:14:16.772 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l[65149]: starting mds.l at 2024-05-17T09:14:16.772 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 bash[65134]: 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b 2024-05-17T09:14:16.772 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:14:16 smithi079 systemd[1]: Started Ceph mds.l for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:17.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.79:6832/1681609094,v1:172.21.15.79:6833/1681609094] up:boot 2024-05-17T09:14:17.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: fsmap 1 up:standby 2024-05-17T09:14:17.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T09:14:17.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3183934835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:17.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: Deploying daemon mds.k on smithi069 2024-05-17T09:14:17.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:17 smithi019 ceph-mon[31620]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.79:6832/1681609094,v1:172.21.15.79:6833/1681609094] up:boot 2024-05-17T09:14:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: fsmap 1 up:standby 2024-05-17T09:14:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T09:14:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3183934835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:17.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:17.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:17.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: Deploying daemon mds.k on smithi069 2024-05-17T09:14:17.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:17 smithi069 ceph-mon[35081]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.79:6832/1681609094,v1:172.21.15.79:6833/1681609094] up:boot 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: fsmap 1 up:standby 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3183934835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:17.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:17.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:17.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:17.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: Deploying daemon mds.k on smithi069 2024-05-17T09:14:17.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:17 smithi079 ceph-mon[35502]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:18.876 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:18 smithi069 systemd[1]: Starting Ceph mds.k for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:19.127 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:18 smithi069 podman[63808]: 2024-05-17 09:14:18.779327026 +0000 UTC m=+0.017423398 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:19.127 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:18 smithi069 podman[63808]: 2024-05-17 09:14:18.899864839 +0000 UTC m=+0.137961192 container create 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-17T09:14:19.197 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:19.430 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:19 smithi069 podman[63808]: 2024-05-17 09:14:19.127905909 +0000 UTC m=+0.366002272 container init 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-17T09:14:19.430 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:19 smithi069 podman[63808]: 2024-05-17 09:14:19.131228461 +0000 UTC m=+0.369324830 container start 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-17T09:14:19.430 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:19 smithi069 ceph-mds[63829]: -- 172.21.15.69:0/1764663033 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce5080da40 con 0x55ce50855600 2024-05-17T09:14:19.430 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k[63825]: starting mds.k at 2024-05-17T09:14:19.430 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:19 smithi069 bash[63808]: 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 2024-05-17T09:14:19.430 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:14:19 smithi069 systemd[1]: Started Ceph mds.k for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: Deploying daemon mds.j on smithi019 2024-05-17T09:14:20.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6832/700922121,v1:172.21.15.69:6833/700922121] up:boot 2024-05-17T09:14:20.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: fsmap 2 up:standby 2024-05-17T09:14:20.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T09:14:20.539 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:20.539 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.539 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.539 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.539 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:20.540 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:20.540 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:20.540 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: Deploying daemon mds.j on smithi019 2024-05-17T09:14:20.540 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6832/700922121,v1:172.21.15.69:6833/700922121] up:boot 2024-05-17T09:14:20.540 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: fsmap 2 up:standby 2024-05-17T09:14:20.540 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T09:14:20.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:20.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:20.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:20.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:20.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:20.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: Deploying daemon mds.j on smithi019 2024-05-17T09:14:20.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6832/700922121,v1:172.21.15.69:6833/700922121] up:boot 2024-05-17T09:14:20.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: fsmap 2 up:standby 2024-05-17T09:14:20.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 -- 172.21.15.19:0/2183286764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 msgr2=0x7f1810108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 --2- 172.21.15.19:0/2183286764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f1810108210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f18040099c0 tx=0x7f1804009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 -- 172.21.15.19:0/2183286764 shutdown_connections 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 --2- 172.21.15.19:0/2183286764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1810108750 0x7f18101100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 --2- 172.21.15.19:0/2183286764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18101034d0 0x7f18101038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 --2- 172.21.15.19:0/2183286764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f1810108210 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 -- 172.21.15.19:0/2183286764 >> 172.21.15.19:0/2183286764 conn(0x7f18100fef20 msgr2=0x7f1810101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:21.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 -- 172.21.15.19:0/2183286764 shutdown_connections 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.547+0000 7f18167da700 1 -- 172.21.15.19:0/2183286764 wait complete. 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 Processor -- start 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 -- start start 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18101034d0 0x7f181019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f181019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1810108750 0x7f181019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113210 con 0x7f1810107e00 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113090 con 0x7f1810108750 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.548+0000 7f18167da700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113390 con 0x7f18101034d0 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f181019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f181019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50356/0 (socket says 172.21.15.19:50356) 2024-05-17T09:14:21.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 -- 172.21.15.19:0/2625813173 learned_addr learned my addr 172.21.15.19:0/2625813173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f1814d77700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1810108750 0x7f181019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180ffff700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18101034d0 0x7f181019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 -- 172.21.15.19:0/2625813173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18101034d0 msgr2=0x7f181019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18101034d0 0x7f181019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 -- 172.21.15.19:0/2625813173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1810108750 msgr2=0x7f181019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1810108750 0x7f181019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 -- 172.21.15.19:0/2625813173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181019fc00 con 0x7f1810107e00 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180ffff700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18101034d0 0x7f181019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180f7fe700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f181019b2d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18040045e0 tx=0x7f1804004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804026070 con 0x7f1810107e00 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f180400f460 con 0x7f1810107e00 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181019fe90 con 0x7f1810107e00 2024-05-17T09:14:21.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804009e60 con 0x7f1810107e00 2024-05-17T09:14:21.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.549+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101a0370 con 0x7f1810107e00 2024-05-17T09:14:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.550+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18100512b0 con 0x7f1810107e00 2024-05-17T09:14:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.552+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1804029070 con 0x7f1810107e00 2024-05-17T09:14:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.553+0000 7f180d7fa700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17f80b2880 0x7f17f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.553+0000 7f180ffff700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17f80b2880 0x7f17f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.553+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f18040e2980 con 0x7f1810107e00 2024-05-17T09:14:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.554+0000 7f180ffff700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17f80b2880 0x7f17f80b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1800009be0 tx=0x7f1800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.556+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f180402f020 con 0x7f1810107e00 2024-05-17T09:14:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.817+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f181002e5e0 con 0x7f1810107e00 2024-05-17T09:14:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.817+0000 7f180d7fa700 1 -- 172.21.15.19:0/2625813173 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f1804013070 con 0x7f1810107e00 2024-05-17T09:14:21.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:21.818 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.819+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17f80b2880 msgr2=0x7f17f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.819+0000 7f18167da700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17f80b2880 0x7f17f80b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1800009be0 tx=0x7f1800009260 comp rx=0 tx=0).stop 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 msgr2=0x7f181019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f181019b2d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18040045e0 tx=0x7f1804004610 comp rx=0 tx=0).stop 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 shutdown_connections 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17f80b2880 0x7f17f80b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18101034d0 0x7f181019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1810108750 0x7f181019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 --2- 172.21.15.19:0/2625813173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1810107e00 0x7f181019b2d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 >> 172.21.15.19:0/2625813173 conn(0x7f18100fef20 msgr2=0x7f181006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 shutdown_connections 2024-05-17T09:14:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:21.820+0000 7f18167da700 1 -- 172.21.15.19:0/2625813173 wait complete. 2024-05-17T09:14:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-05-17T09:14:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:22 smithi069 ceph-mon[35081]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:22.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:22 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2625813173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:22 smithi019 ceph-mon[31620]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:22 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2625813173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:22 smithi079 ceph-mon[35502]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:22 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2625813173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:23.358 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 systemd[1]: Starting Ceph mds.j for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:23.692 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 podman[69577]: 2024-05-17 09:14:23.472298309 +0000 UTC m=+0.018376433 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:23.692 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 podman[69577]: 2024-05-17 09:14:23.584958329 +0000 UTC m=+0.131036462 container create 8c824afc6b5e6067a8fa4348b4ef65721223b573b0ac13ccfc4a3ae2da5483db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-17T09:14:23.962 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 podman[69577]: 2024-05-17 09:14:23.854233467 +0000 UTC m=+0.400311597 container init 8c824afc6b5e6067a8fa4348b4ef65721223b573b0ac13ccfc4a3ae2da5483db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-17T09:14:23.962 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 podman[69577]: 2024-05-17 09:14:23.857234083 +0000 UTC m=+0.403312195 container start 8c824afc6b5e6067a8fa4348b4ef65721223b573b0ac13ccfc4a3ae2da5483db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:14:23.963 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 ceph-mds[69596]: -- 172.21.15.19:0/3392969816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cb2963a40 con 0x560cb29ab600 2024-05-17T09:14:23.963 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j[69592]: starting mds.j at 2024-05-17T09:14:24.283 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 bash[69577]: 8c824afc6b5e6067a8fa4348b4ef65721223b573b0ac13ccfc4a3ae2da5483db 2024-05-17T09:14:24.283 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:14:23 smithi019 systemd[1]: Started Ceph mds.j for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:24.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:24.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:24.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:24.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:24.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:25.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:25 smithi069 ceph-mon[35081]: Deploying daemon mds.d on smithi019 2024-05-17T09:14:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:25 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.19:6834/4138238464,v1:172.21.15.19:6835/4138238464] up:boot 2024-05-17T09:14:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:25 smithi069 ceph-mon[35081]: fsmap 3 up:standby 2024-05-17T09:14:25.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:25 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T09:14:25.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:25 smithi019 ceph-mon[31620]: Deploying daemon mds.d on smithi019 2024-05-17T09:14:25.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:25 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.19:6834/4138238464,v1:172.21.15.19:6835/4138238464] up:boot 2024-05-17T09:14:25.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:25 smithi019 ceph-mon[31620]: fsmap 3 up:standby 2024-05-17T09:14:25.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:25 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T09:14:25.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:25 smithi079 ceph-mon[35502]: Deploying daemon mds.d on smithi019 2024-05-17T09:14:25.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:25 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.19:6834/4138238464,v1:172.21.15.19:6835/4138238464] up:boot 2024-05-17T09:14:25.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:25 smithi079 ceph-mon[35502]: fsmap 3 up:standby 2024-05-17T09:14:25.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:25 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T09:14:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:26 smithi069 ceph-mon[35081]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:26.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:26 smithi019 ceph-mon[31620]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:26 smithi079 ceph-mon[35502]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 -- 172.21.15.19:0/712445163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff781045e0 msgr2=0x7eff781049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 --2- 172.21.15.19:0/712445163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff781045e0 0x7eff781049f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff6800b0d0 tx=0x7eff6800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 -- 172.21.15.19:0/712445163 shutdown_connections 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 --2- 172.21.15.19:0/712445163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff78109860 0x7eff781111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 --2- 172.21.15.19:0/712445163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff781045e0 0x7eff781049f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 --2- 172.21.15.19:0/712445163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff78108f10 0x7eff78109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 -- 172.21.15.19:0/712445163 >> 172.21.15.19:0/712445163 conn(0x7eff780ffeb0 msgr2=0x7eff78102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 -- 172.21.15.19:0/712445163 shutdown_connections 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.641+0000 7eff7cd67700 1 -- 172.21.15.19:0/712445163 wait complete. 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.642+0000 7eff7cd67700 1 Processor -- start 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.642+0000 7eff7cd67700 1 -- start start 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7cd67700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff781045e0 0x7eff7819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7cd67700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff78108f10 0x7eff7819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7cd67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 0x7eff7819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7cd67700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff78114390 con 0x7eff78109860 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7cd67700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff78114210 con 0x7eff781045e0 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7cd67700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7eff78114510 con 0x7eff78108f10 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 0x7eff7819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 0x7eff7819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33646/0 (socket says 172.21.15.19:33646) 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 -- 172.21.15.19:0/1434716644 learned_addr learned my addr 172.21.15.19:0/1434716644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff7659c700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff781045e0 0x7eff7819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff75d9b700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff78108f10 0x7eff7819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 -- 172.21.15.19:0/1434716644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff78108f10 msgr2=0x7eff7819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff78108f10 0x7eff7819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 -- 172.21.15.19:0/1434716644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff781045e0 msgr2=0x7eff7819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff781045e0 0x7eff7819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 -- 172.21.15.19:0/1434716644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7819fc50 con 0x7eff78109860 2024-05-17T09:14:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.643+0000 7eff76d9d700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 0x7eff7819f5b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d980 tx=0x7eff7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.644+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70017070 con 0x7eff78109860 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.644+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff70005020 con 0x7eff78109860 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.644+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7819ff40 con 0x7eff78109860 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.644+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7000f980 con 0x7eff78109860 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.644+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff781a7860 con 0x7eff78109860 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.645+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff780512b0 con 0x7eff78109860 2024-05-17T09:14:27.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.647+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7eff70004a30 con 0x7eff78109860 2024-05-17T09:14:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.648+0000 7eff677fe700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff600b2900 0x7eff600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.648+0000 7eff7659c700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff600b2900 0x7eff600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.648+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7eff700e1e10 con 0x7eff78109860 2024-05-17T09:14:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.648+0000 7eff7659c700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff600b2900 0x7eff600b4e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff68009fd0 tx=0x7eff6800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.652+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff70020020 con 0x7eff78109860 2024-05-17T09:14:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.929+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff7819c0e0 con 0x7eff78109860 2024-05-17T09:14:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.930+0000 7eff677fe700 1 -- 172.21.15.19:0/1434716644 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7eff700aa050 con 0x7eff78109860 2024-05-17T09:14:27.930 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:27.931 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-17T09:14:27.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff600b2900 msgr2=0x7eff600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:27.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff600b2900 0x7eff600b4e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff68009fd0 tx=0x7eff6800b040 comp rx=0 tx=0).stop 2024-05-17T09:14:27.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 msgr2=0x7eff7819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 0x7eff7819f5b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d980 tx=0x7eff7000dc90 comp rx=0 tx=0).stop 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 shutdown_connections 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff78109860 0x7eff7819f5b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff781045e0 0x7eff7819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff600b2900 0x7eff600b4e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 --2- 172.21.15.19:0/1434716644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff78108f10 0x7eff7819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 >> 172.21.15.19:0/1434716644 conn(0x7eff780ffeb0 msgr2=0x7eff781022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 shutdown_connections 2024-05-17T09:14:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:27.933+0000 7eff7cd67700 1 -- 172.21.15.19:0/1434716644 wait complete. 2024-05-17T09:14:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1434716644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:28.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1434716644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1434716644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:28.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:28.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:28.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:29.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:29 smithi069 ceph-mon[35081]: Deploying daemon mds.e on smithi069 2024-05-17T09:14:29.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:29 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.19:6836/2308476934,v1:172.21.15.19:6837/2308476934] up:boot 2024-05-17T09:14:29.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:29 smithi069 ceph-mon[35081]: fsmap 4 up:standby 2024-05-17T09:14:29.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:29 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T09:14:29.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:29 smithi019 ceph-mon[31620]: Deploying daemon mds.e on smithi069 2024-05-17T09:14:29.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:29 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.19:6836/2308476934,v1:172.21.15.19:6837/2308476934] up:boot 2024-05-17T09:14:29.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:29 smithi019 ceph-mon[31620]: fsmap 4 up:standby 2024-05-17T09:14:29.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:29 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T09:14:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:29 smithi079 ceph-mon[35502]: Deploying daemon mds.e on smithi069 2024-05-17T09:14:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:29 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.19:6836/2308476934,v1:172.21.15.19:6837/2308476934] up:boot 2024-05-17T09:14:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:29 smithi079 ceph-mon[35502]: fsmap 4 up:standby 2024-05-17T09:14:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:29 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T09:14:30.583 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:30 smithi069 ceph-mon[35081]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:30.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:30 smithi079 ceph-mon[35502]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:30 smithi019 ceph-mon[31620]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:30.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:30.931 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:30 smithi069 podman[64126]: 2024-05-17 09:14:30.814121219 +0000 UTC m=+0.127510806 container create d569de8c19bf0f1bbcfcf4bfee4329348c0de2383d3cce7871cd6e5ba20a7e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:14:31.097 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:31.430 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:31 smithi069 podman[64126]: 2024-05-17 09:14:31.067410566 +0000 UTC m=+0.380800108 container init d569de8c19bf0f1bbcfcf4bfee4329348c0de2383d3cce7871cd6e5ba20a7e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-17T09:14:31.430 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:31 smithi069 podman[64126]: 2024-05-17 09:14:31.070508727 +0000 UTC m=+0.383898267 container start d569de8c19bf0f1bbcfcf4bfee4329348c0de2383d3cce7871cd6e5ba20a7e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-17T09:14:31.430 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:31 smithi069 ceph-mds[64145]: -- 172.21.15.69:0/3174787257 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eff0219a40 con 0x55eff0261600 2024-05-17T09:14:31.431 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e[64141]: starting mds.e at 2024-05-17T09:14:31.431 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:31 smithi069 bash[64126]: d569de8c19bf0f1bbcfcf4bfee4329348c0de2383d3cce7871cd6e5ba20a7e7d 2024-05-17T09:14:31.431 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:14:31 smithi069 systemd[1]: Started Ceph mds.e for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: Deploying daemon mds.f on smithi079 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6834/437548660,v1:172.21.15.69:6835/437548660] up:boot 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: fsmap 5 up:standby 2024-05-17T09:14:32.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: Deploying daemon mds.f on smithi079 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6834/437548660,v1:172.21.15.69:6835/437548660] up:boot 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: fsmap 5 up:standby 2024-05-17T09:14:32.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T09:14:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:32.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:32.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:32.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: Deploying daemon mds.f on smithi079 2024-05-17T09:14:32.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6834/437548660,v1:172.21.15.69:6835/437548660] up:boot 2024-05-17T09:14:32.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: fsmap 5 up:standby 2024-05-17T09:14:32.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T09:14:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 -- 172.21.15.19:0/1084354814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 msgr2=0x7fef801081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 --2- 172.21.15.19:0/1084354814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef801081f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef740099c0 tx=0x7fef74009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 -- 172.21.15.19:0/1084354814 shutdown_connections 2024-05-17T09:14:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 --2- 172.21.15.19:0/1084354814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef801034b0 0x7fef801038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 --2- 172.21.15.19:0/1084354814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef80108730 0x7fef80110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 --2- 172.21.15.19:0/1084354814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef801081f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 -- 172.21.15.19:0/1084354814 >> 172.21.15.19:0/1084354814 conn(0x7fef800fede0 msgr2=0x7fef80101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 -- 172.21.15.19:0/1084354814 shutdown_connections 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.815+0000 7fef86f18700 1 -- 172.21.15.19:0/1084354814 wait complete. 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 Processor -- start 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 -- start start 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef801034b0 0x7fef8019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef8019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef80108730 0x7fef8019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef801131f0 con 0x7fef80107de0 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef80113070 con 0x7fef801034b0 2024-05-17T09:14:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef86f18700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fef80113370 con 0x7fef80108730 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef8019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef8019c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33688/0 (socket says 172.21.15.19:33688) 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef7ffff700 1 -- 172.21.15.19:0/2207035133 learned_addr learned my addr 172.21.15.19:0/2207035133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef854b5700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef80108730 0x7fef8019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.816+0000 7fef84cb4700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef801034b0 0x7fef8019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7ffff700 1 -- 172.21.15.19:0/2207035133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef80108730 msgr2=0x7fef8019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7ffff700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef80108730 0x7fef8019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7ffff700 1 -- 172.21.15.19:0/2207035133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef801034b0 msgr2=0x7fef8019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7ffff700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef801034b0 0x7fef8019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7ffff700 1 -- 172.21.15.19:0/2207035133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801a6f00 con 0x7fef80107de0 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7ffff700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef8019c0d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef740045e0 tx=0x7fef74004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74026070 con 0x7fef80107de0 2024-05-17T09:14:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801a7190 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef7400f460 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74009e60 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.817+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a7640 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.818+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef800512b0 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.820+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fef74029070 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.821+0000 7fef7dffb700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fef680b2880 0x7fef680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.821+0000 7fef84cb4700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fef680b2880 0x7fef680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.821+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fef740e27f0 con 0x7fef80107de0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.821+0000 7fef84cb4700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fef680b2880 0x7fef680b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef70009c60 tx=0x7fef70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:32.824+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef7405edf0 con 0x7fef80107de0 2024-05-17T09:14:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.098+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef8002e5e0 con 0x7fef80107de0 2024-05-17T09:14:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.099+0000 7fef7dffb700 1 -- 172.21.15.19:0/2207035133 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7fef74013070 con 0x7fef80107de0 2024-05-17T09:14:33.099 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:33.099 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fef680b2880 msgr2=0x7fef680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fef680b2880 0x7fef680b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef70009c60 tx=0x7fef70009260 comp rx=0 tx=0).stop 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 msgr2=0x7fef8019c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef8019c0d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef740045e0 tx=0x7fef74004610 comp rx=0 tx=0).stop 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 shutdown_connections 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fef680b2880 0x7fef680b4d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef80108730 0x7fef8019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef801034b0 0x7fef8019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 --2- 172.21.15.19:0/2207035133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef80107de0 0x7fef8019c0d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 >> 172.21.15.19:0/2207035133 conn(0x7fef800fede0 msgr2=0x7fef80101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 shutdown_connections 2024-05-17T09:14:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:33.101+0000 7fef86f18700 1 -- 172.21.15.19:0/2207035133 wait complete. 2024-05-17T09:14:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-05-17T09:14:33.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:33 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2207035133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:33.514 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:33 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2207035133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:33.515 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 systemd[1]: Starting Ceph mds.f for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:33 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2207035133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:33.772 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 podman[65447]: 2024-05-17 09:14:33.417309169 +0000 UTC m=+0.017224587 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:33.772 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 podman[65447]: 2024-05-17 09:14:33.541084586 +0000 UTC m=+0.141000026 container create 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-17T09:14:34.215 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 podman[65447]: 2024-05-17 09:14:33.819004305 +0000 UTC m=+0.418919704 container init 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-17T09:14:34.215 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 podman[65447]: 2024-05-17 09:14:33.821809621 +0000 UTC m=+0.421725022 container start 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-17T09:14:34.215 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 ceph-mds[65467]: -- 172.21.15.79:0/3922502596 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e0137da40 con 0x555e013c5a80 2024-05-17T09:14:34.215 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f[65463]: starting mds.f at 2024-05-17T09:14:34.215 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 bash[65447]: 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 2024-05-17T09:14:34.215 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:14:33 smithi079 systemd[1]: Started Ceph mds.f for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:34.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:34.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:34.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:34.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:34.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:35.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:35 smithi079 ceph-mon[35502]: Deploying daemon mds.c on smithi079 2024-05-17T09:14:35.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:35 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.79:6834/1132484996,v1:172.21.15.79:6835/1132484996] up:boot 2024-05-17T09:14:35.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:35 smithi079 ceph-mon[35502]: fsmap 6 up:standby 2024-05-17T09:14:35.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T09:14:35.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:35 smithi069 ceph-mon[35081]: Deploying daemon mds.c on smithi079 2024-05-17T09:14:35.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:35 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.79:6834/1132484996,v1:172.21.15.79:6835/1132484996] up:boot 2024-05-17T09:14:35.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:35 smithi069 ceph-mon[35081]: fsmap 6 up:standby 2024-05-17T09:14:35.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T09:14:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:35 smithi019 ceph-mon[31620]: Deploying daemon mds.c on smithi079 2024-05-17T09:14:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:35 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.79:6834/1132484996,v1:172.21.15.79:6835/1132484996] up:boot 2024-05-17T09:14:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:35 smithi019 ceph-mon[31620]: fsmap 6 up:standby 2024-05-17T09:14:35.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T09:14:35.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:36.228 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 systemd[1]: Starting Ceph mds.c for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:36.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:36 smithi079 ceph-mon[35502]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:36.507 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 podman[65750]: 2024-05-17 09:14:36.142012733 +0000 UTC m=+0.017505596 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:36.507 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 podman[65750]: 2024-05-17 09:14:36.254779683 +0000 UTC m=+0.130272526 container create 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True) 2024-05-17T09:14:36.507 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 podman[65750]: 2024-05-17 09:14:36.507685542 +0000 UTC m=+0.383178412 container init 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-17T09:14:36.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:36 smithi019 ceph-mon[31620]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:36 smithi069 ceph-mon[35081]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:36.772 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 podman[65750]: 2024-05-17 09:14:36.510713283 +0000 UTC m=+0.386206133 container start 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True) 2024-05-17T09:14:36.772 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 ceph-mds[65769]: -- 172.21.15.79:0/2263967568 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ee7f5fa40 con 0x558ee7fa7a80 2024-05-17T09:14:36.772 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c[65765]: starting mds.c at 2024-05-17T09:14:36.773 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 bash[65750]: 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf 2024-05-17T09:14:36.773 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:14:36 smithi079 systemd[1]: Started Ceph mds.c for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 -- 172.21.15.19:0/3932830692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 msgr2=0x7fd0a4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/3932830692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a4076130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009990 tx=0x7fd0a0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 -- 172.21.15.19:0/3932830692 shutdown_connections 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/3932830692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4110b80 0x7fd0a4112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/3932830692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a4076130 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/3932830692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 0x7fd0a4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 -- 172.21.15.19:0/3932830692 >> 172.21.15.19:0/3932830692 conn(0x7fd0a40ffe60 msgr2=0x7fd0a41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:37.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 -- 172.21.15.19:0/3932830692 shutdown_connections 2024-05-17T09:14:37.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.780+0000 7fd0ab35c700 1 -- 172.21.15.19:0/3932830692 wait complete. 2024-05-17T09:14:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 Processor -- start 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 -- start start 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 0x7fd0a419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4110b80 0x7fd0a41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4116520 con 0x7fd0a4075d20 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41163a0 con 0x7fd0a4076670 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.781+0000 7fd0ab35c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41166a0 con 0x7fd0a4110b80 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a90f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 0x7fd0a419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a90f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37450/0 (socket says 172.21.15.19:37450) 2024-05-17T09:14:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a90f8700 1 -- 172.21.15.19:0/1789059298 learned_addr learned my addr 172.21.15.19:0/1789059298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a98f9700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4110b80 0x7fd0a41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 -- 172.21.15.19:0/1789059298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4110b80 msgr2=0x7fd0a41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4110b80 0x7fd0a41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 -- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 msgr2=0x7fd0a419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 -- 172.21.15.19:0/1789059298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41a1dc0 con 0x7fd0a4076670 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a90f8700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0a88f7700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 0x7fd0a419d490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd09400d950 tx=0x7fd09400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094017070 con 0x7fd0a4076670 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd094009dc0 con 0x7fd0a4076670 2024-05-17T09:14:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41a20b0 con 0x7fd0a4076670 2024-05-17T09:14:37.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.782+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094010cd0 con 0x7fd0a4076670 2024-05-17T09:14:37.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.783+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a99d0 con 0x7fd0a4076670 2024-05-17T09:14:37.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.783+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a40512b0 con 0x7fd0a4076670 2024-05-17T09:14:37.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.785+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd09401a070 con 0x7fd0a4076670 2024-05-17T09:14:37.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.786+0000 7fd09a7fc700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0900b2880 0x7fd0900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:37.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.786+0000 7fd0a90f8700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0900b2880 0x7fd0900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:37.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.786+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd0940e17c0 con 0x7fd0a4076670 2024-05-17T09:14:37.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.786+0000 7fd0a90f8700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0900b2880 0x7fd0900b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009750 tx=0x7fd0a0009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:37.789+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0940a98e0 con 0x7fd0a4076670 2024-05-17T09:14:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:37.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:37.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: Deploying daemon mds.h on smithi069 2024-05-17T09:14:37.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:37.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.79:6836/720193094,v1:172.21.15.79:6837/720193094] up:boot 2024-05-17T09:14:37.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: fsmap 7 up:standby 2024-05-17T09:14:37.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:37 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: Deploying daemon mds.h on smithi069 2024-05-17T09:14:37.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:37.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.79:6836/720193094,v1:172.21.15.79:6837/720193094] up:boot 2024-05-17T09:14:37.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: fsmap 7 up:standby 2024-05-17T09:14:37.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:37 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T09:14:38.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:38.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:38.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:38.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:38.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:38.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:38.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: Deploying daemon mds.h on smithi069 2024-05-17T09:14:38.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:38.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.79:6836/720193094,v1:172.21.15.79:6837/720193094] up:boot 2024-05-17T09:14:38.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: fsmap 7 up:standby 2024-05-17T09:14:38.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:37 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T09:14:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.055+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a402e5e0 con 0x7fd0a4076670 2024-05-17T09:14:38.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.056+0000 7fd09a7fc700 1 -- 172.21.15.19:0/1789059298 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6334 (secure 0 0 0) 0x7fd0940a91e0 con 0x7fd0a4076670 2024-05-17T09:14:38.058 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:38.058 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-17T09:14:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0900b2880 msgr2=0x7fd0900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0900b2880 0x7fd0900b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009750 tx=0x7fd0a0009ee0 comp rx=0 tx=0).stop 2024-05-17T09:14:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 msgr2=0x7fd0a419d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 0x7fd0a419d490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd09400d950 tx=0x7fd09400dc60 comp rx=0 tx=0).stop 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 shutdown_connections 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0900b2880 0x7fd0900b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4110b80 0x7fd0a41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0a4075d20 0x7fd0a419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 --2- 172.21.15.19:0/1789059298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a4076670 0x7fd0a419d490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 >> 172.21.15.19:0/1789059298 conn(0x7fd0a40ffe60 msgr2=0x7fd0a4101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 shutdown_connections 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:38.059+0000 7fd0ab35c700 1 -- 172.21.15.19:0/1789059298 wait complete. 2024-05-17T09:14:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-05-17T09:14:39.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:38 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1789059298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:39.127 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:38 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1789059298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:39.127 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:38 smithi069 systemd[1]: Starting Ceph mds.h for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:39.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:38 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1789059298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:39.393 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 podman[64428]: 2024-05-17 09:14:39.030421501 +0000 UTC m=+0.017142754 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:39.393 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 podman[64428]: 2024-05-17 09:14:39.164224654 +0000 UTC m=+0.150945929 container create fcc5974a4e124246e336b67980ec2d6fc6c136c45a8cdcdcd1ce91a9923f7ce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-17T09:14:39.393 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 podman[64428]: 2024-05-17 09:14:39.393888551 +0000 UTC m=+0.380609799 container init fcc5974a4e124246e336b67980ec2d6fc6c136c45a8cdcdcd1ce91a9923f7ce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD) 2024-05-17T09:14:39.680 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 podman[64428]: 2024-05-17 09:14:39.396817364 +0000 UTC m=+0.383538622 container start fcc5974a4e124246e336b67980ec2d6fc6c136c45a8cdcdcd1ce91a9923f7ce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:14:39.680 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mds[64448]: -- 172.21.15.69:0/2890958455 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587b8b49a40 con 0x5587b8b91a80 2024-05-17T09:14:39.681 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h[64444]: starting mds.h at 2024-05-17T09:14:39.681 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 bash[64428]: fcc5974a4e124246e336b67980ec2d6fc6c136c45a8cdcdcd1ce91a9923f7ce2 2024-05-17T09:14:39.681 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:14:39 smithi069 systemd[1]: Started Ceph mds.h for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:40.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:39 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:40.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:40.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:39 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:40.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:39 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:40.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:40.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:40 smithi079 ceph-mon[35502]: Deploying daemon mds.i on smithi079 2024-05-17T09:14:40.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:40 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:boot 2024-05-17T09:14:40.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:40 smithi079 ceph-mon[35502]: fsmap 8 up:standby 2024-05-17T09:14:40.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:40 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T09:14:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:41.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:40 smithi019 ceph-mon[31620]: Deploying daemon mds.i on smithi079 2024-05-17T09:14:41.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:40 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:boot 2024-05-17T09:14:41.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:40 smithi019 ceph-mon[31620]: fsmap 8 up:standby 2024-05-17T09:14:41.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:40 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T09:14:41.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:40 smithi069 ceph-mon[35081]: Deploying daemon mds.i on smithi079 2024-05-17T09:14:41.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:40 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:boot 2024-05-17T09:14:41.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:40 smithi069 ceph-mon[35081]: fsmap 8 up:standby 2024-05-17T09:14:41.182 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:40 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T09:14:41.747 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:41 smithi079 systemd[1]: Starting Ceph mds.i for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:41.996 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:41 smithi079 ceph-mon[35502]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:41.996 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:41 smithi079 podman[66054]: 2024-05-17 09:14:41.790203297 +0000 UTC m=+0.017757524 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:41.997 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:41 smithi079 podman[66054]: 2024-05-17 09:14:41.902088441 +0000 UTC m=+0.129642651 container create 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-17T09:14:42.085 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:41 smithi019 ceph-mon[31620]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:42.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:41 smithi069 ceph-mon[35081]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:42.272 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:42 smithi079 podman[66054]: 2024-05-17 09:14:42.149430647 +0000 UTC m=+0.376984865 container init 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-17T09:14:42.272 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:42 smithi079 podman[66054]: 2024-05-17 09:14:42.152202869 +0000 UTC m=+0.379757087 container start 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-17T09:14:42.273 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:42 smithi079 ceph-mds[66073]: -- 172.21.15.79:0/1503993158 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eaef1f1a40 con 0x55eaef239a80 2024-05-17T09:14:42.273 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i[66069]: starting mds.i at 2024-05-17T09:14:42.273 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:42 smithi079 bash[66054]: 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 2024-05-17T09:14:42.273 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:14:42 smithi079 systemd[1]: Started Ceph mds.i for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 -- 172.21.15.19:0/187497983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc105e20 msgr2=0x7f97cc10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 --2- 172.21.15.19:0/187497983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc105e20 0x7f97cc10e370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009990 tx=0x7f97bc009ca0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 -- 172.21.15.19:0/187497983 shutdown_connections 2024-05-17T09:14:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 --2- 172.21.15.19:0/187497983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97cc10e900 0x7f97cc110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 --2- 172.21.15.19:0/187497983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc1054d0 0x7f97cc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 --2- 172.21.15.19:0/187497983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc105e20 0x7f97cc10e370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 -- 172.21.15.19:0/187497983 >> 172.21.15.19:0/187497983 conn(0x7f97cc0fef20 msgr2=0x7f97cc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 -- 172.21.15.19:0/187497983 shutdown_connections 2024-05-17T09:14:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.702+0000 7f97d43cf700 1 -- 172.21.15.19:0/187497983 wait complete. 2024-05-17T09:14:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 Processor -- start 2024-05-17T09:14:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 -- start start 2024-05-17T09:14:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97cc1054d0 0x7f97cc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc105e20 0x7f97cc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 0x7f97cc19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc114310 con 0x7f97cc10e900 2024-05-17T09:14:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc114190 con 0x7f97cc105e20 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d43cf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc114490 con 0x7f97cc1054d0 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 0x7f97cc19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 0x7f97cc19f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37464/0 (socket says 172.21.15.19:37464) 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 -- 172.21.15.19:0/643344537 learned_addr learned my addr 172.21.15.19:0/643344537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d216b700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97cc1054d0 0x7f97cc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d196a700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc105e20 0x7f97cc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 -- 172.21.15.19:0/643344537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97cc1054d0 msgr2=0x7f97cc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97cc1054d0 0x7f97cc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 -- 172.21.15.19:0/643344537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc105e20 msgr2=0x7f97cc19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc105e20 0x7f97cc19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d296c700 1 -- 172.21.15.19:0/643344537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97cc19fc30 con 0x7f97cc10e900 2024-05-17T09:14:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.703+0000 7f97d196a700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc105e20 0x7f97cc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.704+0000 7f97d296c700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 0x7f97cc19f590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97c400ef30 tx=0x7f97c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.704+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40092e0 con 0x7f97cc10e900 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.704+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97cc19fec0 con 0x7f97cc10e900 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.704+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97c4004bb0 con 0x7f97cc10e900 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.704+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4004020 con 0x7f97cc10e900 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.704+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97cc1a7840 con 0x7f97cc10e900 2024-05-17T09:14:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.705+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97cc0512b0 con 0x7f97cc10e900 2024-05-17T09:14:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.707+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f97c401b030 con 0x7f97cc10e900 2024-05-17T09:14:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.707+0000 7f97c37fe700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97b80b2880 0x7f97b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.708+0000 7f97d216b700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97b80b2880 0x7f97b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.708+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f97c40e2230 con 0x7f97cc10e900 2024-05-17T09:14:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.708+0000 7f97d216b700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97b80b2880 0x7f97b80b4d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009c60 tx=0x7f97c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.711+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97c4022020 con 0x7f97cc10e900 2024-05-17T09:14:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.994+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97cc02e5e0 con 0x7f97cc10e900 2024-05-17T09:14:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.995+0000 7f97c37fe700 1 -- 172.21.15.19:0/643344537 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7986 (secure 0 0 0) 0x7f97c4010040 con 0x7f97cc10e900 2024-05-17T09:14:42.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:42.996 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97b80b2880 msgr2=0x7f97b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97b80b2880 0x7f97b80b4d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009c60 tx=0x7f97c8009260 comp rx=0 tx=0).stop 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 msgr2=0x7f97cc19f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 0x7f97cc19f590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97c400ef30 tx=0x7f97c400c3a0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 shutdown_connections 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97b80b2880 0x7f97b80b4d90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97cc1054d0 0x7f97cc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97cc10e900 0x7f97cc19f590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 --2- 172.21.15.19:0/643344537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc105e20 0x7f97cc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 >> 172.21.15.19:0/643344537 conn(0x7f97cc0fef20 msgr2=0x7f97cc0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 shutdown_connections 2024-05-17T09:14:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:42.998+0000 7f97d43cf700 1 -- 172.21.15.19:0/643344537 wait complete. 2024-05-17T09:14:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-05-17T09:14:43.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: Deploying daemon mds.g on smithi019 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] up:boot 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: fsmap 9 up:standby 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T09:14:43.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:43 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/643344537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:43.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: Deploying daemon mds.g on smithi019 2024-05-17T09:14:43.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] up:boot 2024-05-17T09:14:43.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: fsmap 9 up:standby 2024-05-17T09:14:43.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T09:14:43.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:43 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/643344537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:43.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: Deploying daemon mds.g on smithi019 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] up:boot 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: fsmap 9 up:standby 2024-05-17T09:14:43.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T09:14:43.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:43 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/643344537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:44 smithi019 ceph-mon[31620]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:44 smithi069 ceph-mon[35081]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:44 smithi079 ceph-mon[35502]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:45.442 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 systemd[1]: Starting Ceph mds.g for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:45.776 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 podman[70829]: 2024-05-17 09:14:45.424252484 +0000 UTC m=+0.017976670 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:45.777 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 podman[70829]: 2024-05-17 09:14:45.523654089 +0000 UTC m=+0.117378318 container create 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-17T09:14:45.777 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 podman[70829]: 2024-05-17 09:14:45.776719298 +0000 UTC m=+0.370443479 container init 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=) 2024-05-17T09:14:46.192 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 podman[70829]: 2024-05-17 09:14:45.780124145 +0000 UTC m=+0.373848311 container start 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=) 2024-05-17T09:14:46.192 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 ceph-mds[70848]: -- 172.21.15.19:0/3129421755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f86f93a40 con 0x559f86fdb180 2024-05-17T09:14:46.192 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g[70844]: starting mds.g at 2024-05-17T09:14:46.192 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 bash[70829]: 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c 2024-05-17T09:14:46.192 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:14:45 smithi019 systemd[1]: Started Ceph mds.g for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:46.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:46.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:46.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:46.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:47.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:47 smithi019 ceph-mon[31620]: Deploying daemon mds.a on smithi019 2024-05-17T09:14:47.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:47 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.19:6838/1964664313,v1:172.21.15.19:6839/1964664313] up:boot 2024-05-17T09:14:47.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:47 smithi019 ceph-mon[31620]: fsmap 10 up:standby 2024-05-17T09:14:47.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T09:14:47.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:47 smithi069 ceph-mon[35081]: Deploying daemon mds.a on smithi019 2024-05-17T09:14:47.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:47 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.19:6838/1964664313,v1:172.21.15.19:6839/1964664313] up:boot 2024-05-17T09:14:47.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:47 smithi069 ceph-mon[35081]: fsmap 10 up:standby 2024-05-17T09:14:47.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T09:14:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:47 smithi079 ceph-mon[35502]: Deploying daemon mds.a on smithi019 2024-05-17T09:14:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:47 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.19:6838/1964664313,v1:172.21.15.19:6839/1964664313] up:boot 2024-05-17T09:14:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:47 smithi079 ceph-mon[35502]: fsmap 10 up:standby 2024-05-17T09:14:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T09:14:48.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:48 smithi019 ceph-mon[31620]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:48 smithi069 ceph-mon[35081]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:48 smithi079 ceph-mon[35502]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:49.518 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 podman[71248]: 2024-05-17 09:14:49.516067255 +0000 UTC m=+0.254587440 container create 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 -- 172.21.15.19:0/689805547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3001034b0 msgr2=0x7ff3001038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 --2- 172.21.15.19:0/689805547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3001034b0 0x7ff3001038c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00099c0 tx=0x7ff2f0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 -- 172.21.15.19:0/689805547 shutdown_connections 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 --2- 172.21.15.19:0/689805547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300108730 0x7ff300110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 --2- 172.21.15.19:0/689805547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3001034b0 0x7ff3001038c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 --2- 172.21.15.19:0/689805547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff300107de0 0x7ff3001081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 -- 172.21.15.19:0/689805547 >> 172.21.15.19:0/689805547 conn(0x7ff3000fede0 msgr2=0x7ff300101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 -- 172.21.15.19:0/689805547 shutdown_connections 2024-05-17T09:14:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.517+0000 7ff30496b700 1 -- 172.21.15.19:0/689805547 wait complete. 2024-05-17T09:14:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 Processor -- start 2024-05-17T09:14:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 -- start start 2024-05-17T09:14:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3001034b0 0x7ff30019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:50.054 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 podman[71248]: 2024-05-17 09:14:49.802628403 +0000 UTC m=+0.541148597 container init 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:14:50.054 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 podman[71248]: 2024-05-17 09:14:49.805712603 +0000 UTC m=+0.544232805 container start 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-17T09:14:50.054 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 ceph-mds[71292]: -- 172.21.15.19:0/1686786684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f23df9da40 con 0x55f23dfe5a80 2024-05-17T09:14:50.055 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a[71288]: starting mds.a at 2024-05-17T09:14:50.055 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 bash[71248]: 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 2024-05-17T09:14:50.055 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:14:49 smithi019 systemd[1]: Started Ceph mds.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff300107de0 0x7ff30019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 0x7ff30019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff300113210 con 0x7ff300108730 2024-05-17T09:14:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.518+0000 7ff30496b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff300113090 con 0x7ff3001034b0 2024-05-17T09:14:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff30496b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff300113390 con 0x7ff300107de0 2024-05-17T09:14:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 0x7ff30019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 0x7ff30019f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40604/0 (socket says 172.21.15.19:40604) 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 -- 172.21.15.19:0/2274666656 learned_addr learned my addr 172.21.15.19:0/2274666656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fdd9b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff300107de0 0x7ff30019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fe59c700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3001034b0 0x7ff30019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 -- 172.21.15.19:0/2274666656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff300107de0 msgr2=0x7ff30019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff300107de0 0x7ff30019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 -- 172.21.15.19:0/2274666656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3001034b0 msgr2=0x7ff30019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3001034b0 0x7ff30019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 -- 172.21.15.19:0/2274666656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30019fc20 con 0x7ff300108730 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.519+0000 7ff2fed9d700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 0x7ff30019f580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800ada0 tx=0x7ff2f800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.520+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8017020 con 0x7ff300108730 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.520+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2f800cd60 con 0x7ff300108730 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.520+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8005280 con 0x7ff300108730 2024-05-17T09:14:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.520+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30019ff10 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.520+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001a7830 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.521+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3000512b0 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.522+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff2f800c8d0 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.523+0000 7ff2ef7fe700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2e80b2900 0x7ff2e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.523+0000 7ff2fe59c700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2e80b2900 0x7ff2e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.523+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff2f80e1470 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.524+0000 7ff2fe59c700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2e80b2900 0x7ff2e80b4e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000c00 tx=0x7ff2f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.528+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2f8021020 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.808+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff30002e5e0 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.808+0000 7ff2ef7fe700 1 -- 172.21.15.19:0/2274666656 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8812 (secure 0 0 0) 0x7ff2f80a96c0 con 0x7ff300108730 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2e80b2900 msgr2=0x7ff2e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2e80b2900 0x7ff2e80b4e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000c00 tx=0x7ff2f0005fb0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 msgr2=0x7ff30019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 0x7ff30019f580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800ada0 tx=0x7ff2f800c330 comp rx=0 tx=0).stop 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 shutdown_connections 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff300108730 0x7ff30019f580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3001034b0 0x7ff30019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2e80b2900 0x7ff2e80b4e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 --2- 172.21.15.19:0/2274666656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff300107de0 0x7ff30019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 >> 172.21.15.19:0/2274666656 conn(0x7ff3000fede0 msgr2=0x7ff300106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 shutdown_connections 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:49.811+0000 7ff30496b700 1 -- 172.21.15.19:0/2274666656 wait complete. 2024-05-17T09:14:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2274666656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:50.630 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2274666656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:50.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2274666656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T09:14:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:51.633 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:51 smithi069 ceph-mon[35081]: Deploying daemon mds.b on smithi069 2024-05-17T09:14:51.633 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:51 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.19:6840/2849498318,v1:172.21.15.19:6841/2849498318] up:boot 2024-05-17T09:14:51.633 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:51 smithi069 ceph-mon[35081]: fsmap 11 up:standby 2024-05-17T09:14:51.633 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:51 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T09:14:51.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:51 smithi019 ceph-mon[31620]: Deploying daemon mds.b on smithi069 2024-05-17T09:14:51.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:51 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.19:6840/2849498318,v1:172.21.15.19:6841/2849498318] up:boot 2024-05-17T09:14:51.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:51 smithi019 ceph-mon[31620]: fsmap 11 up:standby 2024-05-17T09:14:51.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:51 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T09:14:51.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:51 smithi079 ceph-mon[35502]: Deploying daemon mds.b on smithi069 2024-05-17T09:14:51.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:51 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.19:6840/2849498318,v1:172.21.15.19:6841/2849498318] up:boot 2024-05-17T09:14:51.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:51 smithi079 ceph-mon[35502]: fsmap 11 up:standby 2024-05-17T09:14:51.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:51 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T09:14:52.333 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 systemd[1]: Starting Ceph mds.b for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:14:52.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:52 smithi069 ceph-mon[35081]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:52.603 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 podman[64731]: 2024-05-17 09:14:52.246014289 +0000 UTC m=+0.017171455 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:14:52.603 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 podman[64731]: 2024-05-17 09:14:52.356669791 +0000 UTC m=+0.127826946 container create b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-17T09:14:52.603 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 podman[64731]: 2024-05-17 09:14:52.567424877 +0000 UTC m=+0.338582032 container init b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-17T09:14:52.603 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 podman[64731]: 2024-05-17 09:14:52.570345618 +0000 UTC m=+0.341502784 container start b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-17T09:14:52.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:52 smithi019 ceph-mon[31620]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:52.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:14:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:52 smithi079 ceph-mon[35502]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:52.930 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 ceph-mds[64750]: -- 172.21.15.69:0/3185875859 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca26ed7a40 con 0x55ca26f1f180 2024-05-17T09:14:52.930 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b[64746]: starting mds.b at 2024-05-17T09:14:52.930 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 bash[64731]: b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 2024-05-17T09:14:52.930 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:14:52 smithi069 systemd[1]: Started Ceph mds.b for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:14:52.976 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:53.987 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] up:boot 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: fsmap 12 up:standby 2024-05-17T09:14:53.988 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:53 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T09:14:53.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:53.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:53.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:53.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] up:boot 2024-05-17T09:14:53.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: fsmap 12 up:standby 2024-05-17T09:14:53.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:53 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T09:14:54.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:54.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:54.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:54.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:54.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:54.130 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:54.130 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:54.130 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] up:boot 2024-05-17T09:14:54.130 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: fsmap 12 up:standby 2024-05-17T09:14:54.130 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:53 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T09:14:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.798+0000 7f1f79d55700 1 -- 172.21.15.19:0/885558062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f741029b0 msgr2=0x7f1f74102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.798+0000 7f1f79d55700 1 --2- 172.21.15.19:0/885558062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f741029b0 0x7f1f74102dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098f0 tx=0x7f1f5c009c00 comp rx=0 tx=0).stop 2024-05-17T09:14:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 -- 172.21.15.19:0/885558062 shutdown_connections 2024-05-17T09:14:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 --2- 172.21.15.19:0/885558062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f74103300 0x7f1f7410f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 --2- 172.21.15.19:0/885558062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741089e0 0x7f1f74108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 --2- 172.21.15.19:0/885558062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f741029b0 0x7f1f74102dc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 -- 172.21.15.19:0/885558062 >> 172.21.15.19:0/885558062 conn(0x7f1f740fe6e0 msgr2=0x7f1f74100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 -- 172.21.15.19:0/885558062 shutdown_connections 2024-05-17T09:14:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 -- 172.21.15.19:0/885558062 wait complete. 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 Processor -- start 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.799+0000 7f1f79d55700 1 -- start start 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f79d55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741029b0 0x7f1f7419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f79d55700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 0x7f1f7419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f79d55700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f741089e0 0x7f1f7419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f79d55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741129e0 con 0x7f1f741029b0 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f79d55700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74112860 con 0x7f1f74103300 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f79d55700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74112b60 con 0x7f1f741089e0 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741029b0 0x7f1f7419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 0x7f1f7419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741029b0 0x7f1f7419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54886/0 (socket says 172.21.15.19:54886) 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 0x7f1f7419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49538/0 (socket says 172.21.15.19:49538) 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f737fe700 1 -- 172.21.15.19:0/2681581237 learned_addr learned my addr 172.21.15.19:0/2681581237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.800+0000 7f1f73fff700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f741089e0 0x7f1f7419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f6affd700 1 -- 172.21.15.19:0/2681581237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f741089e0 msgr2=0x7f1f7419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f6affd700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f741089e0 0x7f1f7419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f6affd700 1 -- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741029b0 msgr2=0x7f1f7419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f6affd700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741029b0 0x7f1f7419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f6affd700 1 -- 172.21.15.19:0/2681581237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7419fcc0 con 0x7f1f74103300 2024-05-17T09:14:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f6affd700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 0x7f1f7419b390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000c00 tx=0x7f1f5c00f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:54.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00b5e0 con 0x7f1f74103300 2024-05-17T09:14:54.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f5c005040 con 0x7f1f74103300 2024-05-17T09:14:54.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7419ff50 con 0x7f1f74103300 2024-05-17T09:14:54.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0038e0 con 0x7f1f74103300 2024-05-17T09:14:54.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.801+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f741a0450 con 0x7f1f74103300 2024-05-17T09:14:54.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.802+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f740512b0 con 0x7f1f74103300 2024-05-17T09:14:54.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.804+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1f5c0051f0 con 0x7f1f74103300 2024-05-17T09:14:54.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.804+0000 7f1f717fa700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f600b2900 0x7f1f600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:54.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.804+0000 7f1f737fe700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f600b2900 0x7f1f600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:54.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.804+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1f5c0e2da0 con 0x7f1f74103300 2024-05-17T09:14:54.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.805+0000 7f1f737fe700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f600b2900 0x7f1f600b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009750 tx=0x7f1f64009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:54.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:54.808+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f741a0450 con 0x7f1f74103300 2024-05-17T09:14:55.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.096+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f740ffbe0 con 0x7f1f74103300 2024-05-17T09:14:55.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.097+0000 7f1f717fa700 1 -- 172.21.15.19:0/2681581237 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10461 (secure 0 0 0) 0x7f1f5c013030 con 0x7f1f74103300 2024-05-17T09:14:55.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:55.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f600b2900 msgr2=0x7f1f600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f600b2900 0x7f1f600b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009750 tx=0x7f1f64009310 comp rx=0 tx=0).stop 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 msgr2=0x7f1f7419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 0x7f1f7419b390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000c00 tx=0x7f1f5c00f990 comp rx=0 tx=0).stop 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 shutdown_connections 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f600b2900 0x7f1f600b4e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f741089e0 0x7f1f7419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f741029b0 0x7f1f7419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 --2- 172.21.15.19:0/2681581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f74103300 0x7f1f7419b390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.099+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 >> 172.21.15.19:0/2681581237 conn(0x7f1f740fe6e0 msgr2=0x7f1f7406ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:55.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.100+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 shutdown_connections 2024-05-17T09:14:55.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:55.100+0000 7f1f79d55700 1 -- 172.21.15.19:0/2681581237 wait complete. 2024-05-17T09:14:55.101 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-05-17T09:14:55.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:55 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:55 smithi019 ceph-mon[31620]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:55.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:55 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2681581237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:55 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:55 smithi069 ceph-mon[35081]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:55 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2681581237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:55 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:56.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:55 smithi079 ceph-mon[35502]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:56.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:55 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2681581237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:14:56.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-17T09:14:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:14:57.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:57 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:57.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:57 smithi019 ceph-mon[31620]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:57.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:57 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:57.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:57 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:57 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:57 smithi069 ceph-mon[35081]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:58.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:57 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.181 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:57 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/2676091666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d58107de0 msgr2=0x7f5d581081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/2676091666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d58107de0 0x7f5d581081f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5d480099c0 tx=0x7f5d48009cd0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/2676091666 shutdown_connections 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/2676091666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58108730 0x7f5d58110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/2676091666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d581034b0 0x7f5d581038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/2676091666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d58107de0 0x7f5d581081f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/2676091666 >> 172.21.15.19:0/2676091666 conn(0x7f5d580fedb0 msgr2=0x7f5d581011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/2676091666 shutdown_connections 2024-05-17T09:14:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.236+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/2676091666 wait complete. 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.237+0000 7f5d5f3cb700 1 Processor -- start 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.237+0000 7f5d5f3cb700 1 -- start start 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5f3cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 0x7f5d5819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5f3cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58107de0 0x7f5d5819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5f3cb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d58108730 0x7f5d581a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5f3cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d581131f0 con 0x7f5d581034b0 2024-05-17T09:14:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5f3cb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58113070 con 0x7f5d58108730 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5f3cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58113370 con 0x7f5d58107de0 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 0x7f5d5819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 0x7f5d5819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54910/0 (socket says 172.21.15.19:54910) 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 -- 172.21.15.19:0/380734715 learned_addr learned my addr 172.21.15.19:0/380734715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5c966700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58107de0 0x7f5d5819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 -- 172.21.15.19:0/380734715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58107de0 msgr2=0x7f5d5819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58107de0 0x7f5d5819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 -- 172.21.15.19:0/380734715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d58108730 msgr2=0x7f5d581a1790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d58108730 0x7f5d581a1790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 -- 172.21.15.19:0/380734715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d581a1d80 con 0x7f5d581034b0 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5c966700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58107de0 0x7f5d5819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:14:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.238+0000 7f5d5d167700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 0x7f5d5819d000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400c9b0 tx=0x7f5d5400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.239+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54007b00 con 0x7f5d581034b0 2024-05-17T09:14:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.239+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d581a2070 con 0x7f5d581034b0 2024-05-17T09:14:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.239+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581a25d0 con 0x7f5d581034b0 2024-05-17T09:14:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.239+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d54007ca0 con 0x7f5d581034b0 2024-05-17T09:14:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.239+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54018660 con 0x7f5d581034b0 2024-05-17T09:14:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.240+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c005360 con 0x7f5d581034b0 2024-05-17T09:14:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.242+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5d540188c0 con 0x7f5d581034b0 2024-05-17T09:14:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.242+0000 7f5d4e7fc700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d440b2900 0x7f5d440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:14:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.243+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5d540e1fd0 con 0x7f5d581034b0 2024-05-17T09:14:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.243+0000 7f5d5c966700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d440b2900 0x7f5d440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:14:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.243+0000 7f5d5c966700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d440b2900 0x7f5d440b4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009990 tx=0x7f5d48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:14:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.246+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d54014030 con 0x7f5d581034b0 2024-05-17T09:14:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:57 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:57 smithi079 ceph-mon[35502]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:14:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:57 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:57 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.467+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5d3c005920 con 0x7f5d581034b0 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.467+0000 7f5d4e7fc700 1 -- 172.21.15.19:0/380734715 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f5d540aa2a0 con 0x7f5d581034b0 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:14:58.468 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-05-17T09:14:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.469+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d440b2900 msgr2=0x7f5d440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.469+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d440b2900 0x7f5d440b4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009990 tx=0x7f5d48005fd0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.469+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 msgr2=0x7f5d5819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:14:58.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 0x7f5d5819d000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400c9b0 tx=0x7f5d5400cd70 comp rx=0 tx=0).stop 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 shutdown_connections 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d58108730 0x7f5d581a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581034b0 0x7f5d5819d000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d440b2900 0x7f5d440b4e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 --2- 172.21.15.19:0/380734715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d58107de0 0x7f5d5819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 >> 172.21.15.19:0/380734715 conn(0x7f5d580fedb0 msgr2=0x7f5d581066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 shutdown_connections 2024-05-17T09:14:58.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:14:58.470+0000 7f5d5f3cb700 1 -- 172.21.15.19:0/380734715 wait complete. 2024-05-17T09:14:59.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:59.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:59.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:59.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:58 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/380734715' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T09:14:59.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:58 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:59.157 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T09:14:59.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T09:14:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:58 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/380734715' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T09:14:59.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:58 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:14:59.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:14:59.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:14:59.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:14:59.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:58 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/380734715' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T09:14:59.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:58 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:15:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:00.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:14:59 smithi069 ceph-mon[35081]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:00.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:14:59 smithi019 ceph-mon[31620]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:00.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:14:59 smithi079 ceph-mon[35502]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:01.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.470 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.471 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.471 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.471 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 -- 172.21.15.19:0/3643418366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff8108730 msgr2=0x7f5ff8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 --2- 172.21.15.19:0/3643418366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff8108730 0x7f5ff8110090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b0d0 tx=0x7f5ff000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 -- 172.21.15.19:0/3643418366 shutdown_connections 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 --2- 172.21.15.19:0/3643418366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff81034b0 0x7f5ff81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 --2- 172.21.15.19:0/3643418366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff8108730 0x7f5ff8110090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 --2- 172.21.15.19:0/3643418366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 -- 172.21.15.19:0/3643418366 >> 172.21.15.19:0/3643418366 conn(0x7f5ff80fedb0 msgr2=0x7f5ff81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 -- 172.21.15.19:0/3643418366 shutdown_connections 2024-05-17T09:15:01.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.686+0000 7f5fffe14700 1 -- 172.21.15.19:0/3643418366 wait complete. 2024-05-17T09:15:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.687+0000 7f5fffe14700 1 Processor -- start 2024-05-17T09:15:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.687+0000 7f5fffe14700 1 -- start start 2024-05-17T09:15:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5fffe14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 0x7f5ff819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5fffe14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5fffe14700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8108730 0x7f5ff8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5fffe14700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8113260 con 0x7f5ff81034b0 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5fffe14700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff81130e0 con 0x7f5ff8107de0 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5fffe14700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff81133e0 con 0x7f5ff8108730 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5ffdbb0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 0x7f5ff819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5ffdbb0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 0x7f5ff819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54932/0 (socket says 172.21.15.19:54932) 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5ffdbb0700 1 -- 172.21.15.19:0/142386438 learned_addr learned my addr 172.21.15.19:0/142386438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5ffd3af700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.688+0000 7f5ffe3b1700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8108730 0x7f5ff8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5ffdbb0700 1 -- 172.21.15.19:0/142386438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8108730 msgr2=0x7f5ff8199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5ffdbb0700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8108730 0x7f5ff8199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5ffdbb0700 1 -- 172.21.15.19:0/142386438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 msgr2=0x7f5ff819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5ffdbb0700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5ffdbb0700 1 -- 172.21.15.19:0/142386438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff819a040 con 0x7f5ff81034b0 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5ffdbb0700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 0x7f5ff819f7e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c9b0 tx=0x7f5ff400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4007b00 con 0x7f5ff81034b0 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff819a330 con 0x7f5ff81034b0 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ff400ce50 con 0x7f5ff81034b0 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4018660 con 0x7f5ff81034b0 2024-05-17T09:15:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.689+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff806b600 con 0x7f5ff81034b0 2024-05-17T09:15:01.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.691+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff8104960 con 0x7f5ff81034b0 2024-05-17T09:15:01.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.692+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5ff400f450 con 0x7f5ff81034b0 2024-05-17T09:15:01.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.694+0000 7f5feeffd700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5fe40b2880 0x7f5fe40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:01.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.694+0000 7f5ffd3af700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5fe40b2880 0x7f5fe40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:01.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.694+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5ff40e1d40 con 0x7f5ff81034b0 2024-05-17T09:15:01.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.694+0000 7f5ffd3af700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5fe40b2880 0x7f5fe40b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81023c0 tx=0x7f5fe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.700+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ff4014030 con 0x7f5ff81034b0 2024-05-17T09:15:01.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:01.925+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5ff80512b0 con 0x7f5ff81034b0 2024-05-17T09:15:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:15:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:15:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/142386438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/142386438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:15:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/142386438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T09:15:02.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.912+0000 7f5feeffd700 1 -- 172.21.15.19:0/142386438 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f5ff40a9e60 con 0x7f5ff81034b0 2024-05-17T09:15:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.916+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5fe40b2880 msgr2=0x7f5fe40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.916+0000 7f5fffe14700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5fe40b2880 0x7f5fe40b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81023c0 tx=0x7f5fe8009260 comp rx=0 tx=0).stop 2024-05-17T09:15:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.916+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 msgr2=0x7f5ff819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.916+0000 7f5fffe14700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 0x7f5ff819f7e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c9b0 tx=0x7f5ff400ccc0 comp rx=0 tx=0).stop 2024-05-17T09:15:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 shutdown_connections 2024-05-17T09:15:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5fe40b2880 0x7f5fe40b4d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8108730 0x7f5ff8199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff81034b0 0x7f5ff819f7e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 --2- 172.21.15.19:0/142386438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff8107de0 0x7f5ff819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 >> 172.21.15.19:0/142386438 conn(0x7f5ff80fedb0 msgr2=0x7f5ff8107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 shutdown_connections 2024-05-17T09:15:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:02.917+0000 7f5fffe14700 1 -- 172.21.15.19:0/142386438 wait complete. 2024-05-17T09:15:02.919 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-05-17T09:15:03.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-17T09:15:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:04.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:03 smithi069 ceph-mon[35081]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:04.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:03 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/142386438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T09:15:04.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:03 smithi069 ceph-mon[35081]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T09:15:04.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:03 smithi019 ceph-mon[31620]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:03 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/142386438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T09:15:04.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:03 smithi019 ceph-mon[31620]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T09:15:04.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:03 smithi079 ceph-mon[35502]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:04.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:03 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/142386438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T09:15:04.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:03 smithi079 ceph-mon[35502]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T09:15:05.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:04 smithi069 ceph-mon[35081]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T09:15:05.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:04 smithi019 ceph-mon[31620]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T09:15:05.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:04 smithi079 ceph-mon[35502]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T09:15:05.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 -- 172.21.15.19:0/2292852316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 msgr2=0x7f6704111190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:05.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 --2- 172.21.15.19:0/2292852316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f6704111190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00b0a0 tx=0x7f66fc00b3b0 comp rx=0 tx=0).stop 2024-05-17T09:15:05.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 -- 172.21.15.19:0/2292852316 shutdown_connections 2024-05-17T09:15:05.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 --2- 172.21.15.19:0/2292852316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f6704111190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 --2- 172.21.15.19:0/2292852316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6704109320 0x7f6704109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 --2- 172.21.15.19:0/2292852316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67041027b0 0x7f6704102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 -- 172.21.15.19:0/2292852316 >> 172.21.15.19:0/2292852316 conn(0x7f67040fe500 msgr2=0x7f6704100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 -- 172.21.15.19:0/2292852316 shutdown_connections 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.726+0000 7f6708f93700 1 -- 172.21.15.19:0/2292852316 wait complete. 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 Processor -- start 2024-05-17T09:15:05.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 -- start start 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67041027b0 0x7f670419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f670419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6704109320 0x7f670419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6704114470 con 0x7f6704103100 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67041142f0 con 0x7f6704109320 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f6708f93700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f67041145f0 con 0x7f67041027b0 2024-05-17T09:15:05.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f66fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f670419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f66fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f670419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58338/0 (socket says 172.21.15.19:58338) 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.727+0000 7f66fbfff700 1 -- 172.21.15.19:0/3575881540 learned_addr learned my addr 172.21.15.19:0/3575881540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f6702d9d700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6704109320 0x7f670419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f670259c700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67041027b0 0x7f670419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fbfff700 1 -- 172.21.15.19:0/3575881540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67041027b0 msgr2=0x7f670419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fbfff700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67041027b0 0x7f670419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fbfff700 1 -- 172.21.15.19:0/3575881540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6704109320 msgr2=0x7f670419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fbfff700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6704109320 0x7f670419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fbfff700 1 -- 172.21.15.19:0/3575881540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670419fc80 con 0x7f6704103100 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fbfff700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f670419b3a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00d980 tx=0x7f66ec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec017070 con 0x7f6704103100 2024-05-17T09:15:05.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66ec010460 con 0x7f6704103100 2024-05-17T09:15:05.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00de60 con 0x7f6704103100 2024-05-17T09:15:05.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670419ff70 con 0x7f6704103100 2024-05-17T09:15:05.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.728+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a7890 con 0x7f6704103100 2024-05-17T09:15:05.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.729+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67040512b0 con 0x7f6704103100 2024-05-17T09:15:05.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.731+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f66ec010610 con 0x7f6704103100 2024-05-17T09:15:05.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.732+0000 7f66fb7fe700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66f00b2900 0x7f66f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:05.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.732+0000 7f670259c700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66f00b2900 0x7f66f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:05.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.732+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f66ec0e1970 con 0x7f6704103100 2024-05-17T09:15:05.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.732+0000 7f670259c700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66f00b2900 0x7f66f00b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f670406cac0 tx=0x7f66f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:05.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.735+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66ec0a9a40 con 0x7f6704103100 2024-05-17T09:15:05.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:05.950+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f670402e5e0 con 0x7f6704103100 2024-05-17T09:15:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:05 smithi069 ceph-mon[35081]: pgmap v282: 33 pgs: 6 creating+activating, 6 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:05 smithi069 ceph-mon[35081]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T09:15:06.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:05 smithi069 ceph-mon[35081]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T09:15:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:05 smithi019 ceph-mon[31620]: pgmap v282: 33 pgs: 6 creating+activating, 6 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:05 smithi019 ceph-mon[31620]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T09:15:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:05 smithi019 ceph-mon[31620]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T09:15:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:05 smithi079 ceph-mon[35502]: pgmap v282: 33 pgs: 6 creating+activating, 6 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:05 smithi079 ceph-mon[35502]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T09:15:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:05 smithi079 ceph-mon[35502]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T09:15:06.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.935+0000 7f66fb7fe700 1 -- 172.21.15.19:0/3575881540 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7f66ec0a9340 con 0x7f6704103100 2024-05-17T09:15:06.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.940+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66f00b2900 msgr2=0x7f66f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:06.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.940+0000 7f6708f93700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66f00b2900 0x7f66f00b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f670406cac0 tx=0x7f66f4009260 comp rx=0 tx=0).stop 2024-05-17T09:15:06.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.940+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 msgr2=0x7f670419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:06.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.940+0000 7f6708f93700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f670419b3a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00d980 tx=0x7f66ec00dd40 comp rx=0 tx=0).stop 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 shutdown_connections 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66f00b2900 0x7f66f00b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67041027b0 0x7f670419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6704109320 0x7f670419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 --2- 172.21.15.19:0/3575881540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704103100 0x7f670419b3a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 >> 172.21.15.19:0/3575881540 conn(0x7f67040fe500 msgr2=0x7f67041071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 shutdown_connections 2024-05-17T09:15:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:06.941+0000 7f6708f93700 1 -- 172.21.15.19:0/3575881540 wait complete. 2024-05-17T09:15:06.946 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-05-17T09:15:07.180 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:06 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3575881540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T09:15:07.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3575881540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T09:15:07.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:06 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3575881540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T09:15:07.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T09:15:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:07 smithi079 ceph-mon[35502]: pgmap v284: 33 pgs: 6 creating+activating, 6 creating+peering, 7 unknown, 14 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:07 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3575881540' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T09:15:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:07 smithi079 ceph-mon[35502]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T09:15:08.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:07 smithi019 ceph-mon[31620]: pgmap v284: 33 pgs: 6 creating+activating, 6 creating+peering, 7 unknown, 14 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:08.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:07 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3575881540' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T09:15:08.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:07 smithi019 ceph-mon[31620]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T09:15:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:07 smithi069 ceph-mon[35081]: pgmap v284: 33 pgs: 6 creating+activating, 6 creating+peering, 7 unknown, 14 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:07 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3575881540' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T09:15:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:07 smithi069 ceph-mon[35081]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T09:15:09.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:08 smithi079 ceph-mon[35502]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T09:15:09.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:08 smithi019 ceph-mon[31620]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T09:15:09.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:08 smithi069 ceph-mon[35081]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 -- 172.21.15.19:0/1615986655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1390107de0 msgr2=0x7f13901081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 --2- 172.21.15.19:0/1615986655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1390107de0 0x7f13901081f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f13840099c0 tx=0x7f1384009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 -- 172.21.15.19:0/1615986655 shutdown_connections 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 --2- 172.21.15.19:0/1615986655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390108730 0x7f1390110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 --2- 172.21.15.19:0/1615986655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13901034b0 0x7f13901038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 --2- 172.21.15.19:0/1615986655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1390107de0 0x7f13901081f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 -- 172.21.15.19:0/1615986655 >> 172.21.15.19:0/1615986655 conn(0x7f13900fede0 msgr2=0x7f1390101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 -- 172.21.15.19:0/1615986655 shutdown_connections 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.673+0000 7f1395a17700 1 -- 172.21.15.19:0/1615986655 wait complete. 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 Processor -- start 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 -- start start 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 0x7f13901a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390107de0 0x7f13901a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1390108730 0x7f139019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13901131f0 con 0x7f13901034b0 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1390113070 con 0x7f1390108730 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f1395a17700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1390113370 con 0x7f1390107de0 2024-05-17T09:15:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.674+0000 7f138effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 0x7f13901a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 0x7f13901a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58342/0 (socket says 172.21.15.19:58342) 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 -- 172.21.15.19:0/3870818786 learned_addr learned my addr 172.21.15.19:0/3870818786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138e7fc700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390107de0 0x7f13901a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138f7fe700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1390108730 0x7f139019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 -- 172.21.15.19:0/3870818786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390107de0 msgr2=0x7f13901a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390107de0 0x7f13901a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 -- 172.21.15.19:0/3870818786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1390108730 msgr2=0x7f139019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1390108730 0x7f139019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 -- 172.21.15.19:0/3870818786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139019b2d0 con 0x7f13901034b0 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138e7fc700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390107de0 0x7f13901a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f138effd700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 0x7f13901a09c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9a0 tx=0x7f138000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000cbd0 con 0x7f13901034b0 2024-05-17T09:15:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f138000cd70 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139019b5c0 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380018680 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.675+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13901a7770 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.676+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1390104960 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.678+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1380010950 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.679+0000 7f1394a15700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13780b2900 0x7f13780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.679+0000 7f138e7fc700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13780b2900 0x7f13780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.679+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f13800e25a0 con 0x7f13901034b0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.679+0000 7f138e7fc700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13780b2900 0x7f13780b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1384009990 tx=0x7f138401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.683+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f13800aa3d0 con 0x7f13901034b0 2024-05-17T09:15:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.949+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f13901a7940 con 0x7f13901034b0 2024-05-17T09:15:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.956+0000 7f1394a15700 1 -- 172.21.15.19:0/3870818786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f13800a9cd0 con 0x7f13901034b0 2024-05-17T09:15:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13780b2900 msgr2=0x7f13780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13780b2900 0x7f13780b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1384009990 tx=0x7f138401a040 comp rx=0 tx=0).stop 2024-05-17T09:15:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 msgr2=0x7f13901a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 0x7f13901a09c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9a0 tx=0x7f138000ed60 comp rx=0 tx=0).stop 2024-05-17T09:15:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 shutdown_connections 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1390108730 0x7f139019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13901034b0 0x7f13901a09c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13780b2900 0x7f13780b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 --2- 172.21.15.19:0/3870818786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390107de0 0x7f13901a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 >> 172.21.15.19:0/3870818786 conn(0x7f13900fede0 msgr2=0x7f1390109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 shutdown_connections 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:09.960+0000 7f1395a17700 1 -- 172.21.15.19:0/3870818786 wait complete. 2024-05-17T09:15:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-05-17T09:15:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:09 smithi079 ceph-mon[35502]: pgmap v287: 97 pgs: 6 creating+activating, 6 creating+peering, 38 unknown, 47 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:09 smithi079 ceph-mon[35502]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T09:15:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:09 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3870818786' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T09:15:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:09 smithi079 ceph-mon[35502]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T09:15:10.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:09 smithi019 ceph-mon[31620]: pgmap v287: 97 pgs: 6 creating+activating, 6 creating+peering, 38 unknown, 47 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:10.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:09 smithi019 ceph-mon[31620]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T09:15:10.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:09 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3870818786' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T09:15:10.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:09 smithi019 ceph-mon[31620]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T09:15:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:09 smithi069 ceph-mon[35081]: pgmap v287: 97 pgs: 6 creating+activating, 6 creating+peering, 38 unknown, 47 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:09 smithi069 ceph-mon[35081]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T09:15:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:09 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3870818786' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T09:15:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:09 smithi069 ceph-mon[35081]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T09:15:10.760 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-17T09:15:10.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-17T09:15:11.021 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:11.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:10 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3870818786' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T09:15:11.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:10 smithi019 ceph-mon[31620]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T09:15:11.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:10 smithi019 ceph-mon[31620]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T09:15:11.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:10 smithi019 ceph-mon[31620]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T09:15:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:10 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3870818786' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T09:15:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:10 smithi079 ceph-mon[35502]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T09:15:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:10 smithi079 ceph-mon[35502]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T09:15:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:10 smithi079 ceph-mon[35502]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T09:15:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:10 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3870818786' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T09:15:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:10 smithi069 ceph-mon[35081]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T09:15:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:10 smithi069 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T09:15:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:10 smithi069 ceph-mon[35081]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T09:15:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:11 smithi019 ceph-mon[31620]: pgmap v290: 97 pgs: 6 creating+peering, 19 unknown, 72 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:11 smithi019 ceph-mon[31620]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T09:15:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:11 smithi019 ceph-mon[31620]: Cluster is now healthy 2024-05-17T09:15:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:11 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] up:active 2024-05-17T09:15:12.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:11 smithi019 ceph-mon[31620]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:12.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:11 smithi019 ceph-mon[31620]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:11 smithi079 ceph-mon[35502]: pgmap v290: 97 pgs: 6 creating+peering, 19 unknown, 72 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:11 smithi079 ceph-mon[35502]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T09:15:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:11 smithi079 ceph-mon[35502]: Cluster is now healthy 2024-05-17T09:15:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:11 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] up:active 2024-05-17T09:15:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:11 smithi079 ceph-mon[35502]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:12.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:11 smithi079 ceph-mon[35502]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:11 smithi069 ceph-mon[35081]: pgmap v290: 97 pgs: 6 creating+peering, 19 unknown, 72 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:11 smithi069 ceph-mon[35081]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T09:15:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:11 smithi069 ceph-mon[35081]: Cluster is now healthy 2024-05-17T09:15:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:11 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] up:active 2024-05-17T09:15:12.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:11 smithi069 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:12.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:11 smithi069 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 -- 172.21.15.19:0/1972100449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888109840 msgr2=0x7fe8881111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1972100449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888109840 0x7fe8881111a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8800099c0 tx=0x7fe880009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 -- 172.21.15.19:0/1972100449 shutdown_connections 2024-05-17T09:15:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1972100449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8881045c0 0x7fe8881049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1972100449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888109840 0x7fe8881111a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1972100449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe888108ef0 0x7fe888109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 -- 172.21.15.19:0/1972100449 >> 172.21.15.19:0/1972100449 conn(0x7fe8880ffe90 msgr2=0x7fe8881022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 -- 172.21.15.19:0/1972100449 shutdown_connections 2024-05-17T09:15:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.839+0000 7fe8902e6700 1 -- 172.21.15.19:0/1972100449 wait complete. 2024-05-17T09:15:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.840+0000 7fe8902e6700 1 Processor -- start 2024-05-17T09:15:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.840+0000 7fe8902e6700 1 -- start start 2024-05-17T09:15:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe8902e6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8881045c0 0x7fe88819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe8902e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 0x7fe88819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe8902e6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888109840 0x7fe88819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe8902e6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8881143a0 con 0x7fe888108ef0 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe8902e6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe888114220 con 0x7fe8881045c0 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe8902e6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe888114520 con 0x7fe888109840 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 0x7fe88819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 0x7fe88819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58360/0 (socket says 172.21.15.19:58360) 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 -- 172.21.15.19:0/1313318269 learned_addr learned my addr 172.21.15.19:0/1313318269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88e082700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8881045c0 0x7fe88819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88e883700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888109840 0x7fe88819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 -- 172.21.15.19:0/1313318269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888109840 msgr2=0x7fe88819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888109840 0x7fe88819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 -- 172.21.15.19:0/1313318269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8881045c0 msgr2=0x7fe88819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8881045c0 0x7fe88819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:12.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.841+0000 7fe88d881700 1 -- 172.21.15.19:0/1313318269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88819fbe0 con 0x7fe888108ef0 2024-05-17T09:15:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.842+0000 7fe88d881700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 0x7fe88819b2b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d950 tx=0x7fe87c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.842+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c016070 con 0x7fe888108ef0 2024-05-17T09:15:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.842+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88819fed0 con 0x7fe888108ef0 2024-05-17T09:15:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.842+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe87c004830 con 0x7fe888108ef0 2024-05-17T09:15:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.842+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0053e0 con 0x7fe888108ef0 2024-05-17T09:15:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.842+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881a77f0 con 0x7fe888108ef0 2024-05-17T09:15:12.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.843+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8880512b0 con 0x7fe888108ef0 2024-05-17T09:15:12.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.846+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe87c005580 con 0x7fe888108ef0 2024-05-17T09:15:12.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.846+0000 7fe8877fe700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe8700b2880 0x7fe8700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:12.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.846+0000 7fe88e082700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe8700b2880 0x7fe8700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:12.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.847+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fe87c0e23b0 con 0x7fe888108ef0 2024-05-17T09:15:12.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.847+0000 7fe88e082700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe8700b2880 0x7fe8700b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe87800a400 tx=0x7fe87800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:12.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:12.850+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe87c01f020 con 0x7fe888108ef0 2024-05-17T09:15:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.075+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe88802e5e0 con 0x7fe888108ef0 2024-05-17T09:15:13.680 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:15:13 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b[64746]: 2024-05-17T09:15:13.341+0000 7f927461a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T09:15:13.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.982+0000 7fe8877fe700 1 -- 172.21.15.19:0/1313318269 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7fe87c0aa220 con 0x7fe888108ef0 2024-05-17T09:15:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe8700b2880 msgr2=0x7fe8700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe8700b2880 0x7fe8700b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe87800a400 tx=0x7fe87800d040 comp rx=0 tx=0).stop 2024-05-17T09:15:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 msgr2=0x7fe88819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 0x7fe88819b2b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d950 tx=0x7fe87c00dc60 comp rx=0 tx=0).stop 2024-05-17T09:15:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 shutdown_connections 2024-05-17T09:15:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe8700b2880 0x7fe8700b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888109840 0x7fe88819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8881045c0 0x7fe88819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 --2- 172.21.15.19:0/1313318269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888108ef0 0x7fe88819b2b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.985+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 >> 172.21.15.19:0/1313318269 conn(0x7fe8880ffe90 msgr2=0x7fe88810a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.986+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 shutdown_connections 2024-05-17T09:15:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:13.986+0000 7fe8902e6700 1 -- 172.21.15.19:0/1313318269 wait complete. 2024-05-17T09:15:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:13 smithi079 ceph-mon[35502]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-05-17T09:15:14.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:13 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1313318269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T09:15:14.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:13 smithi019 ceph-mon[31620]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-05-17T09:15:14.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:13 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1313318269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T09:15:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:13 smithi069 ceph-mon[35081]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-05-17T09:15:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:13 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1313318269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T09:15:14.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T09:15:15.089 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:14 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1313318269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T09:15:15.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:14 smithi019 ceph-mon[31620]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T09:15:15.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:14 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1313318269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T09:15:15.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:14 smithi079 ceph-mon[35502]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T09:15:15.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:14 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1313318269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T09:15:15.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:14 smithi069 ceph-mon[35081]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T09:15:16.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:15 smithi079 ceph-mon[35502]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-17T09:15:16.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:15 smithi019 ceph-mon[31620]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-17T09:15:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:15 smithi069 ceph-mon[35081]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-17T09:15:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 -- 172.21.15.19:0/1943279195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 msgr2=0x7f51881081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 --2- 172.21.15.19:0/1943279195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f51881081f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f517c0099c0 tx=0x7f517c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 -- 172.21.15.19:0/1943279195 shutdown_connections 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 --2- 172.21.15.19:0/1943279195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5188108730 0x7f5188110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 --2- 172.21.15.19:0/1943279195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51881034b0 0x7f51881038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 --2- 172.21.15.19:0/1943279195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f51881081f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 -- 172.21.15.19:0/1943279195 >> 172.21.15.19:0/1943279195 conn(0x7f51880fede0 msgr2=0x7f5188101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 -- 172.21.15.19:0/1943279195 shutdown_connections 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.924+0000 7f518c807700 1 -- 172.21.15.19:0/1943279195 wait complete. 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 Processor -- start 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 -- start start 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51881034b0 0x7f518819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f518819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5188108730 0x7f518819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5188113240 con 0x7f5188107de0 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51881130c0 con 0x7f5188108730 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518c807700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51881133c0 con 0x7f51881034b0 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f5185d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f518819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f5185d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f518819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60130/0 (socket says 172.21.15.19:60130) 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f5185d9b700 1 -- 172.21.15.19:0/2021257264 learned_addr learned my addr 172.21.15.19:0/2021257264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f5186d9d700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5188108730 0x7f518819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.925+0000 7f518659c700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51881034b0 0x7f518819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f5185d9b700 1 -- 172.21.15.19:0/2021257264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51881034b0 msgr2=0x7f518819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f5185d9b700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51881034b0 0x7f518819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f5185d9b700 1 -- 172.21.15.19:0/2021257264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5188108730 msgr2=0x7f518819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f5185d9b700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5188108730 0x7f518819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f5185d9b700 1 -- 172.21.15.19:0/2021257264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518819fba0 con 0x7f5188107de0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f5185d9b700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f518819b320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f517c0098e0 tx=0x7f517c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c027070 con 0x7f5188107de0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f517c022470 con 0x7f5188107de0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518819fe30 con 0x7f5188107de0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c017650 con 0x7f5188107de0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.926+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881a0310 con 0x7f5188107de0 2024-05-17T09:15:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.927+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880512b0 con 0x7f5188107de0 2024-05-17T09:15:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.929+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f517c02a070 con 0x7f5188107de0 2024-05-17T09:15:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.930+0000 7f51777fe700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f51700b2880 0x7f51700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.930+0000 7f518659c700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f51700b2880 0x7f51700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.930+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f517c0e3540 con 0x7f5188107de0 2024-05-17T09:15:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.930+0000 7f518659c700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f51700b2880 0x7f51700b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f517800ac50 tx=0x7f517800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:16.933+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f517c031080 con 0x7f5188107de0 2024-05-17T09:15:17.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:17.150+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f518802e5e0 con 0x7f5188107de0 2024-05-17T09:15:18.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:17 smithi079 ceph-mon[35502]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-17T09:15:18.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:17 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2021257264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T09:15:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:17 smithi069 ceph-mon[35081]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-17T09:15:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:17 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2021257264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T09:15:18.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:17 smithi019 ceph-mon[31620]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-17T09:15:18.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:17 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2021257264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T09:15:19.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.010+0000 7f51777fe700 1 -- 172.21.15.19:0/2021257264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f517c013070 con 0x7f5188107de0 2024-05-17T09:15:19.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.013+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f51700b2880 msgr2=0x7f51700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.013+0000 7f518c807700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f51700b2880 0x7f51700b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f517800ac50 tx=0x7f517800a380 comp rx=0 tx=0).stop 2024-05-17T09:15:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.013+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 msgr2=0x7f518819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.013+0000 7f518c807700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f518819b320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f517c0098e0 tx=0x7f517c00f690 comp rx=0 tx=0).stop 2024-05-17T09:15:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 shutdown_connections 2024-05-17T09:15:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f51700b2880 0x7f51700b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51881034b0 0x7f518819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5188108730 0x7f518819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 --2- 172.21.15.19:0/2021257264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5188107de0 0x7f518819b320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 >> 172.21.15.19:0/2021257264 conn(0x7f51880fede0 msgr2=0x7f5188075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 shutdown_connections 2024-05-17T09:15:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:19.014+0000 7f518c807700 1 -- 172.21.15.19:0/2021257264 wait complete. 2024-05-17T09:15:19.016 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' created 2024-05-17T09:15:19.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:19 smithi079 ceph-mon[35502]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T09:15:19.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:19 smithi019 ceph-mon[31620]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T09:15:19.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:19 smithi069 ceph-mon[35081]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T09:15:19.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-17T09:15:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:20 smithi019 ceph-mon[31620]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T09:15:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:20 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2021257264' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T09:15:20.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:20 smithi019 ceph-mon[31620]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T09:15:20.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:20 smithi079 ceph-mon[35502]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T09:15:20.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:20 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2021257264' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T09:15:20.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:20 smithi079 ceph-mon[35502]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T09:15:20.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:20 smithi069 ceph-mon[35081]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T09:15:20.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:20 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2021257264' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T09:15:20.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:20 smithi069 ceph-mon[35081]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T09:15:21.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:21 smithi019 ceph-mon[31620]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T09:15:21.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:21 smithi069 ceph-mon[35081]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T09:15:21.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:21 smithi079 ceph-mon[35502]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T09:15:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 -- 172.21.15.19:0/4261014614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4108f10 msgr2=0x7f5dd4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 --2- 172.21.15.19:0/4261014614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4108f10 0x7f5dd4109320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80099c0 tx=0x7f5dc8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 -- 172.21.15.19:0/4261014614 shutdown_connections 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 --2- 172.21.15.19:0/4261014614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dd41045e0 0x7f5dd41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 --2- 172.21.15.19:0/4261014614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dd4109860 0x7f5dd41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 --2- 172.21.15.19:0/4261014614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4108f10 0x7f5dd4109320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 -- 172.21.15.19:0/4261014614 >> 172.21.15.19:0/4261014614 conn(0x7f5dd4100030 msgr2=0x7f5dd4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 -- 172.21.15.19:0/4261014614 shutdown_connections 2024-05-17T09:15:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 -- 172.21.15.19:0/4261014614 wait complete. 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 Processor -- start 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.980+0000 7f5dda76e700 1 -- start start 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dda76e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dd41045e0 0x7f5dd419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dda76e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dd4108f10 0x7f5dd419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dda76e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 0x7f5dd41a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dda76e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4114320 con 0x7f5dd4109860 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dda76e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd41141a0 con 0x7f5dd41045e0 2024-05-17T09:15:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dda76e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd41144a0 con 0x7f5dd4108f10 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 0x7f5dd41a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 0x7f5dd41a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60154/0 (socket says 172.21.15.19:60154) 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 -- 172.21.15.19:0/1879818879 learned_addr learned my addr 172.21.15.19:0/1879818879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd3fff700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dd41045e0 0x7f5dd419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd37fe700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dd4108f10 0x7f5dd419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 -- 172.21.15.19:0/1879818879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dd4108f10 msgr2=0x7f5dd419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dd4108f10 0x7f5dd419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 -- 172.21.15.19:0/1879818879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dd41045e0 msgr2=0x7f5dd419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.981+0000 7f5dd8d0b700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dd41045e0 0x7f5dd419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dd8d0b700 1 -- 172.21.15.19:0/1879818879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd41a1e80 con 0x7f5dd4109860 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dd8d0b700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 0x7f5dd41a17e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc007ee0 tx=0x7f5dcc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc004bb0 con 0x7f5dd4109860 2024-05-17T09:15:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dcc004d50 con 0x7f5dd4109860 2024-05-17T09:15:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0058b0 con 0x7f5dd4109860 2024-05-17T09:15:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd41a2170 con 0x7f5dd4109860 2024-05-17T09:15:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.982+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd41a9a90 con 0x7f5dd4109860 2024-05-17T09:15:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.983+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd4105a90 con 0x7f5dd4109860 2024-05-17T09:15:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.985+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5dcc01a020 con 0x7f5dd4109860 2024-05-17T09:15:21.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.985+0000 7f5dd17fa700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5dbc0b2880 0x7f5dbc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:21.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.986+0000 7f5dd3fff700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5dbc0b2880 0x7f5dbc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:21.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.986+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f5dcc0e2120 con 0x7f5dd4109860 2024-05-17T09:15:21.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.986+0000 7f5dd3fff700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5dbc0b2880 0x7f5dbc0b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4007be0 tx=0x7f5dc40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:21.989+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5dcc010070 con 0x7f5dd4109860 2024-05-17T09:15:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:22.202+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5dd41a9c60 con 0x7f5dd4109860 2024-05-17T09:15:22.433 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:22 smithi069 ceph-mon[35081]: pgmap v299: 129 pgs: 3 creating+peering, 27 unknown, 99 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T09:15:22.433 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:22 smithi069 ceph-mon[35081]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T09:15:22.433 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:22 smithi069 ceph-mon[35081]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T09:15:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:22 smithi019 ceph-mon[31620]: pgmap v299: 129 pgs: 3 creating+peering, 27 unknown, 99 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T09:15:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:22 smithi019 ceph-mon[31620]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T09:15:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:22 smithi019 ceph-mon[31620]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T09:15:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:22 smithi079 ceph-mon[35502]: pgmap v299: 129 pgs: 3 creating+peering, 27 unknown, 99 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T09:15:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:22 smithi079 ceph-mon[35502]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T09:15:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:22 smithi079 ceph-mon[35502]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T09:15:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.031+0000 7f5dd17fa700 1 -- 172.21.15.19:0/1879818879 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f5dcc0a9c20 con 0x7f5dd4109860 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5dbc0b2880 msgr2=0x7f5dbc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5dbc0b2880 0x7f5dbc0b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4007be0 tx=0x7f5dc40073d0 comp rx=0 tx=0).stop 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 msgr2=0x7f5dd41a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 0x7f5dd41a17e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc007ee0 tx=0x7f5dcc00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 shutdown_connections 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd4109860 0x7f5dd41a17e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dd41045e0 0x7f5dd419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5dbc0b2880 0x7f5dbc0b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 --2- 172.21.15.19:0/1879818879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dd4108f10 0x7f5dd419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 >> 172.21.15.19:0/1879818879 conn(0x7f5dd4100030 msgr2=0x7f5dd410a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:23.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 shutdown_connections 2024-05-17T09:15:23.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:23.034+0000 7f5dda76e700 1 -- 172.21.15.19:0/1879818879 wait complete. 2024-05-17T09:15:23.035 INFO:teuthology.orchestra.run.smithi019.stderr:set pool 4 allow_ec_overwrites to true 2024-05-17T09:15:23.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:23 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1879818879' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T09:15:23.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:23 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1879818879' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T09:15:23.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:23 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1879818879' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T09:15:23.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-17T09:15:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:24.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:24 smithi019 ceph-mon[31620]: pgmap v301: 129 pgs: 3 creating+peering, 4 unknown, 122 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:24.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:24 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1879818879' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T09:15:24.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:24 smithi019 ceph-mon[31620]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T09:15:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:24 smithi069 ceph-mon[35081]: pgmap v301: 129 pgs: 3 creating+peering, 4 unknown, 122 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:24.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:24 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1879818879' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T09:15:24.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:24 smithi069 ceph-mon[35081]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T09:15:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:24 smithi079 ceph-mon[35502]: pgmap v301: 129 pgs: 3 creating+peering, 4 unknown, 122 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:24 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1879818879' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T09:15:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:24 smithi079 ceph-mon[35502]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 -- 172.21.15.19:0/2134736026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 msgr2=0x7f3150110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 --2- 172.21.15.19:0/2134736026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f3150110090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f314800b0d0 tx=0x7f314800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 -- 172.21.15.19:0/2134736026 shutdown_connections 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 --2- 172.21.15.19:0/2134736026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f3150110090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 --2- 172.21.15.19:0/2134736026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31501034b0 0x7f31501038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 --2- 172.21.15.19:0/2134736026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3150107de0 0x7f31501081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 -- 172.21.15.19:0/2134736026 >> 172.21.15.19:0/2134736026 conn(0x7f31500fede0 msgr2=0x7f3150101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:25.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 -- 172.21.15.19:0/2134736026 shutdown_connections 2024-05-17T09:15:25.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.974+0000 7f315767b700 1 -- 172.21.15.19:0/2134736026 wait complete. 2024-05-17T09:15:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 Processor -- start 2024-05-17T09:15:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 -- start start 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31501034b0 0x7f315019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3150107de0 0x7f315019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f31501999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3150113210 con 0x7f3150108730 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3150113090 con 0x7f31501034b0 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f315767b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3150113390 con 0x7f3150107de0 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f3155c18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f31501999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f3155c18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f31501999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55380/0 (socket says 172.21.15.19:55380) 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.975+0000 7f3155c18700 1 -- 172.21.15.19:0/4251264346 learned_addr learned my addr 172.21.15.19:0/4251264346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155417700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31501034b0 0x7f315019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3154c16700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3150107de0 0x7f315019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155c18700 1 -- 172.21.15.19:0/4251264346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3150107de0 msgr2=0x7f315019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155c18700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3150107de0 0x7f315019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155c18700 1 -- 172.21.15.19:0/4251264346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31501034b0 msgr2=0x7f315019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155c18700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31501034b0 0x7f315019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155c18700 1 -- 172.21.15.19:0/4251264346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3150199ff0 con 0x7f3150108730 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f3155c18700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f31501999b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3148009850 tx=0x7f3148007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314801f070 con 0x7f3150108730 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315019a280 con 0x7f3150108730 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3148005500 con 0x7f3150108730 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314801b9a0 con 0x7f3150108730 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.976+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315019a7b0 con 0x7f3150108730 2024-05-17T09:15:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.977+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31500512b0 con 0x7f3150108730 2024-05-17T09:15:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.979+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3148022070 con 0x7f3150108730 2024-05-17T09:15:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.980+0000 7f31427fc700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f313c0b2900 0x7f313c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.980+0000 7f3155417700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f313c0b2900 0x7f313c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.980+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f314800f020 con 0x7f3150108730 2024-05-17T09:15:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.980+0000 7f3155417700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f313c0b2900 0x7f313c0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f315019aaf0 tx=0x7f314c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:25.983+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3148029020 con 0x7f3150108730 2024-05-17T09:15:26.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:26.248+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f315019aee0 con 0x7f3150108730 2024-05-17T09:15:26.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:26 smithi019 ceph-mon[31620]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:26.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:26 smithi069 ceph-mon[35081]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:26 smithi079 ceph-mon[35502]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:27.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.049+0000 7f31427fc700 1 -- 172.21.15.19:0/4251264346 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f31480ab7e0 con 0x7f3150108730 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f313c0b2900 msgr2=0x7f313c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f313c0b2900 0x7f313c0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f315019aaf0 tx=0x7f314c00a380 comp rx=0 tx=0).stop 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 msgr2=0x7f31501999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f31501999b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3148009850 tx=0x7f3148007d90 comp rx=0 tx=0).stop 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 shutdown_connections 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3150108730 0x7f31501999b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31501034b0 0x7f315019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f313c0b2900 0x7f313c0b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 --2- 172.21.15.19:0/4251264346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3150107de0 0x7f315019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 >> 172.21.15.19:0/4251264346 conn(0x7f31500fede0 msgr2=0x7f3150109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.052+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 shutdown_connections 2024-05-17T09:15:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:27.053+0000 7f315767b700 1 -- 172.21.15.19:0/4251264346 wait complete. 2024-05-17T09:15:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:added data pool 4 to fsmap 2024-05-17T09:15:27.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:27 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4251264346' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T09:15:27.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:27 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4251264346' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T09:15:27.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:27 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4251264346' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T09:15:27.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:15:28.113 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:28 smithi019 ceph-mon[31620]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:28 smithi019 ceph-mon[31620]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T09:15:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:28 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4251264346' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T09:15:28.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:28 smithi019 ceph-mon[31620]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:28.331 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:28 smithi069 ceph-mon[35081]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:28.332 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:28 smithi069 ceph-mon[35081]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T09:15:28.332 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:28 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4251264346' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T09:15:28.332 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:28 smithi069 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:28 smithi079 ceph-mon[35502]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:28 smithi079 ceph-mon[35502]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T09:15:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:28 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4251264346' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T09:15:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:28 smithi079 ceph-mon[35502]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:15:29.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:29 smithi019 ceph-mon[31620]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T09:15:29.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:29 smithi019 ceph-mon[31620]: Cluster is now healthy 2024-05-17T09:15:29.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:29 smithi069 ceph-mon[35081]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T09:15:29.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:29 smithi069 ceph-mon[35081]: Cluster is now healthy 2024-05-17T09:15:29.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:29 smithi079 ceph-mon[35502]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T09:15:29.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:29 smithi079 ceph-mon[35502]: Cluster is now healthy 2024-05-17T09:15:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 -- 172.21.15.19:0/4248031144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105eb0 msgr2=0x7f1b4410e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 --2- 172.21.15.19:0/4248031144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105eb0 0x7f1b4410e400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009990 tx=0x7f1b38009ca0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 -- 172.21.15.19:0/4248031144 shutdown_connections 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 --2- 172.21.15.19:0/4248031144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b44105560 0x7f1b44105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 --2- 172.21.15.19:0/4248031144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4410e940 0x7f1b44110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 --2- 172.21.15.19:0/4248031144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105eb0 0x7f1b4410e400 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.045+0000 7f1b49927700 1 -- 172.21.15.19:0/4248031144 >> 172.21.15.19:0/4248031144 conn(0x7f1b44078df0 msgr2=0x7f1b44079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.046+0000 7f1b49927700 1 -- 172.21.15.19:0/4248031144 shutdown_connections 2024-05-17T09:15:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.046+0000 7f1b49927700 1 -- 172.21.15.19:0/4248031144 wait complete. 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.046+0000 7f1b49927700 1 Processor -- start 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.046+0000 7f1b49927700 1 -- start start 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b49927700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 0x7f1b4419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b49927700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b44105eb0 0x7f1b4419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b49927700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4410e940 0x7f1b4419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b49927700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44114320 con 0x7f1b44105560 2024-05-17T09:15:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b49927700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b441141a0 con 0x7f1b4410e940 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b49927700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1b441144a0 con 0x7f1b44105eb0 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 0x7f1b4419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b437fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4410e940 0x7f1b4419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b427fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b44105eb0 0x7f1b4419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 0x7f1b4419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55394/0 (socket says 172.21.15.19:55394) 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 -- 172.21.15.19:0/1981216360 learned_addr learned my addr 172.21.15.19:0/1981216360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 -- 172.21.15.19:0/1981216360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b44105eb0 msgr2=0x7f1b4419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b44105eb0 0x7f1b4419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 -- 172.21.15.19:0/1981216360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4410e940 msgr2=0x7f1b4419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4410e940 0x7f1b4419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 -- 172.21.15.19:0/1981216360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4419fc20 con 0x7f1b44105560 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.047+0000 7f1b42ffd700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 0x7f1b4419ad50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b340060b0 tx=0x7f1b3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.048+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3401a070 con 0x7f1b44105560 2024-05-17T09:15:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.048+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b3400dd20 con 0x7f1b44105560 2024-05-17T09:15:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.048+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4419ff10 con 0x7f1b44105560 2024-05-17T09:15:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.048+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34014600 con 0x7f1b44105560 2024-05-17T09:15:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.048+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441a7830 con 0x7f1b44105560 2024-05-17T09:15:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.049+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b440512b0 con 0x7f1b44105560 2024-05-17T09:15:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.050+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1b3401d070 con 0x7f1b44105560 2024-05-17T09:15:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.051+0000 7f1b48925700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1b2c0b2880 0x7f1b2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.051+0000 7f1b427fc700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1b2c0b2880 0x7f1b2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.051+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f1b340e1e00 con 0x7f1b44105560 2024-05-17T09:15:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.052+0000 7f1b427fc700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1b2c0b2880 0x7f1b2c0b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b5c0 tx=0x7f1b38005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.055+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b34010070 con 0x7f1b44105560 2024-05-17T09:15:30.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.315+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4419c0b0 con 0x7f1b44105560 2024-05-17T09:15:30.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:30 smithi019 ceph-mon[31620]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:30.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.316+0000 7f1b48925700 1 -- 172.21.15.19:0/1981216360 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11779 (secure 0 0 0) 0x7f1b340a9290 con 0x7f1b44105560 2024-05-17T09:15:30.316 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:15:30.316 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:15:26.249217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-05-17T09:15:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1b2c0b2880 msgr2=0x7f1b2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1b2c0b2880 0x7f1b2c0b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b5c0 tx=0x7f1b38005960 comp rx=0 tx=0).stop 2024-05-17T09:15:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 msgr2=0x7f1b4419ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 0x7f1b4419ad50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b340060b0 tx=0x7f1b3400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 shutdown_connections 2024-05-17T09:15:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4410e940 0x7f1b4419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.319+0000 7f1b49927700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b44105560 0x7f1b4419ad50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.320+0000 7f1b49927700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1b2c0b2880 0x7f1b2c0b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.320+0000 7f1b49927700 1 --2- 172.21.15.19:0/1981216360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b44105eb0 0x7f1b4419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.320+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 >> 172.21.15.19:0/1981216360 conn(0x7f1b44078df0 msgr2=0x7f1b44100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.320+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 shutdown_connections 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:30.320+0000 7f1b49927700 1 -- 172.21.15.19:0/1981216360 wait complete. 2024-05-17T09:15:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-05-17T09:15:30.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:30 smithi079 ceph-mon[35502]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:30 smithi069 ceph-mon[35081]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:31.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:15:31.366 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:31.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1981216360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:15:31.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1981216360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:15:31.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1981216360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:15:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:32 smithi069 ceph-mon[35081]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:32.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:32 smithi019 ceph-mon[31620]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:32 smithi079 ceph-mon[35502]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 -- 172.21.15.19:0/2317710368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e741045d0 msgr2=0x7f7e741049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/2317710368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e741045d0 0x7f7e741049e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e700099c0 tx=0x7f7e70009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 -- 172.21.15.19:0/2317710368 shutdown_connections 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/2317710368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e74109850 0x7f7e741111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/2317710368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e741045d0 0x7f7e741049e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/2317710368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e74108f00 0x7f7e74109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 -- 172.21.15.19:0/2317710368 >> 172.21.15.19:0/2317710368 conn(0x7f7e740ffe60 msgr2=0x7f7e741022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 -- 172.21.15.19:0/2317710368 shutdown_connections 2024-05-17T09:15:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.284+0000 7f7e7b54a700 1 -- 172.21.15.19:0/2317710368 wait complete. 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 Processor -- start 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 -- start start 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e741045d0 0x7f7e74075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 0x7f7e74075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e74109850 0x7f7e74076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74114380 con 0x7f7e74108f00 2024-05-17T09:15:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74114200 con 0x7f7e741045d0 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e7b54a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74114500 con 0x7f7e74109850 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e78ae5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 0x7f7e74075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e78ae5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 0x7f7e74075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55418/0 (socket says 172.21.15.19:55418) 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e78ae5700 1 -- 172.21.15.19:0/119723255 learned_addr learned my addr 172.21.15.19:0/119723255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e792e6700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e741045d0 0x7f7e74075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.285+0000 7f7e79ae7700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e74109850 0x7f7e74076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e78ae5700 1 -- 172.21.15.19:0/119723255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e74109850 msgr2=0x7f7e74076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e78ae5700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e74109850 0x7f7e74076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e78ae5700 1 -- 172.21.15.19:0/119723255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e741045d0 msgr2=0x7f7e74075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e78ae5700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e741045d0 0x7f7e74075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e78ae5700 1 -- 172.21.15.19:0/119723255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e741a90c0 con 0x7f7e74108f00 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e78ae5700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 0x7f7e74075ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d980 tx=0x7f7e6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68017070 con 0x7f7e74108f00 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e741a93b0 con 0x7f7e74108f00 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e68009dc0 con 0x7f7e74108f00 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800f3c0 con 0x7f7e74108f00 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.286+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741a9940 con 0x7f7e74108f00 2024-05-17T09:15:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.287+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e740512b0 con 0x7f7e74108f00 2024-05-17T09:15:33.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.290+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7e68009930 con 0x7f7e74108f00 2024-05-17T09:15:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.291+0000 7f7e667fc700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e600b2880 0x7f7e600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.291+0000 7f7e792e6700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e600b2880 0x7f7e600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.291+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f7e680e1f30 con 0x7f7e74108f00 2024-05-17T09:15:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.292+0000 7f7e792e6700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e600b2880 0x7f7e600b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e70000c00 tx=0x7f7e70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:33.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.294+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e680a9a10 con 0x7f7e74108f00 2024-05-17T09:15:33.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.514+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e7402e5e0 con 0x7f7e74108f00 2024-05-17T09:15:33.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.517+0000 7f7e667fc700 1 -- 172.21.15.19:0/119723255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24054 (secure 0 0 0) 0x7f7e680a9310 con 0x7f7e74108f00 2024-05-17T09:15:33.517 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:15:33.517 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:15:26.249054+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e600b2880 msgr2=0x7f7e600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e600b2880 0x7f7e600b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e70000c00 tx=0x7f7e70005fb0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 msgr2=0x7f7e74075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 0x7f7e74075ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d980 tx=0x7f7e6800dc90 comp rx=0 tx=0).stop 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 shutdown_connections 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e600b2880 0x7f7e600b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e74109850 0x7f7e74076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e741045d0 0x7f7e74075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 --2- 172.21.15.19:0/119723255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e74108f00 0x7f7e74075ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.519+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 >> 172.21.15.19:0/119723255 conn(0x7f7e740ffe60 msgr2=0x7f7e74102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.520+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 shutdown_connections 2024-05-17T09:15:33.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:33.520+0000 7f7e7b54a700 1 -- 172.21.15.19:0/119723255 wait complete. 2024-05-17T09:15:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:34 smithi019 ceph-mon[31620]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:34.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:34 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/119723255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:34.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:15:34.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:34 smithi069 ceph-mon[35081]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:34.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:34 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/119723255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:34.486 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:34 smithi079 ceph-mon[35502]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:34 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/119723255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:36 smithi019 ceph-mon[31620]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:36.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:36 smithi069 ceph-mon[35081]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 -- 172.21.15.19:0/4187369763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc108730 msgr2=0x7f03bc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 --2- 172.21.15.19:0/4187369763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc108730 0x7f03bc110090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03b40099c0 tx=0x7f03b4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 -- 172.21.15.19:0/4187369763 shutdown_connections 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 --2- 172.21.15.19:0/4187369763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03bc1034b0 0x7f03bc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 --2- 172.21.15.19:0/4187369763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc108730 0x7f03bc110090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 --2- 172.21.15.19:0/4187369763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc107de0 0x7f03bc1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 -- 172.21.15.19:0/4187369763 >> 172.21.15.19:0/4187369763 conn(0x7f03bc0fedb0 msgr2=0x7f03bc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 -- 172.21.15.19:0/4187369763 shutdown_connections 2024-05-17T09:15:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.432+0000 7f03c142f700 1 -- 172.21.15.19:0/4187369763 wait complete. 2024-05-17T09:15:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.433+0000 7f03c142f700 1 Processor -- start 2024-05-17T09:15:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.433+0000 7f03c142f700 1 -- start start 2024-05-17T09:15:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03c142f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 0x7f03bc1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03c142f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03bc107de0 0x7f03bc1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03c142f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc108730 0x7f03bc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03c142f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113210 con 0x7f03bc1034b0 2024-05-17T09:15:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03c142f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113090 con 0x7f03bc108730 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03c142f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113390 con 0x7f03bc107de0 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 0x7f03bc1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 0x7f03bc1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49676/0 (socket says 172.21.15.19:49676) 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 -- 172.21.15.19:0/1405611710 learned_addr learned my addr 172.21.15.19:0/1405611710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03ba7fc700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03bc107de0 0x7f03bc1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03bb7fe700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc108730 0x7f03bc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 -- 172.21.15.19:0/1405611710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03bc107de0 msgr2=0x7f03bc1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:36.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03bc107de0 0x7f03bc1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 -- 172.21.15.19:0/1405611710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc108730 msgr2=0x7f03bc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc108730 0x7f03bc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03baffd700 1 -- 172.21.15.19:0/1405611710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc19b150 con 0x7f03bc1034b0 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.434+0000 7f03bb7fe700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc108730 0x7f03bc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.435+0000 7f03baffd700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 0x7f03bc1a08a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007e20 tx=0x7f03ac00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.435+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac016020 con 0x7f03bc1034b0 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.435+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03ac00ad40 con 0x7f03bc1034b0 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.435+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc19b440 con 0x7f03bc1034b0 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.435+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0115e0 con 0x7f03bc1034b0 2024-05-17T09:15:36.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.435+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc1a7750 con 0x7f03bc1034b0 2024-05-17T09:15:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.436+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc0512b0 con 0x7f03bc1034b0 2024-05-17T09:15:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.438+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f03ac019020 con 0x7f03bc1034b0 2024-05-17T09:15:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.438+0000 7f03b3fff700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f03a80b2880 0x7f03a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.438+0000 7f03ba7fc700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f03a80b2880 0x7f03a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.438+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f03ac0e2630 con 0x7f03bc1034b0 2024-05-17T09:15:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.439+0000 7f03ba7fc700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f03a80b2880 0x7f03a80b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03bc109820 tx=0x7f03a400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:36.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.442+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03ac020070 con 0x7f03bc1034b0 2024-05-17T09:15:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:36 smithi079 ceph-mon[35502]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.661+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03bc19bfd0 con 0x7f03bc1034b0 2024-05-17T09:15:36.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.663+0000 7f03b3fff700 1 -- 172.21.15.19:0/1405611710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24054 (secure 0 0 0) 0x7f03ac0aa110 con 0x7f03bc1034b0 2024-05-17T09:15:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:15:36.664 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:15:26.249054+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:15:36.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f03a80b2880 msgr2=0x7f03a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:36.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f03a80b2880 0x7f03a80b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03bc109820 tx=0x7f03a400a300 comp rx=0 tx=0).stop 2024-05-17T09:15:36.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 msgr2=0x7f03bc1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:36.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 0x7f03bc1a08a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007e20 tx=0x7f03ac00a340 comp rx=0 tx=0).stop 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 shutdown_connections 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03bc108730 0x7f03bc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03bc1034b0 0x7f03bc1a08a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f03a80b2880 0x7f03a80b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 --2- 172.21.15.19:0/1405611710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03bc107de0 0x7f03bc1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 >> 172.21.15.19:0/1405611710 conn(0x7f03bc0fedb0 msgr2=0x7f03bc06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 shutdown_connections 2024-05-17T09:15:36.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:36.665+0000 7f03c142f700 1 -- 172.21.15.19:0/1405611710 wait complete. 2024-05-17T09:15:37.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:37 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1405611710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:37.438 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:37 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1405611710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:37.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:15:37.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:37 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1405611710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:37.744 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:38.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:38 smithi019 ceph-mon[31620]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:38 smithi069 ceph-mon[35081]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:38 smithi079 ceph-mon[35502]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:39.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 -- 172.21.15.19:0/2467512976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4081045d0 msgr2=0x7fc4081049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:39.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 --2- 172.21.15.19:0/2467512976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4081045d0 0x7fc4081049e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0099c0 tx=0x7fc3fc009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 -- 172.21.15.19:0/2467512976 shutdown_connections 2024-05-17T09:15:39.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 --2- 172.21.15.19:0/2467512976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc408109850 0x7fc4081111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 --2- 172.21.15.19:0/2467512976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4081045d0 0x7fc4081049e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 --2- 172.21.15.19:0/2467512976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc408108f00 0x7fc408109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 -- 172.21.15.19:0/2467512976 >> 172.21.15.19:0/2467512976 conn(0x7fc4080ffee0 msgr2=0x7fc408102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:39.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 -- 172.21.15.19:0/2467512976 shutdown_connections 2024-05-17T09:15:39.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.486+0000 7fc40f045700 1 -- 172.21.15.19:0/2467512976 wait complete. 2024-05-17T09:15:39.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 Processor -- start 2024-05-17T09:15:39.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 -- start start 2024-05-17T09:15:39.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4081045d0 0x7fc408076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 0x7fc408077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc408109850 0x7fc408077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc408114380 con 0x7fc408108f00 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc408114200 con 0x7fc4081045d0 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40f045700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc408114500 con 0x7fc408109850 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc407fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 0x7fc408077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc407fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 0x7fc408077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49694/0 (socket says 172.21.15.19:49694) 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc407fff700 1 -- 172.21.15.19:0/3591517754 learned_addr learned my addr 172.21.15.19:0/3591517754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.487+0000 7fc40cde1700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4081045d0 0x7fc408076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc40d5e2700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc408109850 0x7fc408077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc407fff700 1 -- 172.21.15.19:0/3591517754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc408109850 msgr2=0x7fc408077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc407fff700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc408109850 0x7fc408077640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc407fff700 1 -- 172.21.15.19:0/3591517754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4081045d0 msgr2=0x7fc408076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc407fff700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4081045d0 0x7fc408076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc407fff700 1 -- 172.21.15.19:0/3591517754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4081a9110 con 0x7fc408108f00 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc407fff700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 0x7fc408077100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400d980 tx=0x7fc3f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4017070 con 0x7fc408108f00 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3f4009dc0 con 0x7fc408108f00 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4081a9400 con 0x7fc408108f00 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400f3c0 con 0x7fc408108f00 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.488+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a9940 con 0x7fc408108f00 2024-05-17T09:15:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.489+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4080512b0 con 0x7fc408108f00 2024-05-17T09:15:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.491+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc3f4009930 con 0x7fc408108f00 2024-05-17T09:15:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.492+0000 7fc405ffb700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3f80b2880 0x7fc3f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.492+0000 7fc40cde1700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3f80b2880 0x7fc3f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.492+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fc3f40e1f90 con 0x7fc408108f00 2024-05-17T09:15:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.492+0000 7fc40cde1700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3f80b2880 0x7fc3f80b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc000c00 tx=0x7fc3fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.495+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3f40a9a10 con 0x7fc408108f00 2024-05-17T09:15:39.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.754+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc40802e5e0 con 0x7fc408108f00 2024-05-17T09:15:39.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.755+0000 7fc405ffb700 1 -- 172.21.15.19:0/3591517754 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11779 (secure 0 0 0) 0x7fc3f40a9310 con 0x7fc408108f00 2024-05-17T09:15:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:15:39.756 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:15:26.249217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24497,"qdb_cluster":[24497]},"id":1}]} 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3f80b2880 msgr2=0x7fc3f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3f80b2880 0x7fc3f80b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc000c00 tx=0x7fc3fc01a040 comp rx=0 tx=0).stop 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 msgr2=0x7fc408077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 0x7fc408077100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400d980 tx=0x7fc3f400dc90 comp rx=0 tx=0).stop 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 shutdown_connections 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3f80b2880 0x7fc3f80b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc408109850 0x7fc408077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4081045d0 0x7fc408076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 --2- 172.21.15.19:0/3591517754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408108f00 0x7fc408077100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 >> 172.21.15.19:0/3591517754 conn(0x7fc4080ffee0 msgr2=0x7fc408102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 shutdown_connections 2024-05-17T09:15:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:39.758+0000 7fc40f045700 1 -- 172.21.15.19:0/3591517754 wait complete. 2024-05-17T09:15:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-05-17T09:15:40.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:40 smithi019 ceph-mon[31620]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:40.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:40 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3591517754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:15:40.394 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:15:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:40 smithi069 ceph-mon[35081]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:40 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3591517754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:15:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:40 smithi079 ceph-mon[35502]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:40 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3591517754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:15:40.646 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:42.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:42 smithi019 ceph-mon[31620]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.402+0000 7fddc148c700 1 -- 172.21.15.19:0/3410151167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc108f00 msgr2=0x7fddbc109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.402+0000 7fddc148c700 1 --2- 172.21.15.19:0/3410151167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc108f00 0x7fddbc109310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fddac0099c0 tx=0x7fddac009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 -- 172.21.15.19:0/3410151167 shutdown_connections 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 --2- 172.21.15.19:0/3410151167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddbc1045d0 0x7fddbc1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 --2- 172.21.15.19:0/3410151167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddbc109850 0x7fddbc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 --2- 172.21.15.19:0/3410151167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc108f00 0x7fddbc109310 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 -- 172.21.15.19:0/3410151167 >> 172.21.15.19:0/3410151167 conn(0x7fddbc0ffee0 msgr2=0x7fddbc102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 -- 172.21.15.19:0/3410151167 shutdown_connections 2024-05-17T09:15:42.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.403+0000 7fddc148c700 1 -- 172.21.15.19:0/3410151167 wait complete. 2024-05-17T09:15:42.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 Processor -- start 2024-05-17T09:15:42.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 -- start start 2024-05-17T09:15:42.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddbc1045d0 0x7fddbc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:42.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddbc108f00 0x7fddbc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:42.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 0x7fddbc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114310 con 0x7fddbc109850 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114190 con 0x7fddbc1045d0 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddc148c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114490 con 0x7fddbc108f00 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddbb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 0x7fddbc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddbb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 0x7fddbc19f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49716/0 (socket says 172.21.15.19:49716) 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddbb7fe700 1 -- 172.21.15.19:0/2164636883 learned_addr learned my addr 172.21.15.19:0/2164636883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddbaffd700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddbc1045d0 0x7fddbc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.404+0000 7fddba7fc700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddbc108f00 0x7fddbc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddbb7fe700 1 -- 172.21.15.19:0/2164636883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddbc108f00 msgr2=0x7fddbc19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddbb7fe700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddbc108f00 0x7fddbc19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddbb7fe700 1 -- 172.21.15.19:0/2164636883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddbc1045d0 msgr2=0x7fddbc19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddbb7fe700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddbc1045d0 0x7fddbc19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddbb7fe700 1 -- 172.21.15.19:0/2164636883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc19fc50 con 0x7fddbc109850 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddbb7fe700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 0x7fddbc19f5b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fddb4007ee0 tx=0x7fddb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb400ddd0 con 0x7fddbc109850 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb4004020 con 0x7fddbc109850 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc19ff40 con 0x7fddbc109850 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4015660 con 0x7fddbc109850 2024-05-17T09:15:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.405+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc1a7860 con 0x7fddbc109850 2024-05-17T09:15:42.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.406+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc105a80 con 0x7fddbc109850 2024-05-17T09:15:42.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.409+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fddb40041d0 con 0x7fddbc109850 2024-05-17T09:15:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.410+0000 7fddb3fff700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdda80b2880 0x7fdda80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.410+0000 7fddbaffd700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdda80b2880 0x7fdda80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.410+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fddb40e26b0 con 0x7fddbc109850 2024-05-17T09:15:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.410+0000 7fddbaffd700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdda80b2880 0x7fdda80b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdda4005950 tx=0x7fdda40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.413+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fddb4010070 con 0x7fddbc109850 2024-05-17T09:15:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:42 smithi069 ceph-mon[35081]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:42 smithi079 ceph-mon[35502]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:42.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.635+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddbc02e5e0 con 0x7fddbc109850 2024-05-17T09:15:42.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.637+0000 7fddb3fff700 1 -- 172.21.15.19:0/2164636883 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24054 (secure 0 0 0) 0x7fddb40a5030 con 0x7fddbc109850 2024-05-17T09:15:42.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:15:42.637 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:15:26.249054+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:15:42.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdda80b2880 msgr2=0x7fdda80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:42.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdda80b2880 0x7fdda80b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdda4005950 tx=0x7fdda40058e0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 msgr2=0x7fddbc19f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 0x7fddbc19f5b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fddb4007ee0 tx=0x7fddb400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 shutdown_connections 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddbc109850 0x7fddbc19f5b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddbc1045d0 0x7fddbc19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdda80b2880 0x7fdda80b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 --2- 172.21.15.19:0/2164636883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddbc108f00 0x7fddbc19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 >> 172.21.15.19:0/2164636883 conn(0x7fddbc0ffee0 msgr2=0x7fddbc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 shutdown_connections 2024-05-17T09:15:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:42.639+0000 7fddc148c700 1 -- 172.21.15.19:0/2164636883 wait complete. 2024-05-17T09:15:43.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T09:15:09.950009+0000', 'modified': '2024-05-17T09:15:26.249217+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24497}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.69:6839/814844032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 814844032}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 814844032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24497, 'qdb_cluster': [24497]} 2024-05-17T09:15:43.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T09:15:43.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T09:15:43.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:43 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2164636883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:43.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:43 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2164636883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:43.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:43 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2164636883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:15:43.546 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:44 smithi019 ceph-mon[31620]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:44.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:44 smithi069 ceph-mon[35081]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:44 smithi079 ceph-mon[35502]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.371+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/4047486417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d64108730 msgr2=0x7f7d64110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.371+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/4047486417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d64108730 0x7f7d64110090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0099c0 tx=0x7f7d5c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/4047486417 shutdown_connections 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/4047486417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d64108730 0x7f7d64110090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/4047486417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d641034b0 0x7f7d641038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/4047486417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d64107de0 0x7f7d641081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/4047486417 >> 172.21.15.19:0/4047486417 conn(0x7f7d640fede0 msgr2=0x7f7d64101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/4047486417 shutdown_connections 2024-05-17T09:15:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/4047486417 wait complete. 2024-05-17T09:15:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 Processor -- start 2024-05-17T09:15:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.372+0000 7f7d6bb2b700 1 -- start start 2024-05-17T09:15:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6bb2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 0x7f7d6419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6bb2b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d64107de0 0x7f7d6419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6bb2b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d64108730 0x7f7d641a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6bb2b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64113300 con 0x7f7d641034b0 2024-05-17T09:15:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6bb2b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64113180 con 0x7f7d64107de0 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6bb2b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64113480 con 0x7f7d64108730 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 0x7f7d6419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 0x7f7d6419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36442/0 (socket says 172.21.15.19:36442) 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 -- 172.21.15.19:0/231113181 learned_addr learned my addr 172.21.15.19:0/231113181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d64108730 0x7f7d641a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d690c6700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d64107de0 0x7f7d6419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 -- 172.21.15.19:0/231113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d64108730 msgr2=0x7f7d641a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d64108730 0x7f7d641a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 -- 172.21.15.19:0/231113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d64107de0 msgr2=0x7f7d6419d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d64107de0 0x7f7d6419d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 -- 172.21.15.19:0/231113181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d641a1e60 con 0x7f7d641034b0 2024-05-17T09:15:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.373+0000 7f7d698c7700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 0x7f7d6419cf90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000ba50 tx=0x7f7d6000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.374+0000 7f7d5affd700 1 -- 172.21.15.19:0/231113181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60019070 con 0x7f7d641034b0 2024-05-17T09:15:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.374+0000 7f7d5affd700 1 -- 172.21.15.19:0/231113181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d6000ce40 con 0x7f7d641034b0 2024-05-17T09:15:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.374+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d641a2150 con 0x7f7d641034b0 2024-05-17T09:15:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.374+0000 7f7d5affd700 1 -- 172.21.15.19:0/231113181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60007370 con 0x7f7d641034b0 2024-05-17T09:15:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.374+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a9a70 con 0x7f7d641034b0 2024-05-17T09:15:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.377+0000 7f7d5affd700 1 -- 172.21.15.19:0/231113181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7d6001c070 con 0x7f7d641034b0 2024-05-17T09:15:45.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d5affd700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d500b2880 0x7f7d500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 Processor -- start 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d690c6700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d500b2880 0x7f7d500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d5affd700 1 -- 172.21.15.19:0/231113181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f7d600e1ae0 con 0x7f7d641034b0 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 -- start start 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d690c6700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d500b2880 0x7f7d500b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649db780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f164909c40 con 0x55f16485cdf0 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f1648f97e0 con 0x55f1648da380 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.378+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1648fd820 con 0x55f1649db370 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36444/0 (socket says 172.21.15.19:36444) 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 -- 172.21.15.19:0/3230621580 learned_addr learned my addr 172.21.15.19:0/3230621580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649db780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 -- 172.21.15.19:0/3230621580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 msgr2=0x55f1649db780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649db780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 -- 172.21.15.19:0/3230621580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 msgr2=0x55f1649e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 -- 172.21.15.19:0/3230621580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f164849880 con 0x55f16485cdf0 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e14d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400d6a0 tx=0x7f7d54010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7a4bb9101b980d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d4ffff700 1 -- 172.21.15.19:0/3230621580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54024070 con 0x55f16485cdf0 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d4ffff700 1 -- 172.21.15.19:0/3230621580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d54014ad0 con 0x55f16485cdf0 2024-05-17T09:15:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.379+0000 7f7d4ffff700 1 -- 172.21.15.19:0/3230621580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5401fa20 con 0x55f16485cdf0 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3230621580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 msgr2=0x55f1649e14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e14d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400d6a0 tx=0x7f7d54010000 comp rx=0 tx=0).stop 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3230621580 shutdown_connections 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649db780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3230621580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e14d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3230621580 >> 172.21.15.19:0/3230621580 conn(0x55f16489bc30 msgr2=0x55f1649d5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3230621580 shutdown_connections 2024-05-17T09:15:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.380+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3230621580 wait complete. 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 -- start start 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 Processor -- start 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1648da380 0x55f1649e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1649db370 0x55f1649e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1647fff50 con 0x55f16485cdf0 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f16489b770 con 0x55f1649db370 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1649dc200 con 0x55f1648da380 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.381+0000 7f7d7a540b80 10 client.0 _ll_register_callbacks cb 0x55f1648948d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36452/0 (socket says 172.21.15.19:36452) 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 -- 172.21.15.19:0/3247097238 learned_addr learned my addr 172.21.15.19:0/3247097238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1648da380 0x55f1649e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1649db370 0x55f1649e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 -- 172.21.15.19:0/3247097238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1648da380 msgr2=0x55f1649e5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1648da380 0x55f1649e5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 -- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1649db370 msgr2=0x55f1649e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1649db370 0x55f1649e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 -- 172.21.15.19:0/3247097238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f164898640 con 0x55f16485cdf0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e4bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009fd0 tx=0x7f7d600a93a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ff99607162246b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3247097238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d600a8e60 con 0x55f16485cdf0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3247097238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f7d60014440 con 0x55f16485cdf0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d54014ad0 con 0x55f16485cdf0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.382+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3247097238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7d600ad210 con 0x55f16485cdf0 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 10 client.14688 fetch_fsmap learned FSMap version 17 2024-05-17T09:15:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f1649f25e0 con 0x55f16485cdf0 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3247097238 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d54014ad0 con 0x55f16485cdf0 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3247097238 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7d600a49d0 con 0x55f16485cdf0 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 10 client.14688 fetch_fsmap finished waiting for FSMap version 17 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 10 client.14688 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 20 client.14688 populate_metadata read hostname 'smithi019' 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1649f4c20 0x55f1649f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 4 client.14688 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f16490bb90 con 0x55f1649f4c20 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.383+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1649f4c20 0x55f1649f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.384+0000 7f7d4d7fa700 10 client.14688 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.384+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1649f4c20 0x55f1649f6090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b2adecb4a7ec03 server_cookie=94c7ae5aaff35b76 in_seq=0 out_seq=0 2024-05-17T09:15:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3247097238 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f16490bb90 con 0x55f1649f4c20 2024-05-17T09:15:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d4d7fa700 10 client.14688 handle_command_reply: tid=0 2024-05-17T09:15:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 client.14688 shutdown 2024-05-17T09:15:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 msgr2=0x55f1649e4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e4bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009fd0 tx=0x7f7d600a93a0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 shutdown_connections 2024-05-17T09:15:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1649f4c20 0x55f1649f6090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1649db370 0x55f1649e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f16485cdf0 0x55f1649e4bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3247097238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1648da380 0x55f1649e5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d4d7fa700 0 client.14688 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 >> 172.21.15.19:0/3247097238 conn(0x55f16489bc30 msgr2=0x55f1649d5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 shutdown_connections 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3247097238 wait complete. 2024-05-17T09:15:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.385+0000 7f7d7a540b80 20 client.14688 trim_cache size 0 max 16384 2024-05-17T09:15:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.406+0000 7f7d7a540b80 1 Processor -- start 2024-05-17T09:15:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.406+0000 7f7d7a540b80 1 -- start start 2024-05-17T09:15:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16485cdf0 0x55f1649f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 0x55f1649e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1649f43d0 con 0x55f1649db370 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f1649f4aa0 con 0x55f1648da380 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1649f5170 con 0x55f16485cdf0 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 0x55f1649e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 0x55f1649e01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36466/0 (socket says 172.21.15.19:36466) 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 -- 172.21.15.19:0/443201639 learned_addr learned my addr 172.21.15.19:0/443201639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d698c7700 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16485cdf0 0x55f1649f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d690c6700 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 -- 172.21.15.19:0/443201639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16485cdf0 msgr2=0x55f1649f0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16485cdf0 0x55f1649f0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 -- 172.21.15.19:0/443201639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 msgr2=0x55f1649dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649dfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 -- 172.21.15.19:0/443201639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1649d7c60 con 0x55f1649db370 2024-05-17T09:15:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 0x55f1649e01d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0099c0 tx=0x7f7d5c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876e831833098366 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.407+0000 7f7d4f7fe700 1 -- 172.21.15.19:0/443201639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00b790 con 0x55f1649db370 2024-05-17T09:15:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d4f7fe700 1 -- 172.21.15.19:0/443201639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d5c029070 con 0x55f1649db370 2024-05-17T09:15:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d4f7fe700 1 -- 172.21.15.19:0/443201639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003e30 con 0x55f1649db370 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 -- 172.21.15.19:0/443201639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 msgr2=0x55f1649e01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 0x55f1649e01d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0099c0 tx=0x7f7d5c005040 comp rx=0 tx=0).stop 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 -- 172.21.15.19:0/443201639 shutdown_connections 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16485cdf0 0x55f1649f0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1649db370 0x55f1649e01d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/443201639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1648da380 0x55f1649dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 -- 172.21.15.19:0/443201639 >> 172.21.15.19:0/443201639 conn(0x55f16489bc30 msgr2=0x55f1649e34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 -- 172.21.15.19:0/443201639 shutdown_connections 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.408+0000 7f7d7a540b80 1 -- 172.21.15.19:0/443201639 wait complete. 2024-05-17T09:15:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 -- start start 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 Processor -- start 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16485cdf0 0x55f1649ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 0x55f1649ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1649f5f10 con 0x55f1648da380 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f1649f5840 con 0x55f16485cdf0 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d7a540b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1649f65e0 con 0x55f1649db370 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.409+0000 7f7d690c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 0x55f1649ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 0x55f1649ffcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36474/0 (socket says 172.21.15.19:36474) 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 -- 172.21.15.19:0/3699579951 learned_addr learned my addr 172.21.15.19:0/3699579951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d7a540b80 10 client.0 _ll_register_callbacks cb 0x55f1648948d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16485cdf0 0x55f1649ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d6a0c8700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 -- 172.21.15.19:0/3699579951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 msgr2=0x55f1649fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 -- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16485cdf0 msgr2=0x55f1649ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16485cdf0 0x55f1649ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 -- 172.21.15.19:0/3699579951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1649face0 con 0x55f1648da380 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d690c6700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 0x55f1649ffcc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7d54008010 tx=0x7f7d54004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc4b5570fab51a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3699579951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54024040 con 0x55f1648da380 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3699579951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f7d5400dd40 con 0x55f1648da380 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c029070 con 0x55f1648da380 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3699579951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7d54042050 con 0x55f1648da380 2024-05-17T09:15:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d7a540b80 10 client.14694 fetch_fsmap learned FSMap version 17 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.410+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f164a60e10 con 0x55f1648da380 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3699579951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d5c029070 con 0x55f1648da380 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3699579951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7d5403d030 con 0x55f1648da380 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d7a540b80 10 client.14694 fetch_fsmap finished waiting for FSMap version 17 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d7a540b80 10 client.14694 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d7a540b80 20 client.14694 populate_metadata read hostname 'smithi019' 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f164a63140 0x55f164a645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d7a540b80 4 client.14694 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "status"}) -- 0x55f164905370 con 0x55f164a63140 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.411+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f164a63140 0x55f164a645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d698c7700 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f164a63140 0x55f164a645b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99842535ff3fcc9f server_cookie=c630d45c46cb4f2 in_seq=0 out_seq=0 2024-05-17T09:15:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d4d7fa700 10 client.14694 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d4d7fa700 1 -- 172.21.15.19:0/3699579951 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f164905370 con 0x55f164a63140 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d4d7fa700 10 client.14694 handle_command_reply: tid=0 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 client.14694 shutdown 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 msgr2=0x55f1649ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 0x55f1649ffcc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7d54008010 tx=0x7f7d54004860 comp rx=0 tx=0).stop 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 shutdown_connections 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1649db370 0x55f1649fa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f164a63140 0x55f164a645b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16485cdf0 0x55f1649ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 --2- 172.21.15.19:0/3699579951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1648da380 0x55f1649ffcc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d4d7fa700 0 client.14694 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 >> 172.21.15.19:0/3699579951 conn(0x55f16489bc30 msgr2=0x55f1649de8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 shutdown_connections 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 1 -- 172.21.15.19:0/3699579951 wait complete. 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.412+0000 7f7d7a540b80 20 client.14694 trim_cache size 0 max 16384 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24497, 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:15:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:15:45.421 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 35.454606368, 2024-05-17T09:15:45.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 17, 2024-05-17T09:15:45.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 85, 2024-05-17T09:15:45.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-05-17T09:15:45.421 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 52.806326235999997 2024-05-17T09:15:45.421 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d500b2880 msgr2=0x7f7d500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d500b2880 0x7f7d500b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54008040 comp rx=0 tx=0).stop 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 msgr2=0x7f7d6419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 0x7f7d6419cf90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000ba50 tx=0x7f7d6000bd60 comp rx=0 tx=0).stop 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 shutdown_connections 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7d500b2880 0x7f7d500b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d64108730 0x7f7d641a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d641034b0 0x7f7d6419cf90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 --2- 172.21.15.19:0/231113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d64107de0 0x7f7d6419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 >> 172.21.15.19:0/231113181 conn(0x7f7d640fede0 msgr2=0x7f7d64075bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 shutdown_connections 2024-05-17T09:15:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:45.413+0000 7f7d6bb2b700 1 -- 172.21.15.19:0/231113181 wait complete. 2024-05-17T09:15:46.224 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T09:15:46.224 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:46.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T09:15:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:46 smithi069 ceph-mon[35081]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:46 smithi019 ceph-mon[31620]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:46.480 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:46 smithi079 ceph-mon[35502]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:47.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-05-17T09:15:47.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T09:15:47.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-17T09:15:47.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-17T09:15:47.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-17T09:15:47.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-17T09:15:47.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-05-17T09:15:47.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-17T09:15:47.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-17T09:15:47.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-17T09:15:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-17T09:15:47.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-17T09:15:47.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-05-17T09:15:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-17T09:15:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-17T09:15:47.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-05-17T09:15:48.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-17T09:15:48.173 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-17T09:15:48.173 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-17T09:15:48.173 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:48 smithi019 ceph-mon[31620]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T09:15:48.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 -- 172.21.15.19:0/2759445274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 msgr2=0x7f9e0410fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 --2- 172.21.15.19:0/2759445274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0410fee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00b0d0 tx=0x7f9dfc00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 -- 172.21.15.19:0/2759445274 shutdown_connections 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 --2- 172.21.15.19:0/2759445274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e04103300 0x7f9e04103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 --2- 172.21.15.19:0/2759445274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0410fee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 --2- 172.21.15.19:0/2759445274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04107c30 0x7f9e04108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 -- 172.21.15.19:0/2759445274 >> 172.21.15.19:0/2759445274 conn(0x7f9e04078c30 msgr2=0x7f9e04079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 -- 172.21.15.19:0/2759445274 shutdown_connections 2024-05-17T09:15:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.218+0000 7f9e0919e700 1 -- 172.21.15.19:0/2759445274 wait complete. 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 Processor -- start 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 -- start start 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e04103300 0x7f9e041a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04107c30 0x7f9e041a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e04113060 con 0x7f9e04108580 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e04112ee0 con 0x7f9e04107c30 2024-05-17T09:15:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0919e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041131e0 con 0x7f9e04103300 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0899d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0899d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36506/0 (socket says 172.21.15.19:36506) 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e0899d700 1 -- 172.21.15.19:0/1780524008 learned_addr learned my addr 172.21.15.19:0/1780524008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9dfbfff700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04107c30 0x7f9e041a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.219+0000 7f9e03fff700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e04103300 0x7f9e041a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0899d700 1 -- 172.21.15.19:0/1780524008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e04103300 msgr2=0x7f9e041a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0899d700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e04103300 0x7f9e041a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0899d700 1 -- 172.21.15.19:0/1780524008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04107c30 msgr2=0x7f9e041a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0899d700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04107c30 0x7f9e041a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0899d700 1 -- 172.21.15.19:0/1780524008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0419af90 con 0x7f9e04108580 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0899d700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0419a950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009850 tx=0x7f9dfc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc01f070 con 0x7f9e04108580 2024-05-17T09:15:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dfc004c50 con 0x7f9e04108580 2024-05-17T09:15:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc01b690 con 0x7f9e04108580 2024-05-17T09:15:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0419b220 con 0x7f9e04108580 2024-05-17T09:15:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.220+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0419b750 con 0x7f9e04108580 2024-05-17T09:15:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.221+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e040512b0 con 0x7f9e04108580 2024-05-17T09:15:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.223+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9dfc022070 con 0x7f9e04108580 2024-05-17T09:15:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.223+0000 7f9e01ffb700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9df00b2900 0x7f9df00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.224+0000 7f9e03fff700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9df00b2900 0x7f9df00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.224+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9dfc00f020 con 0x7f9e04108580 2024-05-17T09:15:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.224+0000 7f9e03fff700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9df00b2900 0x7f9df00b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9df4005ea0 tx=0x7f9df4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:48.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.227+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9dfc029020 con 0x7f9e04108580 2024-05-17T09:15:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-05-17T09:15:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T09:15:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-17T09:15:48.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:48 smithi069 ceph-mon[35081]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T09:15:48.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.496+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9e0419be80 con 0x7f9e04108580 2024-05-17T09:15:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.497+0000 7f9e01ffb700 1 -- 172.21.15.19:0/1780524008 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f9dfc0ab4e0 con 0x7f9e04108580 2024-05-17T09:15:48.497 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-17T09:15:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9df00b2900 msgr2=0x7f9df00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9df00b2900 0x7f9df00b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9df4005ea0 tx=0x7f9df4005e30 comp rx=0 tx=0).stop 2024-05-17T09:15:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 msgr2=0x7f9e0419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0419a950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009850 tx=0x7f9dfc007a20 comp rx=0 tx=0).stop 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 shutdown_connections 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9df00b2900 0x7f9df00b4e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e04103300 0x7f9e041a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e04108580 0x7f9e0419a950 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 --2- 172.21.15.19:0/1780524008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e04107c30 0x7f9e041a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 >> 172.21.15.19:0/1780524008 conn(0x7f9e04078c30 msgr2=0x7f9e0406c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 shutdown_connections 2024-05-17T09:15:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:48.499+0000 7f9e0919e700 1 -- 172.21.15.19:0/1780524008 wait complete. 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-17T09:15:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-17T09:15:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-17T09:15:48.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:48 smithi079 ceph-mon[35502]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T09:15:49.159 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-17T09:15:49.159 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T09:15:49.160 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-17T09:15:49.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:49.160 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-05-17T09:15:49.178 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T09:15:49.178 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T09:15:49.178 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T09:15:49.178 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-17T09:15:49.178 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-17T09:15:49.179 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42168sec preferred_lft 42168sec 2024-05-17T09:15:49.180 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-17T09:15:49.180 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-17T09:15:49.180 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T09:15:49.180 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T09:15:49.180 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-05-17T09:15:49.180 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T09:15:49.180 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:15:49.181 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:15:49.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-05-17T09:15:49.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-05-17T09:15:49.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-05-17T09:15:49.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T09:15:49.181 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:15:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:15:49.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:49 smithi019 ceph-mon[31620]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T09:15:49.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:49 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1780524008' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T09:15:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:49.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:49.388 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T09:15:49.414 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-05-17T09:15:49.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:49.416 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-05-17T09:15:49.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:49 smithi069 ceph-mon[35081]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T09:15:49.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:49 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1780524008' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T09:15:49.481 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-05-17T09:15:49.481 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T09:15:49.481 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-17T09:15:49.481 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-17T09:15:49.481 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T09:15:49.482 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:15:49.482 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:15:49.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-17T09:15:49.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-17T09:15:49.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-17T09:15:49.482 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:15:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:49 smithi079 ceph-mon[35502]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T09:15:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:49 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1780524008' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T09:15:49.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:15:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:49.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:49.806 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-17T09:15:49.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:49.823 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-05-17T09:15:49.879 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:15:49.879 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:15:49.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T09:15:49.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-17T09:15:49.879 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:15:49.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:50.000 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-17T09:15:50.000 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:15:50.000 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:15:50.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-17T09:15:50.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T09:15:50.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-17T09:15:50.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-17T09:15:50.001 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-17T09:15:50.001 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:15:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:15:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:50.136 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:15:50.136 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:15:50.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-05-17T09:15:50.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T09:15:50.137 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:15:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:15:50.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:50.218 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-17T09:15:50.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T09:15:50.219 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:50.235 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-17T09:15:50.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T09:15:50.235 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:50.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-17T09:15:50.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:50 smithi019 ceph-mon[31620]: pgmap v318: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:50.357 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T09:15:50.415 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-17T09:15:50.415 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-17T09:15:50.415 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-17T09:15:50.416 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e38ed570e27c2067f159537cd6dbdfa48d8341d814f3621d8480c1538e1a4584/merged 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/915aa4a49a9b427eb59253962f3294adfba243c780d4d98ee3c676f165537864/merged 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:15:50.417 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/771ae42c6c974cb5cabd1b84160e64ee5b728e6f89a90e9084f37af07bf202ca/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/15c08818cc91279bb50691009af2904dd353b8e48fc373a71f6533e7854cb130/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/c475fe19d3080f437062d2798ecff9d5f633dcbca27b63186d2b5ec30bd65b63/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/ae1a36aa63ed826bf37af7568322fabd17e78e97ec6f90ea7927fc98b6237e01/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5b2ae6a0ee44dfc255eae88a62aad5da201f3f85045af9d76bd8f7878ffbfdc0/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/91b165d8ae06b68d3ad58ce9b6cc0c52b9497643b8677875ed636e90c46f5d2a/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/41c98f10fe54919ef49d91ad3a9d0b7ff2951ef2f452b33252b3e1d458a3ae34/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/0a1d060f75c509c6772e39741b59a06bd5e2926810bec6a60fc1fdc83b11e572/merged 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T09:15:50.418 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T09:15:50.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:50.419 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-17T09:15:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:50 smithi069 ceph-mon[35081]: pgmap v318: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:50.473 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T09:15:50.473 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-17T09:15:50.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-17T09:15:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:50 smithi079 ceph-mon[35502]: pgmap v318: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:15:50.542 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-17T09:15:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-17T09:15:50.600 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e38ed570e27c2067f159537cd6dbdfa48d8341d814f3621d8480c1538e1a4584/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/915aa4a49a9b427eb59253962f3294adfba243c780d4d98ee3c676f165537864/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/771ae42c6c974cb5cabd1b84160e64ee5b728e6f89a90e9084f37af07bf202ca/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/15c08818cc91279bb50691009af2904dd353b8e48fc373a71f6533e7854cb130/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/c475fe19d3080f437062d2798ecff9d5f633dcbca27b63186d2b5ec30bd65b63/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/ae1a36aa63ed826bf37af7568322fabd17e78e97ec6f90ea7927fc98b6237e01/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5b2ae6a0ee44dfc255eae88a62aad5da201f3f85045af9d76bd8f7878ffbfdc0/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/91b165d8ae06b68d3ad58ce9b6cc0c52b9497643b8677875ed636e90c46f5d2a/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/41c98f10fe54919ef49d91ad3a9d0b7ff2951ef2f452b33252b3e1d458a3ae34/merged 2024-05-17T09:15:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/0a1d060f75c509c6772e39741b59a06bd5e2926810bec6a60fc1fdc83b11e572/merged 2024-05-17T09:15:50.602 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-17T09:15:50.602 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T09:15:50.602 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T09:15:50.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:50.602 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-17T09:15:50.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:2024-05-17T09:15:50.639+0000 7f5a6a510480 -1 init, newargv = 0x7f5a4c056f20 newargc=15 2024-05-17T09:15:50.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[75165]: starting ceph client 2024-05-17T09:15:50.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[75165]: starting fuse 2024-05-17T09:15:50.656 INFO:teuthology.orchestra.run.smithi019.stdout:107 2024-05-17T09:15:50.656 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-17T09:15:50.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo stdin-killer -- python3 -c ' 2024-05-17T09:15:50.656 DEBUG:teuthology.orchestra.run.smithi019:> import glob 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> import re 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> import subprocess 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> def _find_admin_socket(client_name): 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> files = glob.glob(asok_path) 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> # Given a non-glob path, it better be there 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> if "*" not in asok_path: 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> assert(len(files) == 1) 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> return files[0] 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:15:50.657 DEBUG:teuthology.orchestra.run.smithi019:> for f in files: 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> contents = proc_f.read() 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> if mountpoint in contents: 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> return f 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> print(_find_admin_socket("client.admin")) 2024-05-17T09:15:50.658 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-05-17T09:15:50.778 INFO:teuthology.orchestra.run.smithi019.stdout:/var/run/ceph/ceph-client.admin.75165.asok 2024-05-17T09:15:50.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:50.788 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75165.asok 2024-05-17T09:15:50.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:50.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75165.asok status 2024-05-17T09:15:50.937 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:15:50.937 INFO:teuthology.orchestra.run.smithi019.stdout: "metadata": { 2024-05-17T09:15:50.937 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_sha1": "3790c0feaaca36c33bb1b31470733325d7d91e6a", 2024-05-17T09:15:50.937 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev)", 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "entity_id": "admin", 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": "smithi019", 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "pid": "75165", 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "root": "/" 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_count": 0, 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_pinned_count": 0, 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 14706, 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "inst": { 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": { 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "client", 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 14706 2024-05-17T09:15:50.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2489232459 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2489232459 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "inst_str": "client.14706 192.168.0.1:0/2489232459", 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "addr_str": "192.168.0.1:0/2489232459", 2024-05-17T09:15:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: "inode_count": 1, 2024-05-17T09:15:50.940 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_epoch": 17, 2024-05-17T09:15:50.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch": 87, 2024-05-17T09:15:50.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch_barrier": 0, 2024-05-17T09:15:50.940 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklisted": false, 2024-05-17T09:15:50.940 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs" 2024-05-17T09:15:50.940 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:15:50.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:50.946 DEBUG:teuthology.orchestra.run.smithi019:> stat --file-system '--printf=%T 2024-05-17T09:15:50.946 DEBUG:teuthology.orchestra.run.smithi019:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:50.962 INFO:teuthology.orchestra.run.smithi019.stdout:fuseblk 2024-05-17T09:15:50.963 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:50.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:50.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:51.030 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-17T09:15:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-17T09:15:51.144 INFO:teuthology.orchestra.run.smithi019.stdout:# file: . 2024-05-17T09:15:51.144 INFO:teuthology.orchestra.run.smithi019.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-17T09:15:51.144 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:15:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:15:51.150 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:15:51.150 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T09:15:51.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T09:15:51.461 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:52 smithi069 ceph-mon[35081]: pgmap v319: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:52.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:52 smithi019 ceph-mon[31620]: pgmap v319: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:52 smithi079 ceph-mon[35502]: pgmap v319: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:53.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 -- 172.21.15.19:0/417520752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0107de0 msgr2=0x7f5ee01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/417520752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0107de0 0x7f5ee01081f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80099c0 tx=0x7f5ec8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 -- 172.21.15.19:0/417520752 shutdown_connections 2024-05-17T09:15:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/417520752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee0108730 0x7f5ee0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/417520752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee01034b0 0x7f5ee01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/417520752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0107de0 0x7f5ee01081f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 -- 172.21.15.19:0/417520752 >> 172.21.15.19:0/417520752 conn(0x7f5ee00fede0 msgr2=0x7f5ee0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 -- 172.21.15.19:0/417520752 shutdown_connections 2024-05-17T09:15:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 -- 172.21.15.19:0/417520752 wait complete. 2024-05-17T09:15:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.245+0000 7f5ee6a17700 1 Processor -- start 2024-05-17T09:15:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 -- start start 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee01034b0 0x7f5ee019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee0107de0 0x7f5ee019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 0x7f5ee019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee01143c0 con 0x7f5ee0108730 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0114240 con 0x7f5ee0107de0 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee6a17700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0114540 con 0x7f5ee01034b0 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 0x7f5ee019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 0x7f5ee019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36524/0 (socket says 172.21.15.19:36524) 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 -- 172.21.15.19:0/3778708048 learned_addr learned my addr 172.21.15.19:0/3778708048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5edffff700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee01034b0 0x7f5ee019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ed7fff700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee0107de0 0x7f5ee019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 -- 172.21.15.19:0/3778708048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee01034b0 msgr2=0x7f5ee019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee01034b0 0x7f5ee019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 -- 172.21.15.19:0/3778708048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee0107de0 msgr2=0x7f5ee019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee0107de0 0x7f5ee019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5ee4fb4700 1 -- 172.21.15.19:0/3778708048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee019fc50 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.246+0000 7f5edffff700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee01034b0 0x7f5ee019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.247+0000 7f5ee4fb4700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 0x7f5ee019f660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8007d70 tx=0x7f5ed800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.247+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8016020 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.247+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ed800ad40 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.247+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee019ff40 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.247+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80115e0 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.247+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee01a0400 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.248+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee0104960 con 0x7f5ee0108730 2024-05-17T09:15:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.250+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5ed8019020 con 0x7f5ee0108730 2024-05-17T09:15:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.251+0000 7f5eddffb700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5ecc0b2880 0x7f5ecc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.251+0000 7f5edffff700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5ecc0b2880 0x7f5ecc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.251+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f5ed80e2980 con 0x7f5ee0108730 2024-05-17T09:15:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.251+0000 7f5edffff700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5ecc0b2880 0x7f5ecc0b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005950 tx=0x7f5ed00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.255+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ed8020020 con 0x7f5ee0108730 2024-05-17T09:15:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.483+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ee019c190 con 0x7f5ee0108730 2024-05-17T09:15:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.484+0000 7f5eddffb700 1 -- 172.21.15.19:0/3778708048 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+588 (secure 0 0 0) 0x7f5ed80aa340 con 0x7f5ee0108730 2024-05-17T09:15:53.484 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/21803007 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3426431385 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4126868109 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3686497433 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3120865986 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:53.485 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3378222227 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.486+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5ecc0b2880 msgr2=0x7f5ecc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.486+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5ecc0b2880 0x7f5ecc0b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005950 tx=0x7f5ed00058e0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.486+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 msgr2=0x7f5ee019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.486+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 0x7f5ee019f660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8007d70 tx=0x7f5ed800a340 comp rx=0 tx=0).stop 2024-05-17T09:15:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 shutdown_connections 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5ecc0b2880 0x7f5ecc0b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ee01034b0 0x7f5ee019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ee0108730 0x7f5ee019f660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 --2- 172.21.15.19:0/3778708048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee0107de0 0x7f5ee019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 >> 172.21.15.19:0/3778708048 conn(0x7f5ee00fede0 msgr2=0x7f5ee0101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 shutdown_connections 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:53.487+0000 7f5ee6a17700 1 -- 172.21.15.19:0/3778708048 wait complete. 2024-05-17T09:15:53.488 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-17T09:15:54.232 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:15:54.232 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T09:15:54.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T09:15:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:54 smithi069 ceph-mon[35081]: pgmap v320: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:54 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3778708048' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:15:54.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:54 smithi019 ceph-mon[31620]: pgmap v320: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:54 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3778708048' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:15:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:54 smithi079 ceph-mon[35502]: pgmap v320: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:54 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3778708048' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:15:54.543 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:15:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 -- 172.21.15.19:0/3498779554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa16410e8e0 msgr2=0x7fa164110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 --2- 172.21.15.19:0/3498779554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa16410e8e0 0x7fa164110dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b0a0 tx=0x7fa15c00b3b0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 -- 172.21.15.19:0/3498779554 shutdown_connections 2024-05-17T09:15:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 --2- 172.21.15.19:0/3498779554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa16410e8e0 0x7fa164110dc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 --2- 172.21.15.19:0/3498779554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa164105470 0x7fa164105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 --2- 172.21.15.19:0/3498779554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa164105dc0 0x7fa16410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 -- 172.21.15.19:0/3498779554 >> 172.21.15.19:0/3498779554 conn(0x7fa1640fef20 msgr2=0x7fa164101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 -- 172.21.15.19:0/3498779554 shutdown_connections 2024-05-17T09:15:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.310+0000 7fa169a81700 1 -- 172.21.15.19:0/3498779554 wait complete. 2024-05-17T09:15:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 Processor -- start 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 -- start start 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa164105470 0x7fa164103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 0x7fa1641036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16410e8e0 0x7fa164103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641142f0 con 0x7fa164105dc0 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa164114170 con 0x7fa16410e8e0 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.311+0000 7fa169a81700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa164114470 con 0x7fa164105470 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 0x7fa1641036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 0x7fa1641036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55192/0 (socket says 172.21.15.19:55192) 2024-05-17T09:15:56.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 -- 172.21.15.19:0/4080922028 learned_addr learned my addr 172.21.15.19:0/4080922028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa1637fe700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa164105470 0x7fa164103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa163fff700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16410e8e0 0x7fa164103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 -- 172.21.15.19:0/4080922028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa164105470 msgr2=0x7fa164103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa164105470 0x7fa164103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 -- 172.21.15.19:0/4080922028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16410e8e0 msgr2=0x7fa164103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16410e8e0 0x7fa164103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 -- 172.21.15.19:0/4080922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1641a9060 con 0x7fa164105dc0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa162ffd700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 0x7fa1641036a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa15800d950 tx=0x7fa15800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158016070 con 0x7fa164105dc0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa15800fd20 con 0x7fa164105dc0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1641a92f0 con 0x7fa164105dc0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.312+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15800e3b0 con 0x7fa164105dc0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.313+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1641a9830 con 0x7fa164105dc0 2024-05-17T09:15:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.313+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1640512b0 con 0x7fa164105dc0 2024-05-17T09:15:56.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.315+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa158019070 con 0x7fa164105dc0 2024-05-17T09:15:56.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:56 smithi019 ceph-mon[31620]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.316+0000 7fa160ff9700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa14c0b2880 0x7fa14c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:15:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.317+0000 7fa1637fe700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa14c0b2880 0x7fa14c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:15:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.317+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa1580e23c0 con 0x7fa164105dc0 2024-05-17T09:15:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.317+0000 7fa1637fe700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa14c0b2880 0x7fa14c0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa154009c60 tx=0x7fa154009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:15:56.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.320+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1580a9d20 con 0x7fa164105dc0 2024-05-17T09:15:56.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:56 smithi069 ceph-mon[35081]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:56 smithi079 ceph-mon[35502]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.549+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa16402e5e0 con 0x7fa164105dc0 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.550+0000 7fa160ff9700 1 -- 172.21.15.19:0/4080922028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+588 (secure 0 0 0) 0x7fa1580a9620 con 0x7fa164105dc0 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/21803007 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3426431385 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4126868109 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:56.550 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3686497433 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:56.551 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:56.551 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3120865986 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:56.551 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:56.551 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:15:56.551 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3378222227 2024-05-18T09:07:20.018567+0000 2024-05-17T09:15:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa14c0b2880 msgr2=0x7fa14c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa14c0b2880 0x7fa14c0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa154009c60 tx=0x7fa154009260 comp rx=0 tx=0).stop 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 msgr2=0x7fa1641036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 0x7fa1641036a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa15800d950 tx=0x7fa15800dc60 comp rx=0 tx=0).stop 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 shutdown_connections 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa14c0b2880 0x7fa14c0b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa164105470 0x7fa164103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16410e8e0 0x7fa164103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 --2- 172.21.15.19:0/4080922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa164105dc0 0x7fa1641036a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 >> 172.21.15.19:0/4080922028 conn(0x7fa1640fef20 msgr2=0x7fa164100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:15:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 shutdown_connections 2024-05-17T09:15:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:15:56.552+0000 7fa169a81700 1 -- 172.21.15.19:0/4080922028 wait complete. 2024-05-17T09:15:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-17T09:15:57.236 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi019.front.sepia.ceph.com... 2024-05-17T09:15:57.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:15:57.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-17T09:15:57.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[75165]: fuse finished with error 0 and tester_r 0 2024-05-17T09:15:57.265 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T09:15:57.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:57 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4080922028' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:15:57.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:57 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4080922028' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:15:57.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:57 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4080922028' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:15:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:15:58 smithi069 ceph-mon[35081]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 420 B/s wr, 0 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:58.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:15:58 smithi019 ceph-mon[31620]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 420 B/s wr, 0 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:15:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:15:58 smithi079 ceph-mon[35502]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 420 B/s wr, 0 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-17T09:16:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:00 smithi019 ceph-mon[31620]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 477 B/s wr, 0 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-17T09:16:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:00 smithi079 ceph-mon[35502]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 477 B/s wr, 0 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-17T09:16:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:00 smithi069 ceph-mon[35081]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 477 B/s wr, 0 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-17T09:16:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:02 smithi019 ceph-mon[31620]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-17T09:16:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:16:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:02 smithi079 ceph-mon[35502]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-17T09:16:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:16:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:02 smithi069 ceph-mon[35081]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-17T09:16:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:16:03.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:16:03.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:16:03.368 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-17T09:16:03.405 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:16:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:04 smithi019 ceph-mon[31620]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T09:16:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:04 smithi079 ceph-mon[35502]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T09:16:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:04 smithi069 ceph-mon[35081]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T09:16:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 -- 172.21.15.19:0/657096593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21041089e0 msgr2=0x7f2104108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 --2- 172.21.15.19:0/657096593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21041089e0 0x7f2104108df0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009990 tx=0x7f20f4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 -- 172.21.15.19:0/657096593 shutdown_connections 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 --2- 172.21.15.19:0/657096593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2104103300 0x7f210410f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 --2- 172.21.15.19:0/657096593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21041089e0 0x7f2104108df0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 --2- 172.21.15.19:0/657096593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21041029b0 0x7f2104102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 -- 172.21.15.19:0/657096593 >> 172.21.15.19:0/657096593 conn(0x7f21040fe6e0 msgr2=0x7f2104100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 -- 172.21.15.19:0/657096593 shutdown_connections 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.501+0000 7f2109421700 1 -- 172.21.15.19:0/657096593 wait complete. 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 Processor -- start 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 -- start start 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21041029b0 0x7f210419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 0x7f210419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21041089e0 0x7f210419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21041143c0 con 0x7f2104103300 2024-05-17T09:16:05.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2104114240 con 0x7f21041029b0 2024-05-17T09:16:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f2109421700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2104114540 con 0x7f21041089e0 2024-05-17T09:16:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.502+0000 7f20fa7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 0x7f210419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 0x7f210419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49192/0 (socket says 172.21.15.19:49192) 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 -- 172.21.15.19:0/1144488062 learned_addr learned my addr 172.21.15.19:0/1144488062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f2102ffd700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21041029b0 0x7f210419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f21037fe700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21041089e0 0x7f210419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 -- 172.21.15.19:0/1144488062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21041089e0 msgr2=0x7f210419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21041089e0 0x7f210419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 -- 172.21.15.19:0/1144488062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21041029b0 msgr2=0x7f210419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21041029b0 0x7f210419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.503+0000 7f20fa7fc700 1 -- 172.21.15.19:0/1144488062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210419fc60 con 0x7f2104103300 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.504+0000 7f20fa7fc700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 0x7f210419b330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00c970 tx=0x7f20ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.504+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec004a30 con 0x7f2104103300 2024-05-17T09:16:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.504+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210419ff50 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.504+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ec0054b0 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.504+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec007af0 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.504+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a7870 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.505+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21040512b0 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.507+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f20ec004bd0 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.508+0000 7f2100ff9700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20f00b2980 0x7f20f00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.508+0000 7f2102ffd700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20f00b2980 0x7f20f00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.508+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f20ec0e2460 con 0x7f2104103300 2024-05-17T09:16:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.509+0000 7f2102ffd700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20f00b2980 0x7f20f00b4e90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20f400b5c0 tx=0x7f20f4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.512+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20ec014030 con 0x7f2104103300 2024-05-17T09:16:05.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.734+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f210402e5e0 con 0x7f2104103300 2024-05-17T09:16:05.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.736+0000 7f2100ff9700 1 -- 172.21.15.19:0/1144488062 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24583 (secure 0 0 0) 0x7f20ec005660 con 0x7f2104103300 2024-05-17T09:16:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:05.737 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:15:48.143861+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20f00b2980 msgr2=0x7f20f00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20f00b2980 0x7f20f00b4e90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20f400b5c0 tx=0x7f20f4005960 comp rx=0 tx=0).stop 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 msgr2=0x7f210419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 0x7f210419b330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00c970 tx=0x7f20ec00cc80 comp rx=0 tx=0).stop 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 shutdown_connections 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20f00b2980 0x7f20f00b4e90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21041089e0 0x7f210419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21041029b0 0x7f210419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 --2- 172.21.15.19:0/1144488062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2104103300 0x7f210419b330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.738+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 >> 172.21.15.19:0/1144488062 conn(0x7f21040fe6e0 msgr2=0x7f2104100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.739+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 shutdown_connections 2024-05-17T09:16:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:05.739+0000 7f2109421700 1 -- 172.21.15.19:0/1144488062 wait complete. 2024-05-17T09:16:06.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T09:16:06.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:16:06.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:16:06.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1144488062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T09:16:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:16:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:16:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:06 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1144488062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:06.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:16:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:06 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1144488062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:06.698 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:08.131 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:08 smithi019 ceph-mon[31620]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T09:16:08.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 -- 172.21.15.19:0/3936336147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd101034d0 msgr2=0x7fbd101038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:08.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 --2- 172.21.15.19:0/3936336147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd101034d0 0x7fbd101038e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd000099c0 tx=0x7fbd00009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 -- 172.21.15.19:0/3936336147 shutdown_connections 2024-05-17T09:16:08.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 --2- 172.21.15.19:0/3936336147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd10108750 0x7fbd101100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 --2- 172.21.15.19:0/3936336147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd101034d0 0x7fbd101038e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 --2- 172.21.15.19:0/3936336147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10107e00 0x7fbd10108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 -- 172.21.15.19:0/3936336147 >> 172.21.15.19:0/3936336147 conn(0x7fbd100fef20 msgr2=0x7fbd10101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 -- 172.21.15.19:0/3936336147 shutdown_connections 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.381+0000 7fbd167ef700 1 -- 172.21.15.19:0/3936336147 wait complete. 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 Processor -- start 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 -- start start 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd101034d0 0x7fbd1019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 0x7fbd1019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10108750 0x7fbd1019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10113230 con 0x7fbd10107e00 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd101130b0 con 0x7fbd101034d0 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd167ef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbd101133b0 con 0x7fbd10108750 2024-05-17T09:16:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.382+0000 7fbd07fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 0x7fbd1019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 0x7fbd1019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49208/0 (socket says 172.21.15.19:49208) 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 -- 172.21.15.19:0/1226951647 learned_addr learned my addr 172.21.15.19:0/1226951647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd14d8c700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10108750 0x7fbd1019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd0ffff700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd101034d0 0x7fbd1019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 -- 172.21.15.19:0/1226951647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10108750 msgr2=0x7fbd1019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10108750 0x7fbd1019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 -- 172.21.15.19:0/1226951647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd101034d0 msgr2=0x7fbd1019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd101034d0 0x7fbd1019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 -- 172.21.15.19:0/1226951647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1019fc00 con 0x7fbd10107e00 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd14d8c700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10108750 0x7fbd1019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:16:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd07fff700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 0x7fbd1019b2d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800d980 tx=0x7fbcf800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:08.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8017070 con 0x7fbd10107e00 2024-05-17T09:16:08.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1019fef0 con 0x7fbd10107e00 2024-05-17T09:16:08.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcf8010460 con 0x7fbd10107e00 2024-05-17T09:16:08.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800de60 con 0x7fbd10107e00 2024-05-17T09:16:08.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.383+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101a7810 con 0x7fbd10107e00 2024-05-17T09:16:08.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.385+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd100512b0 con 0x7fbd10107e00 2024-05-17T09:16:08.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.386+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbcf8010610 con 0x7fbd10107e00 2024-05-17T09:16:08.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.387+0000 7fbd0dffb700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcfc0b2880 0x7fbcfc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:08.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.387+0000 7fbd0ffff700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcfc0b2880 0x7fbcfc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:08.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.387+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbcf80e20a0 con 0x7fbd10107e00 2024-05-17T09:16:08.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.387+0000 7fbd0ffff700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcfc0b2880 0x7fbcfc0b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd000058e0 tx=0x7fbd00005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:08.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.390+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcf80a9a60 con 0x7fbd10107e00 2024-05-17T09:16:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:08 smithi069 ceph-mon[35081]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T09:16:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:08 smithi079 ceph-mon[35502]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T09:16:08.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.607+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd1002e5e0 con 0x7fbd10107e00 2024-05-17T09:16:08.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.608+0000 7fbd0dffb700 1 -- 172.21.15.19:0/1226951647 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24583 (secure 0 0 0) 0x7fbcf80a9360 con 0x7fbd10107e00 2024-05-17T09:16:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:08.609 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:15:48.143861+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:16:08.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcfc0b2880 msgr2=0x7fbcfc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcfc0b2880 0x7fbcfc0b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd000058e0 tx=0x7fbd00005940 comp rx=0 tx=0).stop 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 msgr2=0x7fbd1019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 0x7fbd1019b2d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800d980 tx=0x7fbcf800dd40 comp rx=0 tx=0).stop 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 shutdown_connections 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcfc0b2880 0x7fbcfc0b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd10108750 0x7fbd1019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd101034d0 0x7fbd1019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 --2- 172.21.15.19:0/1226951647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd10107e00 0x7fbd1019b2d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 >> 172.21.15.19:0/1226951647 conn(0x7fbd100fef20 msgr2=0x7fbd101063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 shutdown_connections 2024-05-17T09:16:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:08.611+0000 7fbd167ef700 1 -- 172.21.15.19:0/1226951647 wait complete. 2024-05-17T09:16:09.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:09 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1226951647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:09.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-17T09:16:09.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:09 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1226951647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:09.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:09 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1226951647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:09.576 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:10 smithi069 ceph-mon[35081]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T09:16:10.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:10 smithi019 ceph-mon[31620]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T09:16:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:10 smithi079 ceph-mon[35502]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 -- 172.21.15.19:0/1603508871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a01045c0 msgr2=0x7f42a01049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 --2- 172.21.15.19:0/1603508871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a01045c0 0x7f42a01049d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f429c0099c0 tx=0x7f429c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 -- 172.21.15.19:0/1603508871 shutdown_connections 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 --2- 172.21.15.19:0/1603508871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42a0109840 0x7f42a01111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 --2- 172.21.15.19:0/1603508871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a01045c0 0x7f42a01049d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 --2- 172.21.15.19:0/1603508871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42a0108ef0 0x7f42a0109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 -- 172.21.15.19:0/1603508871 >> 172.21.15.19:0/1603508871 conn(0x7f42a00ffe90 msgr2=0x7f42a01022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 -- 172.21.15.19:0/1603508871 shutdown_connections 2024-05-17T09:16:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.484+0000 7f42a80e3700 1 -- 172.21.15.19:0/1603508871 wait complete. 2024-05-17T09:16:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 Processor -- start 2024-05-17T09:16:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 -- start start 2024-05-17T09:16:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42a01045c0 0x7f42a0076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:11.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42a0108ef0 0x7f42a0077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 0x7f42a0077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42a0114320 con 0x7f42a0109840 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42a01141a0 con 0x7f42a0108ef0 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a80e3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42a01144a0 con 0x7f42a01045c0 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a6680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 0x7f42a0077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a6680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 0x7f42a0077640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49220/0 (socket says 172.21.15.19:49220) 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.485+0000 7f42a6680700 1 -- 172.21.15.19:0/2773861223 learned_addr learned my addr 172.21.15.19:0/2773861223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a567e700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42a0108ef0 0x7f42a0077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a5e7f700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42a01045c0 0x7f42a0076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a6680700 1 -- 172.21.15.19:0/2773861223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42a01045c0 msgr2=0x7f42a0076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a6680700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42a01045c0 0x7f42a0076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a6680700 1 -- 172.21.15.19:0/2773861223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42a0108ef0 msgr2=0x7f42a0077100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a6680700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42a0108ef0 0x7f42a0077100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:11.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a6680700 1 -- 172.21.15.19:0/2773861223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a01a90d0 con 0x7f42a0109840 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a6680700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 0x7f42a0077640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4298007ee0 tx=0x7f429800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429800dd20 con 0x7f42a0109840 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4298004020 con 0x7f42a0109840 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a01a93c0 con 0x7f42a0109840 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298015660 con 0x7f42a0109840 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.486+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a01a98b0 con 0x7f42a0109840 2024-05-17T09:16:11.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.488+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a00512b0 con 0x7f42a0109840 2024-05-17T09:16:11.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.490+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f42980041d0 con 0x7f42a0109840 2024-05-17T09:16:11.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.491+0000 7f4292ffd700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f428c0b2880 0x7f428c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:11.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.491+0000 7f42a5e7f700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f428c0b2880 0x7f428c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:11.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.491+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f42980e26f0 con 0x7f42a0109840 2024-05-17T09:16:11.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.491+0000 7f42a5e7f700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f428c0b2880 0x7f428c0b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f429c003760 tx=0x7f429c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:11.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.494+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4298010070 con 0x7f42a0109840 2024-05-17T09:16:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:11.759+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f42a00780f0 con 0x7f42a0109840 2024-05-17T09:16:12.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.163+0000 7f4292ffd700 1 -- 172.21.15.19:0/2773861223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f42980aa050 con 0x7f42a0109840 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f428c0b2880 msgr2=0x7f428c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f428c0b2880 0x7f428c0b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f429c003760 tx=0x7f429c01a040 comp rx=0 tx=0).stop 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 msgr2=0x7f42a0077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 0x7f42a0077640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4298007ee0 tx=0x7f429800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 shutdown_connections 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f428c0b2880 0x7f428c0b4d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42a01045c0 0x7f42a0076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a0109840 0x7f42a0077640 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 --2- 172.21.15.19:0/2773861223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42a0108ef0 0x7f42a0077100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 >> 172.21.15.19:0/2773861223 conn(0x7f42a00ffe90 msgr2=0x7f42a01022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 shutdown_connections 2024-05-17T09:16:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:12.166+0000 7f42a80e3700 1 -- 172.21.15.19:0/2773861223 wait complete. 2024-05-17T09:16:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:12 smithi069 ceph-mon[35081]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:12 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2773861223' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T09:16:12.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:12 smithi019 ceph-mon[31620]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:12.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:12 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2773861223' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T09:16:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:12 smithi079 ceph-mon[35502]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:12 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2773861223' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T09:16:12.995 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-17T09:16:12.995 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-17T09:16:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:13.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:13 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2773861223' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T09:16:13.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:13 smithi019 ceph-mon[31620]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:16:13.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:13 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2773861223' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T09:16:13.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:13 smithi069 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:16:13.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:13 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2773861223' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T09:16:13.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:13 smithi079 ceph-mon[35502]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:16:14.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:14 smithi019 ceph-mon[31620]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:14 smithi079 ceph-mon[35502]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:14 smithi069 ceph-mon[35081]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.186+0000 7f782c09e700 1 -- 172.21.15.19:0/961184281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 msgr2=0x7f7824110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.186+0000 7f782c09e700 1 --2- 172.21.15.19:0/961184281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f7824110dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f781c009990 tx=0x7f781c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 -- 172.21.15.19:0/961184281 shutdown_connections 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 --2- 172.21.15.19:0/961184281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f7824110dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 --2- 172.21.15.19:0/961184281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78241054b0 0x7f78241058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 --2- 172.21.15.19:0/961184281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7824105e00 0x7f782410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 -- 172.21.15.19:0/961184281 >> 172.21.15.19:0/961184281 conn(0x7f78240fedb0 msgr2=0x7f78241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 -- 172.21.15.19:0/961184281 shutdown_connections 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 -- 172.21.15.19:0/961184281 wait complete. 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 Processor -- start 2024-05-17T09:16:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.187+0000 7f782c09e700 1 -- start start 2024-05-17T09:16:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782c09e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78241054b0 0x7f78241a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782c09e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7824105e00 0x7f78241a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782c09e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f782419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782c09e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78241142f0 con 0x7f782410e8e0 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782c09e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7824114170 con 0x7f7824105e00 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782c09e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7824114470 con 0x7f78241054b0 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f782419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f782419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38548/0 (socket says 172.21.15.19:38548) 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 -- 172.21.15.19:0/3865845142 learned_addr learned my addr 172.21.15.19:0/3865845142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f7829e3a700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78241054b0 0x7f78241a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f7829639700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7824105e00 0x7f78241a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 -- 172.21.15.19:0/3865845142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78241054b0 msgr2=0x7f78241a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78241054b0 0x7f78241a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 -- 172.21.15.19:0/3865845142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7824105e00 msgr2=0x7f78241a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7824105e00 0x7f78241a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 -- 172.21.15.19:0/3865845142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782419b160 con 0x7f782410e8e0 2024-05-17T09:16:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.188+0000 7f782a63b700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f782419aab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f781c0059d0 tx=0x7f781c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.189+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c026070 con 0x7f782410e8e0 2024-05-17T09:16:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.189+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f781c021410 con 0x7f782410e8e0 2024-05-17T09:16:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.189+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782419b3f0 con 0x7f782410e8e0 2024-05-17T09:16:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.189+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c017650 con 0x7f782410e8e0 2024-05-17T09:16:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.189+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241a7720 con 0x7f782410e8e0 2024-05-17T09:16:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.190+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78240512b0 con 0x7f782410e8e0 2024-05-17T09:16:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.191+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f781c029070 con 0x7f782410e8e0 2024-05-17T09:16:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.192+0000 7f781affd700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78100b2900 0x7f78100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.192+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f781c0e3a50 con 0x7f782410e8e0 2024-05-17T09:16:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.192+0000 7f7829e3a700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78100b2900 0x7f78100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.193+0000 7f7829e3a700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78100b2900 0x7f78100b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f782419bbf0 tx=0x7f782000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.196+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f781c030020 con 0x7f782410e8e0 2024-05-17T09:16:15.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:15.456+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f782419bfe0 con 0x7f782410e8e0 2024-05-17T09:16:16.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.184+0000 7f781affd700 1 -- 172.21.15.19:0/3865845142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f781c013070 con 0x7f782410e8e0 2024-05-17T09:16:16.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78100b2900 msgr2=0x7f78100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:16.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78100b2900 0x7f78100b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f782419bbf0 tx=0x7f782000b040 comp rx=0 tx=0).stop 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 msgr2=0x7f782419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f782419aab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f781c0059d0 tx=0x7f781c0047b0 comp rx=0 tx=0).stop 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 shutdown_connections 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78100b2900 0x7f78100b4e10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78241054b0 0x7f78241a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f782410e8e0 0x7f782419aab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 --2- 172.21.15.19:0/3865845142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7824105e00 0x7f78241a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 >> 172.21.15.19:0/3865845142 conn(0x7f78240fedb0 msgr2=0x7f78240ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 shutdown_connections 2024-05-17T09:16:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:16.188+0000 7f782c09e700 1 -- 172.21.15.19:0/3865845142 wait complete. 2024-05-17T09:16:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:16 smithi019 ceph-mon[31620]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:16 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3865845142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T09:16:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:16 smithi079 ceph-mon[35502]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:16 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3865845142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T09:16:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:16 smithi069 ceph-mon[35081]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T09:16:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:16 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3865845142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T09:16:16.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-17T09:16:17.152 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:17.498 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:17 smithi019 ceph-mon[31620]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T09:16:17.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:17 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3865845142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T09:16:17.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:17 smithi019 ceph-mon[31620]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T09:16:17.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:17 smithi019 ceph-mon[31620]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:16:17.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:17 smithi019 ceph-mon[31620]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-17T09:16:17.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:17 smithi019 ceph-mon[31620]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-17T09:16:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:17 smithi079 ceph-mon[35502]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T09:16:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:17 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3865845142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T09:16:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:17 smithi079 ceph-mon[35502]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T09:16:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:17 smithi079 ceph-mon[35502]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:16:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:17 smithi079 ceph-mon[35502]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-17T09:16:17.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:17 smithi079 ceph-mon[35502]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-17T09:16:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:17 smithi069 ceph-mon[35081]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T09:16:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:17 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3865845142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T09:16:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:17 smithi069 ceph-mon[35081]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T09:16:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:17 smithi069 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T09:16:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:17 smithi069 ceph-mon[35081]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-17T09:16:17.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:17 smithi069 ceph-mon[35081]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] up:active 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: Cluster is now healthy 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-17T09:16:18.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-17T09:16:18.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:18 smithi019 ceph-mon[31620]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] up:active 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: Cluster is now healthy 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-17T09:16:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:18 smithi079 ceph-mon[35502]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-17T09:16:18.680 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b[64746]: 2024-05-17T09:16:18.346+0000 7f927461a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-17T09:16:18.680 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b[64746]: 2024-05-17T09:16:18.346+0000 7f927461a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-17T09:16:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] up:active 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: Cluster is now healthy 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-17T09:16:18.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:18 smithi069 ceph-mon[35081]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 -- 172.21.15.19:0/3146990386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc107e20 msgr2=0x7fc3cc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/3146990386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc107e20 0x7fc3cc108230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00099c0 tx=0x7fc3c0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 -- 172.21.15.19:0/3146990386 shutdown_connections 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/3146990386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3cc108770 0x7fc3cc1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/3146990386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc103540 0x7fc3cc103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/3146990386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc107e20 0x7fc3cc108230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 -- 172.21.15.19:0/3146990386 >> 172.21.15.19:0/3146990386 conn(0x7fc3cc078df0 msgr2=0x7fc3cc079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 -- 172.21.15.19:0/3146990386 shutdown_connections 2024-05-17T09:16:18.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.812+0000 7fc3d14b0700 1 -- 172.21.15.19:0/3146990386 wait complete. 2024-05-17T09:16:18.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 Processor -- start 2024-05-17T09:16:18.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 -- start start 2024-05-17T09:16:18.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 0x7fc3cc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:18.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc107e20 0x7fc3cc19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:18.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3cc108770 0x7fc3cc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc113230 con 0x7fc3cc103540 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1130b0 con 0x7fc3cc107e20 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.813+0000 7fc3d14b0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1133b0 con 0x7fc3cc108770 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 0x7fc3cc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 0x7fc3cc19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38558/0 (socket says 172.21.15.19:38558) 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 -- 172.21.15.19:0/2248396789 learned_addr learned my addr 172.21.15.19:0/2248396789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3ca7fc700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc107e20 0x7fc3cc19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3cb7fe700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3cc108770 0x7fc3cc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 -- 172.21.15.19:0/2248396789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3cc108770 msgr2=0x7fc3cc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3cc108770 0x7fc3cc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 -- 172.21.15.19:0/2248396789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc107e20 msgr2=0x7fc3cc19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc107e20 0x7fc3cc19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:18.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 -- 172.21.15.19:0/2248396789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc19a120 con 0x7fc3cc103540 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3ca7fc700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc107e20 0x7fc3cc19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3caffd700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 0x7fc3cc19f850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc007910 tx=0x7fc3bc007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc004020 con 0x7fc3cc103540 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3bc004830 con 0x7fc3cc103540 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc19a410 con 0x7fc3cc103540 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0053a0 con 0x7fc3cc103540 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.814+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc0759e0 con 0x7fc3cc103540 2024-05-17T09:16:18.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.815+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc1049a0 con 0x7fc3cc103540 2024-05-17T09:16:18.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.817+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc3bc01f020 con 0x7fc3cc103540 2024-05-17T09:16:18.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.818+0000 7fc3b3fff700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3b40b2880 0x7fc3b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:18.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.818+0000 7fc3ca7fc700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3b40b2880 0x7fc3b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:18.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.818+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc3bc0e4140 con 0x7fc3cc103540 2024-05-17T09:16:18.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.818+0000 7fc3ca7fc700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3b40b2880 0x7fc3b40b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009990 tx=0x7fc3c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:18.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:18.822+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3bc026030 con 0x7fc3cc103540 2024-05-17T09:16:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.088+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc3cc0512b0 con 0x7fc3cc103540 2024-05-17T09:16:19.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.215+0000 7fc3b3fff700 1 -- 172.21.15.19:0/2248396789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fc3bc0abb00 con 0x7fc3cc103540 2024-05-17T09:16:19.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3b40b2880 msgr2=0x7fc3b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:19.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3b40b2880 0x7fc3b40b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009990 tx=0x7fc3c0005fd0 comp rx=0 tx=0).stop 2024-05-17T09:16:19.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 msgr2=0x7fc3cc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:19.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 0x7fc3cc19f850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc007910 tx=0x7fc3bc007cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 shutdown_connections 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc3b40b2880 0x7fc3b40b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3cc108770 0x7fc3cc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3cc103540 0x7fc3cc19f850 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 --2- 172.21.15.19:0/2248396789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3cc107e20 0x7fc3cc19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 >> 172.21.15.19:0/2248396789 conn(0x7fc3cc078df0 msgr2=0x7fc3cc100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 shutdown_connections 2024-05-17T09:16:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:19.218+0000 7fc3d14b0700 1 -- 172.21.15.19:0/2248396789 wait complete. 2024-05-17T09:16:19.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:19 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:active 2024-05-17T09:16:19.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:19 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-17T09:16:19.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:19 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2248396789' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T09:16:19.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:19 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:active 2024-05-17T09:16:19.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:19 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-17T09:16:19.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:19 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2248396789' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T09:16:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:19 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:active 2024-05-17T09:16:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:19 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-17T09:16:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:19 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2248396789' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T09:16:19.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-17T09:16:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:20 smithi079 ceph-mon[35502]: pgmap v333: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-17T09:16:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:20 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2248396789' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T09:16:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:20 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-17T09:16:20.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:20 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:16:20.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:20 smithi019 ceph-mon[31620]: pgmap v333: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-17T09:16:20.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:20 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2248396789' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T09:16:20.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:20 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-17T09:16:20.540 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:20 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:16:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:20 smithi069 ceph-mon[35081]: pgmap v333: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-17T09:16:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:20 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2248396789' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T09:16:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:20 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-17T09:16:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:20 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.883+0000 7f8788379700 1 -- 172.21.15.19:0/15001373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780107de0 msgr2=0x7f87801081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.883+0000 7f8788379700 1 --2- 172.21.15.19:0/15001373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780107de0 0x7f87801081f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87700099c0 tx=0x7f8770009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 -- 172.21.15.19:0/15001373 shutdown_connections 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 --2- 172.21.15.19:0/15001373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8780108730 0x7f8780110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 --2- 172.21.15.19:0/15001373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87801034b0 0x7f87801038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 --2- 172.21.15.19:0/15001373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780107de0 0x7f87801081f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 -- 172.21.15.19:0/15001373 >> 172.21.15.19:0/15001373 conn(0x7f87800fedb0 msgr2=0x7f87801011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 -- 172.21.15.19:0/15001373 shutdown_connections 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 -- 172.21.15.19:0/15001373 wait complete. 2024-05-17T09:16:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.884+0000 7f8788379700 1 Processor -- start 2024-05-17T09:16:21.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 -- start start 2024-05-17T09:16:21.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 0x7f87800ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780107de0 0x7f87800ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 0x7f87800ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8780113240 con 0x7f8780108730 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87801130c0 con 0x7f8780107de0 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8788379700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87801133c0 con 0x7f87801034b0 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786916700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 0x7f87800ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 0x7f87800ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786916700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 0x7f87800ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38570/0 (socket says 172.21.15.19:38570) 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786916700 1 -- 172.21.15.19:0/1552586990 learned_addr learned my addr 172.21.15.19:0/1552586990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 0x7f87800ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33142/0 (socket says 172.21.15.19:33142) 2024-05-17T09:16:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8785914700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780107de0 0x7f87800ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 -- 172.21.15.19:0/1552586990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780107de0 msgr2=0x7f87800ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780107de0 0x7f87800ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 -- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 msgr2=0x7f87800ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 0x7f87800ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.885+0000 7f8786115700 1 -- 172.21.15.19:0/1552586990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87801a90a0 con 0x7f87801034b0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f8786916700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 0x7f87800ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f8786115700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 0x7f87800ff290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f877c00d750 tx=0x7f877c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c014030 con 0x7f87801034b0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87801a9390 con 0x7f87801034b0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f877c0044b0 con 0x7f87801034b0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87801a98f0 con 0x7f87801034b0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.886+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c005280 con 0x7f87801034b0 2024-05-17T09:16:21.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.887+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764005360 con 0x7f87801034b0 2024-05-17T09:16:21.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.889+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f877c0054e0 con 0x7f87801034b0 2024-05-17T09:16:21.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.890+0000 7f87777fe700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f876c0b2880 0x7f876c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:21.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.890+0000 7f8785914700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f876c0b2880 0x7f876c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:21.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.890+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f877c0e2220 con 0x7f87801034b0 2024-05-17T09:16:21.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.890+0000 7f8785914700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f876c0b2880 0x7f876c0b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8770009990 tx=0x7f8770005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:21.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:21.894+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f877c0a9b80 con 0x7f87801034b0 2024-05-17T09:16:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.154+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8764005650 con 0x7f87801034b0 2024-05-17T09:16:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.224+0000 7f87777fe700 1 -- 172.21.15.19:0/1552586990 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f877c0a9480 con 0x7f87801034b0 2024-05-17T09:16:22.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f876c0b2880 msgr2=0x7f876c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:22.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f876c0b2880 0x7f876c0b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8770009990 tx=0x7f8770005fb0 comp rx=0 tx=0).stop 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 msgr2=0x7f87800ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 0x7f87800ff290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f877c00d750 tx=0x7f877c00da60 comp rx=0 tx=0).stop 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 shutdown_connections 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f876c0b2880 0x7f876c0b4d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87801034b0 0x7f87800ff290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780108730 0x7f87800ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 --2- 172.21.15.19:0/1552586990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780107de0 0x7f87800ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 >> 172.21.15.19:0/1552586990 conn(0x7f87800fedb0 msgr2=0x7f878006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 shutdown_connections 2024-05-17T09:16:22.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:22.227+0000 7f8788379700 1 -- 172.21.15.19:0/1552586990 wait complete. 2024-05-17T09:16:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:22 smithi019 ceph-mon[31620]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T09:16:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:22 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1552586990' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T09:16:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:22 smithi019 ceph-mon[31620]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T09:16:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:22 smithi079 ceph-mon[35502]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T09:16:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:22 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1552586990' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T09:16:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:22 smithi079 ceph-mon[35502]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T09:16:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:22 smithi069 ceph-mon[35081]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T09:16:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:22 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1552586990' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T09:16:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:22 smithi069 ceph-mon[35081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T09:16:22.946 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-17T09:16:22.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-17T09:16:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:23.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:23 smithi019 ceph-mon[31620]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T09:16:23.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:23 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:16:23.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:23 smithi079 ceph-mon[35502]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T09:16:23.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:23 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:16:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:23 smithi069 ceph-mon[35081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T09:16:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:23 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:16:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:24 smithi019 ceph-mon[31620]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-17T09:16:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:24 smithi079 ceph-mon[35502]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-17T09:16:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:24 smithi069 ceph-mon[35081]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-17T09:16:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 -- 172.21.15.19:0/1517639944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 msgr2=0x7fac081100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:25.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/1517639944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac081100b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac0000b0d0 tx=0x7fac0000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 -- 172.21.15.19:0/1517639944 shutdown_connections 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/1517639944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac081034d0 0x7fac081038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/1517639944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac081100b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/1517639944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac08107e00 0x7fac08108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 -- 172.21.15.19:0/1517639944 >> 172.21.15.19:0/1517639944 conn(0x7fac080fef20 msgr2=0x7fac08101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 -- 172.21.15.19:0/1517639944 shutdown_connections 2024-05-17T09:16:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.057+0000 7fac0e8f2700 1 -- 172.21.15.19:0/1517639944 wait complete. 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 Processor -- start 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 -- start start 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac081034d0 0x7fac0819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac08107e00 0x7fac0819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac0819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113230 con 0x7fac08108750 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac081130b0 con 0x7fac081034d0 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0e8f2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac081133b0 con 0x7fac08107e00 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0ce8f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac0819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0ce8f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac0819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50898/0 (socket says 172.21.15.19:50898) 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac0ce8f700 1 -- 172.21.15.19:0/2901167824 learned_addr learned my addr 172.21.15.19:0/2901167824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac077fe700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac08107e00 0x7fac0819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.058+0000 7fac07fff700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac081034d0 0x7fac0819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0ce8f700 1 -- 172.21.15.19:0/2901167824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac08107e00 msgr2=0x7fac0819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0ce8f700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac08107e00 0x7fac0819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0ce8f700 1 -- 172.21.15.19:0/2901167824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac081034d0 msgr2=0x7fac0819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0ce8f700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac081034d0 0x7fac0819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0ce8f700 1 -- 172.21.15.19:0/2901167824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0819fc00 con 0x7fac08108750 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0ce8f700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac0819f560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac00009850 tx=0x7fac00007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0001f070 con 0x7fac08108750 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0819fe90 con 0x7fac08108750 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac00005500 con 0x7fac08108750 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0001b8f0 con 0x7fac08108750 2024-05-17T09:16:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.059+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a0370 con 0x7fac08108750 2024-05-17T09:16:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.060+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac080512b0 con 0x7fac08108750 2024-05-17T09:16:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.063+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fac00022070 con 0x7fac08108750 2024-05-17T09:16:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.063+0000 7fac057fa700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabf00b2880 0x7fabf00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.064+0000 7fac07fff700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabf00b2880 0x7fabf00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.064+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fac0000f020 con 0x7fac08108750 2024-05-17T09:16:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.064+0000 7fac07fff700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabf00b2880 0x7fabf00b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabf8005950 tx=0x7fabf800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:25.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.067+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac00029020 con 0x7fac08108750 2024-05-17T09:16:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.320+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fac08063bf0 con 0x7fabf00b2880 2024-05-17T09:16:25.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.352+0000 7fac057fa700 1 -- 172.21.15.19:0/2901167824 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fac08063bf0 con 0x7fabf00b2880 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabf00b2880 msgr2=0x7fabf00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabf00b2880 0x7fabf00b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabf8005950 tx=0x7fabf800b410 comp rx=0 tx=0).stop 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 msgr2=0x7fac0819f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac0819f560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac00009850 tx=0x7fac00007d90 comp rx=0 tx=0).stop 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 shutdown_connections 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac08108750 0x7fac0819f560 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac081034d0 0x7fac0819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabf00b2880 0x7fabf00b4d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 --2- 172.21.15.19:0/2901167824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac08107e00 0x7fac0819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 >> 172.21.15.19:0/2901167824 conn(0x7fac080fef20 msgr2=0x7fac0806c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:25.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 shutdown_connections 2024-05-17T09:16:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:25.354+0000 7fac0e8f2700 1 -- 172.21.15.19:0/2901167824 wait complete. 2024-05-17T09:16:25.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:16:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:16:25.326+0000 7f803b519700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T09:16:25.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:16:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:16:25.326+0000 7f803b519700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T09:16:25.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:16:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:16:25.326+0000 7f803b519700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T09:16:25.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:16:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:16:25.326+0000 7f803b519700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T09:16:25.692 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:16:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x[31851]: 2024-05-17T09:16:25.326+0000 7f803b519700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T09:16:26.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-17T09:16:26.423 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:26.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:26 smithi019 ceph-mon[31620]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T09:16:26.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:26 smithi019 ceph-mon[31620]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:26 smithi079 ceph-mon[35502]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T09:16:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:26 smithi079 ceph-mon[35502]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:26 smithi069 ceph-mon[35081]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T09:16:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:26 smithi069 ceph-mon[35081]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.200+0000 7f070f99e700 1 -- 172.21.15.19:0/1726729398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 msgr2=0x7f07081038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.200+0000 7f070f99e700 1 --2- 172.21.15.19:0/1726729398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07081038c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b0d0 tx=0x7f06f800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.200+0000 7f070f99e700 1 -- 172.21.15.19:0/1726729398 shutdown_connections 2024-05-17T09:16:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.200+0000 7f070f99e700 1 --2- 172.21.15.19:0/1726729398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0708108730 0x7f0708110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.200+0000 7f070f99e700 1 --2- 172.21.15.19:0/1726729398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07081038c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.200+0000 7f070f99e700 1 --2- 172.21.15.19:0/1726729398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708107de0 0x7f07081081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.201+0000 7f070f99e700 1 -- 172.21.15.19:0/1726729398 >> 172.21.15.19:0/1726729398 conn(0x7f07080fedb0 msgr2=0x7f07081011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.201+0000 7f070f99e700 1 -- 172.21.15.19:0/1726729398 shutdown_connections 2024-05-17T09:16:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.201+0000 7f070f99e700 1 -- 172.21.15.19:0/1726729398 wait complete. 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.201+0000 7f070f99e700 1 Processor -- start 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.201+0000 7f070f99e700 1 -- start start 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070f99e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07080ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070f99e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708107de0 0x7f07080ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070f99e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0708108730 0x7f07080ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070f99e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0708113210 con 0x7f07081034b0 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070f99e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0708113090 con 0x7f0708108730 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070f99e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0708113390 con 0x7f0708107de0 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07080ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07080ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50934/0 (socket says 172.21.15.19:50934) 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 -- 172.21.15.19:0/2728159856 learned_addr learned my addr 172.21.15.19:0/2728159856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070cf39700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708107de0 0x7f07080ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 -- 172.21.15.19:0/2728159856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708107de0 msgr2=0x7f07080ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708107de0 0x7f07080ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 -- 172.21.15.19:0/2728159856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0708108730 msgr2=0x7f07080ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0708108730 0x7f07080ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 -- 172.21.15.19:0/2728159856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07081a9130 con 0x7f07081034b0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.202+0000 7f070d73a700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07080ff290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06f8007d90 tx=0x7f06f8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.203+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f801f070 con 0x7f07081034b0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.203+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06f8005500 con 0x7f07081034b0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.203+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f801b860 con 0x7f07081034b0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.203+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07081a93c0 con 0x7f07081034b0 2024-05-17T09:16:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.203+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07081a97c0 con 0x7f07081034b0 2024-05-17T09:16:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.204+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07080512b0 con 0x7f07081034b0 2024-05-17T09:16:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.205+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f06f8022070 con 0x7f07081034b0 2024-05-17T09:16:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.206+0000 7f07067fc700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06f00b2910 0x7f06f00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.206+0000 7f070cf39700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06f00b2910 0x7f06f00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.206+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f06f800f020 con 0x7f07081034b0 2024-05-17T09:16:28.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.207+0000 7f070cf39700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06f00b2910 0x7f06f00b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0708107c40 tx=0x7f06fc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.211+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06f8029020 con 0x7f07081034b0 2024-05-17T09:16:28.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:28 smithi019 ceph-mon[31620]: pgmap v337: 129 pgs: 129 active+clean; 585 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T09:16:28.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:28 smithi019 ceph-mon[31620]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T09:16:28.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.460+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f070802e6a0 con 0x7f06f00b2910 2024-05-17T09:16:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.487+0000 7f07067fc700 1 -- 172.21.15.19:0/2728159856 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f070802e6a0 con 0x7f06f00b2910 2024-05-17T09:16:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06f00b2910 msgr2=0x7f06f00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06f00b2910 0x7f06f00b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0708107c40 tx=0x7f06fc00a300 comp rx=0 tx=0).stop 2024-05-17T09:16:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 msgr2=0x7f07080ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07080ff290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06f8007d90 tx=0x7f06f8007dc0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 shutdown_connections 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0708108730 0x7f07080ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07081034b0 0x7f07080ff290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f06f00b2910 0x7f06f00b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 --2- 172.21.15.19:0/2728159856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708107de0 0x7f07080ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 >> 172.21.15.19:0/2728159856 conn(0x7f07080fedb0 msgr2=0x7f0708106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 shutdown_connections 2024-05-17T09:16:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:28.489+0000 7f070f99e700 1 -- 172.21.15.19:0/2728159856 wait complete. 2024-05-17T09:16:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:28 smithi079 ceph-mon[35502]: pgmap v337: 129 pgs: 129 active+clean; 585 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T09:16:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:28 smithi079 ceph-mon[35502]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T09:16:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:28 smithi069 ceph-mon[35081]: pgmap v337: 129 pgs: 129 active+clean; 585 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-17T09:16:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:28 smithi069 ceph-mon[35081]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T09:16:29.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-17T09:16:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:30 smithi079 ceph-mon[35502]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:30 smithi079 ceph-mon[35502]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-17T09:16:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:30 smithi069 ceph-mon[35081]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:30 smithi069 ceph-mon[35081]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-17T09:16:30.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:30 smithi019 ceph-mon[31620]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:30.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:30 smithi019 ceph-mon[31620]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 -- 172.21.15.19:0/2027031424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac104640 msgr2=0x7f79ac104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 --2- 172.21.15.19:0/2027031424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac104640 0x7f79ac104a50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f79a00099c0 tx=0x7f79a0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 -- 172.21.15.19:0/2027031424 shutdown_connections 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 --2- 172.21.15.19:0/2027031424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79ac109870 0x7f79ac1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 --2- 172.21.15.19:0/2027031424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac104640 0x7f79ac104a50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 --2- 172.21.15.19:0/2027031424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79ac108f20 0x7f79ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 -- 172.21.15.19:0/2027031424 >> 172.21.15.19:0/2027031424 conn(0x7f79ac079ef0 msgr2=0x7f79ac07a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 -- 172.21.15.19:0/2027031424 shutdown_connections 2024-05-17T09:16:31.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.359+0000 7f79b309c700 1 -- 172.21.15.19:0/2027031424 wait complete. 2024-05-17T09:16:31.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 Processor -- start 2024-05-17T09:16:31.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 -- start start 2024-05-17T09:16:31.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79ac104640 0x7f79ac077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79ac108f20 0x7f79ac075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 0x7f79ac076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1143c0 con 0x7f79ac109870 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac114240 con 0x7f79ac108f20 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.360+0000 7f79b309c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac114540 con 0x7f79ac104640 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 0x7f79ac076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 0x7f79ac076090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50956/0 (socket says 172.21.15.19:50956) 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 -- 172.21.15.19:0/2070520084 learned_addr learned my addr 172.21.15.19:0/2070520084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79abfff700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79ac108f20 0x7f79ac075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b0e38700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79ac104640 0x7f79ac077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 -- 172.21.15.19:0/2070520084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79ac104640 msgr2=0x7f79ac077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79ac104640 0x7f79ac077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 -- 172.21.15.19:0/2070520084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79ac108f20 msgr2=0x7f79ac075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79ac108f20 0x7f79ac075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 -- 172.21.15.19:0/2070520084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac076830 con 0x7f79ac109870 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.361+0000 7f79b1639700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 0x7f79ac076090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f79a400ada0 tx=0x7f79a400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.362+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4017020 con 0x7f79ac109870 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.362+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac1ad7a0 con 0x7f79ac109870 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.362+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79a400cd60 con 0x7f79ac109870 2024-05-17T09:16:31.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.362+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4005280 con 0x7f79ac109870 2024-05-17T09:16:31.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.362+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac1add30 con 0x7f79ac109870 2024-05-17T09:16:31.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.363+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ac105aa0 con 0x7f79ac109870 2024-05-17T09:16:31.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.364+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f79a400c8d0 con 0x7f79ac109870 2024-05-17T09:16:31.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.365+0000 7f79a9ffb700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f799c0b2990 0x7f799c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:31.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.365+0000 7f79b0e38700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f799c0b2990 0x7f799c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:31.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.365+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f79a40e1d30 con 0x7f79ac109870 2024-05-17T09:16:31.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.366+0000 7f79b0e38700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f799c0b2990 0x7f799c0b4ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f79a0000c00 tx=0x7f79a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:31.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.370+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79a40e6070 con 0x7f79ac109870 2024-05-17T09:16:31.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.620+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f79ac063bf0 con 0x7f799c0b2990 2024-05-17T09:16:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.625+0000 7f79a9ffb700 1 -- 172.21.15.19:0/2070520084 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f79ac063bf0 con 0x7f799c0b2990 2024-05-17T09:16:31.626 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.627+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f799c0b2990 msgr2=0x7f799c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.627+0000 7f79b309c700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f799c0b2990 0x7f799c0b4ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f79a0000c00 tx=0x7f79a0005fb0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 msgr2=0x7f79ac076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 0x7f79ac076090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f79a400ada0 tx=0x7f79a400c330 comp rx=0 tx=0).stop 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 shutdown_connections 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f799c0b2990 0x7f799c0b4ea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79ac104640 0x7f79ac077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ac109870 0x7f79ac076090 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 --2- 172.21.15.19:0/2070520084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79ac108f20 0x7f79ac075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 >> 172.21.15.19:0/2070520084 conn(0x7f79ac079ef0 msgr2=0x7f79ac102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 shutdown_connections 2024-05-17T09:16:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:31.628+0000 7f79b309c700 1 -- 172.21.15.19:0/2070520084 wait complete. 2024-05-17T09:16:32.321 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-17T09:16:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:32 smithi079 ceph-mon[35502]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-17T09:16:32.573 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:32 smithi069 ceph-mon[35081]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-17T09:16:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:32 smithi019 ceph-mon[31620]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-17T09:16:33.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:33 smithi019 ceph-mon[31620]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:33.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:33 smithi079 ceph-mon[35502]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:33 smithi069 ceph-mon[35081]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:34.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:34 smithi019 ceph-mon[31620]: pgmap v340: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-17T09:16:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 -- 172.21.15.19:0/3989424408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c10a3d0 msgr2=0x7fda1c10a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 --2- 172.21.15.19:0/3989424408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c10a3d0 0x7fda1c10a7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009990 tx=0x7fda0c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 -- 172.21.15.19:0/3989424408 shutdown_connections 2024-05-17T09:16:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 --2- 172.21.15.19:0/3989424408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c103120 0x7fda1c111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 --2- 172.21.15.19:0/3989424408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c10a3d0 0x7fda1c10a7e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 --2- 172.21.15.19:0/3989424408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 0x7fda1c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 -- 172.21.15.19:0/3989424408 >> 172.21.15.19:0/3989424408 conn(0x7fda1c0fe6e0 msgr2=0x7fda1c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 -- 172.21.15.19:0/3989424408 shutdown_connections 2024-05-17T09:16:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.462+0000 7fda20d28700 1 -- 172.21.15.19:0/3989424408 wait complete. 2024-05-17T09:16:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.463+0000 7fda20d28700 1 Processor -- start 2024-05-17T09:16:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.463+0000 7fda20d28700 1 -- start start 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda20d28700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 0x7fda1c1039b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda20d28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c103120 0x7fda1c103ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda20d28700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c10a3d0 0x7fda1c104430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda20d28700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c114320 con 0x7fda1c103120 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda20d28700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1141a0 con 0x7fda1c10a3d0 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda20d28700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1144a0 con 0x7fda1c1027d0 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda19d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c103120 0x7fda1c103ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda19d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c103120 0x7fda1c103ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38070/0 (socket says 172.21.15.19:38070) 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda19d9b700 1 -- 172.21.15.19:0/2153679253 learned_addr learned my addr 172.21.15.19:0/2153679253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 0x7fda1c1039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1ad9d700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c10a3d0 0x7fda1c104430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 -- 172.21.15.19:0/2153679253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c10a3d0 msgr2=0x7fda1c104430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c10a3d0 0x7fda1c104430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 -- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c103120 msgr2=0x7fda1c103ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c103120 0x7fda1c103ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 -- 172.21.15.19:0/2153679253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c1a9150 con 0x7fda1c1027d0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1ad9d700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c10a3d0 0x7fda1c104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.464+0000 7fda1a59c700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 0x7fda1c1039b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda0c000c00 tx=0x7fda0c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.465+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c026070 con 0x7fda1c1027d0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.465+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c1a93e0 con 0x7fda1c1027d0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.465+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda0c021410 con 0x7fda1c1027d0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.465+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c017650 con 0x7fda1c1027d0 2024-05-17T09:16:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.465+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1a9830 con 0x7fda1c1027d0 2024-05-17T09:16:34.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.466+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c10c840 con 0x7fda1c1027d0 2024-05-17T09:16:34.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.467+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fda0c029070 con 0x7fda1c1027d0 2024-05-17T09:16:34.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.468+0000 7fda0b7fe700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda040b2910 0x7fda040b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:34.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.468+0000 7fda19d9b700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda040b2910 0x7fda040b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:34.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.468+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fda0c0e3ce0 con 0x7fda1c1027d0 2024-05-17T09:16:34.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.469+0000 7fda19d9b700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda040b2910 0x7fda040b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda1c100410 tx=0x7fda10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.472+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda0c013070 con 0x7fda1c1027d0 2024-05-17T09:16:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:34 smithi069 ceph-mon[35081]: pgmap v340: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-17T09:16:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.724+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda1c104ee0 con 0x7fda040b2910 2024-05-17T09:16:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.754+0000 7fda0b7fe700 1 -- 172.21.15.19:0/2153679253 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda1c104ee0 con 0x7fda040b2910 2024-05-17T09:16:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda040b2910 msgr2=0x7fda040b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda040b2910 0x7fda040b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda1c100410 tx=0x7fda10009260 comp rx=0 tx=0).stop 2024-05-17T09:16:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 msgr2=0x7fda1c1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 0x7fda1c1039b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda0c000c00 tx=0x7fda0c0047b0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 shutdown_connections 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda040b2910 0x7fda040b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda1c1027d0 0x7fda1c1039b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda1c10a3d0 0x7fda1c104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 --2- 172.21.15.19:0/2153679253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda1c103120 0x7fda1c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 >> 172.21.15.19:0/2153679253 conn(0x7fda1c0fe6e0 msgr2=0x7fda1c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 shutdown_connections 2024-05-17T09:16:34.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:34.757+0000 7fda20d28700 1 -- 172.21.15.19:0/2153679253 wait complete. 2024-05-17T09:16:34.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:34 smithi079 ceph-mon[35502]: pgmap v340: 129 pgs: 129 active+clean; 605 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-17T09:16:35.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-17T09:16:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:36.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:36 smithi019 ceph-mon[31620]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:36.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:36 smithi019 ceph-mon[31620]: pgmap v341: 129 pgs: 129 active+clean; 611 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-17T09:16:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:36 smithi069 ceph-mon[35081]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:36 smithi069 ceph-mon[35081]: pgmap v341: 129 pgs: 129 active+clean; 611 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-17T09:16:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:36 smithi079 ceph-mon[35502]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:36 smithi079 ceph-mon[35502]: pgmap v341: 129 pgs: 129 active+clean; 611 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-17T09:16:37.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 -- 172.21.15.19:0/1935633359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c107e00 msgr2=0x7ff75c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:37.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 --2- 172.21.15.19:0/1935633359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c107e00 0x7ff75c108210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7500099c0 tx=0x7ff750009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 -- 172.21.15.19:0/1935633359 shutdown_connections 2024-05-17T09:16:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 --2- 172.21.15.19:0/1935633359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75c1034d0 0x7ff75c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 --2- 172.21.15.19:0/1935633359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c108750 0x7ff75c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 --2- 172.21.15.19:0/1935633359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c107e00 0x7ff75c108210 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 -- 172.21.15.19:0/1935633359 >> 172.21.15.19:0/1935633359 conn(0x7ff75c0fef20 msgr2=0x7ff75c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.644+0000 7ff7608fc700 1 -- 172.21.15.19:0/1935633359 shutdown_connections 2024-05-17T09:16:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.645+0000 7ff7608fc700 1 -- 172.21.15.19:0/1935633359 wait complete. 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 Processor -- start 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 -- start start 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 0x7ff75c110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75c107e00 0x7ff75c110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c108750 0x7ff75c111510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c114380 con 0x7ff75c1034d0 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c114200 con 0x7ff75c108750 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff7608fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c114500 con 0x7ff75c107e00 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 0x7ff75c110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 0x7ff75c110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38084/0 (socket says 172.21.15.19:38084) 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 -- 172.21.15.19:0/789068769 learned_addr learned my addr 172.21.15.19:0/789068769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75ad9d700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c108750 0x7ff75c111510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff759d9b700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75c107e00 0x7ff75c110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 -- 172.21.15.19:0/789068769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75c107e00 msgr2=0x7ff75c110fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75c107e00 0x7ff75c110fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 -- 172.21.15.19:0/789068769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c108750 msgr2=0x7ff75c111510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c108750 0x7ff75c111510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.646+0000 7ff75a59c700 1 -- 172.21.15.19:0/789068769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c1a9030 con 0x7ff75c1034d0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.647+0000 7ff75a59c700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 0x7ff75c110a90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff74c007e20 tx=0x7ff74c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.647+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c016020 con 0x7ff75c1034d0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.647+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff74c00ac90 con 0x7ff75c1034d0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.647+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c1a9320 con 0x7ff75c1034d0 2024-05-17T09:16:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.647+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0115e0 con 0x7ff75c1034d0 2024-05-17T09:16:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.647+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1a9830 con 0x7ff75c1034d0 2024-05-17T09:16:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.648+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c0512b0 con 0x7ff75c1034d0 2024-05-17T09:16:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.650+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff74c019020 con 0x7ff75c1034d0 2024-05-17T09:16:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.650+0000 7ff74b7fe700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7440b2910 0x7ff7440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.651+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff74c0e27a0 con 0x7ff75c1034d0 2024-05-17T09:16:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.651+0000 7ff759d9b700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7440b2910 0x7ff7440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.651+0000 7ff759d9b700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7440b2910 0x7ff7440b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff750009990 tx=0x7ff750005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.654+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff74c020070 con 0x7ff75c1034d0 2024-05-17T09:16:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.905+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff75c063bf0 con 0x7ff7440b2910 2024-05-17T09:16:37.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.910+0000 7ff74b7fe700 1 -- 172.21.15.19:0/789068769 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff75c063bf0 con 0x7ff7440b2910 2024-05-17T09:16:37.911 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/8fa9e892-296d-4b26-983c-404512153705 2024-05-17T09:16:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7440b2910 msgr2=0x7ff7440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7440b2910 0x7ff7440b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff750009990 tx=0x7ff750005fb0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 msgr2=0x7ff75c110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 0x7ff75c110a90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff74c007e20 tx=0x7ff74c00a340 comp rx=0 tx=0).stop 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 shutdown_connections 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c108750 0x7ff75c111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75c1034d0 0x7ff75c110a90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7440b2910 0x7ff7440b4e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 --2- 172.21.15.19:0/789068769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75c107e00 0x7ff75c110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 >> 172.21.15.19:0/789068769 conn(0x7ff75c0fef20 msgr2=0x7ff75c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 shutdown_connections 2024-05-17T09:16:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:37.913+0000 7ff7608fc700 1 -- 172.21.15.19:0/789068769 wait complete. 2024-05-17T09:16:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:38 smithi069 ceph-mon[35081]: pgmap v342: 129 pgs: 129 active+clean; 611 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-17T09:16:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:38 smithi019 ceph-mon[31620]: pgmap v342: 129 pgs: 129 active+clean; 611 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-17T09:16:38.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:16:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:38 smithi079 ceph-mon[35502]: pgmap v342: 129 pgs: 129 active+clean; 611 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-17T09:16:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:39.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:39 smithi019 ceph-mon[31620]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:39.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:39 smithi069 ceph-mon[35081]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:39 smithi079 ceph-mon[35502]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:16:40.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:40 smithi019 ceph-mon[31620]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-05-17T09:16:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:40 smithi069 ceph-mon[35081]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-05-17T09:16:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 -- 172.21.15.19:0/285811732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54109840 msgr2=0x7f6c541111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 --2- 172.21.15.19:0/285811732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54109840 0x7f6c541111a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0099c0 tx=0x7f6c4c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 -- 172.21.15.19:0/285811732 shutdown_connections 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 --2- 172.21.15.19:0/285811732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c541045c0 0x7f6c541049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 --2- 172.21.15.19:0/285811732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54109840 0x7f6c541111a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 --2- 172.21.15.19:0/285811732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c54108ef0 0x7f6c54109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 -- 172.21.15.19:0/285811732 >> 172.21.15.19:0/285811732 conn(0x7f6c540ffe90 msgr2=0x7f6c541022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 -- 172.21.15.19:0/285811732 shutdown_connections 2024-05-17T09:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.770+0000 7f6c5989b700 1 -- 172.21.15.19:0/285811732 wait complete. 2024-05-17T09:16:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:40 smithi079 ceph-mon[35502]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-05-17T09:16:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 Processor -- start 2024-05-17T09:16:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 -- start start 2024-05-17T09:16:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c541045c0 0x7f6c54076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 0x7f6c54077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c54109840 0x7f6c54077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54114320 con 0x7f6c54108ef0 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541141a0 con 0x7f6c54109840 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.771+0000 7f6c5989b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541144a0 con 0x7f6c541045c0 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 0x7f6c54077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 0x7f6c54077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38114/0 (socket says 172.21.15.19:38114) 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 -- 172.21.15.19:0/2767950230 learned_addr learned my addr 172.21.15.19:0/2767950230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c52ffd700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c541045c0 0x7f6c54076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c537fe700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c54109840 0x7f6c54077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 -- 172.21.15.19:0/2767950230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c541045c0 msgr2=0x7f6c54076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c541045c0 0x7f6c54076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 -- 172.21.15.19:0/2767950230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c54109840 msgr2=0x7f6c54077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c54109840 0x7f6c54077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 -- 172.21.15.19:0/2767950230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c541a8fc0 con 0x7f6c54108ef0 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c527fc700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 0x7f6c54077100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c9e0 tx=0x7f6c3c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c007aa0 con 0x7f6c54108ef0 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c3c007c40 con 0x7f6c54108ef0 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c018400 con 0x7f6c54108ef0 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.772+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c541a92b0 con 0x7f6c54108ef0 2024-05-17T09:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.773+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541a9840 con 0x7f6c54108ef0 2024-05-17T09:16:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.773+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c540512b0 con 0x7f6c54108ef0 2024-05-17T09:16:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.775+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6c3c00f450 con 0x7f6c54108ef0 2024-05-17T09:16:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.776+0000 7f6c58899700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c400b2990 0x7f6c400b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.776+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6c3c0e27a0 con 0x7f6c54108ef0 2024-05-17T09:16:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.776+0000 7f6c52ffd700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c400b2990 0x7f6c400b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.776+0000 7f6c52ffd700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c400b2990 0x7f6c400b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c440060b0 tx=0x7f6c4400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:40.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:40.779+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c3c014030 con 0x7f6c54108ef0 2024-05-17T09:16:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.049+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c540780f0 con 0x7f6c54108ef0 2024-05-17T09:16:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.050+0000 7f6c58899700 1 -- 172.21.15.19:0/2767950230 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f6c3c0aa110 con 0x7f6c54108ef0 2024-05-17T09:16:41.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:41.051 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:16:22.157327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c400b2990 msgr2=0x7f6c400b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c400b2990 0x7f6c400b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c440060b0 tx=0x7f6c4400c040 comp rx=0 tx=0).stop 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 msgr2=0x7f6c54077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 0x7f6c54077100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c9e0 tx=0x7f6c3c00ccf0 comp rx=0 tx=0).stop 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 shutdown_connections 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c400b2990 0x7f6c400b4ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c541045c0 0x7f6c54076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c54109840 0x7f6c54077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 --2- 172.21.15.19:0/2767950230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c54108ef0 0x7f6c54077100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.052+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 >> 172.21.15.19:0/2767950230 conn(0x7f6c540ffe90 msgr2=0x7f6c541086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.053+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 shutdown_connections 2024-05-17T09:16:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:41.053+0000 7f6c5989b700 1 -- 172.21.15.19:0/2767950230 wait complete. 2024-05-17T09:16:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:16:41.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:41 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2767950230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:16:41.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:41 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2767950230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:16:41.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:41 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2767950230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:16:41.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:16:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:42.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:42 smithi019 ceph-mon[31620]: pgmap v344: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-05-17T09:16:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:42 smithi069 ceph-mon[35081]: pgmap v344: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-05-17T09:16:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:42 smithi079 ceph-mon[35502]: pgmap v344: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-05-17T09:16:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 -- 172.21.15.19:0/3365066530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64108ef0 msgr2=0x7fec64109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 --2- 172.21.15.19:0/3365066530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64108ef0 0x7fec64109300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0099c0 tx=0x7fec4c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 -- 172.21.15.19:0/3365066530 shutdown_connections 2024-05-17T09:16:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 --2- 172.21.15.19:0/3365066530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec64109840 0x7fec641111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 --2- 172.21.15.19:0/3365066530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec641045c0 0x7fec641049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 --2- 172.21.15.19:0/3365066530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64108ef0 0x7fec64109300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.830+0000 7fec68a13700 1 -- 172.21.15.19:0/3365066530 >> 172.21.15.19:0/3365066530 conn(0x7fec640ffe90 msgr2=0x7fec641022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.831+0000 7fec68a13700 1 -- 172.21.15.19:0/3365066530 shutdown_connections 2024-05-17T09:16:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.831+0000 7fec68a13700 1 -- 172.21.15.19:0/3365066530 wait complete. 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 Processor -- start 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 -- start start 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec641045c0 0x7fec6419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec64108ef0 0x7fec6419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 0x7fec6419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec64114300 con 0x7fec64109840 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec64114180 con 0x7fec64108ef0 2024-05-17T09:16:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec68a13700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fec64114480 con 0x7fec641045c0 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 0x7fec6419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 0x7fec6419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43806/0 (socket says 172.21.15.19:43806) 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec62d9d700 1 -- 172.21.15.19:0/2880970606 learned_addr learned my addr 172.21.15.19:0/2880970606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec6259c700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec641045c0 0x7fec6419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.832+0000 7fec61d9b700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec64108ef0 0x7fec6419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec62d9d700 1 -- 172.21.15.19:0/2880970606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec641045c0 msgr2=0x7fec6419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec62d9d700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec641045c0 0x7fec6419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec62d9d700 1 -- 172.21.15.19:0/2880970606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec64108ef0 msgr2=0x7fec6419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec62d9d700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec64108ef0 0x7fec6419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec62d9d700 1 -- 172.21.15.19:0/2880970606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6419fca0 con 0x7fec64109840 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec62d9d700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 0x7fec6419f600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c970 tx=0x7fec5c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004a30 con 0x7fec64109840 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6419ff90 con 0x7fec64109840 2024-05-17T09:16:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec5c0054b0 con 0x7fec64109840 2024-05-17T09:16:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00eac0 con 0x7fec64109840 2024-05-17T09:16:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.833+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641a78b0 con 0x7fec64109840 2024-05-17T09:16:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.835+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec640512b0 con 0x7fec64109840 2024-05-17T09:16:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.836+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fec5c00eca0 con 0x7fec64109840 2024-05-17T09:16:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.837+0000 7fec5b7fe700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fec500b2910 0x7fec500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.837+0000 7fec6259c700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fec500b2910 0x7fec500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.838+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fec5c0e25b0 con 0x7fec64109840 2024-05-17T09:16:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.838+0000 7fec6259c700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fec500b2910 0x7fec500b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fec54009550 tx=0x7fec540091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:43.840+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec5c014030 con 0x7fec64109840 2024-05-17T09:16:44.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.058+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec6419c130 con 0x7fec64109840 2024-05-17T09:16:44.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.060+0000 7fec5b7fe700 1 -- 172.21.15.19:0/2880970606 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24583 (secure 0 0 0) 0x7fec5c005660 con 0x7fec64109840 2024-05-17T09:16:44.061 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:44.061 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:15:48.143861+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:16:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fec500b2910 msgr2=0x7fec500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fec500b2910 0x7fec500b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fec54009550 tx=0x7fec540091b0 comp rx=0 tx=0).stop 2024-05-17T09:16:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 msgr2=0x7fec6419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 0x7fec6419f600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c970 tx=0x7fec5c00cc80 comp rx=0 tx=0).stop 2024-05-17T09:16:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 shutdown_connections 2024-05-17T09:16:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fec500b2910 0x7fec500b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec641045c0 0x7fec6419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec64109840 0x7fec6419f600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 --2- 172.21.15.19:0/2880970606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec64108ef0 0x7fec6419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 >> 172.21.15.19:0/2880970606 conn(0x7fec640ffe90 msgr2=0x7fec641022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 shutdown_connections 2024-05-17T09:16:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:44.063+0000 7fec68a13700 1 -- 172.21.15.19:0/2880970606 wait complete. 2024-05-17T09:16:44.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:44 smithi019 ceph-mon[31620]: pgmap v345: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-17T09:16:44.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:44 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2880970606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:44 smithi069 ceph-mon[35081]: pgmap v345: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-17T09:16:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:44 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2880970606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:44 smithi079 ceph-mon[35502]: pgmap v345: 129 pgs: 129 active+clean; 636 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-05-17T09:16:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:44 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2880970606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:16:44.811 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T09:16:44.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:16:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:46.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:46 smithi019 ceph-mon[31620]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-17T09:16:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:46 smithi069 ceph-mon[35081]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-17T09:16:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:46 smithi079 ceph-mon[35502]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-17T09:16:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 -- 172.21.15.19:0/1432215349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 msgr2=0x7ff4381038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 --2- 172.21.15.19:0/1432215349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff4381038c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4280099c0 tx=0x7ff428009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 -- 172.21.15.19:0/1432215349 shutdown_connections 2024-05-17T09:16:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 --2- 172.21.15.19:0/1432215349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 0x7ff438110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 --2- 172.21.15.19:0/1432215349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff4381038c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 --2- 172.21.15.19:0/1432215349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff438107de0 0x7ff4381081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 -- 172.21.15.19:0/1432215349 >> 172.21.15.19:0/1432215349 conn(0x7ff4380fede0 msgr2=0x7ff438101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 -- 172.21.15.19:0/1432215349 shutdown_connections 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.968+0000 7ff43db99700 1 -- 172.21.15.19:0/1432215349 wait complete. 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.969+0000 7ff43db99700 1 Processor -- start 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.969+0000 7ff43db99700 1 -- start start 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff43db99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff43819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff43db99700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff438107de0 0x7ff43819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff43db99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 0x7ff438199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff43db99700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113210 con 0x7ff4381034b0 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff43db99700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113090 con 0x7ff438108730 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff43db99700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113390 con 0x7ff438107de0 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff43819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff43819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43826/0 (socket says 172.21.15.19:43826) 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 -- 172.21.15.19:0/1551785251 learned_addr learned my addr 172.21.15.19:0/1551785251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff42ffff700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff438107de0 0x7ff43819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff437fff700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 0x7ff438199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 -- 172.21.15.19:0/1551785251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff438107de0 msgr2=0x7ff43819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff438107de0 0x7ff43819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 -- 172.21.15.19:0/1551785251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 msgr2=0x7ff438199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 0x7ff438199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 -- 172.21.15.19:0/1551785251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43819a0c0 con 0x7ff4381034b0 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff437fff700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 0x7ff438199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4377fe700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff43819f7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff428000c00 tx=0x7ff4280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.970+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42801a840 con 0x7ff4381034b0 2024-05-17T09:16:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.971+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff428004830 con 0x7ff4381034b0 2024-05-17T09:16:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.971+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428005380 con 0x7ff4381034b0 2024-05-17T09:16:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.971+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43819a350 con 0x7ff4381034b0 2024-05-17T09:16:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.971+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43819a7d0 con 0x7ff4381034b0 2024-05-17T09:16:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.972+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4380512b0 con 0x7ff4381034b0 2024-05-17T09:16:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.974+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff42802c060 con 0x7ff4381034b0 2024-05-17T09:16:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.975+0000 7ff4357fa700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4240b2910 0x7ff4240b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.975+0000 7ff42ffff700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4240b2910 0x7ff4240b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.975+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff4280e57d0 con 0x7ff4381034b0 2024-05-17T09:16:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.975+0000 7ff42ffff700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4240b2910 0x7ff4240b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4381068b0 tx=0x7ff42000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:46.978+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff428061df0 con 0x7ff4381034b0 2024-05-17T09:16:47.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.190+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff43810b1d0 con 0x7ff4240b2910 2024-05-17T09:16:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.203+0000 7ff4357fa700 1 -- 172.21.15.19:0/1551785251 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340605 (secure 0 0 0) 0x7ff43810b1d0 con 0x7ff4240b2910 2024-05-17T09:16:47.208 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4240b2910 msgr2=0x7ff4240b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4240b2910 0x7ff4240b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4381068b0 tx=0x7ff42000b410 comp rx=0 tx=0).stop 2024-05-17T09:16:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 msgr2=0x7ff43819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff43819f7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff428000c00 tx=0x7ff4280047b0 comp rx=0 tx=0).stop 2024-05-17T09:16:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 shutdown_connections 2024-05-17T09:16:47.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff438108730 0x7ff438199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4381034b0 0x7ff43819f7f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4240b2910 0x7ff4240b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 --2- 172.21.15.19:0/1551785251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff438107de0 0x7ff43819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 >> 172.21.15.19:0/1551785251 conn(0x7ff4380fede0 msgr2=0x7ff438101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 shutdown_connections 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:47.206+0000 7ff43db99700 1 -- 172.21.15.19:0/1551785251 wait complete. 2024-05-17T09:16:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:16:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-17T09:16:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-05-17T09:16:47.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-17T09:16:47.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-17T09:16:47.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-05-17T09:16:47.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-17T09:16:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-17T09:16:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-05-17T09:16:47.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-17T09:16:47.947 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-05-17T09:16:46.895385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652877,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":259,"num_read_kb":218,"num_write":289,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348332,"kb_used_data":11660,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593492,"statfs":{"total":1151940427776,"available":1151583735808,"internally_reserved":0,"allocated":11939840,"data_stored":7887069,"data_compressed":424623,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27655,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":73,"num_read_kb":58,"num_write":20,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003566"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153939+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:15:48.153939+0000","last_peered":"2024-05-17T09:15:48.153939+0000","last_clean":"2024-05-17T09:15:48.153939+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:15:48.153939+0000","last_undegraded":"2024-05-17T09:15:48.153939+0000","last_fullsized":"2024-05-17T09:15:48.153939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147535+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:15:48.147535+0000","last_peered":"2024-05-17T09:15:48.147535+0000","last_clean":"2024-05-17T09:15:48.147535+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:15:48.147535+0000","last_undegraded":"2024-05-17T09:15:48.147535+0000","last_fullsized":"2024-05-17T09:15:48.147535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147304+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:15:48.147304+0000","last_peered":"2024-05-17T09:15:48.147304+0000","last_clean":"2024-05-17T09:15:48.147304+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:15:48.147304+0000","last_undegraded":"2024-05-17T09:15:48.147304+0000","last_fullsized":"2024-05-17T09:15:48.147304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148854+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:15:48.148854+0000","last_peered":"2024-05-17T09:15:48.148854+0000","last_clean":"2024-05-17T09:15:48.148854+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:15:48.148854+0000","last_undegraded":"2024-05-17T09:15:48.148854+0000","last_fullsized":"2024-05-17T09:15:48.148854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150013+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:15:48.150013+0000","last_peered":"2024-05-17T09:15:48.150013+0000","last_clean":"2024-05-17T09:15:48.150013+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:15:48.150013+0000","last_undegraded":"2024-05-17T09:15:48.150013+0000","last_fullsized":"2024-05-17T09:15:48.150013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153857+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:15:48.153857+0000","last_peered":"2024-05-17T09:15:48.153857+0000","last_clean":"2024-05-17T09:15:48.153857+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:15:48.153857+0000","last_undegraded":"2024-05-17T09:15:48.153857+0000","last_fullsized":"2024-05-17T09:15:48.153857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151240+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:15:48.151240+0000","last_peered":"2024-05-17T09:15:48.151240+0000","last_clean":"2024-05-17T09:15:48.151240+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:15:48.151240+0000","last_undegraded":"2024-05-17T09:15:48.151240+0000","last_fullsized":"2024-05-17T09:15:48.151240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154109+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:15:48.154109+0000","last_peered":"2024-05-17T09:15:48.154109+0000","last_clean":"2024-05-17T09:15:48.154109+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:15:48.154109+0000","last_undegraded":"2024-05-17T09:15:48.154109+0000","last_fullsized":"2024-05-17T09:15:48.154109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.402155+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:15:48.402155+0000","last_peered":"2024-05-17T09:15:48.402155+0000","last_clean":"2024-05-17T09:15:48.402155+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:15:48.402155+0000","last_undegraded":"2024-05-17T09:15:48.402155+0000","last_fullsized":"2024-05-17T09:15:48.402155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154198+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:15:48.154198+0000","last_peered":"2024-05-17T09:15:48.154198+0000","last_clean":"2024-05-17T09:15:48.154198+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:15:48.154198+0000","last_undegraded":"2024-05-17T09:15:48.154198+0000","last_fullsized":"2024-05-17T09:15:48.154198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152024+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:15:48.152024+0000","last_peered":"2024-05-17T09:15:48.152024+0000","last_clean":"2024-05-17T09:15:48.152024+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:15:48.152024+0000","last_undegraded":"2024-05-17T09:15:48.152024+0000","last_fullsized":"2024-05-17T09:15:48.152024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148672+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:15:48.148672+0000","last_peered":"2024-05-17T09:15:48.148672+0000","last_clean":"2024-05-17T09:15:48.148672+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:15:48.148672+0000","last_undegraded":"2024-05-17T09:15:48.148672+0000","last_fullsized":"2024-05-17T09:15:48.148672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.525966+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:15:48.525966+0000","last_peered":"2024-05-17T09:15:48.525966+0000","last_clean":"2024-05-17T09:15:48.525966+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:15:48.525966+0000","last_undegraded":"2024-05-17T09:15:48.525966+0000","last_fullsized":"2024-05-17T09:15:48.525966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147157+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:15:48.147157+0000","last_peered":"2024-05-17T09:15:48.147157+0000","last_clean":"2024-05-17T09:15:48.147157+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:15:48.147157+0000","last_undegraded":"2024-05-17T09:15:48.147157+0000","last_fullsized":"2024-05-17T09:15:48.147157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.159911+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:15:48.159911+0000","last_peered":"2024-05-17T09:15:48.159911+0000","last_clean":"2024-05-17T09:15:48.159911+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:15:48.159911+0000","last_undegraded":"2024-05-17T09:15:48.159911+0000","last_fullsized":"2024-05-17T09:15:48.159911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401695+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:15:48.401695+0000","last_peered":"2024-05-17T09:15:48.401695+0000","last_clean":"2024-05-17T09:15:48.401695+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:15:48.401695+0000","last_undegraded":"2024-05-17T09:15:48.401695+0000","last_fullsized":"2024-05-17T09:15:48.401695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153652+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:15:48.153652+0000","last_peered":"2024-05-17T09:15:48.153652+0000","last_clean":"2024-05-17T09:15:48.153652+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:15:48.153652+0000","last_undegraded":"2024-05-17T09:15:48.153652+0000","last_fullsized":"2024-05-17T09:15:48.153652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.308540+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:15:48.308540+0000","last_peered":"2024-05-17T09:15:48.308540+0000","last_clean":"2024-05-17T09:15:48.308540+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:15:48.308540+0000","last_undegraded":"2024-05-17T09:15:48.308540+0000","last_fullsized":"2024-05-17T09:15:48.308540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.158729+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:15:48.158729+0000","last_peered":"2024-05-17T09:15:48.158729+0000","last_clean":"2024-05-17T09:15:48.158729+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:15:48.158729+0000","last_undegraded":"2024-05-17T09:15:48.158729+0000","last_fullsized":"2024-05-17T09:15:48.158729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.157893+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:15:48.157893+0000","last_peered":"2024-05-17T09:15:48.157893+0000","last_clean":"2024-05-17T09:15:48.157893+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:15:48.157893+0000","last_undegraded":"2024-05-17T09:15:48.157893+0000","last_fullsized":"2024-05-17T09:15:48.157893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.221934+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:17.221934+0000","last_peered":"2024-05-17T09:16:17.221934+0000","last_clean":"2024-05-17T09:16:17.221934+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:17.221934+0000","last_undegraded":"2024-05-17T09:16:17.221934+0000","last_fullsized":"2024-05-17T09:16:17.221934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154170+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:15:48.154170+0000","last_peered":"2024-05-17T09:15:48.154170+0000","last_clean":"2024-05-17T09:15:48.154170+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:15:48.154170+0000","last_undegraded":"2024-05-17T09:15:48.154170+0000","last_fullsized":"2024-05-17T09:15:48.154170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147916+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:15:48.147916+0000","last_peered":"2024-05-17T09:15:48.147916+0000","last_clean":"2024-05-17T09:15:48.147916+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:15:48.147916+0000","last_undegraded":"2024-05-17T09:15:48.147916+0000","last_fullsized":"2024-05-17T09:15:48.147916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154696+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:15:48.154696+0000","last_peered":"2024-05-17T09:15:48.154696+0000","last_clean":"2024-05-17T09:15:48.154696+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:15:48.154696+0000","last_undegraded":"2024-05-17T09:15:48.154696+0000","last_fullsized":"2024-05-17T09:15:48.154696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147606+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:15:48.147606+0000","last_peered":"2024-05-17T09:15:48.147606+0000","last_clean":"2024-05-17T09:15:48.147606+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:15:48.147606+0000","last_undegraded":"2024-05-17T09:15:48.147606+0000","last_fullsized":"2024-05-17T09:15:48.147606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148057+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:15:48.148057+0000","last_peered":"2024-05-17T09:15:48.148057+0000","last_clean":"2024-05-17T09:15:48.148057+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:15:48.148057+0000","last_undegraded":"2024-05-17T09:15:48.148057+0000","last_fullsized":"2024-05-17T09:15:48.148057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.219306+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:17.219306+0000","last_peered":"2024-05-17T09:16:17.219306+0000","last_clean":"2024-05-17T09:16:17.219306+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:17.219306+0000","last_undegraded":"2024-05-17T09:16:17.219306+0000","last_fullsized":"2024-05-17T09:16:17.219306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401535+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:15:48.401535+0000","last_peered":"2024-05-17T09:15:48.401535+0000","last_clean":"2024-05-17T09:15:48.401535+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:15:48.401535+0000","last_undegraded":"2024-05-17T09:15:48.401535+0000","last_fullsized":"2024-05-17T09:15:48.401535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147985+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:15:48.147985+0000","last_peered":"2024-05-17T09:15:48.147985+0000","last_clean":"2024-05-17T09:15:48.147985+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:15:48.147985+0000","last_undegraded":"2024-05-17T09:15:48.147985+0000","last_fullsized":"2024-05-17T09:15:48.147985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149354+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:15:48.149354+0000","last_peered":"2024-05-17T09:15:48.149354+0000","last_clean":"2024-05-17T09:15:48.149354+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:15:48.149354+0000","last_undegraded":"2024-05-17T09:15:48.149354+0000","last_fullsized":"2024-05-17T09:15:48.149354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152793+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:15:48.152793+0000","last_peered":"2024-05-17T09:15:48.152793+0000","last_clean":"2024-05-17T09:15:48.152793+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:15:48.152793+0000","last_undegraded":"2024-05-17T09:15:48.152793+0000","last_fullsized":"2024-05-17T09:15:48.152793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.158512+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:15:48.158512+0000","last_peered":"2024-05-17T09:15:48.158512+0000","last_clean":"2024-05-17T09:15:48.158512+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:15:48.158512+0000","last_undegraded":"2024-05-17T09:15:48.158512+0000","last_fullsized":"2024-05-17T09:15:48.158512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:45.271034+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:16:45.271034+0000","last_peered":"2024-05-17T09:16:45.271034+0000","last_clean":"2024-05-17T09:16:45.271034+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:16:45.271034+0000","last_undegraded":"2024-05-17T09:16:45.271034+0000","last_fullsized":"2024-05-17T09:16:45.271034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149996+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:15:48.149996+0000","last_peered":"2024-05-17T09:15:48.149996+0000","last_clean":"2024-05-17T09:15:48.149996+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:15:48.149996+0000","last_undegraded":"2024-05-17T09:15:48.149996+0000","last_fullsized":"2024-05-17T09:15:48.149996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.156699+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:15:48.156699+0000","last_peered":"2024-05-17T09:15:48.156699+0000","last_clean":"2024-05-17T09:15:48.156699+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:15:48.156699+0000","last_undegraded":"2024-05-17T09:15:48.156699+0000","last_fullsized":"2024-05-17T09:15:48.156699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.220933+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:17.220933+0000","last_peered":"2024-05-17T09:16:17.220933+0000","last_clean":"2024-05-17T09:16:17.220933+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:17.220933+0000","last_undegraded":"2024-05-17T09:16:17.220933+0000","last_fullsized":"2024-05-17T09:16:17.220933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.527228+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:15:48.527228+0000","last_peered":"2024-05-17T09:15:48.527228+0000","last_clean":"2024-05-17T09:15:48.527228+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:15:48.527228+0000","last_undegraded":"2024-05-17T09:15:48.527228+0000","last_fullsized":"2024-05-17T09:15:48.527228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.160379+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:15:48.160379+0000","last_peered":"2024-05-17T09:15:48.160379+0000","last_clean":"2024-05-17T09:15:48.160379+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:15:48.160379+0000","last_undegraded":"2024-05-17T09:15:48.160379+0000","last_fullsized":"2024-05-17T09:15:48.160379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.220990+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:17.220990+0000","last_peered":"2024-05-17T09:16:17.220990+0000","last_clean":"2024-05-17T09:16:17.220990+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:17.220990+0000","last_undegraded":"2024-05-17T09:16:17.220990+0000","last_fullsized":"2024-05-17T09:16:17.220990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153804+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:15:48.153804+0000","last_peered":"2024-05-17T09:15:48.153804+0000","last_clean":"2024-05-17T09:15:48.153804+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:15:48.153804+0000","last_undegraded":"2024-05-17T09:15:48.153804+0000","last_fullsized":"2024-05-17T09:15:48.153804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147390+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:15:48.147390+0000","last_peered":"2024-05-17T09:15:48.147390+0000","last_clean":"2024-05-17T09:15:48.147390+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:15:48.147390+0000","last_undegraded":"2024-05-17T09:15:48.147390+0000","last_fullsized":"2024-05-17T09:15:48.147390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152523+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:15:48.152523+0000","last_peered":"2024-05-17T09:15:48.152523+0000","last_clean":"2024-05-17T09:15:48.152523+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:15:48.152523+0000","last_undegraded":"2024-05-17T09:15:48.152523+0000","last_fullsized":"2024-05-17T09:15:48.152523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.307989+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:15:48.307989+0000","last_peered":"2024-05-17T09:15:48.307989+0000","last_clean":"2024-05-17T09:15:48.307989+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:15:48.307989+0000","last_undegraded":"2024-05-17T09:15:48.307989+0000","last_fullsized":"2024-05-17T09:15:48.307989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153742+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:15:48.153742+0000","last_peered":"2024-05-17T09:15:48.153742+0000","last_clean":"2024-05-17T09:15:48.153742+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:15:48.153742+0000","last_undegraded":"2024-05-17T09:15:48.153742+0000","last_fullsized":"2024-05-17T09:15:48.153742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:42.265989+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:16:42.265989+0000","last_peered":"2024-05-17T09:16:42.265989+0000","last_clean":"2024-05-17T09:16:42.265989+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:16:42.265989+0000","last_undegraded":"2024-05-17T09:16:42.265989+0000","last_fullsized":"2024-05-17T09:16:42.265989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.402150+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:15:48.402150+0000","last_peered":"2024-05-17T09:15:48.402150+0000","last_clean":"2024-05-17T09:15:48.402150+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:15:48.402150+0000","last_undegraded":"2024-05-17T09:15:48.402150+0000","last_fullsized":"2024-05-17T09:15:48.402150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.307837+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:15:48.307837+0000","last_peered":"2024-05-17T09:15:48.307837+0000","last_clean":"2024-05-17T09:15:48.307837+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:15:48.307837+0000","last_undegraded":"2024-05-17T09:15:48.307837+0000","last_fullsized":"2024-05-17T09:15:48.307837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.206882+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:16.206882+0000","last_peered":"2024-05-17T09:16:16.206882+0000","last_clean":"2024-05-17T09:16:16.206882+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:16.206882+0000","last_undegraded":"2024-05-17T09:16:16.206882+0000","last_fullsized":"2024-05-17T09:16:16.206882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147464+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:15:48.147464+0000","last_peered":"2024-05-17T09:15:48.147464+0000","last_clean":"2024-05-17T09:15:48.147464+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:15:48.147464+0000","last_undegraded":"2024-05-17T09:15:48.147464+0000","last_fullsized":"2024-05-17T09:15:48.147464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148524+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:15:48.148524+0000","last_peered":"2024-05-17T09:15:48.148524+0000","last_clean":"2024-05-17T09:15:48.148524+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:15:48.148524+0000","last_undegraded":"2024-05-17T09:15:48.148524+0000","last_fullsized":"2024-05-17T09:15:48.148524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.205842+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:16.205842+0000","last_peered":"2024-05-17T09:16:16.205842+0000","last_clean":"2024-05-17T09:16:16.205842+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:16.205842+0000","last_undegraded":"2024-05-17T09:16:16.205842+0000","last_fullsized":"2024-05-17T09:16:16.205842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401613+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:15:48.401613+0000","last_peered":"2024-05-17T09:15:48.401613+0000","last_clean":"2024-05-17T09:15:48.401613+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:15:48.401613+0000","last_undegraded":"2024-05-17T09:15:48.401613+0000","last_fullsized":"2024-05-17T09:15:48.401613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.159007+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:15:48.159007+0000","last_peered":"2024-05-17T09:15:48.159007+0000","last_clean":"2024-05-17T09:15:48.159007+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:15:48.159007+0000","last_undegraded":"2024-05-17T09:15:48.159007+0000","last_fullsized":"2024-05-17T09:15:48.159007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":129,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:43.268352+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:16:43.268352+0000","last_peered":"2024-05-17T09:16:43.268352+0000","last_clean":"2024-05-17T09:16:43.268352+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:16:43.268352+0000","last_undegraded":"2024-05-17T09:16:43.268352+0000","last_fullsized":"2024-05-17T09:16:43.268352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57883,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":85,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149548+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:15:48.149548+0000","last_peered":"2024-05-17T09:15:48.149548+0000","last_clean":"2024-05-17T09:15:48.149548+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:15:48.149548+0000","last_undegraded":"2024-05-17T09:15:48.149548+0000","last_fullsized":"2024-05-17T09:15:48.149548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153247+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:15:48.153247+0000","last_peered":"2024-05-17T09:15:48.153247+0000","last_clean":"2024-05-17T09:15:48.153247+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:15:48.153247+0000","last_undegraded":"2024-05-17T09:15:48.153247+0000","last_fullsized":"2024-05-17T09:15:48.153247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.220032+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:17.220032+0000","last_peered":"2024-05-17T09:16:17.220032+0000","last_clean":"2024-05-17T09:16:17.220032+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:17.220032+0000","last_undegraded":"2024-05-17T09:16:17.220032+0000","last_fullsized":"2024-05-17T09:16:17.220032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526199+0000","last_change":"2024-05-17T09:15:07.962444+0000","last_active":"2024-05-17T09:15:48.526199+0000","last_peered":"2024-05-17T09:15:48.526199+0000","last_clean":"2024-05-17T09:15:48.526199+0000","last_became_active":"2024-05-17T09:15:07.962064+0000","last_became_peered":"2024-05-17T09:15:07.962064+0000","last_unstale":"2024-05-17T09:15:48.526199+0000","last_undegraded":"2024-05-17T09:15:48.526199+0000","last_fullsized":"2024-05-17T09:15:48.526199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:39:59.922322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151162+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:15:48.151162+0000","last_peered":"2024-05-17T09:15:48.151162+0000","last_clean":"2024-05-17T09:15:48.151162+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:15:48.151162+0000","last_undegraded":"2024-05-17T09:15:48.151162+0000","last_fullsized":"2024-05-17T09:15:48.151162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.207037+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:16.207037+0000","last_peered":"2024-05-17T09:16:16.207037+0000","last_clean":"2024-05-17T09:16:16.207037+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:16.207037+0000","last_undegraded":"2024-05-17T09:16:16.207037+0000","last_fullsized":"2024-05-17T09:16:16.207037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150672+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:15:48.150672+0000","last_peered":"2024-05-17T09:15:48.150672+0000","last_clean":"2024-05-17T09:15:48.150672+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:15:48.150672+0000","last_undegraded":"2024-05-17T09:15:48.150672+0000","last_fullsized":"2024-05-17T09:15:48.150672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.160250+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:15:48.160250+0000","last_peered":"2024-05-17T09:15:48.160250+0000","last_clean":"2024-05-17T09:15:48.160250+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:15:48.160250+0000","last_undegraded":"2024-05-17T09:15:48.160250+0000","last_fullsized":"2024-05-17T09:15:48.160250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:45.270604+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:16:45.270604+0000","last_peered":"2024-05-17T09:16:45.270604+0000","last_clean":"2024-05-17T09:16:45.270604+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:16:45.270604+0000","last_undegraded":"2024-05-17T09:16:45.270604+0000","last_fullsized":"2024-05-17T09:16:45.270604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151534+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:15:48.151534+0000","last_peered":"2024-05-17T09:15:48.151534+0000","last_clean":"2024-05-17T09:15:48.151534+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:15:48.151534+0000","last_undegraded":"2024-05-17T09:15:48.151534+0000","last_fullsized":"2024-05-17T09:15:48.151534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526554+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:15:48.526554+0000","last_peered":"2024-05-17T09:15:48.526554+0000","last_clean":"2024-05-17T09:15:48.526554+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:15:48.526554+0000","last_undegraded":"2024-05-17T09:15:48.526554+0000","last_fullsized":"2024-05-17T09:15:48.526554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.221352+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:17.221352+0000","last_peered":"2024-05-17T09:16:17.221352+0000","last_clean":"2024-05-17T09:16:17.221352+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:17.221352+0000","last_undegraded":"2024-05-17T09:16:17.221352+0000","last_fullsized":"2024-05-17T09:16:17.221352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152657+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:15:48.152657+0000","last_peered":"2024-05-17T09:15:48.152657+0000","last_clean":"2024-05-17T09:15:48.152657+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:15:48.152657+0000","last_undegraded":"2024-05-17T09:15:48.152657+0000","last_fullsized":"2024-05-17T09:15:48.152657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151954+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:15:48.151954+0000","last_peered":"2024-05-17T09:15:48.151954+0000","last_clean":"2024-05-17T09:15:48.151954+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:15:48.151954+0000","last_undegraded":"2024-05-17T09:15:48.151954+0000","last_fullsized":"2024-05-17T09:15:48.151954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:18.214956+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:18.214956+0000","last_peered":"2024-05-17T09:16:18.214956+0000","last_clean":"2024-05-17T09:16:18.214956+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:18.214956+0000","last_undegraded":"2024-05-17T09:16:18.214956+0000","last_fullsized":"2024-05-17T09:16:18.214956+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152172+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:15:48.152172+0000","last_peered":"2024-05-17T09:15:48.152172+0000","last_clean":"2024-05-17T09:15:48.152172+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:15:48.152172+0000","last_undegraded":"2024-05-17T09:15:48.152172+0000","last_fullsized":"2024-05-17T09:15:48.152172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149303+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:15:48.149303+0000","last_peered":"2024-05-17T09:15:48.149303+0000","last_clean":"2024-05-17T09:15:48.149303+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:15:48.149303+0000","last_undegraded":"2024-05-17T09:15:48.149303+0000","last_fullsized":"2024-05-17T09:15:48.149303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.308206+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:15:48.308206+0000","last_peered":"2024-05-17T09:15:48.308206+0000","last_clean":"2024-05-17T09:15:48.308206+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:15:48.308206+0000","last_undegraded":"2024-05-17T09:15:48.308206+0000","last_fullsized":"2024-05-17T09:15:48.308206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153529+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:15:48.153529+0000","last_peered":"2024-05-17T09:15:48.153529+0000","last_clean":"2024-05-17T09:15:48.153529+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:15:48.153529+0000","last_undegraded":"2024-05-17T09:15:48.153529+0000","last_fullsized":"2024-05-17T09:15:48.153529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526078+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:15:48.526078+0000","last_peered":"2024-05-17T09:15:48.526078+0000","last_clean":"2024-05-17T09:15:48.526078+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:15:48.526078+0000","last_undegraded":"2024-05-17T09:15:48.526078+0000","last_fullsized":"2024-05-17T09:15:48.526078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151630+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:15:48.151630+0000","last_peered":"2024-05-17T09:15:48.151630+0000","last_clean":"2024-05-17T09:15:48.151630+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:15:48.151630+0000","last_undegraded":"2024-05-17T09:15:48.151630+0000","last_fullsized":"2024-05-17T09:15:48.151630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153159+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:15:48.153159+0000","last_peered":"2024-05-17T09:15:48.153159+0000","last_clean":"2024-05-17T09:15:48.153159+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:15:48.153159+0000","last_undegraded":"2024-05-17T09:15:48.153159+0000","last_fullsized":"2024-05-17T09:15:48.153159+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526518+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:15:48.526518+0000","last_peered":"2024-05-17T09:15:48.526518+0000","last_clean":"2024-05-17T09:15:48.526518+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:15:48.526518+0000","last_undegraded":"2024-05-17T09:15:48.526518+0000","last_fullsized":"2024-05-17T09:15:48.526518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152829+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:15:48.152829+0000","last_peered":"2024-05-17T09:15:48.152829+0000","last_clean":"2024-05-17T09:15:48.152829+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:15:48.152829+0000","last_undegraded":"2024-05-17T09:15:48.152829+0000","last_fullsized":"2024-05-17T09:15:48.152829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:41.263792+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:16:41.263792+0000","last_peered":"2024-05-17T09:16:41.263792+0000","last_clean":"2024-05-17T09:16:41.263792+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:16:41.263792+0000","last_undegraded":"2024-05-17T09:16:41.263792+0000","last_fullsized":"2024-05-17T09:16:41.263792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.310131+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:15:48.310131+0000","last_peered":"2024-05-17T09:15:48.310131+0000","last_clean":"2024-05-17T09:15:48.310131+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:15:48.310131+0000","last_undegraded":"2024-05-17T09:15:48.310131+0000","last_fullsized":"2024-05-17T09:15:48.310131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154656+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:15:48.154656+0000","last_peered":"2024-05-17T09:15:48.154656+0000","last_clean":"2024-05-17T09:15:48.154656+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:15:48.154656+0000","last_undegraded":"2024-05-17T09:15:48.154656+0000","last_fullsized":"2024-05-17T09:15:48.154656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:19.227245+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:19.227245+0000","last_peered":"2024-05-17T09:16:19.227245+0000","last_clean":"2024-05-17T09:16:19.227245+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:19.227245+0000","last_undegraded":"2024-05-17T09:16:19.227245+0000","last_fullsized":"2024-05-17T09:16:19.227245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151629+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:15:48.151629+0000","last_peered":"2024-05-17T09:15:48.151629+0000","last_clean":"2024-05-17T09:15:48.151629+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:15:48.151629+0000","last_undegraded":"2024-05-17T09:15:48.151629+0000","last_fullsized":"2024-05-17T09:15:48.151629+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153293+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:15:48.153293+0000","last_peered":"2024-05-17T09:15:48.153293+0000","last_clean":"2024-05-17T09:15:48.153293+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:15:48.153293+0000","last_undegraded":"2024-05-17T09:15:48.153293+0000","last_fullsized":"2024-05-17T09:15:48.153293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.525741+0000","last_change":"2024-05-17T09:15:07.966554+0000","last_active":"2024-05-17T09:15:48.525741+0000","last_peered":"2024-05-17T09:15:48.525741+0000","last_clean":"2024-05-17T09:15:48.525741+0000","last_became_active":"2024-05-17T09:15:07.966039+0000","last_became_peered":"2024-05-17T09:15:07.966039+0000","last_unstale":"2024-05-17T09:15:48.525741+0000","last_undegraded":"2024-05-17T09:15:48.525741+0000","last_fullsized":"2024-05-17T09:15:48.525741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:50:59.186435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.219060+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:17.219060+0000","last_peered":"2024-05-17T09:16:17.219060+0000","last_clean":"2024-05-17T09:16:17.219060+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:17.219060+0000","last_undegraded":"2024-05-17T09:16:17.219060+0000","last_fullsized":"2024-05-17T09:16:17.219060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151152+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:15:48.151152+0000","last_peered":"2024-05-17T09:15:48.151152+0000","last_clean":"2024-05-17T09:15:48.151152+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:15:48.151152+0000","last_undegraded":"2024-05-17T09:15:48.151152+0000","last_fullsized":"2024-05-17T09:15:48.151152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.204261+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:16.204261+0000","last_peered":"2024-05-17T09:16:16.204261+0000","last_clean":"2024-05-17T09:16:16.204261+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:16.204261+0000","last_undegraded":"2024-05-17T09:16:16.204261+0000","last_fullsized":"2024-05-17T09:16:16.204261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526268+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:15:48.526268+0000","last_peered":"2024-05-17T09:15:48.526268+0000","last_clean":"2024-05-17T09:15:48.526268+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:15:48.526268+0000","last_undegraded":"2024-05-17T09:15:48.526268+0000","last_fullsized":"2024-05-17T09:15:48.526268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153756+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:15:48.153756+0000","last_peered":"2024-05-17T09:15:48.153756+0000","last_clean":"2024-05-17T09:15:48.153756+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:15:48.153756+0000","last_undegraded":"2024-05-17T09:15:48.153756+0000","last_fullsized":"2024-05-17T09:15:48.153756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:37.977805+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:16:37.977805+0000","last_peered":"2024-05-17T09:16:37.977805+0000","last_clean":"2024-05-17T09:16:37.977805+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:16:37.977805+0000","last_undegraded":"2024-05-17T09:16:37.977805+0000","last_fullsized":"2024-05-17T09:16:37.977805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154873+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:15:48.154873+0000","last_peered":"2024-05-17T09:15:48.154873+0000","last_clean":"2024-05-17T09:15:48.154873+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:15:48.154873+0000","last_undegraded":"2024-05-17T09:15:48.154873+0000","last_fullsized":"2024-05-17T09:15:48.154873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.157756+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:15:48.157756+0000","last_peered":"2024-05-17T09:15:48.157756+0000","last_clean":"2024-05-17T09:15:48.157756+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:15:48.157756+0000","last_undegraded":"2024-05-17T09:15:48.157756+0000","last_fullsized":"2024-05-17T09:15:48.157756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.206578+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:16.206578+0000","last_peered":"2024-05-17T09:16:16.206578+0000","last_clean":"2024-05-17T09:16:16.206578+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:16.206578+0000","last_undegraded":"2024-05-17T09:16:16.206578+0000","last_fullsized":"2024-05-17T09:16:16.206578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147637+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:15:48.147637+0000","last_peered":"2024-05-17T09:15:48.147637+0000","last_clean":"2024-05-17T09:15:48.147637+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:15:48.147637+0000","last_undegraded":"2024-05-17T09:15:48.147637+0000","last_fullsized":"2024-05-17T09:15:48.147637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148340+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:15:48.148340+0000","last_peered":"2024-05-17T09:15:48.148340+0000","last_clean":"2024-05-17T09:15:48.148340+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:15:48.148340+0000","last_undegraded":"2024-05-17T09:15:48.148340+0000","last_fullsized":"2024-05-17T09:15:48.148340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.207401+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:16.207401+0000","last_peered":"2024-05-17T09:16:16.207401+0000","last_clean":"2024-05-17T09:16:16.207401+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:16.207401+0000","last_undegraded":"2024-05-17T09:16:16.207401+0000","last_fullsized":"2024-05-17T09:16:16.207401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149196+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:15:48.149196+0000","last_peered":"2024-05-17T09:15:48.149196+0000","last_clean":"2024-05-17T09:15:48.149196+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:15:48.149196+0000","last_undegraded":"2024-05-17T09:15:48.149196+0000","last_fullsized":"2024-05-17T09:15:48.149196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.308433+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:15:48.308433+0000","last_peered":"2024-05-17T09:15:48.308433+0000","last_clean":"2024-05-17T09:15:48.308433+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:15:48.308433+0000","last_undegraded":"2024-05-17T09:15:48.308433+0000","last_fullsized":"2024-05-17T09:15:48.308433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.206437+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:16.206437+0000","last_peered":"2024-05-17T09:16:16.206437+0000","last_clean":"2024-05-17T09:16:16.206437+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:16.206437+0000","last_undegraded":"2024-05-17T09:16:16.206437+0000","last_fullsized":"2024-05-17T09:16:16.206437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150336+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:15:48.150336+0000","last_peered":"2024-05-17T09:15:48.150336+0000","last_clean":"2024-05-17T09:15:48.150336+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:15:48.150336+0000","last_undegraded":"2024-05-17T09:15:48.150336+0000","last_fullsized":"2024-05-17T09:15:48.150336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148214+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:15:48.148214+0000","last_peered":"2024-05-17T09:15:48.148214+0000","last_clean":"2024-05-17T09:15:48.148214+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:15:48.148214+0000","last_undegraded":"2024-05-17T09:15:48.148214+0000","last_fullsized":"2024-05-17T09:15:48.148214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.156549+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:15:48.156549+0000","last_peered":"2024-05-17T09:15:48.156549+0000","last_clean":"2024-05-17T09:15:48.156549+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:15:48.156549+0000","last_undegraded":"2024-05-17T09:15:48.156549+0000","last_fullsized":"2024-05-17T09:15:48.156549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154146+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:15:48.154146+0000","last_peered":"2024-05-17T09:15:48.154146+0000","last_clean":"2024-05-17T09:15:48.154146+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:15:48.154146+0000","last_undegraded":"2024-05-17T09:15:48.154146+0000","last_fullsized":"2024-05-17T09:15:48.154146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148297+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:15:48.148297+0000","last_peered":"2024-05-17T09:15:48.148297+0000","last_clean":"2024-05-17T09:15:48.148297+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:15:48.148297+0000","last_undegraded":"2024-05-17T09:15:48.148297+0000","last_fullsized":"2024-05-17T09:15:48.148297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.205311+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:16.205311+0000","last_peered":"2024-05-17T09:16:16.205311+0000","last_clean":"2024-05-17T09:16:16.205311+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:16.205311+0000","last_undegraded":"2024-05-17T09:16:16.205311+0000","last_fullsized":"2024-05-17T09:16:16.205311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151767+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:15:48.151767+0000","last_peered":"2024-05-17T09:15:48.151767+0000","last_clean":"2024-05-17T09:15:48.151767+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:15:48.151767+0000","last_undegraded":"2024-05-17T09:15:48.151767+0000","last_fullsized":"2024-05-17T09:15:48.151767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401434+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:15:48.401434+0000","last_peered":"2024-05-17T09:15:48.401434+0000","last_clean":"2024-05-17T09:15:48.401434+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:15:48.401434+0000","last_undegraded":"2024-05-17T09:15:48.401434+0000","last_fullsized":"2024-05-17T09:15:48.401434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.220941+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:17.220941+0000","last_peered":"2024-05-17T09:16:17.220941+0000","last_clean":"2024-05-17T09:16:17.220941+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:17.220941+0000","last_undegraded":"2024-05-17T09:16:17.220941+0000","last_fullsized":"2024-05-17T09:16:17.220941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152691+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:15:48.152691+0000","last_peered":"2024-05-17T09:15:48.152691+0000","last_clean":"2024-05-17T09:15:48.152691+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:15:48.152691+0000","last_undegraded":"2024-05-17T09:15:48.152691+0000","last_fullsized":"2024-05-17T09:15:48.152691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147185+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:15:48.147185+0000","last_peered":"2024-05-17T09:15:48.147185+0000","last_clean":"2024-05-17T09:15:48.147185+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:15:48.147185+0000","last_undegraded":"2024-05-17T09:15:48.147185+0000","last_fullsized":"2024-05-17T09:15:48.147185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152959+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:15:48.152959+0000","last_peered":"2024-05-17T09:15:48.152959+0000","last_clean":"2024-05-17T09:15:48.152959+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:15:48.152959+0000","last_undegraded":"2024-05-17T09:15:48.152959+0000","last_fullsized":"2024-05-17T09:15:48.152959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150754+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:15:48.150754+0000","last_peered":"2024-05-17T09:15:48.150754+0000","last_clean":"2024-05-17T09:15:48.150754+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:15:48.150754+0000","last_undegraded":"2024-05-17T09:15:48.150754+0000","last_fullsized":"2024-05-17T09:15:48.150754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150682+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:15:48.150682+0000","last_peered":"2024-05-17T09:15:48.150682+0000","last_clean":"2024-05-17T09:15:48.150682+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:15:48.150682+0000","last_undegraded":"2024-05-17T09:15:48.150682+0000","last_fullsized":"2024-05-17T09:15:48.150682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526542+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:15:48.526542+0000","last_peered":"2024-05-17T09:15:48.526542+0000","last_clean":"2024-05-17T09:15:48.526542+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:15:48.526542+0000","last_undegraded":"2024-05-17T09:15:48.526542+0000","last_fullsized":"2024-05-17T09:15:48.526542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152488+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:15:48.152488+0000","last_peered":"2024-05-17T09:15:48.152488+0000","last_clean":"2024-05-17T09:15:48.152488+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:15:48.152488+0000","last_undegraded":"2024-05-17T09:15:48.152488+0000","last_fullsized":"2024-05-17T09:15:48.152488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150478+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:15:48.150478+0000","last_peered":"2024-05-17T09:15:48.150478+0000","last_clean":"2024-05-17T09:15:48.150478+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:15:48.150478+0000","last_undegraded":"2024-05-17T09:15:48.150478+0000","last_fullsized":"2024-05-17T09:15:48.150478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.158198+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:15:48.158198+0000","last_peered":"2024-05-17T09:15:48.158198+0000","last_clean":"2024-05-17T09:15:48.158198+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:15:48.158198+0000","last_undegraded":"2024-05-17T09:15:48.158198+0000","last_fullsized":"2024-05-17T09:15:48.158198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.156298+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:15:48.156298+0000","last_peered":"2024-05-17T09:15:48.156298+0000","last_clean":"2024-05-17T09:15:48.156298+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:15:48.156298+0000","last_undegraded":"2024-05-17T09:15:48.156298+0000","last_fullsized":"2024-05-17T09:15:48.156298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147682+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:15:48.147682+0000","last_peered":"2024-05-17T09:15:48.147682+0000","last_clean":"2024-05-17T09:15:48.147682+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:15:48.147682+0000","last_undegraded":"2024-05-17T09:15:48.147682+0000","last_fullsized":"2024-05-17T09:15:48.147682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526391+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:15:48.526391+0000","last_peered":"2024-05-17T09:15:48.526391+0000","last_clean":"2024-05-17T09:15:48.526391+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:15:48.526391+0000","last_undegraded":"2024-05-17T09:15:48.526391+0000","last_fullsized":"2024-05-17T09:15:48.526391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151950+0000","last_change":"2024-05-17T09:15:07.959003+0000","last_active":"2024-05-17T09:15:48.151950+0000","last_peered":"2024-05-17T09:15:48.151950+0000","last_clean":"2024-05-17T09:15:48.151950+0000","last_became_active":"2024-05-17T09:15:07.958653+0000","last_became_peered":"2024-05-17T09:15:07.958653+0000","last_unstale":"2024-05-17T09:15:48.151950+0000","last_undegraded":"2024-05-17T09:15:48.151950+0000","last_fullsized":"2024-05-17T09:15:48.151950+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:40:55.013374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.157394+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:15:48.157394+0000","last_peered":"2024-05-17T09:15:48.157394+0000","last_clean":"2024-05-17T09:15:48.157394+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:15:48.157394+0000","last_undegraded":"2024-05-17T09:15:48.157394+0000","last_fullsized":"2024-05-17T09:15:48.157394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.155006+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:15:48.155006+0000","last_peered":"2024-05-17T09:15:48.155006+0000","last_clean":"2024-05-17T09:15:48.155006+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:15:48.155006+0000","last_undegraded":"2024-05-17T09:15:48.155006+0000","last_fullsized":"2024-05-17T09:15:48.155006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152553+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:15:48.152553+0000","last_peered":"2024-05-17T09:15:48.152553+0000","last_clean":"2024-05-17T09:15:48.152553+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:15:48.152553+0000","last_undegraded":"2024-05-17T09:15:48.152553+0000","last_fullsized":"2024-05-17T09:15:48.152553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153980+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:15:48.153980+0000","last_peered":"2024-05-17T09:15:48.153980+0000","last_clean":"2024-05-17T09:15:48.153980+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:15:48.153980+0000","last_undegraded":"2024-05-17T09:15:48.153980+0000","last_fullsized":"2024-05-17T09:15:48.153980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401677+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:15:48.401677+0000","last_peered":"2024-05-17T09:15:48.401677+0000","last_clean":"2024-05-17T09:15:48.401677+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:15:48.401677+0000","last_undegraded":"2024-05-17T09:15:48.401677+0000","last_fullsized":"2024-05-17T09:15:48.401677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.526085+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:15:48.526085+0000","last_peered":"2024-05-17T09:15:48.526085+0000","last_clean":"2024-05-17T09:15:48.526085+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:15:48.526085+0000","last_undegraded":"2024-05-17T09:15:48.526085+0000","last_fullsized":"2024-05-17T09:15:48.526085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152398+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:15:48.152398+0000","last_peered":"2024-05-17T09:15:48.152398+0000","last_clean":"2024-05-17T09:15:48.152398+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:15:48.152398+0000","last_undegraded":"2024-05-17T09:15:48.152398+0000","last_fullsized":"2024-05-17T09:15:48.152398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62275,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":183,"num_read_kb":154,"num_write":164,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":185687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":496288,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906983,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1144732,"data_compressed":62182,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070509,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":500564,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266738,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":496366,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1092688,"data_compressed":62182,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":491644,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":548299,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":493032,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":551144,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":491838,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":496685,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1083789,"data_compressed":62182,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:16:47.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:16:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:48.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:48 smithi019 ceph-mon[31620]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-17T09:16:48.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:48 smithi019 ceph-mon[31620]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:48.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-17T09:16:48.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-05-17T09:16:48.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-17T09:16:48.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:48 smithi019 ceph-mon[31620]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T09:16:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:48 smithi069 ceph-mon[35081]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-17T09:16:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:48 smithi069 ceph-mon[35081]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-17T09:16:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-05-17T09:16:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-17T09:16:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:48 smithi069 ceph-mon[35081]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T09:16:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:48 smithi079 ceph-mon[35502]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-17T09:16:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:48 smithi079 ceph-mon[35502]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-17T09:16:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-05-17T09:16:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-17T09:16:48.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:48 smithi079 ceph-mon[35502]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T09:16:49.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:49 smithi019 ceph-mon[31620]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T09:16:49.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:49 smithi069 ceph-mon[35081]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T09:16:49.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:49 smithi079 ceph-mon[35502]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T09:16:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 -- 172.21.15.19:0/930799632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad41034b0 msgr2=0x7f9ad41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 --2- 172.21.15.19:0/930799632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad41034b0 0x7f9ad41038c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40099c0 tx=0x7f9ac4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 -- 172.21.15.19:0/930799632 shutdown_connections 2024-05-17T09:16:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 --2- 172.21.15.19:0/930799632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad4108730 0x7f9ad4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 --2- 172.21.15.19:0/930799632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad41034b0 0x7f9ad41038c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 --2- 172.21.15.19:0/930799632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ad4107de0 0x7f9ad41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 -- 172.21.15.19:0/930799632 >> 172.21.15.19:0/930799632 conn(0x7f9ad40fedb0 msgr2=0x7f9ad41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 -- 172.21.15.19:0/930799632 shutdown_connections 2024-05-17T09:16:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 -- 172.21.15.19:0/930799632 wait complete. 2024-05-17T09:16:50.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 Processor -- start 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.059+0000 7f9adae18700 1 -- start start 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9adae18700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad41034b0 0x7f9ad40ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9adae18700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ad4107de0 0x7f9ad40ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9adae18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 0x7f9ad40ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9adae18700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4113300 con 0x7f9ad4108730 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9adae18700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4113180 con 0x7f9ad41034b0 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9adae18700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4113480 con 0x7f9ad4107de0 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 0x7f9ad40ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 0x7f9ad40ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43850/0 (socket says 172.21.15.19:43850) 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 -- 172.21.15.19:0/2845776013 learned_addr learned my addr 172.21.15.19:0/2845776013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad8bb4700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad41034b0 0x7f9ad40ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad3fff700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ad4107de0 0x7f9ad40ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 -- 172.21.15.19:0/2845776013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ad4107de0 msgr2=0x7f9ad40ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ad4107de0 0x7f9ad40ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 -- 172.21.15.19:0/2845776013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad41034b0 msgr2=0x7f9ad40ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad41034b0 0x7f9ad40ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.060+0000 7f9ad93b5700 1 -- 172.21.15.19:0/2845776013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad41a9180 con 0x7f9ad4108730 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.061+0000 7f9ad93b5700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 0x7f9ad40ffd10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00b970 tx=0x7f9acc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.061+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00d860 con 0x7f9ad4108730 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.061+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9acc00e430 con 0x7f9ad4108730 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.061+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad41a9470 con 0x7f9ad4108730 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.061+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc017660 con 0x7f9ad4108730 2024-05-17T09:16:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.061+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad41a9960 con 0x7f9ad4108730 2024-05-17T09:16:50.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.062+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad40512b0 con 0x7f9ad4108730 2024-05-17T09:16:50.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.063+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9acc00da00 con 0x7f9ad4108730 2024-05-17T09:16:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.064+0000 7f9ad1ffb700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ac00b2910 0x7f9ac00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.064+0000 7f9ad8bb4700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ac00b2910 0x7f9ac00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.064+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9acc0e23e0 con 0x7f9ad4108730 2024-05-17T09:16:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.065+0000 7f9ad8bb4700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ac00b2910 0x7f9ac00b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.069+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9acc013030 con 0x7f9ad4108730 2024-05-17T09:16:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.283+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ad4063bf0 con 0x7f9ac00b2910 2024-05-17T09:16:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.292+0000 7f9ad1ffb700 1 -- 172.21.15.19:0/2845776013 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340534 (secure 0 0 0) 0x7f9ac407b250 con 0x7f9ac00b2910 2024-05-17T09:16:50.297 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ac00b2910 msgr2=0x7f9ac00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ac00b2910 0x7f9ac00b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac4005fb0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 msgr2=0x7f9ad40ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 0x7f9ad40ffd10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00b970 tx=0x7f9acc00bc80 comp rx=0 tx=0).stop 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 shutdown_connections 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad4108730 0x7f9ad40ffd10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad41034b0 0x7f9ad40ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ac00b2910 0x7f9ac00b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 --2- 172.21.15.19:0/2845776013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ad4107de0 0x7f9ad40ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 >> 172.21.15.19:0/2845776013 conn(0x7f9ad40fedb0 msgr2=0x7f9ad4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 shutdown_connections 2024-05-17T09:16:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:50.296+0000 7f9adae18700 1 -- 172.21.15.19:0/2845776013 wait complete. 2024-05-17T09:16:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:16:50.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:50 smithi019 ceph-mon[31620]: pgmap v350: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-17T09:16:50.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:50 smithi019 ceph-mon[31620]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T09:16:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:50 smithi069 ceph-mon[35081]: pgmap v350: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-17T09:16:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:50 smithi069 ceph-mon[35081]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T09:16:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:50 smithi079 ceph-mon[35502]: pgmap v350: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-17T09:16:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:50 smithi079 ceph-mon[35502]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-17T09:16:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-05-17T09:16:48.895933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659130,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":280,"num_read_kb":240,"num_write":293,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348404,"kb_used_data":11716,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593420,"statfs":{"total":1151940427776,"available":1151583662080,"internally_reserved":0,"allocated":11997184,"data_stored":7933391,"data_compressed":436502,"data_compressed_allocated":593920,"data_compressed_original":2686976,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7391,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":39,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002310"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153939+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:15:48.153939+0000","last_peered":"2024-05-17T09:15:48.153939+0000","last_clean":"2024-05-17T09:15:48.153939+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:15:48.153939+0000","last_undegraded":"2024-05-17T09:15:48.153939+0000","last_fullsized":"2024-05-17T09:15:48.153939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494299+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:16:47.494299+0000","last_peered":"2024-05-17T09:16:47.494299+0000","last_clean":"2024-05-17T09:16:47.494299+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:16:47.494299+0000","last_undegraded":"2024-05-17T09:16:47.494299+0000","last_fullsized":"2024-05-17T09:16:47.494299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494122+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:16:47.494122+0000","last_peered":"2024-05-17T09:16:47.494122+0000","last_clean":"2024-05-17T09:16:47.494122+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:16:47.494122+0000","last_undegraded":"2024-05-17T09:16:47.494122+0000","last_fullsized":"2024-05-17T09:16:47.494122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148854+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:15:48.148854+0000","last_peered":"2024-05-17T09:15:48.148854+0000","last_clean":"2024-05-17T09:15:48.148854+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:15:48.148854+0000","last_undegraded":"2024-05-17T09:15:48.148854+0000","last_fullsized":"2024-05-17T09:15:48.148854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150013+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:15:48.150013+0000","last_peered":"2024-05-17T09:15:48.150013+0000","last_clean":"2024-05-17T09:15:48.150013+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:15:48.150013+0000","last_undegraded":"2024-05-17T09:15:48.150013+0000","last_fullsized":"2024-05-17T09:15:48.150013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320631+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:16:48.320631+0000","last_peered":"2024-05-17T09:16:48.320631+0000","last_clean":"2024-05-17T09:16:48.320631+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:16:48.320631+0000","last_undegraded":"2024-05-17T09:16:48.320631+0000","last_fullsized":"2024-05-17T09:16:48.320631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312941+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:16:47.312941+0000","last_peered":"2024-05-17T09:16:47.312941+0000","last_clean":"2024-05-17T09:16:47.312941+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:16:47.312941+0000","last_undegraded":"2024-05-17T09:16:47.312941+0000","last_fullsized":"2024-05-17T09:16:47.312941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154109+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:15:48.154109+0000","last_peered":"2024-05-17T09:15:48.154109+0000","last_clean":"2024-05-17T09:15:48.154109+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:15:48.154109+0000","last_undegraded":"2024-05-17T09:15:48.154109+0000","last_fullsized":"2024-05-17T09:15:48.154109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.402155+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:15:48.402155+0000","last_peered":"2024-05-17T09:15:48.402155+0000","last_clean":"2024-05-17T09:15:48.402155+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:15:48.402155+0000","last_undegraded":"2024-05-17T09:15:48.402155+0000","last_fullsized":"2024-05-17T09:15:48.402155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321155+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:16:48.321155+0000","last_peered":"2024-05-17T09:16:48.321155+0000","last_clean":"2024-05-17T09:16:48.321155+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:16:48.321155+0000","last_undegraded":"2024-05-17T09:16:48.321155+0000","last_fullsized":"2024-05-17T09:16:48.321155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313745+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:16:47.313745+0000","last_peered":"2024-05-17T09:16:47.313745+0000","last_clean":"2024-05-17T09:16:47.313745+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:16:47.313745+0000","last_undegraded":"2024-05-17T09:16:47.313745+0000","last_fullsized":"2024-05-17T09:16:47.313745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148672+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:15:48.148672+0000","last_peered":"2024-05-17T09:15:48.148672+0000","last_clean":"2024-05-17T09:15:48.148672+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:15:48.148672+0000","last_undegraded":"2024-05-17T09:15:48.148672+0000","last_fullsized":"2024-05-17T09:15:48.148672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315237+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:16:47.315237+0000","last_peered":"2024-05-17T09:16:47.315237+0000","last_clean":"2024-05-17T09:16:47.315237+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:16:47.315237+0000","last_undegraded":"2024-05-17T09:16:47.315237+0000","last_fullsized":"2024-05-17T09:16:47.315237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494016+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:16:47.494016+0000","last_peered":"2024-05-17T09:16:47.494016+0000","last_clean":"2024-05-17T09:16:47.494016+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:16:47.494016+0000","last_undegraded":"2024-05-17T09:16:47.494016+0000","last_fullsized":"2024-05-17T09:16:47.494016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494045+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:16:47.494045+0000","last_peered":"2024-05-17T09:16:47.494045+0000","last_clean":"2024-05-17T09:16:47.494045+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:16:47.494045+0000","last_undegraded":"2024-05-17T09:16:47.494045+0000","last_fullsized":"2024-05-17T09:16:47.494045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401695+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:15:48.401695+0000","last_peered":"2024-05-17T09:15:48.401695+0000","last_clean":"2024-05-17T09:15:48.401695+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:15:48.401695+0000","last_undegraded":"2024-05-17T09:15:48.401695+0000","last_fullsized":"2024-05-17T09:15:48.401695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153652+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:15:48.153652+0000","last_peered":"2024-05-17T09:15:48.153652+0000","last_clean":"2024-05-17T09:15:48.153652+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:15:48.153652+0000","last_undegraded":"2024-05-17T09:15:48.153652+0000","last_fullsized":"2024-05-17T09:15:48.153652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.308540+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:15:48.308540+0000","last_peered":"2024-05-17T09:15:48.308540+0000","last_clean":"2024-05-17T09:15:48.308540+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:15:48.308540+0000","last_undegraded":"2024-05-17T09:15:48.308540+0000","last_fullsized":"2024-05-17T09:15:48.308540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.158729+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:15:48.158729+0000","last_peered":"2024-05-17T09:15:48.158729+0000","last_clean":"2024-05-17T09:15:48.158729+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:15:48.158729+0000","last_undegraded":"2024-05-17T09:15:48.158729+0000","last_fullsized":"2024-05-17T09:15:48.158729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.157893+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:15:48.157893+0000","last_peered":"2024-05-17T09:15:48.157893+0000","last_clean":"2024-05-17T09:15:48.157893+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:15:48.157893+0000","last_undegraded":"2024-05-17T09:15:48.157893+0000","last_fullsized":"2024-05-17T09:15:48.157893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.221934+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:17.221934+0000","last_peered":"2024-05-17T09:16:17.221934+0000","last_clean":"2024-05-17T09:16:17.221934+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:17.221934+0000","last_undegraded":"2024-05-17T09:16:17.221934+0000","last_fullsized":"2024-05-17T09:16:17.221934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154170+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:15:48.154170+0000","last_peered":"2024-05-17T09:15:48.154170+0000","last_clean":"2024-05-17T09:15:48.154170+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:15:48.154170+0000","last_undegraded":"2024-05-17T09:15:48.154170+0000","last_fullsized":"2024-05-17T09:15:48.154170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147916+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:15:48.147916+0000","last_peered":"2024-05-17T09:15:48.147916+0000","last_clean":"2024-05-17T09:15:48.147916+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:15:48.147916+0000","last_undegraded":"2024-05-17T09:15:48.147916+0000","last_fullsized":"2024-05-17T09:15:48.147916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154696+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:15:48.154696+0000","last_peered":"2024-05-17T09:15:48.154696+0000","last_clean":"2024-05-17T09:15:48.154696+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:15:48.154696+0000","last_undegraded":"2024-05-17T09:15:48.154696+0000","last_fullsized":"2024-05-17T09:15:48.154696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494351+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:16:47.494351+0000","last_peered":"2024-05-17T09:16:47.494351+0000","last_clean":"2024-05-17T09:16:47.494351+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:16:47.494351+0000","last_undegraded":"2024-05-17T09:16:47.494351+0000","last_fullsized":"2024-05-17T09:16:47.494351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148057+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:15:48.148057+0000","last_peered":"2024-05-17T09:15:48.148057+0000","last_clean":"2024-05-17T09:15:48.148057+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:15:48.148057+0000","last_undegraded":"2024-05-17T09:15:48.148057+0000","last_fullsized":"2024-05-17T09:15:48.148057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.219306+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:17.219306+0000","last_peered":"2024-05-17T09:16:17.219306+0000","last_clean":"2024-05-17T09:16:17.219306+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:17.219306+0000","last_undegraded":"2024-05-17T09:16:17.219306+0000","last_fullsized":"2024-05-17T09:16:17.219306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401535+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:15:48.401535+0000","last_peered":"2024-05-17T09:15:48.401535+0000","last_clean":"2024-05-17T09:15:48.401535+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:15:48.401535+0000","last_undegraded":"2024-05-17T09:15:48.401535+0000","last_fullsized":"2024-05-17T09:15:48.401535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.147985+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:15:48.147985+0000","last_peered":"2024-05-17T09:15:48.147985+0000","last_clean":"2024-05-17T09:15:48.147985+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:15:48.147985+0000","last_undegraded":"2024-05-17T09:15:48.147985+0000","last_fullsized":"2024-05-17T09:15:48.147985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149354+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:15:48.149354+0000","last_peered":"2024-05-17T09:15:48.149354+0000","last_clean":"2024-05-17T09:15:48.149354+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:15:48.149354+0000","last_undegraded":"2024-05-17T09:15:48.149354+0000","last_fullsized":"2024-05-17T09:15:48.149354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152793+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:15:48.152793+0000","last_peered":"2024-05-17T09:15:48.152793+0000","last_clean":"2024-05-17T09:15:48.152793+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:15:48.152793+0000","last_undegraded":"2024-05-17T09:15:48.152793+0000","last_fullsized":"2024-05-17T09:15:48.152793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.158512+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:15:48.158512+0000","last_peered":"2024-05-17T09:15:48.158512+0000","last_clean":"2024-05-17T09:15:48.158512+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:15:48.158512+0000","last_undegraded":"2024-05-17T09:15:48.158512+0000","last_fullsized":"2024-05-17T09:15:48.158512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:45.271034+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:16:45.271034+0000","last_peered":"2024-05-17T09:16:45.271034+0000","last_clean":"2024-05-17T09:16:45.271034+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:16:45.271034+0000","last_undegraded":"2024-05-17T09:16:45.271034+0000","last_fullsized":"2024-05-17T09:16:45.271034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149996+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:15:48.149996+0000","last_peered":"2024-05-17T09:15:48.149996+0000","last_clean":"2024-05-17T09:15:48.149996+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:15:48.149996+0000","last_undegraded":"2024-05-17T09:15:48.149996+0000","last_fullsized":"2024-05-17T09:15:48.149996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.156699+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:15:48.156699+0000","last_peered":"2024-05-17T09:15:48.156699+0000","last_clean":"2024-05-17T09:15:48.156699+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:15:48.156699+0000","last_undegraded":"2024-05-17T09:15:48.156699+0000","last_fullsized":"2024-05-17T09:15:48.156699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320555+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320555+0000","last_peered":"2024-05-17T09:16:48.320555+0000","last_clean":"2024-05-17T09:16:48.320555+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320555+0000","last_undegraded":"2024-05-17T09:16:48.320555+0000","last_fullsized":"2024-05-17T09:16:48.320555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.527228+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:15:48.527228+0000","last_peered":"2024-05-17T09:15:48.527228+0000","last_clean":"2024-05-17T09:15:48.527228+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:15:48.527228+0000","last_undegraded":"2024-05-17T09:15:48.527228+0000","last_fullsized":"2024-05-17T09:15:48.527228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.160379+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:15:48.160379+0000","last_peered":"2024-05-17T09:15:48.160379+0000","last_clean":"2024-05-17T09:15:48.160379+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:15:48.160379+0000","last_undegraded":"2024-05-17T09:15:48.160379+0000","last_fullsized":"2024-05-17T09:15:48.160379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315252+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:47.315252+0000","last_peered":"2024-05-17T09:16:47.315252+0000","last_clean":"2024-05-17T09:16:47.315252+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:47.315252+0000","last_undegraded":"2024-05-17T09:16:47.315252+0000","last_fullsized":"2024-05-17T09:16:47.315252+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153804+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:15:48.153804+0000","last_peered":"2024-05-17T09:15:48.153804+0000","last_clean":"2024-05-17T09:15:48.153804+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:15:48.153804+0000","last_undegraded":"2024-05-17T09:15:48.153804+0000","last_fullsized":"2024-05-17T09:15:48.153804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494270+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:16:47.494270+0000","last_peered":"2024-05-17T09:16:47.494270+0000","last_clean":"2024-05-17T09:16:47.494270+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:16:47.494270+0000","last_undegraded":"2024-05-17T09:16:47.494270+0000","last_fullsized":"2024-05-17T09:16:47.494270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152523+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:15:48.152523+0000","last_peered":"2024-05-17T09:15:48.152523+0000","last_clean":"2024-05-17T09:15:48.152523+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:15:48.152523+0000","last_undegraded":"2024-05-17T09:15:48.152523+0000","last_fullsized":"2024-05-17T09:15:48.152523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.307989+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:15:48.307989+0000","last_peered":"2024-05-17T09:15:48.307989+0000","last_clean":"2024-05-17T09:15:48.307989+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:15:48.307989+0000","last_undegraded":"2024-05-17T09:15:48.307989+0000","last_fullsized":"2024-05-17T09:15:48.307989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153742+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:15:48.153742+0000","last_peered":"2024-05-17T09:15:48.153742+0000","last_clean":"2024-05-17T09:15:48.153742+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:15:48.153742+0000","last_undegraded":"2024-05-17T09:15:48.153742+0000","last_fullsized":"2024-05-17T09:15:48.153742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320974+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:16:48.320974+0000","last_peered":"2024-05-17T09:16:48.320974+0000","last_clean":"2024-05-17T09:16:48.320974+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:16:48.320974+0000","last_undegraded":"2024-05-17T09:16:48.320974+0000","last_fullsized":"2024-05-17T09:16:48.320974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.402150+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:15:48.402150+0000","last_peered":"2024-05-17T09:15:48.402150+0000","last_clean":"2024-05-17T09:15:48.402150+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:15:48.402150+0000","last_undegraded":"2024-05-17T09:15:48.402150+0000","last_fullsized":"2024-05-17T09:15:48.402150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.307837+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:15:48.307837+0000","last_peered":"2024-05-17T09:15:48.307837+0000","last_clean":"2024-05-17T09:15:48.307837+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:15:48.307837+0000","last_undegraded":"2024-05-17T09:15:48.307837+0000","last_fullsized":"2024-05-17T09:15:48.307837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313691+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:47.313691+0000","last_peered":"2024-05-17T09:16:47.313691+0000","last_clean":"2024-05-17T09:16:47.313691+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:47.313691+0000","last_undegraded":"2024-05-17T09:16:47.313691+0000","last_fullsized":"2024-05-17T09:16:47.313691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494341+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:16:47.494341+0000","last_peered":"2024-05-17T09:16:47.494341+0000","last_clean":"2024-05-17T09:16:47.494341+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:16:47.494341+0000","last_undegraded":"2024-05-17T09:16:47.494341+0000","last_fullsized":"2024-05-17T09:16:47.494341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148524+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:15:48.148524+0000","last_peered":"2024-05-17T09:15:48.148524+0000","last_clean":"2024-05-17T09:15:48.148524+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:15:48.148524+0000","last_undegraded":"2024-05-17T09:15:48.148524+0000","last_fullsized":"2024-05-17T09:15:48.148524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.205842+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:16.205842+0000","last_peered":"2024-05-17T09:16:16.205842+0000","last_clean":"2024-05-17T09:16:16.205842+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:16.205842+0000","last_undegraded":"2024-05-17T09:16:16.205842+0000","last_fullsized":"2024-05-17T09:16:16.205842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401613+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:15:48.401613+0000","last_peered":"2024-05-17T09:15:48.401613+0000","last_clean":"2024-05-17T09:15:48.401613+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:15:48.401613+0000","last_undegraded":"2024-05-17T09:15:48.401613+0000","last_fullsized":"2024-05-17T09:15:48.401613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320362+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320362+0000","last_peered":"2024-05-17T09:16:48.320362+0000","last_clean":"2024-05-17T09:16:48.320362+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320362+0000","last_undegraded":"2024-05-17T09:16:48.320362+0000","last_fullsized":"2024-05-17T09:16:48.320362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320785+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:16:48.320785+0000","last_peered":"2024-05-17T09:16:48.320785+0000","last_clean":"2024-05-17T09:16:48.320785+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:16:48.320785+0000","last_undegraded":"2024-05-17T09:16:48.320785+0000","last_fullsized":"2024-05-17T09:16:48.320785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":97,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149548+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:15:48.149548+0000","last_peered":"2024-05-17T09:15:48.149548+0000","last_clean":"2024-05-17T09:15:48.149548+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:15:48.149548+0000","last_undegraded":"2024-05-17T09:15:48.149548+0000","last_fullsized":"2024-05-17T09:15:48.149548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153247+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:15:48.153247+0000","last_peered":"2024-05-17T09:15:48.153247+0000","last_clean":"2024-05-17T09:15:48.153247+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:15:48.153247+0000","last_undegraded":"2024-05-17T09:15:48.153247+0000","last_fullsized":"2024-05-17T09:15:48.153247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.220032+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:17.220032+0000","last_peered":"2024-05-17T09:16:17.220032+0000","last_clean":"2024-05-17T09:16:17.220032+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:17.220032+0000","last_undegraded":"2024-05-17T09:16:17.220032+0000","last_fullsized":"2024-05-17T09:16:17.220032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"peering","last_fresh":"2024-05-17T09:15:48.526248+0000","last_change":"2024-05-17T09:16:47.315594+0000","last_active":"2024-05-17T09:16:47.315594+0000","last_peered":"2024-05-17T09:15:48.526248+0000","last_clean":"2024-05-17T09:15:48.526248+0000","last_became_active":"2024-05-17T09:15:07.962064+0000","last_became_peered":"2024-05-17T09:15:07.962064+0000","last_unstale":"2024-05-17T09:15:48.526248+0000","last_undegraded":"2024-05-17T09:15:48.526248+0000","last_fullsized":"2024-05-17T09:15:48.526248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151162+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:15:48.151162+0000","last_peered":"2024-05-17T09:15:48.151162+0000","last_clean":"2024-05-17T09:15:48.151162+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:15:48.151162+0000","last_undegraded":"2024-05-17T09:15:48.151162+0000","last_fullsized":"2024-05-17T09:15:48.151162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.207037+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:16.207037+0000","last_peered":"2024-05-17T09:16:16.207037+0000","last_clean":"2024-05-17T09:16:16.207037+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:16.207037+0000","last_undegraded":"2024-05-17T09:16:16.207037+0000","last_fullsized":"2024-05-17T09:16:16.207037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313217+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:16:47.313217+0000","last_peered":"2024-05-17T09:16:47.313217+0000","last_clean":"2024-05-17T09:16:47.313217+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:16:47.313217+0000","last_undegraded":"2024-05-17T09:16:47.313217+0000","last_fullsized":"2024-05-17T09:16:47.313217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.160250+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:15:48.160250+0000","last_peered":"2024-05-17T09:15:48.160250+0000","last_clean":"2024-05-17T09:15:48.160250+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:15:48.160250+0000","last_undegraded":"2024-05-17T09:15:48.160250+0000","last_fullsized":"2024-05-17T09:15:48.160250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:45.270604+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:16:45.270604+0000","last_peered":"2024-05-17T09:16:45.270604+0000","last_clean":"2024-05-17T09:16:45.270604+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:16:45.270604+0000","last_undegraded":"2024-05-17T09:16:45.270604+0000","last_fullsized":"2024-05-17T09:16:45.270604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312906+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:16:47.312906+0000","last_peered":"2024-05-17T09:16:47.312906+0000","last_clean":"2024-05-17T09:16:47.312906+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:16:47.312906+0000","last_undegraded":"2024-05-17T09:16:47.312906+0000","last_fullsized":"2024-05-17T09:16:47.312906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315645+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:47.315645+0000","last_peered":"2024-05-17T09:16:47.315645+0000","last_clean":"2024-05-17T09:16:47.315645+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:47.315645+0000","last_undegraded":"2024-05-17T09:16:47.315645+0000","last_fullsized":"2024-05-17T09:16:47.315645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.221352+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:17.221352+0000","last_peered":"2024-05-17T09:16:17.221352+0000","last_clean":"2024-05-17T09:16:17.221352+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:17.221352+0000","last_undegraded":"2024-05-17T09:16:17.221352+0000","last_fullsized":"2024-05-17T09:16:17.221352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152657+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:15:48.152657+0000","last_peered":"2024-05-17T09:15:48.152657+0000","last_clean":"2024-05-17T09:15:48.152657+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:15:48.152657+0000","last_undegraded":"2024-05-17T09:15:48.152657+0000","last_fullsized":"2024-05-17T09:15:48.152657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151954+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:15:48.151954+0000","last_peered":"2024-05-17T09:15:48.151954+0000","last_clean":"2024-05-17T09:15:48.151954+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:15:48.151954+0000","last_undegraded":"2024-05-17T09:15:48.151954+0000","last_fullsized":"2024-05-17T09:15:48.151954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:18.214956+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:18.214956+0000","last_peered":"2024-05-17T09:16:18.214956+0000","last_clean":"2024-05-17T09:16:18.214956+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:18.214956+0000","last_undegraded":"2024-05-17T09:16:18.214956+0000","last_fullsized":"2024-05-17T09:16:18.214956+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152172+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:15:48.152172+0000","last_peered":"2024-05-17T09:15:48.152172+0000","last_clean":"2024-05-17T09:15:48.152172+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:15:48.152172+0000","last_undegraded":"2024-05-17T09:15:48.152172+0000","last_fullsized":"2024-05-17T09:15:48.152172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149303+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:15:48.149303+0000","last_peered":"2024-05-17T09:15:48.149303+0000","last_clean":"2024-05-17T09:15:48.149303+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:15:48.149303+0000","last_undegraded":"2024-05-17T09:15:48.149303+0000","last_fullsized":"2024-05-17T09:15:48.149303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.308206+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:15:48.308206+0000","last_peered":"2024-05-17T09:15:48.308206+0000","last_clean":"2024-05-17T09:15:48.308206+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:15:48.308206+0000","last_undegraded":"2024-05-17T09:15:48.308206+0000","last_fullsized":"2024-05-17T09:15:48.308206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320891+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:16:48.320891+0000","last_peered":"2024-05-17T09:16:48.320891+0000","last_clean":"2024-05-17T09:16:48.320891+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:16:48.320891+0000","last_undegraded":"2024-05-17T09:16:48.320891+0000","last_fullsized":"2024-05-17T09:16:48.320891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315390+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:16:47.315390+0000","last_peered":"2024-05-17T09:16:47.315390+0000","last_clean":"2024-05-17T09:16:47.315390+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:16:47.315390+0000","last_undegraded":"2024-05-17T09:16:47.315390+0000","last_fullsized":"2024-05-17T09:16:47.315390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313086+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:16:47.313086+0000","last_peered":"2024-05-17T09:16:47.313086+0000","last_clean":"2024-05-17T09:16:47.313086+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:16:47.313086+0000","last_undegraded":"2024-05-17T09:16:47.313086+0000","last_fullsized":"2024-05-17T09:16:47.313086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315648+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:47.315648+0000","last_peered":"2024-05-17T09:16:47.315648+0000","last_clean":"2024-05-17T09:16:47.315648+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:47.315648+0000","last_undegraded":"2024-05-17T09:16:47.315648+0000","last_fullsized":"2024-05-17T09:16:47.315648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152829+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:15:48.152829+0000","last_peered":"2024-05-17T09:15:48.152829+0000","last_clean":"2024-05-17T09:15:48.152829+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:15:48.152829+0000","last_undegraded":"2024-05-17T09:15:48.152829+0000","last_fullsized":"2024-05-17T09:15:48.152829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315103+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:16:47.315103+0000","last_peered":"2024-05-17T09:16:47.315103+0000","last_clean":"2024-05-17T09:16:47.315103+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:16:47.315103+0000","last_undegraded":"2024-05-17T09:16:47.315103+0000","last_fullsized":"2024-05-17T09:16:47.315103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313524+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:16:47.313524+0000","last_peered":"2024-05-17T09:16:47.313524+0000","last_clean":"2024-05-17T09:16:47.313524+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:16:47.313524+0000","last_undegraded":"2024-05-17T09:16:47.313524+0000","last_fullsized":"2024-05-17T09:16:47.313524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154656+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:15:48.154656+0000","last_peered":"2024-05-17T09:15:48.154656+0000","last_clean":"2024-05-17T09:15:48.154656+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:15:48.154656+0000","last_undegraded":"2024-05-17T09:15:48.154656+0000","last_fullsized":"2024-05-17T09:15:48.154656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:19.227245+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:19.227245+0000","last_peered":"2024-05-17T09:16:19.227245+0000","last_clean":"2024-05-17T09:16:19.227245+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:19.227245+0000","last_undegraded":"2024-05-17T09:16:19.227245+0000","last_fullsized":"2024-05-17T09:16:19.227245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312761+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:47.312761+0000","last_peered":"2024-05-17T09:16:47.312761+0000","last_clean":"2024-05-17T09:16:47.312761+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:47.312761+0000","last_undegraded":"2024-05-17T09:16:47.312761+0000","last_fullsized":"2024-05-17T09:16:47.312761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153293+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:15:48.153293+0000","last_peered":"2024-05-17T09:15:48.153293+0000","last_clean":"2024-05-17T09:15:48.153293+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:15:48.153293+0000","last_undegraded":"2024-05-17T09:15:48.153293+0000","last_fullsized":"2024-05-17T09:15:48.153293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"peering","last_fresh":"2024-05-17T09:15:48.525805+0000","last_change":"2024-05-17T09:16:47.316182+0000","last_active":"2024-05-17T09:16:47.316182+0000","last_peered":"2024-05-17T09:15:48.525805+0000","last_clean":"2024-05-17T09:15:48.525805+0000","last_became_active":"2024-05-17T09:15:07.966039+0000","last_became_peered":"2024-05-17T09:15:07.966039+0000","last_unstale":"2024-05-17T09:15:48.525805+0000","last_undegraded":"2024-05-17T09:15:48.525805+0000","last_fullsized":"2024-05-17T09:15:48.525805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312901+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:47.312901+0000","last_peered":"2024-05-17T09:16:47.312901+0000","last_clean":"2024-05-17T09:16:47.312901+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:47.312901+0000","last_undegraded":"2024-05-17T09:16:47.312901+0000","last_fullsized":"2024-05-17T09:16:47.312901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151152+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:15:48.151152+0000","last_peered":"2024-05-17T09:15:48.151152+0000","last_clean":"2024-05-17T09:15:48.151152+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:15:48.151152+0000","last_undegraded":"2024-05-17T09:15:48.151152+0000","last_fullsized":"2024-05-17T09:15:48.151152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.204261+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:16.204261+0000","last_peered":"2024-05-17T09:16:16.204261+0000","last_clean":"2024-05-17T09:16:16.204261+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:16.204261+0000","last_undegraded":"2024-05-17T09:16:16.204261+0000","last_fullsized":"2024-05-17T09:16:16.204261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315532+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:16:47.315532+0000","last_peered":"2024-05-17T09:16:47.315532+0000","last_clean":"2024-05-17T09:16:47.315532+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:16:47.315532+0000","last_undegraded":"2024-05-17T09:16:47.315532+0000","last_fullsized":"2024-05-17T09:16:47.315532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153756+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:15:48.153756+0000","last_peered":"2024-05-17T09:15:48.153756+0000","last_clean":"2024-05-17T09:15:48.153756+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:15:48.153756+0000","last_undegraded":"2024-05-17T09:15:48.153756+0000","last_fullsized":"2024-05-17T09:15:48.153756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:40.978273+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:16:40.978273+0000","last_peered":"2024-05-17T09:16:40.978273+0000","last_clean":"2024-05-17T09:16:40.978273+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:16:40.978273+0000","last_undegraded":"2024-05-17T09:16:40.978273+0000","last_fullsized":"2024-05-17T09:16:40.978273+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154873+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:15:48.154873+0000","last_peered":"2024-05-17T09:15:48.154873+0000","last_clean":"2024-05-17T09:15:48.154873+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:15:48.154873+0000","last_undegraded":"2024-05-17T09:15:48.154873+0000","last_fullsized":"2024-05-17T09:15:48.154873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.157756+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:15:48.157756+0000","last_peered":"2024-05-17T09:15:48.157756+0000","last_clean":"2024-05-17T09:15:48.157756+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:15:48.157756+0000","last_undegraded":"2024-05-17T09:15:48.157756+0000","last_fullsized":"2024-05-17T09:15:48.157756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.206578+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:16.206578+0000","last_peered":"2024-05-17T09:16:16.206578+0000","last_clean":"2024-05-17T09:16:16.206578+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:16.206578+0000","last_undegraded":"2024-05-17T09:16:16.206578+0000","last_fullsized":"2024-05-17T09:16:16.206578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494415+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:16:47.494415+0000","last_peered":"2024-05-17T09:16:47.494415+0000","last_clean":"2024-05-17T09:16:47.494415+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:16:47.494415+0000","last_undegraded":"2024-05-17T09:16:47.494415+0000","last_fullsized":"2024-05-17T09:16:47.494415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148340+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:15:48.148340+0000","last_peered":"2024-05-17T09:15:48.148340+0000","last_clean":"2024-05-17T09:15:48.148340+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:15:48.148340+0000","last_undegraded":"2024-05-17T09:15:48.148340+0000","last_fullsized":"2024-05-17T09:15:48.148340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.207401+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:16.207401+0000","last_peered":"2024-05-17T09:16:16.207401+0000","last_clean":"2024-05-17T09:16:16.207401+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:16.207401+0000","last_undegraded":"2024-05-17T09:16:16.207401+0000","last_fullsized":"2024-05-17T09:16:16.207401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.149196+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:15:48.149196+0000","last_peered":"2024-05-17T09:15:48.149196+0000","last_clean":"2024-05-17T09:15:48.149196+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:15:48.149196+0000","last_undegraded":"2024-05-17T09:15:48.149196+0000","last_fullsized":"2024-05-17T09:15:48.149196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.308433+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:15:48.308433+0000","last_peered":"2024-05-17T09:15:48.308433+0000","last_clean":"2024-05-17T09:15:48.308433+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:15:48.308433+0000","last_undegraded":"2024-05-17T09:15:48.308433+0000","last_fullsized":"2024-05-17T09:15:48.308433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.206437+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:16.206437+0000","last_peered":"2024-05-17T09:16:16.206437+0000","last_clean":"2024-05-17T09:16:16.206437+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:16.206437+0000","last_undegraded":"2024-05-17T09:16:16.206437+0000","last_fullsized":"2024-05-17T09:16:16.206437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150336+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:15:48.150336+0000","last_peered":"2024-05-17T09:15:48.150336+0000","last_clean":"2024-05-17T09:15:48.150336+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:15:48.150336+0000","last_undegraded":"2024-05-17T09:15:48.150336+0000","last_fullsized":"2024-05-17T09:15:48.150336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148214+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:15:48.148214+0000","last_peered":"2024-05-17T09:15:48.148214+0000","last_clean":"2024-05-17T09:15:48.148214+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:15:48.148214+0000","last_undegraded":"2024-05-17T09:15:48.148214+0000","last_fullsized":"2024-05-17T09:15:48.148214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.156549+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:15:48.156549+0000","last_peered":"2024-05-17T09:15:48.156549+0000","last_clean":"2024-05-17T09:15:48.156549+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:15:48.156549+0000","last_undegraded":"2024-05-17T09:15:48.156549+0000","last_fullsized":"2024-05-17T09:15:48.156549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.154146+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:15:48.154146+0000","last_peered":"2024-05-17T09:15:48.154146+0000","last_clean":"2024-05-17T09:15:48.154146+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:15:48.154146+0000","last_undegraded":"2024-05-17T09:15:48.154146+0000","last_fullsized":"2024-05-17T09:15:48.154146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.148297+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:15:48.148297+0000","last_peered":"2024-05-17T09:15:48.148297+0000","last_clean":"2024-05-17T09:15:48.148297+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:15:48.148297+0000","last_undegraded":"2024-05-17T09:15:48.148297+0000","last_fullsized":"2024-05-17T09:15:48.148297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:16.205311+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:16.205311+0000","last_peered":"2024-05-17T09:16:16.205311+0000","last_clean":"2024-05-17T09:16:16.205311+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:16.205311+0000","last_undegraded":"2024-05-17T09:16:16.205311+0000","last_fullsized":"2024-05-17T09:16:16.205311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313203+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:16:47.313203+0000","last_peered":"2024-05-17T09:16:47.313203+0000","last_clean":"2024-05-17T09:16:47.313203+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:16:47.313203+0000","last_undegraded":"2024-05-17T09:16:47.313203+0000","last_fullsized":"2024-05-17T09:16:47.313203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401434+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:15:48.401434+0000","last_peered":"2024-05-17T09:15:48.401434+0000","last_clean":"2024-05-17T09:15:48.401434+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:15:48.401434+0000","last_undegraded":"2024-05-17T09:15:48.401434+0000","last_fullsized":"2024-05-17T09:15:48.401434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:16:17.220941+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:17.220941+0000","last_peered":"2024-05-17T09:16:17.220941+0000","last_clean":"2024-05-17T09:16:17.220941+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:17.220941+0000","last_undegraded":"2024-05-17T09:16:17.220941+0000","last_fullsized":"2024-05-17T09:16:17.220941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313815+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:16:47.313815+0000","last_peered":"2024-05-17T09:16:47.313815+0000","last_clean":"2024-05-17T09:16:47.313815+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:16:47.313815+0000","last_undegraded":"2024-05-17T09:16:47.313815+0000","last_fullsized":"2024-05-17T09:16:47.313815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494082+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:47.494082+0000","last_peered":"2024-05-17T09:16:47.494082+0000","last_clean":"2024-05-17T09:16:47.494082+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:47.494082+0000","last_undegraded":"2024-05-17T09:16:47.494082+0000","last_fullsized":"2024-05-17T09:16:47.494082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152959+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:15:48.152959+0000","last_peered":"2024-05-17T09:15:48.152959+0000","last_clean":"2024-05-17T09:15:48.152959+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:15:48.152959+0000","last_undegraded":"2024-05-17T09:15:48.152959+0000","last_fullsized":"2024-05-17T09:15:48.152959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313435+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:16:47.313435+0000","last_peered":"2024-05-17T09:16:47.313435+0000","last_clean":"2024-05-17T09:16:47.313435+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:16:47.313435+0000","last_undegraded":"2024-05-17T09:16:47.313435+0000","last_fullsized":"2024-05-17T09:16:47.313435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150682+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:15:48.150682+0000","last_peered":"2024-05-17T09:15:48.150682+0000","last_clean":"2024-05-17T09:15:48.150682+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:15:48.150682+0000","last_undegraded":"2024-05-17T09:15:48.150682+0000","last_fullsized":"2024-05-17T09:15:48.150682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315638+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:47.315638+0000","last_peered":"2024-05-17T09:16:47.315638+0000","last_clean":"2024-05-17T09:16:47.315638+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:47.315638+0000","last_undegraded":"2024-05-17T09:16:47.315638+0000","last_fullsized":"2024-05-17T09:16:47.315638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313810+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:16:47.313810+0000","last_peered":"2024-05-17T09:16:47.313810+0000","last_clean":"2024-05-17T09:16:47.313810+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:16:47.313810+0000","last_undegraded":"2024-05-17T09:16:47.313810+0000","last_fullsized":"2024-05-17T09:16:47.313810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.150478+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:15:48.150478+0000","last_peered":"2024-05-17T09:15:48.150478+0000","last_clean":"2024-05-17T09:15:48.150478+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:15:48.150478+0000","last_undegraded":"2024-05-17T09:15:48.150478+0000","last_fullsized":"2024-05-17T09:15:48.150478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.158198+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:15:48.158198+0000","last_peered":"2024-05-17T09:15:48.158198+0000","last_clean":"2024-05-17T09:15:48.158198+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:15:48.158198+0000","last_undegraded":"2024-05-17T09:15:48.158198+0000","last_fullsized":"2024-05-17T09:15:48.158198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.156298+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:15:48.156298+0000","last_peered":"2024-05-17T09:15:48.156298+0000","last_clean":"2024-05-17T09:15:48.156298+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:15:48.156298+0000","last_undegraded":"2024-05-17T09:15:48.156298+0000","last_fullsized":"2024-05-17T09:15:48.156298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494269+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:16:47.494269+0000","last_peered":"2024-05-17T09:16:47.494269+0000","last_clean":"2024-05-17T09:16:47.494269+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:16:47.494269+0000","last_undegraded":"2024-05-17T09:16:47.494269+0000","last_fullsized":"2024-05-17T09:16:47.494269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315558+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:16:47.315558+0000","last_peered":"2024-05-17T09:16:47.315558+0000","last_clean":"2024-05-17T09:16:47.315558+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:16:47.315558+0000","last_undegraded":"2024-05-17T09:16:47.315558+0000","last_fullsized":"2024-05-17T09:16:47.315558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.151950+0000","last_change":"2024-05-17T09:15:07.959003+0000","last_active":"2024-05-17T09:15:48.151950+0000","last_peered":"2024-05-17T09:15:48.151950+0000","last_clean":"2024-05-17T09:15:48.151950+0000","last_became_active":"2024-05-17T09:15:07.958653+0000","last_became_peered":"2024-05-17T09:15:07.958653+0000","last_unstale":"2024-05-17T09:15:48.151950+0000","last_undegraded":"2024-05-17T09:15:48.151950+0000","last_fullsized":"2024-05-17T09:15:48.151950+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:40:55.013374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.157394+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:15:48.157394+0000","last_peered":"2024-05-17T09:15:48.157394+0000","last_clean":"2024-05-17T09:15:48.157394+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:15:48.157394+0000","last_undegraded":"2024-05-17T09:15:48.157394+0000","last_fullsized":"2024-05-17T09:15:48.157394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.155006+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:15:48.155006+0000","last_peered":"2024-05-17T09:15:48.155006+0000","last_clean":"2024-05-17T09:15:48.155006+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:15:48.155006+0000","last_undegraded":"2024-05-17T09:15:48.155006+0000","last_fullsized":"2024-05-17T09:15:48.155006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152553+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:15:48.152553+0000","last_peered":"2024-05-17T09:15:48.152553+0000","last_clean":"2024-05-17T09:15:48.152553+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:15:48.152553+0000","last_undegraded":"2024-05-17T09:15:48.152553+0000","last_fullsized":"2024-05-17T09:15:48.152553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.153980+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:15:48.153980+0000","last_peered":"2024-05-17T09:15:48.153980+0000","last_clean":"2024-05-17T09:15:48.153980+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:15:48.153980+0000","last_undegraded":"2024-05-17T09:15:48.153980+0000","last_fullsized":"2024-05-17T09:15:48.153980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.401677+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:15:48.401677+0000","last_peered":"2024-05-17T09:15:48.401677+0000","last_clean":"2024-05-17T09:15:48.401677+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:15:48.401677+0000","last_undegraded":"2024-05-17T09:15:48.401677+0000","last_fullsized":"2024-05-17T09:15:48.401677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.315461+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:16:47.315461+0000","last_peered":"2024-05-17T09:16:47.315461+0000","last_clean":"2024-05-17T09:16:47.315461+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:16:47.315461+0000","last_undegraded":"2024-05-17T09:16:47.315461+0000","last_fullsized":"2024-05-17T09:16:47.315461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T09:15:48.152398+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:15:48.152398+0000","last_peered":"2024-05-17T09:15:48.152398+0000","last_clean":"2024-05-17T09:15:48.152398+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:15:48.152398+0000","last_undegraded":"2024-05-17T09:15:48.152398+0000","last_fullsized":"2024-05-17T09:15:48.152398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68528,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":168,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":193078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":496288,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1170984,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070510,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":500564,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266738,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":496366,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1102154,"data_compressed":65150,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":491644,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":549437,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":502498,"data_compressed":29421,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":551144,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":491838,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":496685,"data_compressed":26453,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1083789,"data_compressed":62182,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:16:51.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:16:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:51.604 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:51 smithi019 ceph-mon[31620]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:51.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:51 smithi069 ceph-mon[35081]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:51.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:51 smithi079 ceph-mon[35502]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:52.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:52 smithi019 ceph-mon[31620]: pgmap v351: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-17T09:16:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:52 smithi069 ceph-mon[35081]: pgmap v351: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-17T09:16:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:52 smithi079 ceph-mon[35502]: pgmap v351: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 -- 172.21.15.19:0/3808945052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 msgr2=0x7f69bc1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 --2- 172.21.15.19:0/3808945052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc1038c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b0d0 tx=0x7f69ac00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 -- 172.21.15.19:0/3808945052 shutdown_connections 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 --2- 172.21.15.19:0/3808945052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69bc108730 0x7f69bc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 --2- 172.21.15.19:0/3808945052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc1038c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 --2- 172.21.15.19:0/3808945052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69bc107de0 0x7f69bc1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 -- 172.21.15.19:0/3808945052 >> 172.21.15.19:0/3808945052 conn(0x7f69bc0fede0 msgr2=0x7f69bc101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 -- 172.21.15.19:0/3808945052 shutdown_connections 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 -- 172.21.15.19:0/3808945052 wait complete. 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.254+0000 7f69c2237700 1 Processor -- start 2024-05-17T09:16:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 -- start start 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69bc107de0 0x7f69bc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69bc108730 0x7f69bc19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc113210 con 0x7f69bc1034b0 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc113090 con 0x7f69bc108730 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69c2237700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc113390 con 0x7f69bc107de0 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43870/0 (socket says 172.21.15.19:43870) 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 -- 172.21.15.19:0/4092154132 learned_addr learned my addr 172.21.15.19:0/4092154132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69baffd700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69bc107de0 0x7f69bc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bbfff700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69bc108730 0x7f69bc19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 -- 172.21.15.19:0/4092154132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69bc107de0 msgr2=0x7f69bc19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69bc107de0 0x7f69bc19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 -- 172.21.15.19:0/4092154132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69bc108730 msgr2=0x7f69bc19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69bc108730 0x7f69bc19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.255+0000 7f69bb7fe700 1 -- 172.21.15.19:0/4092154132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc19fc20 con 0x7f69bc1034b0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.256+0000 7f69bb7fe700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc19ad50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f69ac007a20 tx=0x7f69ac007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.256+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac01f070 con 0x7f69bc1034b0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.256+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69ac004c50 con 0x7f69bc1034b0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.256+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc19feb0 con 0x7f69bc1034b0 2024-05-17T09:16:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.256+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac01b690 con 0x7f69bc1034b0 2024-05-17T09:16:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.256+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1a0390 con 0x7f69bc1034b0 2024-05-17T09:16:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.257+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc0512b0 con 0x7f69bc1034b0 2024-05-17T09:16:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.259+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f69ac022070 con 0x7f69bc1034b0 2024-05-17T09:16:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.260+0000 7f69b8ff9700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69a80b2910 0x7f69a80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.260+0000 7f69baffd700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69a80b2910 0x7f69a80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.260+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f69ac00f020 con 0x7f69bc1034b0 2024-05-17T09:16:53.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.261+0000 7f69baffd700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69a80b2910 0x7f69a80b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69bc1099e0 tx=0x7f69a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:53.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.265+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69ac029020 con 0x7f69bc1034b0 2024-05-17T09:16:53.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:53 smithi019 ceph-mon[31620]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 6 op/s 2024-05-17T09:16:53.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:53 smithi019 ceph-mon[31620]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-17T09:16:53.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:53 smithi019 ceph-mon[31620]: Cluster is now healthy 2024-05-17T09:16:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.480+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69bc19c0b0 con 0x7f69a80b2910 2024-05-17T09:16:53.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.489+0000 7f69b8ff9700 1 -- 172.21.15.19:0/4092154132 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340607 (secure 0 0 0) 0x7f69bc19c0b0 con 0x7f69a80b2910 2024-05-17T09:16:53.494 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69a80b2910 msgr2=0x7f69a80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69a80b2910 0x7f69a80b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69bc1099e0 tx=0x7f69a400b410 comp rx=0 tx=0).stop 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 msgr2=0x7f69bc19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc19ad50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f69ac007a20 tx=0x7f69ac007a50 comp rx=0 tx=0).stop 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 shutdown_connections 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69bc108730 0x7f69bc19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69bc1034b0 0x7f69bc19ad50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69a80b2910 0x7f69a80b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 --2- 172.21.15.19:0/4092154132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69bc107de0 0x7f69bc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 >> 172.21.15.19:0/4092154132 conn(0x7f69bc0fede0 msgr2=0x7f69bc075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 shutdown_connections 2024-05-17T09:16:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:53.494+0000 7f69c2237700 1 -- 172.21.15.19:0/4092154132 wait complete. 2024-05-17T09:16:53.498 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:16:53.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:53 smithi069 ceph-mon[35081]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 6 op/s 2024-05-17T09:16:53.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:53 smithi069 ceph-mon[35081]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-17T09:16:53.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:53 smithi069 ceph-mon[35081]: Cluster is now healthy 2024-05-17T09:16:53.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:53 smithi079 ceph-mon[35502]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 6 op/s 2024-05-17T09:16:53.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:53 smithi079 ceph-mon[35502]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-17T09:16:53.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:53 smithi079 ceph-mon[35502]: Cluster is now healthy 2024-05-17T09:16:54.171 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-05-17T09:16:52.897125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659130,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":295,"num_read_kb":245,"num_write":293,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348688,"kb_used_data":11936,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593136,"statfs":{"total":1151940427776,"available":1151583371264,"internally_reserved":0,"allocated":12222464,"data_stored":8106251,"data_compressed":487014,"data_compressed_allocated":663552,"data_compressed_original":2826240,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6253,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":27,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002308"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322463+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:16:48.322463+0000","last_peered":"2024-05-17T09:16:48.322463+0000","last_clean":"2024-05-17T09:16:48.322463+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:16:48.322463+0000","last_undegraded":"2024-05-17T09:16:48.322463+0000","last_fullsized":"2024-05-17T09:16:48.322463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494299+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:16:47.494299+0000","last_peered":"2024-05-17T09:16:47.494299+0000","last_clean":"2024-05-17T09:16:47.494299+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:16:47.494299+0000","last_undegraded":"2024-05-17T09:16:47.494299+0000","last_fullsized":"2024-05-17T09:16:47.494299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494122+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:16:47.494122+0000","last_peered":"2024-05-17T09:16:47.494122+0000","last_clean":"2024-05-17T09:16:47.494122+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:16:47.494122+0000","last_undegraded":"2024-05-17T09:16:47.494122+0000","last_fullsized":"2024-05-17T09:16:47.494122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731397+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:16:48.731397+0000","last_peered":"2024-05-17T09:16:48.731397+0000","last_clean":"2024-05-17T09:16:48.731397+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:16:48.731397+0000","last_undegraded":"2024-05-17T09:16:48.731397+0000","last_fullsized":"2024-05-17T09:16:48.731397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829710+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:16:48.829710+0000","last_peered":"2024-05-17T09:16:48.829710+0000","last_clean":"2024-05-17T09:16:48.829710+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:16:48.829710+0000","last_undegraded":"2024-05-17T09:16:48.829710+0000","last_fullsized":"2024-05-17T09:16:48.829710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320631+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:16:48.320631+0000","last_peered":"2024-05-17T09:16:48.320631+0000","last_clean":"2024-05-17T09:16:48.320631+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:16:48.320631+0000","last_undegraded":"2024-05-17T09:16:48.320631+0000","last_fullsized":"2024-05-17T09:16:48.320631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312941+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:16:47.312941+0000","last_peered":"2024-05-17T09:16:47.312941+0000","last_clean":"2024-05-17T09:16:47.312941+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:16:47.312941+0000","last_undegraded":"2024-05-17T09:16:47.312941+0000","last_fullsized":"2024-05-17T09:16:47.312941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755154+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:16:48.755154+0000","last_peered":"2024-05-17T09:16:48.755154+0000","last_clean":"2024-05-17T09:16:48.755154+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:16:48.755154+0000","last_undegraded":"2024-05-17T09:16:48.755154+0000","last_fullsized":"2024-05-17T09:16:48.755154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860025+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:16:48.860025+0000","last_peered":"2024-05-17T09:16:48.860025+0000","last_clean":"2024-05-17T09:16:48.860025+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:16:48.860025+0000","last_undegraded":"2024-05-17T09:16:48.860025+0000","last_fullsized":"2024-05-17T09:16:48.860025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321155+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:16:48.321155+0000","last_peered":"2024-05-17T09:16:48.321155+0000","last_clean":"2024-05-17T09:16:48.321155+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:16:48.321155+0000","last_undegraded":"2024-05-17T09:16:48.321155+0000","last_fullsized":"2024-05-17T09:16:48.321155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313745+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:16:47.313745+0000","last_peered":"2024-05-17T09:16:47.313745+0000","last_clean":"2024-05-17T09:16:47.313745+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:16:47.313745+0000","last_undegraded":"2024-05-17T09:16:47.313745+0000","last_fullsized":"2024-05-17T09:16:47.313745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731487+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:16:48.731487+0000","last_peered":"2024-05-17T09:16:48.731487+0000","last_clean":"2024-05-17T09:16:48.731487+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:16:48.731487+0000","last_undegraded":"2024-05-17T09:16:48.731487+0000","last_fullsized":"2024-05-17T09:16:48.731487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319970+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:16:48.319970+0000","last_peered":"2024-05-17T09:16:48.319970+0000","last_clean":"2024-05-17T09:16:48.319970+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:16:48.319970+0000","last_undegraded":"2024-05-17T09:16:48.319970+0000","last_fullsized":"2024-05-17T09:16:48.319970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494016+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:16:47.494016+0000","last_peered":"2024-05-17T09:16:47.494016+0000","last_clean":"2024-05-17T09:16:47.494016+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:16:47.494016+0000","last_undegraded":"2024-05-17T09:16:47.494016+0000","last_fullsized":"2024-05-17T09:16:47.494016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494045+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:16:47.494045+0000","last_peered":"2024-05-17T09:16:47.494045+0000","last_clean":"2024-05-17T09:16:47.494045+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:16:47.494045+0000","last_undegraded":"2024-05-17T09:16:47.494045+0000","last_fullsized":"2024-05-17T09:16:47.494045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859390+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:16:48.859390+0000","last_peered":"2024-05-17T09:16:48.859390+0000","last_clean":"2024-05-17T09:16:48.859390+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:16:48.859390+0000","last_undegraded":"2024-05-17T09:16:48.859390+0000","last_fullsized":"2024-05-17T09:16:48.859390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322303+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:16:48.322303+0000","last_peered":"2024-05-17T09:16:48.322303+0000","last_clean":"2024-05-17T09:16:48.322303+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:16:48.322303+0000","last_undegraded":"2024-05-17T09:16:48.322303+0000","last_fullsized":"2024-05-17T09:16:48.322303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318054+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:16:48.318054+0000","last_peered":"2024-05-17T09:16:48.318054+0000","last_clean":"2024-05-17T09:16:48.318054+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:16:48.318054+0000","last_undegraded":"2024-05-17T09:16:48.318054+0000","last_fullsized":"2024-05-17T09:16:48.318054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319276+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:16:48.319276+0000","last_peered":"2024-05-17T09:16:48.319276+0000","last_clean":"2024-05-17T09:16:48.319276+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:16:48.319276+0000","last_undegraded":"2024-05-17T09:16:48.319276+0000","last_fullsized":"2024-05-17T09:16:48.319276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318651+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318651+0000","last_peered":"2024-05-17T09:16:48.318651+0000","last_clean":"2024-05-17T09:16:48.318651+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318651+0000","last_undegraded":"2024-05-17T09:16:48.318651+0000","last_fullsized":"2024-05-17T09:16:48.318651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317714+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:48.317714+0000","last_peered":"2024-05-17T09:16:48.317714+0000","last_clean":"2024-05-17T09:16:48.317714+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:48.317714+0000","last_undegraded":"2024-05-17T09:16:48.317714+0000","last_fullsized":"2024-05-17T09:16:48.317714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755182+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:16:48.755182+0000","last_peered":"2024-05-17T09:16:48.755182+0000","last_clean":"2024-05-17T09:16:48.755182+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:16:48.755182+0000","last_undegraded":"2024-05-17T09:16:48.755182+0000","last_fullsized":"2024-05-17T09:16:48.755182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730880+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730880+0000","last_peered":"2024-05-17T09:16:48.730880+0000","last_clean":"2024-05-17T09:16:48.730880+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730880+0000","last_undegraded":"2024-05-17T09:16:48.730880+0000","last_fullsized":"2024-05-17T09:16:48.730880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755604+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:16:48.755604+0000","last_peered":"2024-05-17T09:16:48.755604+0000","last_clean":"2024-05-17T09:16:48.755604+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:16:48.755604+0000","last_undegraded":"2024-05-17T09:16:48.755604+0000","last_fullsized":"2024-05-17T09:16:48.755604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494351+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:16:47.494351+0000","last_peered":"2024-05-17T09:16:47.494351+0000","last_clean":"2024-05-17T09:16:47.494351+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:16:47.494351+0000","last_undegraded":"2024-05-17T09:16:47.494351+0000","last_fullsized":"2024-05-17T09:16:47.494351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730914+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.730914+0000","last_peered":"2024-05-17T09:16:48.730914+0000","last_clean":"2024-05-17T09:16:48.730914+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.730914+0000","last_undegraded":"2024-05-17T09:16:48.730914+0000","last_fullsized":"2024-05-17T09:16:48.730914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830320+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830320+0000","last_peered":"2024-05-17T09:16:48.830320+0000","last_clean":"2024-05-17T09:16:48.830320+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830320+0000","last_undegraded":"2024-05-17T09:16:48.830320+0000","last_fullsized":"2024-05-17T09:16:48.830320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859534+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:16:48.859534+0000","last_peered":"2024-05-17T09:16:48.859534+0000","last_clean":"2024-05-17T09:16:48.859534+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:16:48.859534+0000","last_undegraded":"2024-05-17T09:16:48.859534+0000","last_fullsized":"2024-05-17T09:16:48.859534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730836+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730836+0000","last_peered":"2024-05-17T09:16:48.730836+0000","last_clean":"2024-05-17T09:16:48.730836+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730836+0000","last_undegraded":"2024-05-17T09:16:48.730836+0000","last_fullsized":"2024-05-17T09:16:48.730836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829757+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829757+0000","last_peered":"2024-05-17T09:16:48.829757+0000","last_clean":"2024-05-17T09:16:48.829757+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829757+0000","last_undegraded":"2024-05-17T09:16:48.829757+0000","last_fullsized":"2024-05-17T09:16:48.829757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423730+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:16:48.423730+0000","last_peered":"2024-05-17T09:16:48.423730+0000","last_clean":"2024-05-17T09:16:48.423730+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:16:48.423730+0000","last_undegraded":"2024-05-17T09:16:48.423730+0000","last_fullsized":"2024-05-17T09:16:48.423730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319264+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319264+0000","last_peered":"2024-05-17T09:16:48.319264+0000","last_clean":"2024-05-17T09:16:48.319264+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319264+0000","last_undegraded":"2024-05-17T09:16:48.319264+0000","last_fullsized":"2024-05-17T09:16:48.319264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":85,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:50.279050+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:16:50.279050+0000","last_peered":"2024-05-17T09:16:50.279050+0000","last_clean":"2024-05-17T09:16:50.279050+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:16:50.279050+0000","last_undegraded":"2024-05-17T09:16:50.279050+0000","last_fullsized":"2024-05-17T09:16:50.279050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829757+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:16:48.829757+0000","last_peered":"2024-05-17T09:16:48.829757+0000","last_clean":"2024-05-17T09:16:48.829757+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:16:48.829757+0000","last_undegraded":"2024-05-17T09:16:48.829757+0000","last_fullsized":"2024-05-17T09:16:48.829757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319091+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319091+0000","last_peered":"2024-05-17T09:16:48.319091+0000","last_clean":"2024-05-17T09:16:48.319091+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319091+0000","last_undegraded":"2024-05-17T09:16:48.319091+0000","last_fullsized":"2024-05-17T09:16:48.319091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320555+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320555+0000","last_peered":"2024-05-17T09:16:48.320555+0000","last_clean":"2024-05-17T09:16:48.320555+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320555+0000","last_undegraded":"2024-05-17T09:16:48.320555+0000","last_fullsized":"2024-05-17T09:16:48.320555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423621+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:16:48.423621+0000","last_peered":"2024-05-17T09:16:48.423621+0000","last_clean":"2024-05-17T09:16:48.423621+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:16:48.423621+0000","last_undegraded":"2024-05-17T09:16:48.423621+0000","last_fullsized":"2024-05-17T09:16:48.423621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322263+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:16:48.322263+0000","last_peered":"2024-05-17T09:16:48.322263+0000","last_clean":"2024-05-17T09:16:48.322263+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:16:48.322263+0000","last_undegraded":"2024-05-17T09:16:48.322263+0000","last_fullsized":"2024-05-17T09:16:48.322263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320431+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320431+0000","last_peered":"2024-05-17T09:16:48.320431+0000","last_clean":"2024-05-17T09:16:48.320431+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320431+0000","last_undegraded":"2024-05-17T09:16:48.320431+0000","last_fullsized":"2024-05-17T09:16:48.320431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755011+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:16:48.755011+0000","last_peered":"2024-05-17T09:16:48.755011+0000","last_clean":"2024-05-17T09:16:48.755011+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:16:48.755011+0000","last_undegraded":"2024-05-17T09:16:48.755011+0000","last_fullsized":"2024-05-17T09:16:48.755011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494270+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:16:47.494270+0000","last_peered":"2024-05-17T09:16:47.494270+0000","last_clean":"2024-05-17T09:16:47.494270+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:16:47.494270+0000","last_undegraded":"2024-05-17T09:16:47.494270+0000","last_fullsized":"2024-05-17T09:16:47.494270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423103+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423103+0000","last_peered":"2024-05-17T09:16:48.423103+0000","last_clean":"2024-05-17T09:16:48.423103+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423103+0000","last_undegraded":"2024-05-17T09:16:48.423103+0000","last_fullsized":"2024-05-17T09:16:48.423103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317416+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:16:48.317416+0000","last_peered":"2024-05-17T09:16:48.317416+0000","last_clean":"2024-05-17T09:16:48.317416+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:16:48.317416+0000","last_undegraded":"2024-05-17T09:16:48.317416+0000","last_fullsized":"2024-05-17T09:16:48.317416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755023+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755023+0000","last_peered":"2024-05-17T09:16:48.755023+0000","last_clean":"2024-05-17T09:16:48.755023+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755023+0000","last_undegraded":"2024-05-17T09:16:48.755023+0000","last_fullsized":"2024-05-17T09:16:48.755023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320974+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:16:48.320974+0000","last_peered":"2024-05-17T09:16:48.320974+0000","last_clean":"2024-05-17T09:16:48.320974+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:16:48.320974+0000","last_undegraded":"2024-05-17T09:16:48.320974+0000","last_fullsized":"2024-05-17T09:16:48.320974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859982+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:16:48.859982+0000","last_peered":"2024-05-17T09:16:48.859982+0000","last_clean":"2024-05-17T09:16:48.859982+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:16:48.859982+0000","last_undegraded":"2024-05-17T09:16:48.859982+0000","last_fullsized":"2024-05-17T09:16:48.859982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317388+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317388+0000","last_peered":"2024-05-17T09:16:48.317388+0000","last_clean":"2024-05-17T09:16:48.317388+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317388+0000","last_undegraded":"2024-05-17T09:16:48.317388+0000","last_fullsized":"2024-05-17T09:16:48.317388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313691+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:47.313691+0000","last_peered":"2024-05-17T09:16:47.313691+0000","last_clean":"2024-05-17T09:16:47.313691+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:47.313691+0000","last_undegraded":"2024-05-17T09:16:47.313691+0000","last_fullsized":"2024-05-17T09:16:47.313691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494341+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:16:47.494341+0000","last_peered":"2024-05-17T09:16:47.494341+0000","last_clean":"2024-05-17T09:16:47.494341+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:16:47.494341+0000","last_undegraded":"2024-05-17T09:16:47.494341+0000","last_fullsized":"2024-05-17T09:16:47.494341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731308+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731308+0000","last_peered":"2024-05-17T09:16:48.731308+0000","last_clean":"2024-05-17T09:16:48.731308+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731308+0000","last_undegraded":"2024-05-17T09:16:48.731308+0000","last_fullsized":"2024-05-17T09:16:48.731308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423574+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423574+0000","last_peered":"2024-05-17T09:16:48.423574+0000","last_clean":"2024-05-17T09:16:48.423574+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423574+0000","last_undegraded":"2024-05-17T09:16:48.423574+0000","last_fullsized":"2024-05-17T09:16:48.423574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859592+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:16:48.859592+0000","last_peered":"2024-05-17T09:16:48.859592+0000","last_clean":"2024-05-17T09:16:48.859592+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:16:48.859592+0000","last_undegraded":"2024-05-17T09:16:48.859592+0000","last_fullsized":"2024-05-17T09:16:48.859592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320362+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320362+0000","last_peered":"2024-05-17T09:16:48.320362+0000","last_clean":"2024-05-17T09:16:48.320362+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320362+0000","last_undegraded":"2024-05-17T09:16:48.320362+0000","last_fullsized":"2024-05-17T09:16:48.320362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320785+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:16:48.320785+0000","last_peered":"2024-05-17T09:16:48.320785+0000","last_clean":"2024-05-17T09:16:48.320785+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:16:48.320785+0000","last_undegraded":"2024-05-17T09:16:48.320785+0000","last_fullsized":"2024-05-17T09:16:48.320785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":97,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830194+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:16:48.830194+0000","last_peered":"2024-05-17T09:16:48.830194+0000","last_clean":"2024-05-17T09:16:48.830194+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:16:48.830194+0000","last_undegraded":"2024-05-17T09:16:48.830194+0000","last_fullsized":"2024-05-17T09:16:48.830194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423882+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423882+0000","last_peered":"2024-05-17T09:16:48.423882+0000","last_clean":"2024-05-17T09:16:48.423882+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423882+0000","last_undegraded":"2024-05-17T09:16:48.423882+0000","last_fullsized":"2024-05-17T09:16:48.423882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321702+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321702+0000","last_peered":"2024-05-17T09:16:48.321702+0000","last_clean":"2024-05-17T09:16:48.321702+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321702+0000","last_undegraded":"2024-05-17T09:16:48.321702+0000","last_fullsized":"2024-05-17T09:16:48.321702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.596820+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:16:48.596820+0000","last_peered":"2024-05-17T09:16:48.596820+0000","last_clean":"2024-05-17T09:16:48.596820+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:16:48.596820+0000","last_undegraded":"2024-05-17T09:16:48.596820+0000","last_fullsized":"2024-05-17T09:16:48.596820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321819+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321819+0000","last_peered":"2024-05-17T09:16:48.321819+0000","last_clean":"2024-05-17T09:16:48.321819+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321819+0000","last_undegraded":"2024-05-17T09:16:48.321819+0000","last_fullsized":"2024-05-17T09:16:48.321819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423617+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423617+0000","last_peered":"2024-05-17T09:16:48.423617+0000","last_clean":"2024-05-17T09:16:48.423617+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423617+0000","last_undegraded":"2024-05-17T09:16:48.423617+0000","last_fullsized":"2024-05-17T09:16:48.423617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313217+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:16:47.313217+0000","last_peered":"2024-05-17T09:16:47.313217+0000","last_clean":"2024-05-17T09:16:47.313217+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:16:47.313217+0000","last_undegraded":"2024-05-17T09:16:47.313217+0000","last_fullsized":"2024-05-17T09:16:47.313217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829920+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.829920+0000","last_peered":"2024-05-17T09:16:48.829920+0000","last_clean":"2024-05-17T09:16:48.829920+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.829920+0000","last_undegraded":"2024-05-17T09:16:48.829920+0000","last_fullsized":"2024-05-17T09:16:48.829920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:50.278615+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:16:50.278615+0000","last_peered":"2024-05-17T09:16:50.278615+0000","last_clean":"2024-05-17T09:16:50.278615+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:16:50.278615+0000","last_undegraded":"2024-05-17T09:16:50.278615+0000","last_fullsized":"2024-05-17T09:16:50.278615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312906+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:16:47.312906+0000","last_peered":"2024-05-17T09:16:47.312906+0000","last_clean":"2024-05-17T09:16:47.312906+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:16:47.312906+0000","last_undegraded":"2024-05-17T09:16:47.312906+0000","last_fullsized":"2024-05-17T09:16:47.312906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320591+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320591+0000","last_peered":"2024-05-17T09:16:48.320591+0000","last_clean":"2024-05-17T09:16:48.320591+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320591+0000","last_undegraded":"2024-05-17T09:16:48.320591+0000","last_fullsized":"2024-05-17T09:16:48.320591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829568+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829568+0000","last_peered":"2024-05-17T09:16:48.829568+0000","last_clean":"2024-05-17T09:16:48.829568+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829568+0000","last_undegraded":"2024-05-17T09:16:48.829568+0000","last_fullsized":"2024-05-17T09:16:48.829568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423245+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:16:48.423245+0000","last_peered":"2024-05-17T09:16:48.423245+0000","last_clean":"2024-05-17T09:16:48.423245+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:16:48.423245+0000","last_undegraded":"2024-05-17T09:16:48.423245+0000","last_fullsized":"2024-05-17T09:16:48.423245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423178+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423178+0000","last_peered":"2024-05-17T09:16:48.423178+0000","last_clean":"2024-05-17T09:16:48.423178+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423178+0000","last_undegraded":"2024-05-17T09:16:48.423178+0000","last_fullsized":"2024-05-17T09:16:48.423178+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322139+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322139+0000","last_peered":"2024-05-17T09:16:48.322139+0000","last_clean":"2024-05-17T09:16:48.322139+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322139+0000","last_undegraded":"2024-05-17T09:16:48.322139+0000","last_fullsized":"2024-05-17T09:16:48.322139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322131+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:16:48.322131+0000","last_peered":"2024-05-17T09:16:48.322131+0000","last_clean":"2024-05-17T09:16:48.322131+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:16:48.322131+0000","last_undegraded":"2024-05-17T09:16:48.322131+0000","last_fullsized":"2024-05-17T09:16:48.322131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829595+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:16:48.829595+0000","last_peered":"2024-05-17T09:16:48.829595+0000","last_clean":"2024-05-17T09:16:48.829595+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:16:48.829595+0000","last_undegraded":"2024-05-17T09:16:48.829595+0000","last_fullsized":"2024-05-17T09:16:48.829595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317645+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:16:48.317645+0000","last_peered":"2024-05-17T09:16:48.317645+0000","last_clean":"2024-05-17T09:16:48.317645+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:16:48.317645+0000","last_undegraded":"2024-05-17T09:16:48.317645+0000","last_fullsized":"2024-05-17T09:16:48.317645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320891+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:16:48.320891+0000","last_peered":"2024-05-17T09:16:48.320891+0000","last_clean":"2024-05-17T09:16:48.320891+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:16:48.320891+0000","last_undegraded":"2024-05-17T09:16:48.320891+0000","last_fullsized":"2024-05-17T09:16:48.320891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320147+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:16:48.320147+0000","last_peered":"2024-05-17T09:16:48.320147+0000","last_clean":"2024-05-17T09:16:48.320147+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:16:48.320147+0000","last_undegraded":"2024-05-17T09:16:48.320147+0000","last_fullsized":"2024-05-17T09:16:48.320147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313086+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:16:47.313086+0000","last_peered":"2024-05-17T09:16:47.313086+0000","last_clean":"2024-05-17T09:16:47.313086+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:16:47.313086+0000","last_undegraded":"2024-05-17T09:16:47.313086+0000","last_fullsized":"2024-05-17T09:16:47.313086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320212+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320212+0000","last_peered":"2024-05-17T09:16:48.320212+0000","last_clean":"2024-05-17T09:16:48.320212+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320212+0000","last_undegraded":"2024-05-17T09:16:48.320212+0000","last_fullsized":"2024-05-17T09:16:48.320212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423393+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:16:48.423393+0000","last_peered":"2024-05-17T09:16:48.423393+0000","last_clean":"2024-05-17T09:16:48.423393+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:16:48.423393+0000","last_undegraded":"2024-05-17T09:16:48.423393+0000","last_fullsized":"2024-05-17T09:16:48.423393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:51.279458+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:16:51.279458+0000","last_peered":"2024-05-17T09:16:51.279458+0000","last_clean":"2024-05-17T09:16:51.279458+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:16:51.279458+0000","last_undegraded":"2024-05-17T09:16:51.279458+0000","last_fullsized":"2024-05-17T09:16:51.279458+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313524+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:16:47.313524+0000","last_peered":"2024-05-17T09:16:47.313524+0000","last_clean":"2024-05-17T09:16:47.313524+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:16:47.313524+0000","last_undegraded":"2024-05-17T09:16:47.313524+0000","last_fullsized":"2024-05-17T09:16:47.313524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755095+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:16:48.755095+0000","last_peered":"2024-05-17T09:16:48.755095+0000","last_clean":"2024-05-17T09:16:48.755095+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:16:48.755095+0000","last_undegraded":"2024-05-17T09:16:48.755095+0000","last_fullsized":"2024-05-17T09:16:48.755095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731142+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:48.731142+0000","last_peered":"2024-05-17T09:16:48.731142+0000","last_clean":"2024-05-17T09:16:48.731142+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:48.731142+0000","last_undegraded":"2024-05-17T09:16:48.731142+0000","last_fullsized":"2024-05-17T09:16:48.731142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312761+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:47.312761+0000","last_peered":"2024-05-17T09:16:47.312761+0000","last_clean":"2024-05-17T09:16:47.312761+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:47.312761+0000","last_undegraded":"2024-05-17T09:16:47.312761+0000","last_fullsized":"2024-05-17T09:16:47.312761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423792+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423792+0000","last_peered":"2024-05-17T09:16:48.423792+0000","last_clean":"2024-05-17T09:16:48.423792+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423792+0000","last_undegraded":"2024-05-17T09:16:48.423792+0000","last_fullsized":"2024-05-17T09:16:48.423792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322492+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:16:48.322492+0000","last_peered":"2024-05-17T09:16:48.322492+0000","last_clean":"2024-05-17T09:16:48.322492+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:16:48.322492+0000","last_undegraded":"2024-05-17T09:16:48.322492+0000","last_fullsized":"2024-05-17T09:16:48.322492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.312901+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:47.312901+0000","last_peered":"2024-05-17T09:16:47.312901+0000","last_clean":"2024-05-17T09:16:47.312901+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:47.312901+0000","last_undegraded":"2024-05-17T09:16:47.312901+0000","last_fullsized":"2024-05-17T09:16:47.312901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321826+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:16:48.321826+0000","last_peered":"2024-05-17T09:16:48.321826+0000","last_clean":"2024-05-17T09:16:48.321826+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:16:48.321826+0000","last_undegraded":"2024-05-17T09:16:48.321826+0000","last_fullsized":"2024-05-17T09:16:48.321826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830240+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:48.830240+0000","last_peered":"2024-05-17T09:16:48.830240+0000","last_clean":"2024-05-17T09:16:48.830240+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:48.830240+0000","last_undegraded":"2024-05-17T09:16:48.830240+0000","last_fullsized":"2024-05-17T09:16:48.830240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320463+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:16:48.320463+0000","last_peered":"2024-05-17T09:16:48.320463+0000","last_clean":"2024-05-17T09:16:48.320463+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:16:48.320463+0000","last_undegraded":"2024-05-17T09:16:48.320463+0000","last_fullsized":"2024-05-17T09:16:48.320463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.754951+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:16:48.754951+0000","last_peered":"2024-05-17T09:16:48.754951+0000","last_clean":"2024-05-17T09:16:48.754951+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:16:48.754951+0000","last_undegraded":"2024-05-17T09:16:48.754951+0000","last_fullsized":"2024-05-17T09:16:48.754951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318055+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:16:48.318055+0000","last_peered":"2024-05-17T09:16:48.318055+0000","last_clean":"2024-05-17T09:16:48.318055+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:16:48.318055+0000","last_undegraded":"2024-05-17T09:16:48.318055+0000","last_fullsized":"2024-05-17T09:16:48.318055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755552+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:16:48.755552+0000","last_peered":"2024-05-17T09:16:48.755552+0000","last_clean":"2024-05-17T09:16:48.755552+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:16:48.755552+0000","last_undegraded":"2024-05-17T09:16:48.755552+0000","last_fullsized":"2024-05-17T09:16:48.755552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318614+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:16:48.318614+0000","last_peered":"2024-05-17T09:16:48.318614+0000","last_clean":"2024-05-17T09:16:48.318614+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:16:48.318614+0000","last_undegraded":"2024-05-17T09:16:48.318614+0000","last_fullsized":"2024-05-17T09:16:48.318614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731592+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:48.731592+0000","last_peered":"2024-05-17T09:16:48.731592+0000","last_clean":"2024-05-17T09:16:48.731592+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:48.731592+0000","last_undegraded":"2024-05-17T09:16:48.731592+0000","last_fullsized":"2024-05-17T09:16:48.731592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494415+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:16:47.494415+0000","last_peered":"2024-05-17T09:16:47.494415+0000","last_clean":"2024-05-17T09:16:47.494415+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:16:47.494415+0000","last_undegraded":"2024-05-17T09:16:47.494415+0000","last_fullsized":"2024-05-17T09:16:47.494415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731317+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731317+0000","last_peered":"2024-05-17T09:16:48.731317+0000","last_clean":"2024-05-17T09:16:48.731317+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731317+0000","last_undegraded":"2024-05-17T09:16:48.731317+0000","last_fullsized":"2024-05-17T09:16:48.731317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859466+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:48.859466+0000","last_peered":"2024-05-17T09:16:48.859466+0000","last_clean":"2024-05-17T09:16:48.859466+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:48.859466+0000","last_undegraded":"2024-05-17T09:16:48.859466+0000","last_fullsized":"2024-05-17T09:16:48.859466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829604+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:16:48.829604+0000","last_peered":"2024-05-17T09:16:48.829604+0000","last_clean":"2024-05-17T09:16:48.829604+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:16:48.829604+0000","last_undegraded":"2024-05-17T09:16:48.829604+0000","last_fullsized":"2024-05-17T09:16:48.829604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317594+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317594+0000","last_peered":"2024-05-17T09:16:48.317594+0000","last_clean":"2024-05-17T09:16:48.317594+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317594+0000","last_undegraded":"2024-05-17T09:16:48.317594+0000","last_fullsized":"2024-05-17T09:16:48.317594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755106+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:48.755106+0000","last_peered":"2024-05-17T09:16:48.755106+0000","last_clean":"2024-05-17T09:16:48.755106+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:48.755106+0000","last_undegraded":"2024-05-17T09:16:48.755106+0000","last_fullsized":"2024-05-17T09:16:48.755106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321622+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:16:48.321622+0000","last_peered":"2024-05-17T09:16:48.321622+0000","last_clean":"2024-05-17T09:16:48.321622+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:16:48.321622+0000","last_undegraded":"2024-05-17T09:16:48.321622+0000","last_fullsized":"2024-05-17T09:16:48.321622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731084+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731084+0000","last_peered":"2024-05-17T09:16:48.731084+0000","last_clean":"2024-05-17T09:16:48.731084+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731084+0000","last_undegraded":"2024-05-17T09:16:48.731084+0000","last_fullsized":"2024-05-17T09:16:48.731084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318766+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:16:48.318766+0000","last_peered":"2024-05-17T09:16:48.318766+0000","last_clean":"2024-05-17T09:16:48.318766+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:16:48.318766+0000","last_undegraded":"2024-05-17T09:16:48.318766+0000","last_fullsized":"2024-05-17T09:16:48.318766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755538+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:16:48.755538+0000","last_peered":"2024-05-17T09:16:48.755538+0000","last_clean":"2024-05-17T09:16:48.755538+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:16:48.755538+0000","last_undegraded":"2024-05-17T09:16:48.755538+0000","last_fullsized":"2024-05-17T09:16:48.755538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731363+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731363+0000","last_peered":"2024-05-17T09:16:48.731363+0000","last_clean":"2024-05-17T09:16:48.731363+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731363+0000","last_undegraded":"2024-05-17T09:16:48.731363+0000","last_fullsized":"2024-05-17T09:16:48.731363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829859+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:48.829859+0000","last_peered":"2024-05-17T09:16:48.829859+0000","last_clean":"2024-05-17T09:16:48.829859+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:48.829859+0000","last_undegraded":"2024-05-17T09:16:48.829859+0000","last_fullsized":"2024-05-17T09:16:48.829859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313203+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:16:47.313203+0000","last_peered":"2024-05-17T09:16:47.313203+0000","last_clean":"2024-05-17T09:16:47.313203+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:16:47.313203+0000","last_undegraded":"2024-05-17T09:16:47.313203+0000","last_fullsized":"2024-05-17T09:16:47.313203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859390+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859390+0000","last_peered":"2024-05-17T09:16:48.859390+0000","last_clean":"2024-05-17T09:16:48.859390+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859390+0000","last_undegraded":"2024-05-17T09:16:48.859390+0000","last_fullsized":"2024-05-17T09:16:48.859390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319375+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319375+0000","last_peered":"2024-05-17T09:16:48.319375+0000","last_clean":"2024-05-17T09:16:48.319375+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319375+0000","last_undegraded":"2024-05-17T09:16:48.319375+0000","last_fullsized":"2024-05-17T09:16:48.319375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313815+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:16:47.313815+0000","last_peered":"2024-05-17T09:16:47.313815+0000","last_clean":"2024-05-17T09:16:47.313815+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:16:47.313815+0000","last_undegraded":"2024-05-17T09:16:47.313815+0000","last_fullsized":"2024-05-17T09:16:47.313815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494082+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:47.494082+0000","last_peered":"2024-05-17T09:16:47.494082+0000","last_clean":"2024-05-17T09:16:47.494082+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:47.494082+0000","last_undegraded":"2024-05-17T09:16:47.494082+0000","last_fullsized":"2024-05-17T09:16:47.494082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423747+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423747+0000","last_peered":"2024-05-17T09:16:48.423747+0000","last_clean":"2024-05-17T09:16:48.423747+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423747+0000","last_undegraded":"2024-05-17T09:16:48.423747+0000","last_fullsized":"2024-05-17T09:16:48.423747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313435+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:16:47.313435+0000","last_peered":"2024-05-17T09:16:47.313435+0000","last_clean":"2024-05-17T09:16:47.313435+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:16:47.313435+0000","last_undegraded":"2024-05-17T09:16:47.313435+0000","last_fullsized":"2024-05-17T09:16:47.313435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.422986+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.422986+0000","last_peered":"2024-05-17T09:16:48.422986+0000","last_clean":"2024-05-17T09:16:48.422986+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.422986+0000","last_undegraded":"2024-05-17T09:16:48.422986+0000","last_fullsized":"2024-05-17T09:16:48.422986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320485+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320485+0000","last_peered":"2024-05-17T09:16:48.320485+0000","last_clean":"2024-05-17T09:16:48.320485+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320485+0000","last_undegraded":"2024-05-17T09:16:48.320485+0000","last_fullsized":"2024-05-17T09:16:48.320485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.313810+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:16:47.313810+0000","last_peered":"2024-05-17T09:16:47.313810+0000","last_clean":"2024-05-17T09:16:47.313810+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:16:47.313810+0000","last_undegraded":"2024-05-17T09:16:47.313810+0000","last_fullsized":"2024-05-17T09:16:47.313810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321710+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:16:48.321710+0000","last_peered":"2024-05-17T09:16:48.321710+0000","last_clean":"2024-05-17T09:16:48.321710+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:16:48.321710+0000","last_undegraded":"2024-05-17T09:16:48.321710+0000","last_fullsized":"2024-05-17T09:16:48.321710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319218+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:16:48.319218+0000","last_peered":"2024-05-17T09:16:48.319218+0000","last_clean":"2024-05-17T09:16:48.319218+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:16:48.319218+0000","last_undegraded":"2024-05-17T09:16:48.319218+0000","last_fullsized":"2024-05-17T09:16:48.319218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318902+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:16:48.318902+0000","last_peered":"2024-05-17T09:16:48.318902+0000","last_clean":"2024-05-17T09:16:48.318902+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:16:48.318902+0000","last_undegraded":"2024-05-17T09:16:48.318902+0000","last_fullsized":"2024-05-17T09:16:48.318902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T09:16:47.494269+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:16:47.494269+0000","last_peered":"2024-05-17T09:16:47.494269+0000","last_clean":"2024-05-17T09:16:47.494269+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:16:47.494269+0000","last_undegraded":"2024-05-17T09:16:47.494269+0000","last_fullsized":"2024-05-17T09:16:47.494269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320601+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:16:48.320601+0000","last_peered":"2024-05-17T09:16:48.320601+0000","last_clean":"2024-05-17T09:16:48.320601+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:16:48.320601+0000","last_undegraded":"2024-05-17T09:16:48.320601+0000","last_fullsized":"2024-05-17T09:16:48.320601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.831533+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:16:48.831533+0000","last_peered":"2024-05-17T09:16:48.831533+0000","last_clean":"2024-05-17T09:16:48.831533+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:16:48.831533+0000","last_undegraded":"2024-05-17T09:16:48.831533+0000","last_fullsized":"2024-05-17T09:16:48.831533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319237+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:16:48.319237+0000","last_peered":"2024-05-17T09:16:48.319237+0000","last_clean":"2024-05-17T09:16:48.319237+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:16:48.319237+0000","last_undegraded":"2024-05-17T09:16:48.319237+0000","last_fullsized":"2024-05-17T09:16:48.319237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755633+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:16:48.755633+0000","last_peered":"2024-05-17T09:16:48.755633+0000","last_clean":"2024-05-17T09:16:48.755633+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:16:48.755633+0000","last_undegraded":"2024-05-17T09:16:48.755633+0000","last_fullsized":"2024-05-17T09:16:48.755633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423165+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:16:48.423165+0000","last_peered":"2024-05-17T09:16:48.423165+0000","last_clean":"2024-05-17T09:16:48.423165+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:16:48.423165+0000","last_undegraded":"2024-05-17T09:16:48.423165+0000","last_fullsized":"2024-05-17T09:16:48.423165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755177+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:16:48.755177+0000","last_peered":"2024-05-17T09:16:48.755177+0000","last_clean":"2024-05-17T09:16:48.755177+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:16:48.755177+0000","last_undegraded":"2024-05-17T09:16:48.755177+0000","last_fullsized":"2024-05-17T09:16:48.755177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859751+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:16:48.859751+0000","last_peered":"2024-05-17T09:16:48.859751+0000","last_clean":"2024-05-17T09:16:48.859751+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:16:48.859751+0000","last_undegraded":"2024-05-17T09:16:48.859751+0000","last_fullsized":"2024-05-17T09:16:48.859751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320547+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:16:48.320547+0000","last_peered":"2024-05-17T09:16:48.320547+0000","last_clean":"2024-05-17T09:16:48.320547+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:16:48.320547+0000","last_undegraded":"2024-05-17T09:16:48.320547+0000","last_fullsized":"2024-05-17T09:16:48.320547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321990+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:16:48.321990+0000","last_peered":"2024-05-17T09:16:48.321990+0000","last_clean":"2024-05-17T09:16:48.321990+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:16:48.321990+0000","last_undegraded":"2024-05-17T09:16:48.321990+0000","last_fullsized":"2024-05-17T09:16:48.321990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68528,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":219,"num_read_kb":181,"num_write":168,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1170984,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070511,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":505832,"data_compressed":29421,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1102154,"data_compressed":65150,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626495,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":574551,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":502498,"data_compressed":29421,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":576258,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":515546,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1102650,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:16:54.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph status --format=json 2024-05-17T09:16:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:54 smithi019 ceph-mon[31620]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:54 smithi069 ceph-mon[35081]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:54 smithi079 ceph-mon[35502]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:16:55.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:55 smithi069 ceph-mon[35081]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 895 B/s wr, 6 op/s 2024-05-17T09:16:55.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:55 smithi019 ceph-mon[31620]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 895 B/s wr, 6 op/s 2024-05-17T09:16:55.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:55 smithi079 ceph-mon[35502]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 895 B/s wr, 6 op/s 2024-05-17T09:16:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.121+0000 7f2c03df7700 1 -- 172.21.15.19:0/2355480766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc1034b0 msgr2=0x7f2bfc1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.121+0000 7f2c03df7700 1 --2- 172.21.15.19:0/2355480766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc1034b0 0x7f2bfc1038c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80099c0 tx=0x7f2bf8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 -- 172.21.15.19:0/2355480766 shutdown_connections 2024-05-17T09:16:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 --2- 172.21.15.19:0/2355480766 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 0x7f2bfc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 --2- 172.21.15.19:0/2355480766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc1034b0 0x7f2bfc1038c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 --2- 172.21.15.19:0/2355480766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bfc107de0 0x7f2bfc1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 -- 172.21.15.19:0/2355480766 >> 172.21.15.19:0/2355480766 conn(0x7f2bfc0fedb0 msgr2=0x7f2bfc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 -- 172.21.15.19:0/2355480766 shutdown_connections 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.122+0000 7f2c03df7700 1 -- 172.21.15.19:0/2355480766 wait complete. 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 Processor -- start 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 -- start start 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bfc1034b0 0x7f2bfc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 0x7f2bfc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 0x7f2bfc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc113260 con 0x7f2bfc107de0 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc1130e0 con 0x7f2bfc1034b0 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c03df7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc1133e0 con 0x7f2bfc108730 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 0x7f2bfc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 0x7f2bfc19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60690/0 (socket says 172.21.15.19:60690) 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 -- 172.21.15.19:0/1827117113 learned_addr learned my addr 172.21.15.19:0/1827117113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01b93700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bfc1034b0 0x7f2bfc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c02394700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 0x7f2bfc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 -- 172.21.15.19:0/1827117113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 msgr2=0x7f2bfc19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 0x7f2bfc19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 -- 172.21.15.19:0/1827117113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bfc1034b0 msgr2=0x7f2bfc19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bfc1034b0 0x7f2bfc19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c01392700 1 -- 172.21.15.19:0/1827117113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc19fc50 con 0x7f2bfc107de0 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.123+0000 7f2c02394700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 0x7f2bfc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.124+0000 7f2c01392700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 0x7f2bfc19b320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000d980 tx=0x7f2bf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.124+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0017070 con 0x7f2bfc107de0 2024-05-17T09:16:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.124+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc19ff40 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.124+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bf0009dc0 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.124+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000f3c0 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.124+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1a7860 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.125+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc0512b0 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.126+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2bf0009930 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.127+0000 7f2beeffd700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2be80b2910 0x7f2be80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.127+0000 7f2c01b93700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2be80b2910 0x7f2be80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.127+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2bf00e2270 con 0x7f2bfc107de0 2024-05-17T09:16:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.128+0000 7f2c01b93700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2be80b2910 0x7f2be80b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000c00 tx=0x7f2bf8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.133+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2bf00a9a80 con 0x7f2bfc107de0 2024-05-17T09:16:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.434+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2bfc1a7a30 con 0x7f2bfc107de0 2024-05-17T09:16:56.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.434+0000 7f2beeffd700 1 -- 172.21.15.19:0/1827117113 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1495 (secure 0 0 0) 0x7f2bf00a9380 con 0x7f2bfc107de0 2024-05-17T09:16:56.434 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:56.434 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":491,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":12,"num_up_osds":12,"osd_up_since":1715937243,"num_in_osds":12,"osd_in_since":1715937223,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":659130,"bytes_used":357093376,"bytes_avail":1151583334400,"bytes_total":1151940427776,"read_bytes_sec":4734,"write_bytes_sec":895,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":26,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24449},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24463},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24469},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24470},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24487},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24497}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.19:8443/"}},"servicemap":{"epoch":30,"modified":"2024-05-17T09:16:14.885254+0000","services":{}},"progress_events":{}} 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.436+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2be80b2910 msgr2=0x7f2be80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.436+0000 7f2c03df7700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2be80b2910 0x7f2be80b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000c00 tx=0x7f2bf8005fb0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.436+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 msgr2=0x7f2bfc19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.436+0000 7f2c03df7700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 0x7f2bfc19b320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000d980 tx=0x7f2bf000dc90 comp rx=0 tx=0).stop 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 shutdown_connections 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2be80b2910 0x7f2be80b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bfc108730 0x7f2bfc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bfc1034b0 0x7f2bfc19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 --2- 172.21.15.19:0/1827117113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc107de0 0x7f2bfc19b320 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 >> 172.21.15.19:0/1827117113 conn(0x7f2bfc0fedb0 msgr2=0x7f2bfc109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 shutdown_connections 2024-05-17T09:16:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:56.437+0000 7f2c03df7700 1 -- 172.21.15.19:0/1827117113 wait complete. 2024-05-17T09:16:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:56 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1827117113' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-17T09:16:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:56 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1827117113' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-17T09:16:56.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:56 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1827117113' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-17T09:16:57.130 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-17T09:16:57.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:16:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:16:57.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:57 smithi019 ceph-mon[31620]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 747 B/s wr, 6 op/s 2024-05-17T09:16:57.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:57 smithi079 ceph-mon[35502]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 747 B/s wr, 6 op/s 2024-05-17T09:16:57.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:57 smithi069 ceph-mon[35081]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 747 B/s wr, 6 op/s 2024-05-17T09:16:59.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 -- 172.21.15.19:0/1250745351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730103540 msgr2=0x7f1730103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:59.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 --2- 172.21.15.19:0/1250745351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730103540 0x7f1730103950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17200099c0 tx=0x7f1720009cd0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 -- 172.21.15.19:0/1250745351 shutdown_connections 2024-05-17T09:16:59.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 --2- 172.21.15.19:0/1250745351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1730108770 0x7f17301100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 --2- 172.21.15.19:0/1250745351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730103540 0x7f1730103950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 --2- 172.21.15.19:0/1250745351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1730107e20 0x7f1730108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 -- 172.21.15.19:0/1250745351 >> 172.21.15.19:0/1250745351 conn(0x7f1730078df0 msgr2=0x7f1730079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 -- 172.21.15.19:0/1250745351 shutdown_connections 2024-05-17T09:16:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.121+0000 7f17369c9700 1 -- 172.21.15.19:0/1250745351 wait complete. 2024-05-17T09:16:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 Processor -- start 2024-05-17T09:16:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 -- start start 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1730103540 0x7f1730104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 0x7f1730105400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1730108770 0x7f1730105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17301132a0 con 0x7f1730107e20 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1730113120 con 0x7f1730108770 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f17369c9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1730113420 con 0x7f1730103540 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f1727fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 0x7f1730105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f1727fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 0x7f1730105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60698/0 (socket says 172.21.15.19:60698) 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f1727fff700 1 -- 172.21.15.19:0/347420983 learned_addr learned my addr 172.21.15.19:0/347420983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f172ffff700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1730103540 0x7f1730104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.122+0000 7f1734f66700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1730108770 0x7f1730105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f1727fff700 1 -- 172.21.15.19:0/347420983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1730103540 msgr2=0x7f1730104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f1727fff700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1730103540 0x7f1730104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f1727fff700 1 -- 172.21.15.19:0/347420983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1730108770 msgr2=0x7f1730105940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f1727fff700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1730108770 0x7f1730105940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f1727fff700 1 -- 172.21.15.19:0/347420983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301a9340 con 0x7f1730107e20 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f1727fff700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 0x7f1730105400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f171800e970 tx=0x7f171800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718004a30 con 0x7f1730107e20 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f171800cc70 con 0x7f1730107e20 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718005520 con 0x7f1730107e20 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17301a9630 con 0x7f1730107e20 2024-05-17T09:16:59.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.123+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a9b20 con 0x7f1730107e20 2024-05-17T09:16:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.124+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17300512b0 con 0x7f1730107e20 2024-05-17T09:16:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.126+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f171800c7e0 con 0x7f1730107e20 2024-05-17T09:16:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.127+0000 7f172dffb700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f171c0b2910 0x7f171c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:16:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.127+0000 7f172ffff700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f171c0b2910 0x7f171c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:16:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.127+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f17180e2b20 con 0x7f1730107e20 2024-05-17T09:16:59.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.128+0000 7f172ffff700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f171c0b2910 0x7f171c0b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1720003760 tx=0x7f172001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:16:59.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.131+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1718054df0 con 0x7f1730107e20 2024-05-17T09:16:59.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.340+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17301063f0 con 0x7f171c0b2910 2024-05-17T09:16:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.349+0000 7f172dffb700 1 -- 172.21.15.19:0/347420983 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340609 (secure 0 0 0) 0x7f172007b2a0 con 0x7f171c0b2910 2024-05-17T09:16:59.353 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:16:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.352+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f171c0b2910 msgr2=0x7f171c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.352+0000 7f17369c9700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f171c0b2910 0x7f171c0b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1720003760 tx=0x7f172001a040 comp rx=0 tx=0).stop 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.352+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 msgr2=0x7f1730105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.352+0000 7f17369c9700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 0x7f1730105400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f171800e970 tx=0x7f171800ec80 comp rx=0 tx=0).stop 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.352+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 shutdown_connections 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f171c0b2910 0x7f171c0b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1730103540 0x7f1730104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1730108770 0x7f1730105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 --2- 172.21.15.19:0/347420983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1730107e20 0x7f1730105400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:16:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 >> 172.21.15.19:0/347420983 conn(0x7f1730078df0 msgr2=0x7f1730076040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:16:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 shutdown_connections 2024-05-17T09:16:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:16:59.353+0000 7f17369c9700 1 -- 172.21.15.19:0/347420983 wait complete. 2024-05-17T09:16:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:17:00.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:59 smithi019 ceph-mon[31620]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 677 B/s wr, 7 op/s 2024-05-17T09:17:00.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:16:59 smithi019 ceph-mon[31620]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:17:00.058 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-17T09:16:58.899189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659130,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":340,"num_read_kb":270,"num_write":293,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348724,"kb_used_data":11972,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593100,"statfs":{"total":1151940427776,"available":1151583334400,"internally_reserved":0,"allocated":12259328,"data_stored":8134436,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6253,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":47,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.583209"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322463+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:16:48.322463+0000","last_peered":"2024-05-17T09:16:48.322463+0000","last_clean":"2024-05-17T09:16:48.322463+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:16:48.322463+0000","last_undegraded":"2024-05-17T09:16:48.322463+0000","last_fullsized":"2024-05-17T09:16:48.322463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860199+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:16:48.860199+0000","last_peered":"2024-05-17T09:16:48.860199+0000","last_clean":"2024-05-17T09:16:48.860199+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:16:48.860199+0000","last_undegraded":"2024-05-17T09:16:48.860199+0000","last_fullsized":"2024-05-17T09:16:48.860199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860037+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:16:48.860037+0000","last_peered":"2024-05-17T09:16:48.860037+0000","last_clean":"2024-05-17T09:16:48.860037+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:16:48.860037+0000","last_undegraded":"2024-05-17T09:16:48.860037+0000","last_fullsized":"2024-05-17T09:16:48.860037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731397+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:16:48.731397+0000","last_peered":"2024-05-17T09:16:48.731397+0000","last_clean":"2024-05-17T09:16:48.731397+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:16:48.731397+0000","last_undegraded":"2024-05-17T09:16:48.731397+0000","last_fullsized":"2024-05-17T09:16:48.731397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829710+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:16:48.829710+0000","last_peered":"2024-05-17T09:16:48.829710+0000","last_clean":"2024-05-17T09:16:48.829710+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:16:48.829710+0000","last_undegraded":"2024-05-17T09:16:48.829710+0000","last_fullsized":"2024-05-17T09:16:48.829710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320631+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:16:48.320631+0000","last_peered":"2024-05-17T09:16:48.320631+0000","last_clean":"2024-05-17T09:16:48.320631+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:16:48.320631+0000","last_undegraded":"2024-05-17T09:16:48.320631+0000","last_fullsized":"2024-05-17T09:16:48.320631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595280+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:16:48.595280+0000","last_peered":"2024-05-17T09:16:48.595280+0000","last_clean":"2024-05-17T09:16:48.595280+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:16:48.595280+0000","last_undegraded":"2024-05-17T09:16:48.595280+0000","last_fullsized":"2024-05-17T09:16:48.595280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755154+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:16:48.755154+0000","last_peered":"2024-05-17T09:16:48.755154+0000","last_clean":"2024-05-17T09:16:48.755154+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:16:48.755154+0000","last_undegraded":"2024-05-17T09:16:48.755154+0000","last_fullsized":"2024-05-17T09:16:48.755154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860025+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:16:48.860025+0000","last_peered":"2024-05-17T09:16:48.860025+0000","last_clean":"2024-05-17T09:16:48.860025+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:16:48.860025+0000","last_undegraded":"2024-05-17T09:16:48.860025+0000","last_fullsized":"2024-05-17T09:16:48.860025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321155+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:16:48.321155+0000","last_peered":"2024-05-17T09:16:48.321155+0000","last_clean":"2024-05-17T09:16:48.321155+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:16:48.321155+0000","last_undegraded":"2024-05-17T09:16:48.321155+0000","last_fullsized":"2024-05-17T09:16:48.321155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595890+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:16:48.595890+0000","last_peered":"2024-05-17T09:16:48.595890+0000","last_clean":"2024-05-17T09:16:48.595890+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:16:48.595890+0000","last_undegraded":"2024-05-17T09:16:48.595890+0000","last_fullsized":"2024-05-17T09:16:48.595890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731487+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:16:48.731487+0000","last_peered":"2024-05-17T09:16:48.731487+0000","last_clean":"2024-05-17T09:16:48.731487+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:16:48.731487+0000","last_undegraded":"2024-05-17T09:16:48.731487+0000","last_fullsized":"2024-05-17T09:16:48.731487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319970+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:16:48.319970+0000","last_peered":"2024-05-17T09:16:48.319970+0000","last_clean":"2024-05-17T09:16:48.319970+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:16:48.319970+0000","last_undegraded":"2024-05-17T09:16:48.319970+0000","last_fullsized":"2024-05-17T09:16:48.319970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859824+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:16:48.859824+0000","last_peered":"2024-05-17T09:16:48.859824+0000","last_clean":"2024-05-17T09:16:48.859824+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:16:48.859824+0000","last_undegraded":"2024-05-17T09:16:48.859824+0000","last_fullsized":"2024-05-17T09:16:48.859824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859839+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:16:48.859839+0000","last_peered":"2024-05-17T09:16:48.859839+0000","last_clean":"2024-05-17T09:16:48.859839+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:16:48.859839+0000","last_undegraded":"2024-05-17T09:16:48.859839+0000","last_fullsized":"2024-05-17T09:16:48.859839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859390+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:16:48.859390+0000","last_peered":"2024-05-17T09:16:48.859390+0000","last_clean":"2024-05-17T09:16:48.859390+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:16:48.859390+0000","last_undegraded":"2024-05-17T09:16:48.859390+0000","last_fullsized":"2024-05-17T09:16:48.859390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322303+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:16:48.322303+0000","last_peered":"2024-05-17T09:16:48.322303+0000","last_clean":"2024-05-17T09:16:48.322303+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:16:48.322303+0000","last_undegraded":"2024-05-17T09:16:48.322303+0000","last_fullsized":"2024-05-17T09:16:48.322303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318054+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:16:48.318054+0000","last_peered":"2024-05-17T09:16:48.318054+0000","last_clean":"2024-05-17T09:16:48.318054+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:16:48.318054+0000","last_undegraded":"2024-05-17T09:16:48.318054+0000","last_fullsized":"2024-05-17T09:16:48.318054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319276+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:16:48.319276+0000","last_peered":"2024-05-17T09:16:48.319276+0000","last_clean":"2024-05-17T09:16:48.319276+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:16:48.319276+0000","last_undegraded":"2024-05-17T09:16:48.319276+0000","last_fullsized":"2024-05-17T09:16:48.319276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318651+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318651+0000","last_peered":"2024-05-17T09:16:48.318651+0000","last_clean":"2024-05-17T09:16:48.318651+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318651+0000","last_undegraded":"2024-05-17T09:16:48.318651+0000","last_fullsized":"2024-05-17T09:16:48.318651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317714+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:48.317714+0000","last_peered":"2024-05-17T09:16:48.317714+0000","last_clean":"2024-05-17T09:16:48.317714+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:48.317714+0000","last_undegraded":"2024-05-17T09:16:48.317714+0000","last_fullsized":"2024-05-17T09:16:48.317714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755182+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:16:48.755182+0000","last_peered":"2024-05-17T09:16:48.755182+0000","last_clean":"2024-05-17T09:16:48.755182+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:16:48.755182+0000","last_undegraded":"2024-05-17T09:16:48.755182+0000","last_fullsized":"2024-05-17T09:16:48.755182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730880+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730880+0000","last_peered":"2024-05-17T09:16:48.730880+0000","last_clean":"2024-05-17T09:16:48.730880+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730880+0000","last_undegraded":"2024-05-17T09:16:48.730880+0000","last_fullsized":"2024-05-17T09:16:48.730880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755604+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:16:48.755604+0000","last_peered":"2024-05-17T09:16:48.755604+0000","last_clean":"2024-05-17T09:16:48.755604+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:16:48.755604+0000","last_undegraded":"2024-05-17T09:16:48.755604+0000","last_fullsized":"2024-05-17T09:16:48.755604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860196+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:16:48.860196+0000","last_peered":"2024-05-17T09:16:48.860196+0000","last_clean":"2024-05-17T09:16:48.860196+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:16:48.860196+0000","last_undegraded":"2024-05-17T09:16:48.860196+0000","last_fullsized":"2024-05-17T09:16:48.860196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730914+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.730914+0000","last_peered":"2024-05-17T09:16:48.730914+0000","last_clean":"2024-05-17T09:16:48.730914+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.730914+0000","last_undegraded":"2024-05-17T09:16:48.730914+0000","last_fullsized":"2024-05-17T09:16:48.730914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830320+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830320+0000","last_peered":"2024-05-17T09:16:48.830320+0000","last_clean":"2024-05-17T09:16:48.830320+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830320+0000","last_undegraded":"2024-05-17T09:16:48.830320+0000","last_fullsized":"2024-05-17T09:16:48.830320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859534+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:16:48.859534+0000","last_peered":"2024-05-17T09:16:48.859534+0000","last_clean":"2024-05-17T09:16:48.859534+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:16:48.859534+0000","last_undegraded":"2024-05-17T09:16:48.859534+0000","last_fullsized":"2024-05-17T09:16:48.859534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730836+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730836+0000","last_peered":"2024-05-17T09:16:48.730836+0000","last_clean":"2024-05-17T09:16:48.730836+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730836+0000","last_undegraded":"2024-05-17T09:16:48.730836+0000","last_fullsized":"2024-05-17T09:16:48.730836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829757+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829757+0000","last_peered":"2024-05-17T09:16:48.829757+0000","last_clean":"2024-05-17T09:16:48.829757+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829757+0000","last_undegraded":"2024-05-17T09:16:48.829757+0000","last_fullsized":"2024-05-17T09:16:48.829757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423730+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:16:48.423730+0000","last_peered":"2024-05-17T09:16:48.423730+0000","last_clean":"2024-05-17T09:16:48.423730+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:16:48.423730+0000","last_undegraded":"2024-05-17T09:16:48.423730+0000","last_fullsized":"2024-05-17T09:16:48.423730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319264+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319264+0000","last_peered":"2024-05-17T09:16:48.319264+0000","last_clean":"2024-05-17T09:16:48.319264+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319264+0000","last_undegraded":"2024-05-17T09:16:48.319264+0000","last_fullsized":"2024-05-17T09:16:48.319264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:55.287048+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:16:55.287048+0000","last_peered":"2024-05-17T09:16:55.287048+0000","last_clean":"2024-05-17T09:16:55.287048+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:16:55.287048+0000","last_undegraded":"2024-05-17T09:16:55.287048+0000","last_fullsized":"2024-05-17T09:16:55.287048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829757+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:16:48.829757+0000","last_peered":"2024-05-17T09:16:48.829757+0000","last_clean":"2024-05-17T09:16:48.829757+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:16:48.829757+0000","last_undegraded":"2024-05-17T09:16:48.829757+0000","last_fullsized":"2024-05-17T09:16:48.829757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319091+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319091+0000","last_peered":"2024-05-17T09:16:48.319091+0000","last_clean":"2024-05-17T09:16:48.319091+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319091+0000","last_undegraded":"2024-05-17T09:16:48.319091+0000","last_fullsized":"2024-05-17T09:16:48.319091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320555+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320555+0000","last_peered":"2024-05-17T09:16:48.320555+0000","last_clean":"2024-05-17T09:16:48.320555+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320555+0000","last_undegraded":"2024-05-17T09:16:48.320555+0000","last_fullsized":"2024-05-17T09:16:48.320555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423621+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:16:48.423621+0000","last_peered":"2024-05-17T09:16:48.423621+0000","last_clean":"2024-05-17T09:16:48.423621+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:16:48.423621+0000","last_undegraded":"2024-05-17T09:16:48.423621+0000","last_fullsized":"2024-05-17T09:16:48.423621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322263+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:16:48.322263+0000","last_peered":"2024-05-17T09:16:48.322263+0000","last_clean":"2024-05-17T09:16:48.322263+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:16:48.322263+0000","last_undegraded":"2024-05-17T09:16:48.322263+0000","last_fullsized":"2024-05-17T09:16:48.322263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320431+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320431+0000","last_peered":"2024-05-17T09:16:48.320431+0000","last_clean":"2024-05-17T09:16:48.320431+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320431+0000","last_undegraded":"2024-05-17T09:16:48.320431+0000","last_fullsized":"2024-05-17T09:16:48.320431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755011+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:16:48.755011+0000","last_peered":"2024-05-17T09:16:48.755011+0000","last_clean":"2024-05-17T09:16:48.755011+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:16:48.755011+0000","last_undegraded":"2024-05-17T09:16:48.755011+0000","last_fullsized":"2024-05-17T09:16:48.755011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860129+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:16:48.860129+0000","last_peered":"2024-05-17T09:16:48.860129+0000","last_clean":"2024-05-17T09:16:48.860129+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:16:48.860129+0000","last_undegraded":"2024-05-17T09:16:48.860129+0000","last_fullsized":"2024-05-17T09:16:48.860129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423103+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423103+0000","last_peered":"2024-05-17T09:16:48.423103+0000","last_clean":"2024-05-17T09:16:48.423103+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423103+0000","last_undegraded":"2024-05-17T09:16:48.423103+0000","last_fullsized":"2024-05-17T09:16:48.423103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317416+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:16:48.317416+0000","last_peered":"2024-05-17T09:16:48.317416+0000","last_clean":"2024-05-17T09:16:48.317416+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:16:48.317416+0000","last_undegraded":"2024-05-17T09:16:48.317416+0000","last_fullsized":"2024-05-17T09:16:48.317416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755023+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755023+0000","last_peered":"2024-05-17T09:16:48.755023+0000","last_clean":"2024-05-17T09:16:48.755023+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755023+0000","last_undegraded":"2024-05-17T09:16:48.755023+0000","last_fullsized":"2024-05-17T09:16:48.755023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:57.289543+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:16:57.289543+0000","last_peered":"2024-05-17T09:16:57.289543+0000","last_clean":"2024-05-17T09:16:57.289543+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:16:57.289543+0000","last_undegraded":"2024-05-17T09:16:57.289543+0000","last_fullsized":"2024-05-17T09:16:57.289543+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859982+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:16:48.859982+0000","last_peered":"2024-05-17T09:16:48.859982+0000","last_clean":"2024-05-17T09:16:48.859982+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:16:48.859982+0000","last_undegraded":"2024-05-17T09:16:48.859982+0000","last_fullsized":"2024-05-17T09:16:48.859982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317388+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317388+0000","last_peered":"2024-05-17T09:16:48.317388+0000","last_clean":"2024-05-17T09:16:48.317388+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317388+0000","last_undegraded":"2024-05-17T09:16:48.317388+0000","last_fullsized":"2024-05-17T09:16:48.317388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595729+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595729+0000","last_peered":"2024-05-17T09:16:48.595729+0000","last_clean":"2024-05-17T09:16:48.595729+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595729+0000","last_undegraded":"2024-05-17T09:16:48.595729+0000","last_fullsized":"2024-05-17T09:16:48.595729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860152+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:16:48.860152+0000","last_peered":"2024-05-17T09:16:48.860152+0000","last_clean":"2024-05-17T09:16:48.860152+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:16:48.860152+0000","last_undegraded":"2024-05-17T09:16:48.860152+0000","last_fullsized":"2024-05-17T09:16:48.860152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731308+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731308+0000","last_peered":"2024-05-17T09:16:48.731308+0000","last_clean":"2024-05-17T09:16:48.731308+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731308+0000","last_undegraded":"2024-05-17T09:16:48.731308+0000","last_fullsized":"2024-05-17T09:16:48.731308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423574+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423574+0000","last_peered":"2024-05-17T09:16:48.423574+0000","last_clean":"2024-05-17T09:16:48.423574+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423574+0000","last_undegraded":"2024-05-17T09:16:48.423574+0000","last_fullsized":"2024-05-17T09:16:48.423574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859592+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:16:48.859592+0000","last_peered":"2024-05-17T09:16:48.859592+0000","last_clean":"2024-05-17T09:16:48.859592+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:16:48.859592+0000","last_undegraded":"2024-05-17T09:16:48.859592+0000","last_fullsized":"2024-05-17T09:16:48.859592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320362+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320362+0000","last_peered":"2024-05-17T09:16:48.320362+0000","last_clean":"2024-05-17T09:16:48.320362+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320362+0000","last_undegraded":"2024-05-17T09:16:48.320362+0000","last_fullsized":"2024-05-17T09:16:48.320362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":166,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:58.291138+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:16:58.291138+0000","last_peered":"2024-05-17T09:16:58.291138+0000","last_clean":"2024-05-17T09:16:58.291138+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:16:58.291138+0000","last_undegraded":"2024-05-17T09:16:58.291138+0000","last_fullsized":"2024-05-17T09:16:58.291138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":89,"num_read_kb":107,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830194+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:16:48.830194+0000","last_peered":"2024-05-17T09:16:48.830194+0000","last_clean":"2024-05-17T09:16:48.830194+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:16:48.830194+0000","last_undegraded":"2024-05-17T09:16:48.830194+0000","last_fullsized":"2024-05-17T09:16:48.830194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423882+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423882+0000","last_peered":"2024-05-17T09:16:48.423882+0000","last_clean":"2024-05-17T09:16:48.423882+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423882+0000","last_undegraded":"2024-05-17T09:16:48.423882+0000","last_fullsized":"2024-05-17T09:16:48.423882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321702+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321702+0000","last_peered":"2024-05-17T09:16:48.321702+0000","last_clean":"2024-05-17T09:16:48.321702+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321702+0000","last_undegraded":"2024-05-17T09:16:48.321702+0000","last_fullsized":"2024-05-17T09:16:48.321702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.596820+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:16:48.596820+0000","last_peered":"2024-05-17T09:16:48.596820+0000","last_clean":"2024-05-17T09:16:48.596820+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:16:48.596820+0000","last_undegraded":"2024-05-17T09:16:48.596820+0000","last_fullsized":"2024-05-17T09:16:48.596820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321819+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321819+0000","last_peered":"2024-05-17T09:16:48.321819+0000","last_clean":"2024-05-17T09:16:48.321819+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321819+0000","last_undegraded":"2024-05-17T09:16:48.321819+0000","last_fullsized":"2024-05-17T09:16:48.321819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423617+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423617+0000","last_peered":"2024-05-17T09:16:48.423617+0000","last_clean":"2024-05-17T09:16:48.423617+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423617+0000","last_undegraded":"2024-05-17T09:16:48.423617+0000","last_fullsized":"2024-05-17T09:16:48.423617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595473+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:16:48.595473+0000","last_peered":"2024-05-17T09:16:48.595473+0000","last_clean":"2024-05-17T09:16:48.595473+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:16:48.595473+0000","last_undegraded":"2024-05-17T09:16:48.595473+0000","last_fullsized":"2024-05-17T09:16:48.595473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829920+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.829920+0000","last_peered":"2024-05-17T09:16:48.829920+0000","last_clean":"2024-05-17T09:16:48.829920+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.829920+0000","last_undegraded":"2024-05-17T09:16:48.829920+0000","last_fullsized":"2024-05-17T09:16:48.829920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:55.286454+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:16:55.286454+0000","last_peered":"2024-05-17T09:16:55.286454+0000","last_clean":"2024-05-17T09:16:55.286454+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:16:55.286454+0000","last_undegraded":"2024-05-17T09:16:55.286454+0000","last_fullsized":"2024-05-17T09:16:55.286454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595368+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:16:48.595368+0000","last_peered":"2024-05-17T09:16:48.595368+0000","last_clean":"2024-05-17T09:16:48.595368+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:16:48.595368+0000","last_undegraded":"2024-05-17T09:16:48.595368+0000","last_fullsized":"2024-05-17T09:16:48.595368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320591+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320591+0000","last_peered":"2024-05-17T09:16:48.320591+0000","last_clean":"2024-05-17T09:16:48.320591+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320591+0000","last_undegraded":"2024-05-17T09:16:48.320591+0000","last_fullsized":"2024-05-17T09:16:48.320591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829568+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829568+0000","last_peered":"2024-05-17T09:16:48.829568+0000","last_clean":"2024-05-17T09:16:48.829568+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829568+0000","last_undegraded":"2024-05-17T09:16:48.829568+0000","last_fullsized":"2024-05-17T09:16:48.829568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423245+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:16:48.423245+0000","last_peered":"2024-05-17T09:16:48.423245+0000","last_clean":"2024-05-17T09:16:48.423245+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:16:48.423245+0000","last_undegraded":"2024-05-17T09:16:48.423245+0000","last_fullsized":"2024-05-17T09:16:48.423245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423178+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423178+0000","last_peered":"2024-05-17T09:16:48.423178+0000","last_clean":"2024-05-17T09:16:48.423178+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423178+0000","last_undegraded":"2024-05-17T09:16:48.423178+0000","last_fullsized":"2024-05-17T09:16:48.423178+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322139+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322139+0000","last_peered":"2024-05-17T09:16:48.322139+0000","last_clean":"2024-05-17T09:16:48.322139+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322139+0000","last_undegraded":"2024-05-17T09:16:48.322139+0000","last_fullsized":"2024-05-17T09:16:48.322139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322131+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:16:48.322131+0000","last_peered":"2024-05-17T09:16:48.322131+0000","last_clean":"2024-05-17T09:16:48.322131+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:16:48.322131+0000","last_undegraded":"2024-05-17T09:16:48.322131+0000","last_fullsized":"2024-05-17T09:16:48.322131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829595+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:16:48.829595+0000","last_peered":"2024-05-17T09:16:48.829595+0000","last_clean":"2024-05-17T09:16:48.829595+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:16:48.829595+0000","last_undegraded":"2024-05-17T09:16:48.829595+0000","last_fullsized":"2024-05-17T09:16:48.829595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317645+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:16:48.317645+0000","last_peered":"2024-05-17T09:16:48.317645+0000","last_clean":"2024-05-17T09:16:48.317645+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:16:48.317645+0000","last_undegraded":"2024-05-17T09:16:48.317645+0000","last_fullsized":"2024-05-17T09:16:48.317645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320891+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:16:48.320891+0000","last_peered":"2024-05-17T09:16:48.320891+0000","last_clean":"2024-05-17T09:16:48.320891+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:16:48.320891+0000","last_undegraded":"2024-05-17T09:16:48.320891+0000","last_fullsized":"2024-05-17T09:16:48.320891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320147+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:16:48.320147+0000","last_peered":"2024-05-17T09:16:48.320147+0000","last_clean":"2024-05-17T09:16:48.320147+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:16:48.320147+0000","last_undegraded":"2024-05-17T09:16:48.320147+0000","last_fullsized":"2024-05-17T09:16:48.320147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595674+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:16:48.595674+0000","last_peered":"2024-05-17T09:16:48.595674+0000","last_clean":"2024-05-17T09:16:48.595674+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:16:48.595674+0000","last_undegraded":"2024-05-17T09:16:48.595674+0000","last_fullsized":"2024-05-17T09:16:48.595674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320212+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320212+0000","last_peered":"2024-05-17T09:16:48.320212+0000","last_clean":"2024-05-17T09:16:48.320212+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320212+0000","last_undegraded":"2024-05-17T09:16:48.320212+0000","last_fullsized":"2024-05-17T09:16:48.320212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423393+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:16:48.423393+0000","last_peered":"2024-05-17T09:16:48.423393+0000","last_clean":"2024-05-17T09:16:48.423393+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:16:48.423393+0000","last_undegraded":"2024-05-17T09:16:48.423393+0000","last_fullsized":"2024-05-17T09:16:48.423393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":86,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:56.287205+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:16:56.287205+0000","last_peered":"2024-05-17T09:16:56.287205+0000","last_clean":"2024-05-17T09:16:56.287205+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:16:56.287205+0000","last_undegraded":"2024-05-17T09:16:56.287205+0000","last_fullsized":"2024-05-17T09:16:56.287205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595535+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:16:48.595535+0000","last_peered":"2024-05-17T09:16:48.595535+0000","last_clean":"2024-05-17T09:16:48.595535+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:16:48.595535+0000","last_undegraded":"2024-05-17T09:16:48.595535+0000","last_fullsized":"2024-05-17T09:16:48.595535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755095+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:16:48.755095+0000","last_peered":"2024-05-17T09:16:48.755095+0000","last_clean":"2024-05-17T09:16:48.755095+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:16:48.755095+0000","last_undegraded":"2024-05-17T09:16:48.755095+0000","last_fullsized":"2024-05-17T09:16:48.755095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731142+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:48.731142+0000","last_peered":"2024-05-17T09:16:48.731142+0000","last_clean":"2024-05-17T09:16:48.731142+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:48.731142+0000","last_undegraded":"2024-05-17T09:16:48.731142+0000","last_fullsized":"2024-05-17T09:16:48.731142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595149+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595149+0000","last_peered":"2024-05-17T09:16:48.595149+0000","last_clean":"2024-05-17T09:16:48.595149+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595149+0000","last_undegraded":"2024-05-17T09:16:48.595149+0000","last_fullsized":"2024-05-17T09:16:48.595149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423792+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423792+0000","last_peered":"2024-05-17T09:16:48.423792+0000","last_clean":"2024-05-17T09:16:48.423792+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423792+0000","last_undegraded":"2024-05-17T09:16:48.423792+0000","last_fullsized":"2024-05-17T09:16:48.423792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322492+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:16:48.322492+0000","last_peered":"2024-05-17T09:16:48.322492+0000","last_clean":"2024-05-17T09:16:48.322492+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:16:48.322492+0000","last_undegraded":"2024-05-17T09:16:48.322492+0000","last_fullsized":"2024-05-17T09:16:48.322492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595307+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:48.595307+0000","last_peered":"2024-05-17T09:16:48.595307+0000","last_clean":"2024-05-17T09:16:48.595307+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:48.595307+0000","last_undegraded":"2024-05-17T09:16:48.595307+0000","last_fullsized":"2024-05-17T09:16:48.595307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321826+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:16:48.321826+0000","last_peered":"2024-05-17T09:16:48.321826+0000","last_clean":"2024-05-17T09:16:48.321826+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:16:48.321826+0000","last_undegraded":"2024-05-17T09:16:48.321826+0000","last_fullsized":"2024-05-17T09:16:48.321826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830240+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:48.830240+0000","last_peered":"2024-05-17T09:16:48.830240+0000","last_clean":"2024-05-17T09:16:48.830240+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:48.830240+0000","last_undegraded":"2024-05-17T09:16:48.830240+0000","last_fullsized":"2024-05-17T09:16:48.830240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320463+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:16:48.320463+0000","last_peered":"2024-05-17T09:16:48.320463+0000","last_clean":"2024-05-17T09:16:48.320463+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:16:48.320463+0000","last_undegraded":"2024-05-17T09:16:48.320463+0000","last_fullsized":"2024-05-17T09:16:48.320463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.754951+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:16:48.754951+0000","last_peered":"2024-05-17T09:16:48.754951+0000","last_clean":"2024-05-17T09:16:48.754951+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:16:48.754951+0000","last_undegraded":"2024-05-17T09:16:48.754951+0000","last_fullsized":"2024-05-17T09:16:48.754951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318055+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:16:48.318055+0000","last_peered":"2024-05-17T09:16:48.318055+0000","last_clean":"2024-05-17T09:16:48.318055+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:16:48.318055+0000","last_undegraded":"2024-05-17T09:16:48.318055+0000","last_fullsized":"2024-05-17T09:16:48.318055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755552+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:16:48.755552+0000","last_peered":"2024-05-17T09:16:48.755552+0000","last_clean":"2024-05-17T09:16:48.755552+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:16:48.755552+0000","last_undegraded":"2024-05-17T09:16:48.755552+0000","last_fullsized":"2024-05-17T09:16:48.755552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318614+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:16:48.318614+0000","last_peered":"2024-05-17T09:16:48.318614+0000","last_clean":"2024-05-17T09:16:48.318614+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:16:48.318614+0000","last_undegraded":"2024-05-17T09:16:48.318614+0000","last_fullsized":"2024-05-17T09:16:48.318614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731592+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:48.731592+0000","last_peered":"2024-05-17T09:16:48.731592+0000","last_clean":"2024-05-17T09:16:48.731592+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:48.731592+0000","last_undegraded":"2024-05-17T09:16:48.731592+0000","last_fullsized":"2024-05-17T09:16:48.731592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860291+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:16:48.860291+0000","last_peered":"2024-05-17T09:16:48.860291+0000","last_clean":"2024-05-17T09:16:48.860291+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:16:48.860291+0000","last_undegraded":"2024-05-17T09:16:48.860291+0000","last_fullsized":"2024-05-17T09:16:48.860291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731317+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731317+0000","last_peered":"2024-05-17T09:16:48.731317+0000","last_clean":"2024-05-17T09:16:48.731317+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731317+0000","last_undegraded":"2024-05-17T09:16:48.731317+0000","last_fullsized":"2024-05-17T09:16:48.731317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859466+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:48.859466+0000","last_peered":"2024-05-17T09:16:48.859466+0000","last_clean":"2024-05-17T09:16:48.859466+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:48.859466+0000","last_undegraded":"2024-05-17T09:16:48.859466+0000","last_fullsized":"2024-05-17T09:16:48.859466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829604+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:16:48.829604+0000","last_peered":"2024-05-17T09:16:48.829604+0000","last_clean":"2024-05-17T09:16:48.829604+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:16:48.829604+0000","last_undegraded":"2024-05-17T09:16:48.829604+0000","last_fullsized":"2024-05-17T09:16:48.829604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317594+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317594+0000","last_peered":"2024-05-17T09:16:48.317594+0000","last_clean":"2024-05-17T09:16:48.317594+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317594+0000","last_undegraded":"2024-05-17T09:16:48.317594+0000","last_fullsized":"2024-05-17T09:16:48.317594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755106+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:48.755106+0000","last_peered":"2024-05-17T09:16:48.755106+0000","last_clean":"2024-05-17T09:16:48.755106+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:48.755106+0000","last_undegraded":"2024-05-17T09:16:48.755106+0000","last_fullsized":"2024-05-17T09:16:48.755106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321622+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:16:48.321622+0000","last_peered":"2024-05-17T09:16:48.321622+0000","last_clean":"2024-05-17T09:16:48.321622+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:16:48.321622+0000","last_undegraded":"2024-05-17T09:16:48.321622+0000","last_fullsized":"2024-05-17T09:16:48.321622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731084+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731084+0000","last_peered":"2024-05-17T09:16:48.731084+0000","last_clean":"2024-05-17T09:16:48.731084+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731084+0000","last_undegraded":"2024-05-17T09:16:48.731084+0000","last_fullsized":"2024-05-17T09:16:48.731084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318766+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:16:48.318766+0000","last_peered":"2024-05-17T09:16:48.318766+0000","last_clean":"2024-05-17T09:16:48.318766+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:16:48.318766+0000","last_undegraded":"2024-05-17T09:16:48.318766+0000","last_fullsized":"2024-05-17T09:16:48.318766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755538+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:16:48.755538+0000","last_peered":"2024-05-17T09:16:48.755538+0000","last_clean":"2024-05-17T09:16:48.755538+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:16:48.755538+0000","last_undegraded":"2024-05-17T09:16:48.755538+0000","last_fullsized":"2024-05-17T09:16:48.755538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731363+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731363+0000","last_peered":"2024-05-17T09:16:48.731363+0000","last_clean":"2024-05-17T09:16:48.731363+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731363+0000","last_undegraded":"2024-05-17T09:16:48.731363+0000","last_fullsized":"2024-05-17T09:16:48.731363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829859+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:48.829859+0000","last_peered":"2024-05-17T09:16:48.829859+0000","last_clean":"2024-05-17T09:16:48.829859+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:48.829859+0000","last_undegraded":"2024-05-17T09:16:48.829859+0000","last_fullsized":"2024-05-17T09:16:48.829859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595791+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:16:48.595791+0000","last_peered":"2024-05-17T09:16:48.595791+0000","last_clean":"2024-05-17T09:16:48.595791+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:16:48.595791+0000","last_undegraded":"2024-05-17T09:16:48.595791+0000","last_fullsized":"2024-05-17T09:16:48.595791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859390+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859390+0000","last_peered":"2024-05-17T09:16:48.859390+0000","last_clean":"2024-05-17T09:16:48.859390+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859390+0000","last_undegraded":"2024-05-17T09:16:48.859390+0000","last_fullsized":"2024-05-17T09:16:48.859390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319375+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319375+0000","last_peered":"2024-05-17T09:16:48.319375+0000","last_clean":"2024-05-17T09:16:48.319375+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319375+0000","last_undegraded":"2024-05-17T09:16:48.319375+0000","last_fullsized":"2024-05-17T09:16:48.319375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.596054+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:16:48.596054+0000","last_peered":"2024-05-17T09:16:48.596054+0000","last_clean":"2024-05-17T09:16:48.596054+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:16:48.596054+0000","last_undegraded":"2024-05-17T09:16:48.596054+0000","last_fullsized":"2024-05-17T09:16:48.596054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859820+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859820+0000","last_peered":"2024-05-17T09:16:48.859820+0000","last_clean":"2024-05-17T09:16:48.859820+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859820+0000","last_undegraded":"2024-05-17T09:16:48.859820+0000","last_fullsized":"2024-05-17T09:16:48.859820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423747+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423747+0000","last_peered":"2024-05-17T09:16:48.423747+0000","last_clean":"2024-05-17T09:16:48.423747+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423747+0000","last_undegraded":"2024-05-17T09:16:48.423747+0000","last_fullsized":"2024-05-17T09:16:48.423747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595550+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:16:48.595550+0000","last_peered":"2024-05-17T09:16:48.595550+0000","last_clean":"2024-05-17T09:16:48.595550+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:16:48.595550+0000","last_undegraded":"2024-05-17T09:16:48.595550+0000","last_fullsized":"2024-05-17T09:16:48.595550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.422986+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.422986+0000","last_peered":"2024-05-17T09:16:48.422986+0000","last_clean":"2024-05-17T09:16:48.422986+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.422986+0000","last_undegraded":"2024-05-17T09:16:48.422986+0000","last_fullsized":"2024-05-17T09:16:48.422986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320485+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320485+0000","last_peered":"2024-05-17T09:16:48.320485+0000","last_clean":"2024-05-17T09:16:48.320485+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320485+0000","last_undegraded":"2024-05-17T09:16:48.320485+0000","last_fullsized":"2024-05-17T09:16:48.320485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595825+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:16:48.595825+0000","last_peered":"2024-05-17T09:16:48.595825+0000","last_clean":"2024-05-17T09:16:48.595825+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:16:48.595825+0000","last_undegraded":"2024-05-17T09:16:48.595825+0000","last_fullsized":"2024-05-17T09:16:48.595825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321710+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:16:48.321710+0000","last_peered":"2024-05-17T09:16:48.321710+0000","last_clean":"2024-05-17T09:16:48.321710+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:16:48.321710+0000","last_undegraded":"2024-05-17T09:16:48.321710+0000","last_fullsized":"2024-05-17T09:16:48.321710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319218+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:16:48.319218+0000","last_peered":"2024-05-17T09:16:48.319218+0000","last_clean":"2024-05-17T09:16:48.319218+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:16:48.319218+0000","last_undegraded":"2024-05-17T09:16:48.319218+0000","last_fullsized":"2024-05-17T09:16:48.319218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318902+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:16:48.318902+0000","last_peered":"2024-05-17T09:16:48.318902+0000","last_clean":"2024-05-17T09:16:48.318902+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:16:48.318902+0000","last_undegraded":"2024-05-17T09:16:48.318902+0000","last_fullsized":"2024-05-17T09:16:48.318902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860079+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:16:48.860079+0000","last_peered":"2024-05-17T09:16:48.860079+0000","last_clean":"2024-05-17T09:16:48.860079+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:16:48.860079+0000","last_undegraded":"2024-05-17T09:16:48.860079+0000","last_fullsized":"2024-05-17T09:16:48.860079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320601+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:16:48.320601+0000","last_peered":"2024-05-17T09:16:48.320601+0000","last_clean":"2024-05-17T09:16:48.320601+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:16:48.320601+0000","last_undegraded":"2024-05-17T09:16:48.320601+0000","last_fullsized":"2024-05-17T09:16:48.320601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.831533+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:16:48.831533+0000","last_peered":"2024-05-17T09:16:48.831533+0000","last_clean":"2024-05-17T09:16:48.831533+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:16:48.831533+0000","last_undegraded":"2024-05-17T09:16:48.831533+0000","last_fullsized":"2024-05-17T09:16:48.831533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319237+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:16:48.319237+0000","last_peered":"2024-05-17T09:16:48.319237+0000","last_clean":"2024-05-17T09:16:48.319237+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:16:48.319237+0000","last_undegraded":"2024-05-17T09:16:48.319237+0000","last_fullsized":"2024-05-17T09:16:48.319237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755633+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:16:48.755633+0000","last_peered":"2024-05-17T09:16:48.755633+0000","last_clean":"2024-05-17T09:16:48.755633+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:16:48.755633+0000","last_undegraded":"2024-05-17T09:16:48.755633+0000","last_fullsized":"2024-05-17T09:16:48.755633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423165+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:16:48.423165+0000","last_peered":"2024-05-17T09:16:48.423165+0000","last_clean":"2024-05-17T09:16:48.423165+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:16:48.423165+0000","last_undegraded":"2024-05-17T09:16:48.423165+0000","last_fullsized":"2024-05-17T09:16:48.423165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755177+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:16:48.755177+0000","last_peered":"2024-05-17T09:16:48.755177+0000","last_clean":"2024-05-17T09:16:48.755177+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:16:48.755177+0000","last_undegraded":"2024-05-17T09:16:48.755177+0000","last_fullsized":"2024-05-17T09:16:48.755177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859751+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:16:48.859751+0000","last_peered":"2024-05-17T09:16:48.859751+0000","last_clean":"2024-05-17T09:16:48.859751+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:16:48.859751+0000","last_undegraded":"2024-05-17T09:16:48.859751+0000","last_fullsized":"2024-05-17T09:16:48.859751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320547+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:16:48.320547+0000","last_peered":"2024-05-17T09:16:48.320547+0000","last_clean":"2024-05-17T09:16:48.320547+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:16:48.320547+0000","last_undegraded":"2024-05-17T09:16:48.320547+0000","last_fullsized":"2024-05-17T09:16:48.320547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321990+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:16:48.321990+0000","last_peered":"2024-05-17T09:16:48.321990+0000","last_clean":"2024-05-17T09:16:48.321990+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:16:48.321990+0000","last_undegraded":"2024-05-17T09:16:48.321990+0000","last_fullsized":"2024-05-17T09:16:48.321990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68528,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":264,"num_read_kb":206,"num_write":168,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710757,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1170984,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266740,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1111549,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":574551,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":576258,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":515546,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1102650,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:17:00.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:59 smithi079 ceph-mon[35502]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 677 B/s wr, 7 op/s 2024-05-17T09:17:00.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:16:59 smithi079 ceph-mon[35502]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:17:00.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:59 smithi069 ceph-mon[35081]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 677 B/s wr, 7 op/s 2024-05-17T09:17:00.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:16:59 smithi069 ceph-mon[35081]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:17:02.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:01 smithi019 ceph-mon[31620]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:02.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:01 smithi079 ceph-mon[35502]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:01 smithi069 ceph-mon[35081]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:03.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:17:03.335 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:04.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:03 smithi079 ceph-mon[35502]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:04.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:03 smithi019 ceph-mon[31620]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:03 smithi069 ceph-mon[35081]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 -- 172.21.15.19:0/220694135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c107e10 msgr2=0x7fbe4c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 --2- 172.21.15.19:0/220694135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c107e10 0x7fbe4c108220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe400099c0 tx=0x7fbe40009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 -- 172.21.15.19:0/220694135 shutdown_connections 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 --2- 172.21.15.19:0/220694135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 0x7fbe4c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 --2- 172.21.15.19:0/220694135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe4c103530 0x7fbe4c103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 --2- 172.21.15.19:0/220694135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c107e10 0x7fbe4c108220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 -- 172.21.15.19:0/220694135 >> 172.21.15.19:0/220694135 conn(0x7fbe4c078e40 msgr2=0x7fbe4c079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 -- 172.21.15.19:0/220694135 shutdown_connections 2024-05-17T09:17:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.145+0000 7fbe52415700 1 -- 172.21.15.19:0/220694135 wait complete. 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 Processor -- start 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 -- start start 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 0x7fbe4c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe4c107e10 0x7fbe4c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 0x7fbe4c101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c113220 con 0x7fbe4c103530 2024-05-17T09:17:05.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1130a0 con 0x7fbe4c107e10 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe52415700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1133a0 con 0x7fbe4c108760 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 0x7fbe4c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 0x7fbe4c100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43644/0 (socket says 172.21.15.19:43644) 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe4bfff700 1 -- 172.21.15.19:0/2540003416 learned_addr learned my addr 172.21.15.19:0/2540003416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.146+0000 7fbe4b7fe700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe4c107e10 0x7fbe4c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe509b2700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 0x7fbe4c101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe4bfff700 1 -- 172.21.15.19:0/2540003416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 msgr2=0x7fbe4c101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe4bfff700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 0x7fbe4c101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe4bfff700 1 -- 172.21.15.19:0/2540003416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe4c107e10 msgr2=0x7fbe4c100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe4bfff700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe4c107e10 0x7fbe4c100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe4bfff700 1 -- 172.21.15.19:0/2540003416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c1a9160 con 0x7fbe4c103530 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe509b2700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 0x7fbe4c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe4bfff700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 0x7fbe4c100710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00d950 tx=0x7fbe3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c016070 con 0x7fbe4c103530 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe3c009dd0 con 0x7fbe4c103530 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4c1a9450 con 0x7fbe4c103530 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00f3c0 con 0x7fbe4c103530 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.147+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4c1a9960 con 0x7fbe4c103530 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.149+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4c104990 con 0x7fbe4c103530 2024-05-17T09:17:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.150+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbe3c019070 con 0x7fbe4c103530 2024-05-17T09:17:05.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.151+0000 7fbe497fa700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbe340b2910 0x7fbe340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:05.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.151+0000 7fbe4b7fe700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbe340b2910 0x7fbe340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:05.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.151+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbe3c0e23e0 con 0x7fbe4c103530 2024-05-17T09:17:05.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.152+0000 7fbe4b7fe700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbe340b2910 0x7fbe340b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009990 tx=0x7fbe40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:05.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.155+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe3c0a9b90 con 0x7fbe4c103530 2024-05-17T09:17:05.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.371+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe4c101c40 con 0x7fbe340b2910 2024-05-17T09:17:05.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.380+0000 7fbe497fa700 1 -- 172.21.15.19:0/2540003416 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340606 (secure 0 0 0) 0x7fbe4007b290 con 0x7fbe340b2910 2024-05-17T09:17:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:17:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbe340b2910 msgr2=0x7fbe340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbe340b2910 0x7fbe340b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009990 tx=0x7fbe40005fb0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 msgr2=0x7fbe4c100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 0x7fbe4c100710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00d950 tx=0x7fbe3c00dc60 comp rx=0 tx=0).stop 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 shutdown_connections 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbe340b2910 0x7fbe340b4e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe4c108760 0x7fbe4c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe4c103530 0x7fbe4c100710 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 --2- 172.21.15.19:0/2540003416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe4c107e10 0x7fbe4c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 >> 172.21.15.19:0/2540003416 conn(0x7fbe4c078e40 msgr2=0x7fbe4c075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 shutdown_connections 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:05.384+0000 7fbe52415700 1 -- 172.21.15.19:0/2540003416 wait complete. 2024-05-17T09:17:05.387 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:17:06.115 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-05-17T09:17:04.900987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659130,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":370,"num_read_kb":285,"num_write":293,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348724,"kb_used_data":11972,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593100,"statfs":{"total":1151940427776,"available":1151583334400,"internally_reserved":0,"allocated":12259328,"data_stored":8134436,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003861"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322463+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:16:48.322463+0000","last_peered":"2024-05-17T09:16:48.322463+0000","last_clean":"2024-05-17T09:16:48.322463+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:16:48.322463+0000","last_undegraded":"2024-05-17T09:16:48.322463+0000","last_fullsized":"2024-05-17T09:16:48.322463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860199+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:16:48.860199+0000","last_peered":"2024-05-17T09:16:48.860199+0000","last_clean":"2024-05-17T09:16:48.860199+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:16:48.860199+0000","last_undegraded":"2024-05-17T09:16:48.860199+0000","last_fullsized":"2024-05-17T09:16:48.860199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860037+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:16:48.860037+0000","last_peered":"2024-05-17T09:16:48.860037+0000","last_clean":"2024-05-17T09:16:48.860037+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:16:48.860037+0000","last_undegraded":"2024-05-17T09:16:48.860037+0000","last_fullsized":"2024-05-17T09:16:48.860037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731397+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:16:48.731397+0000","last_peered":"2024-05-17T09:16:48.731397+0000","last_clean":"2024-05-17T09:16:48.731397+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:16:48.731397+0000","last_undegraded":"2024-05-17T09:16:48.731397+0000","last_fullsized":"2024-05-17T09:16:48.731397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829710+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:16:48.829710+0000","last_peered":"2024-05-17T09:16:48.829710+0000","last_clean":"2024-05-17T09:16:48.829710+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:16:48.829710+0000","last_undegraded":"2024-05-17T09:16:48.829710+0000","last_fullsized":"2024-05-17T09:16:48.829710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320631+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:16:48.320631+0000","last_peered":"2024-05-17T09:16:48.320631+0000","last_clean":"2024-05-17T09:16:48.320631+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:16:48.320631+0000","last_undegraded":"2024-05-17T09:16:48.320631+0000","last_fullsized":"2024-05-17T09:16:48.320631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595280+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:16:48.595280+0000","last_peered":"2024-05-17T09:16:48.595280+0000","last_clean":"2024-05-17T09:16:48.595280+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:16:48.595280+0000","last_undegraded":"2024-05-17T09:16:48.595280+0000","last_fullsized":"2024-05-17T09:16:48.595280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755154+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:16:48.755154+0000","last_peered":"2024-05-17T09:16:48.755154+0000","last_clean":"2024-05-17T09:16:48.755154+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:16:48.755154+0000","last_undegraded":"2024-05-17T09:16:48.755154+0000","last_fullsized":"2024-05-17T09:16:48.755154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860025+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:16:48.860025+0000","last_peered":"2024-05-17T09:16:48.860025+0000","last_clean":"2024-05-17T09:16:48.860025+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:16:48.860025+0000","last_undegraded":"2024-05-17T09:16:48.860025+0000","last_fullsized":"2024-05-17T09:16:48.860025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321155+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:16:48.321155+0000","last_peered":"2024-05-17T09:16:48.321155+0000","last_clean":"2024-05-17T09:16:48.321155+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:16:48.321155+0000","last_undegraded":"2024-05-17T09:16:48.321155+0000","last_fullsized":"2024-05-17T09:16:48.321155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595890+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:16:48.595890+0000","last_peered":"2024-05-17T09:16:48.595890+0000","last_clean":"2024-05-17T09:16:48.595890+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:16:48.595890+0000","last_undegraded":"2024-05-17T09:16:48.595890+0000","last_fullsized":"2024-05-17T09:16:48.595890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731487+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:16:48.731487+0000","last_peered":"2024-05-17T09:16:48.731487+0000","last_clean":"2024-05-17T09:16:48.731487+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:16:48.731487+0000","last_undegraded":"2024-05-17T09:16:48.731487+0000","last_fullsized":"2024-05-17T09:16:48.731487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319970+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:16:48.319970+0000","last_peered":"2024-05-17T09:16:48.319970+0000","last_clean":"2024-05-17T09:16:48.319970+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:16:48.319970+0000","last_undegraded":"2024-05-17T09:16:48.319970+0000","last_fullsized":"2024-05-17T09:16:48.319970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859824+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:16:48.859824+0000","last_peered":"2024-05-17T09:16:48.859824+0000","last_clean":"2024-05-17T09:16:48.859824+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:16:48.859824+0000","last_undegraded":"2024-05-17T09:16:48.859824+0000","last_fullsized":"2024-05-17T09:16:48.859824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859839+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:16:48.859839+0000","last_peered":"2024-05-17T09:16:48.859839+0000","last_clean":"2024-05-17T09:16:48.859839+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:16:48.859839+0000","last_undegraded":"2024-05-17T09:16:48.859839+0000","last_fullsized":"2024-05-17T09:16:48.859839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859390+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:16:48.859390+0000","last_peered":"2024-05-17T09:16:48.859390+0000","last_clean":"2024-05-17T09:16:48.859390+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:16:48.859390+0000","last_undegraded":"2024-05-17T09:16:48.859390+0000","last_fullsized":"2024-05-17T09:16:48.859390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322303+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:16:48.322303+0000","last_peered":"2024-05-17T09:16:48.322303+0000","last_clean":"2024-05-17T09:16:48.322303+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:16:48.322303+0000","last_undegraded":"2024-05-17T09:16:48.322303+0000","last_fullsized":"2024-05-17T09:16:48.322303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318054+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:16:48.318054+0000","last_peered":"2024-05-17T09:16:48.318054+0000","last_clean":"2024-05-17T09:16:48.318054+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:16:48.318054+0000","last_undegraded":"2024-05-17T09:16:48.318054+0000","last_fullsized":"2024-05-17T09:16:48.318054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319276+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:16:48.319276+0000","last_peered":"2024-05-17T09:16:48.319276+0000","last_clean":"2024-05-17T09:16:48.319276+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:16:48.319276+0000","last_undegraded":"2024-05-17T09:16:48.319276+0000","last_fullsized":"2024-05-17T09:16:48.319276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318651+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318651+0000","last_peered":"2024-05-17T09:16:48.318651+0000","last_clean":"2024-05-17T09:16:48.318651+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318651+0000","last_undegraded":"2024-05-17T09:16:48.318651+0000","last_fullsized":"2024-05-17T09:16:48.318651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317714+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:48.317714+0000","last_peered":"2024-05-17T09:16:48.317714+0000","last_clean":"2024-05-17T09:16:48.317714+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:48.317714+0000","last_undegraded":"2024-05-17T09:16:48.317714+0000","last_fullsized":"2024-05-17T09:16:48.317714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755182+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:16:48.755182+0000","last_peered":"2024-05-17T09:16:48.755182+0000","last_clean":"2024-05-17T09:16:48.755182+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:16:48.755182+0000","last_undegraded":"2024-05-17T09:16:48.755182+0000","last_fullsized":"2024-05-17T09:16:48.755182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730880+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730880+0000","last_peered":"2024-05-17T09:16:48.730880+0000","last_clean":"2024-05-17T09:16:48.730880+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730880+0000","last_undegraded":"2024-05-17T09:16:48.730880+0000","last_fullsized":"2024-05-17T09:16:48.730880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755604+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:16:48.755604+0000","last_peered":"2024-05-17T09:16:48.755604+0000","last_clean":"2024-05-17T09:16:48.755604+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:16:48.755604+0000","last_undegraded":"2024-05-17T09:16:48.755604+0000","last_fullsized":"2024-05-17T09:16:48.755604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860196+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:16:48.860196+0000","last_peered":"2024-05-17T09:16:48.860196+0000","last_clean":"2024-05-17T09:16:48.860196+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:16:48.860196+0000","last_undegraded":"2024-05-17T09:16:48.860196+0000","last_fullsized":"2024-05-17T09:16:48.860196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730914+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.730914+0000","last_peered":"2024-05-17T09:16:48.730914+0000","last_clean":"2024-05-17T09:16:48.730914+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.730914+0000","last_undegraded":"2024-05-17T09:16:48.730914+0000","last_fullsized":"2024-05-17T09:16:48.730914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830320+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830320+0000","last_peered":"2024-05-17T09:16:48.830320+0000","last_clean":"2024-05-17T09:16:48.830320+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830320+0000","last_undegraded":"2024-05-17T09:16:48.830320+0000","last_fullsized":"2024-05-17T09:16:48.830320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859534+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:16:48.859534+0000","last_peered":"2024-05-17T09:16:48.859534+0000","last_clean":"2024-05-17T09:16:48.859534+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:16:48.859534+0000","last_undegraded":"2024-05-17T09:16:48.859534+0000","last_fullsized":"2024-05-17T09:16:48.859534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730836+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730836+0000","last_peered":"2024-05-17T09:16:48.730836+0000","last_clean":"2024-05-17T09:16:48.730836+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730836+0000","last_undegraded":"2024-05-17T09:16:48.730836+0000","last_fullsized":"2024-05-17T09:16:48.730836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829757+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829757+0000","last_peered":"2024-05-17T09:16:48.829757+0000","last_clean":"2024-05-17T09:16:48.829757+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829757+0000","last_undegraded":"2024-05-17T09:16:48.829757+0000","last_fullsized":"2024-05-17T09:16:48.829757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423730+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:16:48.423730+0000","last_peered":"2024-05-17T09:16:48.423730+0000","last_clean":"2024-05-17T09:16:48.423730+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:16:48.423730+0000","last_undegraded":"2024-05-17T09:16:48.423730+0000","last_fullsized":"2024-05-17T09:16:48.423730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319264+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319264+0000","last_peered":"2024-05-17T09:16:48.319264+0000","last_clean":"2024-05-17T09:16:48.319264+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319264+0000","last_undegraded":"2024-05-17T09:16:48.319264+0000","last_fullsized":"2024-05-17T09:16:48.319264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:00.294782+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:17:00.294782+0000","last_peered":"2024-05-17T09:17:00.294782+0000","last_clean":"2024-05-17T09:17:00.294782+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:17:00.294782+0000","last_undegraded":"2024-05-17T09:17:00.294782+0000","last_fullsized":"2024-05-17T09:17:00.294782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829757+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:16:48.829757+0000","last_peered":"2024-05-17T09:16:48.829757+0000","last_clean":"2024-05-17T09:16:48.829757+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:16:48.829757+0000","last_undegraded":"2024-05-17T09:16:48.829757+0000","last_fullsized":"2024-05-17T09:16:48.829757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319091+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319091+0000","last_peered":"2024-05-17T09:16:48.319091+0000","last_clean":"2024-05-17T09:16:48.319091+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319091+0000","last_undegraded":"2024-05-17T09:16:48.319091+0000","last_fullsized":"2024-05-17T09:16:48.319091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320555+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320555+0000","last_peered":"2024-05-17T09:16:48.320555+0000","last_clean":"2024-05-17T09:16:48.320555+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320555+0000","last_undegraded":"2024-05-17T09:16:48.320555+0000","last_fullsized":"2024-05-17T09:16:48.320555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423621+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:16:48.423621+0000","last_peered":"2024-05-17T09:16:48.423621+0000","last_clean":"2024-05-17T09:16:48.423621+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:16:48.423621+0000","last_undegraded":"2024-05-17T09:16:48.423621+0000","last_fullsized":"2024-05-17T09:16:48.423621+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322263+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:16:48.322263+0000","last_peered":"2024-05-17T09:16:48.322263+0000","last_clean":"2024-05-17T09:16:48.322263+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:16:48.322263+0000","last_undegraded":"2024-05-17T09:16:48.322263+0000","last_fullsized":"2024-05-17T09:16:48.322263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320431+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320431+0000","last_peered":"2024-05-17T09:16:48.320431+0000","last_clean":"2024-05-17T09:16:48.320431+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320431+0000","last_undegraded":"2024-05-17T09:16:48.320431+0000","last_fullsized":"2024-05-17T09:16:48.320431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755011+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:16:48.755011+0000","last_peered":"2024-05-17T09:16:48.755011+0000","last_clean":"2024-05-17T09:16:48.755011+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:16:48.755011+0000","last_undegraded":"2024-05-17T09:16:48.755011+0000","last_fullsized":"2024-05-17T09:16:48.755011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860129+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:16:48.860129+0000","last_peered":"2024-05-17T09:16:48.860129+0000","last_clean":"2024-05-17T09:16:48.860129+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:16:48.860129+0000","last_undegraded":"2024-05-17T09:16:48.860129+0000","last_fullsized":"2024-05-17T09:16:48.860129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423103+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423103+0000","last_peered":"2024-05-17T09:16:48.423103+0000","last_clean":"2024-05-17T09:16:48.423103+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423103+0000","last_undegraded":"2024-05-17T09:16:48.423103+0000","last_fullsized":"2024-05-17T09:16:48.423103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317416+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:16:48.317416+0000","last_peered":"2024-05-17T09:16:48.317416+0000","last_clean":"2024-05-17T09:16:48.317416+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:16:48.317416+0000","last_undegraded":"2024-05-17T09:16:48.317416+0000","last_fullsized":"2024-05-17T09:16:48.317416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755023+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755023+0000","last_peered":"2024-05-17T09:16:48.755023+0000","last_clean":"2024-05-17T09:16:48.755023+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755023+0000","last_undegraded":"2024-05-17T09:16:48.755023+0000","last_fullsized":"2024-05-17T09:16:48.755023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:02.297605+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:17:02.297605+0000","last_peered":"2024-05-17T09:17:02.297605+0000","last_clean":"2024-05-17T09:17:02.297605+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:17:02.297605+0000","last_undegraded":"2024-05-17T09:17:02.297605+0000","last_fullsized":"2024-05-17T09:17:02.297605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859982+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:16:48.859982+0000","last_peered":"2024-05-17T09:16:48.859982+0000","last_clean":"2024-05-17T09:16:48.859982+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:16:48.859982+0000","last_undegraded":"2024-05-17T09:16:48.859982+0000","last_fullsized":"2024-05-17T09:16:48.859982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317388+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317388+0000","last_peered":"2024-05-17T09:16:48.317388+0000","last_clean":"2024-05-17T09:16:48.317388+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317388+0000","last_undegraded":"2024-05-17T09:16:48.317388+0000","last_fullsized":"2024-05-17T09:16:48.317388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595729+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595729+0000","last_peered":"2024-05-17T09:16:48.595729+0000","last_clean":"2024-05-17T09:16:48.595729+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595729+0000","last_undegraded":"2024-05-17T09:16:48.595729+0000","last_fullsized":"2024-05-17T09:16:48.595729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860152+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:16:48.860152+0000","last_peered":"2024-05-17T09:16:48.860152+0000","last_clean":"2024-05-17T09:16:48.860152+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:16:48.860152+0000","last_undegraded":"2024-05-17T09:16:48.860152+0000","last_fullsized":"2024-05-17T09:16:48.860152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731308+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731308+0000","last_peered":"2024-05-17T09:16:48.731308+0000","last_clean":"2024-05-17T09:16:48.731308+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731308+0000","last_undegraded":"2024-05-17T09:16:48.731308+0000","last_fullsized":"2024-05-17T09:16:48.731308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423574+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423574+0000","last_peered":"2024-05-17T09:16:48.423574+0000","last_clean":"2024-05-17T09:16:48.423574+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423574+0000","last_undegraded":"2024-05-17T09:16:48.423574+0000","last_fullsized":"2024-05-17T09:16:48.423574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859592+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:16:48.859592+0000","last_peered":"2024-05-17T09:16:48.859592+0000","last_clean":"2024-05-17T09:16:48.859592+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:16:48.859592+0000","last_undegraded":"2024-05-17T09:16:48.859592+0000","last_fullsized":"2024-05-17T09:16:48.859592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320362+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320362+0000","last_peered":"2024-05-17T09:16:48.320362+0000","last_clean":"2024-05-17T09:16:48.320362+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320362+0000","last_undegraded":"2024-05-17T09:16:48.320362+0000","last_fullsized":"2024-05-17T09:16:48.320362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.299284+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:17:03.299284+0000","last_peered":"2024-05-17T09:17:03.299284+0000","last_clean":"2024-05-17T09:17:03.299284+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:17:03.299284+0000","last_undegraded":"2024-05-17T09:17:03.299284+0000","last_fullsized":"2024-05-17T09:17:03.299284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":99,"num_read_kb":112,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830194+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:16:48.830194+0000","last_peered":"2024-05-17T09:16:48.830194+0000","last_clean":"2024-05-17T09:16:48.830194+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:16:48.830194+0000","last_undegraded":"2024-05-17T09:16:48.830194+0000","last_fullsized":"2024-05-17T09:16:48.830194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423882+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423882+0000","last_peered":"2024-05-17T09:16:48.423882+0000","last_clean":"2024-05-17T09:16:48.423882+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423882+0000","last_undegraded":"2024-05-17T09:16:48.423882+0000","last_fullsized":"2024-05-17T09:16:48.423882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321702+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321702+0000","last_peered":"2024-05-17T09:16:48.321702+0000","last_clean":"2024-05-17T09:16:48.321702+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321702+0000","last_undegraded":"2024-05-17T09:16:48.321702+0000","last_fullsized":"2024-05-17T09:16:48.321702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.596820+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:16:48.596820+0000","last_peered":"2024-05-17T09:16:48.596820+0000","last_clean":"2024-05-17T09:16:48.596820+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:16:48.596820+0000","last_undegraded":"2024-05-17T09:16:48.596820+0000","last_fullsized":"2024-05-17T09:16:48.596820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321819+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321819+0000","last_peered":"2024-05-17T09:16:48.321819+0000","last_clean":"2024-05-17T09:16:48.321819+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321819+0000","last_undegraded":"2024-05-17T09:16:48.321819+0000","last_fullsized":"2024-05-17T09:16:48.321819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423617+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423617+0000","last_peered":"2024-05-17T09:16:48.423617+0000","last_clean":"2024-05-17T09:16:48.423617+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423617+0000","last_undegraded":"2024-05-17T09:16:48.423617+0000","last_fullsized":"2024-05-17T09:16:48.423617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595473+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:16:48.595473+0000","last_peered":"2024-05-17T09:16:48.595473+0000","last_clean":"2024-05-17T09:16:48.595473+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:16:48.595473+0000","last_undegraded":"2024-05-17T09:16:48.595473+0000","last_fullsized":"2024-05-17T09:16:48.595473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829920+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.829920+0000","last_peered":"2024-05-17T09:16:48.829920+0000","last_clean":"2024-05-17T09:16:48.829920+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.829920+0000","last_undegraded":"2024-05-17T09:16:48.829920+0000","last_fullsized":"2024-05-17T09:16:48.829920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:00.294203+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:17:00.294203+0000","last_peered":"2024-05-17T09:17:00.294203+0000","last_clean":"2024-05-17T09:17:00.294203+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:17:00.294203+0000","last_undegraded":"2024-05-17T09:17:00.294203+0000","last_fullsized":"2024-05-17T09:17:00.294203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595368+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:16:48.595368+0000","last_peered":"2024-05-17T09:16:48.595368+0000","last_clean":"2024-05-17T09:16:48.595368+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:16:48.595368+0000","last_undegraded":"2024-05-17T09:16:48.595368+0000","last_fullsized":"2024-05-17T09:16:48.595368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320591+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320591+0000","last_peered":"2024-05-17T09:16:48.320591+0000","last_clean":"2024-05-17T09:16:48.320591+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320591+0000","last_undegraded":"2024-05-17T09:16:48.320591+0000","last_fullsized":"2024-05-17T09:16:48.320591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829568+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829568+0000","last_peered":"2024-05-17T09:16:48.829568+0000","last_clean":"2024-05-17T09:16:48.829568+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829568+0000","last_undegraded":"2024-05-17T09:16:48.829568+0000","last_fullsized":"2024-05-17T09:16:48.829568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423245+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:16:48.423245+0000","last_peered":"2024-05-17T09:16:48.423245+0000","last_clean":"2024-05-17T09:16:48.423245+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:16:48.423245+0000","last_undegraded":"2024-05-17T09:16:48.423245+0000","last_fullsized":"2024-05-17T09:16:48.423245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423178+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423178+0000","last_peered":"2024-05-17T09:16:48.423178+0000","last_clean":"2024-05-17T09:16:48.423178+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423178+0000","last_undegraded":"2024-05-17T09:16:48.423178+0000","last_fullsized":"2024-05-17T09:16:48.423178+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322139+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322139+0000","last_peered":"2024-05-17T09:16:48.322139+0000","last_clean":"2024-05-17T09:16:48.322139+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322139+0000","last_undegraded":"2024-05-17T09:16:48.322139+0000","last_fullsized":"2024-05-17T09:16:48.322139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322131+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:16:48.322131+0000","last_peered":"2024-05-17T09:16:48.322131+0000","last_clean":"2024-05-17T09:16:48.322131+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:16:48.322131+0000","last_undegraded":"2024-05-17T09:16:48.322131+0000","last_fullsized":"2024-05-17T09:16:48.322131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829595+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:16:48.829595+0000","last_peered":"2024-05-17T09:16:48.829595+0000","last_clean":"2024-05-17T09:16:48.829595+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:16:48.829595+0000","last_undegraded":"2024-05-17T09:16:48.829595+0000","last_fullsized":"2024-05-17T09:16:48.829595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317645+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:16:48.317645+0000","last_peered":"2024-05-17T09:16:48.317645+0000","last_clean":"2024-05-17T09:16:48.317645+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:16:48.317645+0000","last_undegraded":"2024-05-17T09:16:48.317645+0000","last_fullsized":"2024-05-17T09:16:48.317645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320891+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:16:48.320891+0000","last_peered":"2024-05-17T09:16:48.320891+0000","last_clean":"2024-05-17T09:16:48.320891+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:16:48.320891+0000","last_undegraded":"2024-05-17T09:16:48.320891+0000","last_fullsized":"2024-05-17T09:16:48.320891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320147+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:16:48.320147+0000","last_peered":"2024-05-17T09:16:48.320147+0000","last_clean":"2024-05-17T09:16:48.320147+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:16:48.320147+0000","last_undegraded":"2024-05-17T09:16:48.320147+0000","last_fullsized":"2024-05-17T09:16:48.320147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595674+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:16:48.595674+0000","last_peered":"2024-05-17T09:16:48.595674+0000","last_clean":"2024-05-17T09:16:48.595674+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:16:48.595674+0000","last_undegraded":"2024-05-17T09:16:48.595674+0000","last_fullsized":"2024-05-17T09:16:48.595674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320212+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320212+0000","last_peered":"2024-05-17T09:16:48.320212+0000","last_clean":"2024-05-17T09:16:48.320212+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320212+0000","last_undegraded":"2024-05-17T09:16:48.320212+0000","last_fullsized":"2024-05-17T09:16:48.320212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423393+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:16:48.423393+0000","last_peered":"2024-05-17T09:16:48.423393+0000","last_clean":"2024-05-17T09:16:48.423393+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:16:48.423393+0000","last_undegraded":"2024-05-17T09:16:48.423393+0000","last_fullsized":"2024-05-17T09:16:48.423393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:01.295058+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:17:01.295058+0000","last_peered":"2024-05-17T09:17:01.295058+0000","last_clean":"2024-05-17T09:17:01.295058+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:17:01.295058+0000","last_undegraded":"2024-05-17T09:17:01.295058+0000","last_fullsized":"2024-05-17T09:17:01.295058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595535+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:16:48.595535+0000","last_peered":"2024-05-17T09:16:48.595535+0000","last_clean":"2024-05-17T09:16:48.595535+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:16:48.595535+0000","last_undegraded":"2024-05-17T09:16:48.595535+0000","last_fullsized":"2024-05-17T09:16:48.595535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755095+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:16:48.755095+0000","last_peered":"2024-05-17T09:16:48.755095+0000","last_clean":"2024-05-17T09:16:48.755095+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:16:48.755095+0000","last_undegraded":"2024-05-17T09:16:48.755095+0000","last_fullsized":"2024-05-17T09:16:48.755095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731142+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:48.731142+0000","last_peered":"2024-05-17T09:16:48.731142+0000","last_clean":"2024-05-17T09:16:48.731142+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:48.731142+0000","last_undegraded":"2024-05-17T09:16:48.731142+0000","last_fullsized":"2024-05-17T09:16:48.731142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595149+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595149+0000","last_peered":"2024-05-17T09:16:48.595149+0000","last_clean":"2024-05-17T09:16:48.595149+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595149+0000","last_undegraded":"2024-05-17T09:16:48.595149+0000","last_fullsized":"2024-05-17T09:16:48.595149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423792+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423792+0000","last_peered":"2024-05-17T09:16:48.423792+0000","last_clean":"2024-05-17T09:16:48.423792+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423792+0000","last_undegraded":"2024-05-17T09:16:48.423792+0000","last_fullsized":"2024-05-17T09:16:48.423792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322492+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:16:48.322492+0000","last_peered":"2024-05-17T09:16:48.322492+0000","last_clean":"2024-05-17T09:16:48.322492+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:16:48.322492+0000","last_undegraded":"2024-05-17T09:16:48.322492+0000","last_fullsized":"2024-05-17T09:16:48.322492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595374+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:48.595374+0000","last_peered":"2024-05-17T09:16:48.595374+0000","last_clean":"2024-05-17T09:16:48.595374+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:48.595374+0000","last_undegraded":"2024-05-17T09:16:48.595374+0000","last_fullsized":"2024-05-17T09:16:48.595374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321826+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:16:48.321826+0000","last_peered":"2024-05-17T09:16:48.321826+0000","last_clean":"2024-05-17T09:16:48.321826+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:16:48.321826+0000","last_undegraded":"2024-05-17T09:16:48.321826+0000","last_fullsized":"2024-05-17T09:16:48.321826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830240+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:48.830240+0000","last_peered":"2024-05-17T09:16:48.830240+0000","last_clean":"2024-05-17T09:16:48.830240+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:48.830240+0000","last_undegraded":"2024-05-17T09:16:48.830240+0000","last_fullsized":"2024-05-17T09:16:48.830240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320463+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:16:48.320463+0000","last_peered":"2024-05-17T09:16:48.320463+0000","last_clean":"2024-05-17T09:16:48.320463+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:16:48.320463+0000","last_undegraded":"2024-05-17T09:16:48.320463+0000","last_fullsized":"2024-05-17T09:16:48.320463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.754951+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:16:48.754951+0000","last_peered":"2024-05-17T09:16:48.754951+0000","last_clean":"2024-05-17T09:16:48.754951+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:16:48.754951+0000","last_undegraded":"2024-05-17T09:16:48.754951+0000","last_fullsized":"2024-05-17T09:16:48.754951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318055+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:16:48.318055+0000","last_peered":"2024-05-17T09:16:48.318055+0000","last_clean":"2024-05-17T09:16:48.318055+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:16:48.318055+0000","last_undegraded":"2024-05-17T09:16:48.318055+0000","last_fullsized":"2024-05-17T09:16:48.318055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755552+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:16:48.755552+0000","last_peered":"2024-05-17T09:16:48.755552+0000","last_clean":"2024-05-17T09:16:48.755552+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:16:48.755552+0000","last_undegraded":"2024-05-17T09:16:48.755552+0000","last_fullsized":"2024-05-17T09:16:48.755552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318614+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:16:48.318614+0000","last_peered":"2024-05-17T09:16:48.318614+0000","last_clean":"2024-05-17T09:16:48.318614+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:16:48.318614+0000","last_undegraded":"2024-05-17T09:16:48.318614+0000","last_fullsized":"2024-05-17T09:16:48.318614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731592+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:48.731592+0000","last_peered":"2024-05-17T09:16:48.731592+0000","last_clean":"2024-05-17T09:16:48.731592+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:48.731592+0000","last_undegraded":"2024-05-17T09:16:48.731592+0000","last_fullsized":"2024-05-17T09:16:48.731592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860291+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:16:48.860291+0000","last_peered":"2024-05-17T09:16:48.860291+0000","last_clean":"2024-05-17T09:16:48.860291+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:16:48.860291+0000","last_undegraded":"2024-05-17T09:16:48.860291+0000","last_fullsized":"2024-05-17T09:16:48.860291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731317+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731317+0000","last_peered":"2024-05-17T09:16:48.731317+0000","last_clean":"2024-05-17T09:16:48.731317+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731317+0000","last_undegraded":"2024-05-17T09:16:48.731317+0000","last_fullsized":"2024-05-17T09:16:48.731317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859466+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:48.859466+0000","last_peered":"2024-05-17T09:16:48.859466+0000","last_clean":"2024-05-17T09:16:48.859466+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:48.859466+0000","last_undegraded":"2024-05-17T09:16:48.859466+0000","last_fullsized":"2024-05-17T09:16:48.859466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829604+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:16:48.829604+0000","last_peered":"2024-05-17T09:16:48.829604+0000","last_clean":"2024-05-17T09:16:48.829604+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:16:48.829604+0000","last_undegraded":"2024-05-17T09:16:48.829604+0000","last_fullsized":"2024-05-17T09:16:48.829604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317594+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317594+0000","last_peered":"2024-05-17T09:16:48.317594+0000","last_clean":"2024-05-17T09:16:48.317594+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317594+0000","last_undegraded":"2024-05-17T09:16:48.317594+0000","last_fullsized":"2024-05-17T09:16:48.317594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755106+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:48.755106+0000","last_peered":"2024-05-17T09:16:48.755106+0000","last_clean":"2024-05-17T09:16:48.755106+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:48.755106+0000","last_undegraded":"2024-05-17T09:16:48.755106+0000","last_fullsized":"2024-05-17T09:16:48.755106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321622+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:16:48.321622+0000","last_peered":"2024-05-17T09:16:48.321622+0000","last_clean":"2024-05-17T09:16:48.321622+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:16:48.321622+0000","last_undegraded":"2024-05-17T09:16:48.321622+0000","last_fullsized":"2024-05-17T09:16:48.321622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731084+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731084+0000","last_peered":"2024-05-17T09:16:48.731084+0000","last_clean":"2024-05-17T09:16:48.731084+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731084+0000","last_undegraded":"2024-05-17T09:16:48.731084+0000","last_fullsized":"2024-05-17T09:16:48.731084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318766+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:16:48.318766+0000","last_peered":"2024-05-17T09:16:48.318766+0000","last_clean":"2024-05-17T09:16:48.318766+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:16:48.318766+0000","last_undegraded":"2024-05-17T09:16:48.318766+0000","last_fullsized":"2024-05-17T09:16:48.318766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755538+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:16:48.755538+0000","last_peered":"2024-05-17T09:16:48.755538+0000","last_clean":"2024-05-17T09:16:48.755538+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:16:48.755538+0000","last_undegraded":"2024-05-17T09:16:48.755538+0000","last_fullsized":"2024-05-17T09:16:48.755538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731363+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731363+0000","last_peered":"2024-05-17T09:16:48.731363+0000","last_clean":"2024-05-17T09:16:48.731363+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731363+0000","last_undegraded":"2024-05-17T09:16:48.731363+0000","last_fullsized":"2024-05-17T09:16:48.731363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829859+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:48.829859+0000","last_peered":"2024-05-17T09:16:48.829859+0000","last_clean":"2024-05-17T09:16:48.829859+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:48.829859+0000","last_undegraded":"2024-05-17T09:16:48.829859+0000","last_fullsized":"2024-05-17T09:16:48.829859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595791+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:16:48.595791+0000","last_peered":"2024-05-17T09:16:48.595791+0000","last_clean":"2024-05-17T09:16:48.595791+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:16:48.595791+0000","last_undegraded":"2024-05-17T09:16:48.595791+0000","last_fullsized":"2024-05-17T09:16:48.595791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859390+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859390+0000","last_peered":"2024-05-17T09:16:48.859390+0000","last_clean":"2024-05-17T09:16:48.859390+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859390+0000","last_undegraded":"2024-05-17T09:16:48.859390+0000","last_fullsized":"2024-05-17T09:16:48.859390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319375+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319375+0000","last_peered":"2024-05-17T09:16:48.319375+0000","last_clean":"2024-05-17T09:16:48.319375+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319375+0000","last_undegraded":"2024-05-17T09:16:48.319375+0000","last_fullsized":"2024-05-17T09:16:48.319375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.596054+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:16:48.596054+0000","last_peered":"2024-05-17T09:16:48.596054+0000","last_clean":"2024-05-17T09:16:48.596054+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:16:48.596054+0000","last_undegraded":"2024-05-17T09:16:48.596054+0000","last_fullsized":"2024-05-17T09:16:48.596054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859820+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859820+0000","last_peered":"2024-05-17T09:16:48.859820+0000","last_clean":"2024-05-17T09:16:48.859820+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859820+0000","last_undegraded":"2024-05-17T09:16:48.859820+0000","last_fullsized":"2024-05-17T09:16:48.859820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423747+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423747+0000","last_peered":"2024-05-17T09:16:48.423747+0000","last_clean":"2024-05-17T09:16:48.423747+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423747+0000","last_undegraded":"2024-05-17T09:16:48.423747+0000","last_fullsized":"2024-05-17T09:16:48.423747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595550+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:16:48.595550+0000","last_peered":"2024-05-17T09:16:48.595550+0000","last_clean":"2024-05-17T09:16:48.595550+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:16:48.595550+0000","last_undegraded":"2024-05-17T09:16:48.595550+0000","last_fullsized":"2024-05-17T09:16:48.595550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.422986+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.422986+0000","last_peered":"2024-05-17T09:16:48.422986+0000","last_clean":"2024-05-17T09:16:48.422986+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.422986+0000","last_undegraded":"2024-05-17T09:16:48.422986+0000","last_fullsized":"2024-05-17T09:16:48.422986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320485+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320485+0000","last_peered":"2024-05-17T09:16:48.320485+0000","last_clean":"2024-05-17T09:16:48.320485+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320485+0000","last_undegraded":"2024-05-17T09:16:48.320485+0000","last_fullsized":"2024-05-17T09:16:48.320485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595825+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:16:48.595825+0000","last_peered":"2024-05-17T09:16:48.595825+0000","last_clean":"2024-05-17T09:16:48.595825+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:16:48.595825+0000","last_undegraded":"2024-05-17T09:16:48.595825+0000","last_fullsized":"2024-05-17T09:16:48.595825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321710+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:16:48.321710+0000","last_peered":"2024-05-17T09:16:48.321710+0000","last_clean":"2024-05-17T09:16:48.321710+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:16:48.321710+0000","last_undegraded":"2024-05-17T09:16:48.321710+0000","last_fullsized":"2024-05-17T09:16:48.321710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319218+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:16:48.319218+0000","last_peered":"2024-05-17T09:16:48.319218+0000","last_clean":"2024-05-17T09:16:48.319218+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:16:48.319218+0000","last_undegraded":"2024-05-17T09:16:48.319218+0000","last_fullsized":"2024-05-17T09:16:48.319218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318902+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:16:48.318902+0000","last_peered":"2024-05-17T09:16:48.318902+0000","last_clean":"2024-05-17T09:16:48.318902+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:16:48.318902+0000","last_undegraded":"2024-05-17T09:16:48.318902+0000","last_fullsized":"2024-05-17T09:16:48.318902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860079+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:16:48.860079+0000","last_peered":"2024-05-17T09:16:48.860079+0000","last_clean":"2024-05-17T09:16:48.860079+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:16:48.860079+0000","last_undegraded":"2024-05-17T09:16:48.860079+0000","last_fullsized":"2024-05-17T09:16:48.860079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320601+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:16:48.320601+0000","last_peered":"2024-05-17T09:16:48.320601+0000","last_clean":"2024-05-17T09:16:48.320601+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:16:48.320601+0000","last_undegraded":"2024-05-17T09:16:48.320601+0000","last_fullsized":"2024-05-17T09:16:48.320601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.831533+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:16:48.831533+0000","last_peered":"2024-05-17T09:16:48.831533+0000","last_clean":"2024-05-17T09:16:48.831533+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:16:48.831533+0000","last_undegraded":"2024-05-17T09:16:48.831533+0000","last_fullsized":"2024-05-17T09:16:48.831533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319237+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:16:48.319237+0000","last_peered":"2024-05-17T09:16:48.319237+0000","last_clean":"2024-05-17T09:16:48.319237+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:16:48.319237+0000","last_undegraded":"2024-05-17T09:16:48.319237+0000","last_fullsized":"2024-05-17T09:16:48.319237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755633+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:16:48.755633+0000","last_peered":"2024-05-17T09:16:48.755633+0000","last_clean":"2024-05-17T09:16:48.755633+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:16:48.755633+0000","last_undegraded":"2024-05-17T09:16:48.755633+0000","last_fullsized":"2024-05-17T09:16:48.755633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423165+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:16:48.423165+0000","last_peered":"2024-05-17T09:16:48.423165+0000","last_clean":"2024-05-17T09:16:48.423165+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:16:48.423165+0000","last_undegraded":"2024-05-17T09:16:48.423165+0000","last_fullsized":"2024-05-17T09:16:48.423165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755177+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:16:48.755177+0000","last_peered":"2024-05-17T09:16:48.755177+0000","last_clean":"2024-05-17T09:16:48.755177+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:16:48.755177+0000","last_undegraded":"2024-05-17T09:16:48.755177+0000","last_fullsized":"2024-05-17T09:16:48.755177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859751+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:16:48.859751+0000","last_peered":"2024-05-17T09:16:48.859751+0000","last_clean":"2024-05-17T09:16:48.859751+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:16:48.859751+0000","last_undegraded":"2024-05-17T09:16:48.859751+0000","last_fullsized":"2024-05-17T09:16:48.859751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320547+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:16:48.320547+0000","last_peered":"2024-05-17T09:16:48.320547+0000","last_clean":"2024-05-17T09:16:48.320547+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:16:48.320547+0000","last_undegraded":"2024-05-17T09:16:48.320547+0000","last_fullsized":"2024-05-17T09:16:48.320547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321990+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:16:48.321990+0000","last_peered":"2024-05-17T09:16:48.321990+0000","last_clean":"2024-05-17T09:16:48.321990+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:16:48.321990+0000","last_undegraded":"2024-05-17T09:16:48.321990+0000","last_fullsized":"2024-05-17T09:16:48.321990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68528,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":294,"num_read_kb":221,"num_write":168,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1170984,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1111549,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626497,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":574551,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":576258,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":515546,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1102650,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:17:06.120 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T09:17:06.120 INFO:tasks.cephadm:Setting up client nodes... 2024-05-17T09:17:06.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-17T09:17:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:05 smithi079 ceph-mon[35502]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:06.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:05 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:17:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:05 smithi069 ceph-mon[35081]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:05 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:17:06.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:05 smithi019 ceph-mon[31620]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:06.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:05 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:17:07.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:06 smithi079 ceph-mon[35502]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:17:07.177 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:06 smithi019 ceph-mon[31620]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:17:07.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:06 smithi069 ceph-mon[35081]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:17:08.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:07 smithi019 ceph-mon[31620]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:08.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:07 smithi079 ceph-mon[35502]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:07 smithi069 ceph-mon[35081]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 -- 172.21.15.19:0/3654593392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc105e00 msgr2=0x7f62dc10c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 --2- 172.21.15.19:0/3654593392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc105e00 0x7f62dc10c6b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62d400b0a0 tx=0x7f62d400b3b0 comp rx=0 tx=0).stop 2024-05-17T09:17:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 -- 172.21.15.19:0/3654593392 shutdown_connections 2024-05-17T09:17:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 --2- 172.21.15.19:0/3654593392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc105e00 0x7f62dc10c6b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 --2- 172.21.15.19:0/3654593392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc104330 0x7f62dc104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 --2- 172.21.15.19:0/3654593392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 -- 172.21.15.19:0/3654593392 >> 172.21.15.19:0/3654593392 conn(0x7f62dc0ff9b0 msgr2=0x7f62dc101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 -- 172.21.15.19:0/3654593392 shutdown_connections 2024-05-17T09:17:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.698+0000 7f62e3d45700 1 -- 172.21.15.19:0/3654593392 wait complete. 2024-05-17T09:17:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 Processor -- start 2024-05-17T09:17:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 -- start start 2024-05-17T09:17:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 0x7f62dc077450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:08.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc105e00 0x7f62dc076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc10fc50 con 0x7f62dc104330 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc10fad0 con 0x7f62dc105e00 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.699+0000 7f62e3d45700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc10fdd0 con 0x7f62dc105460 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e1ae1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 0x7f62dc077450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e1ae1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 0x7f62dc077450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43660/0 (socket says 172.21.15.19:43660) 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e1ae1700 1 -- 172.21.15.19:0/353823955 learned_addr learned my addr 172.21.15.19:0/353823955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:53314/0 (socket says 172.21.15.19:53314) 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e22e2700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc105e00 0x7f62dc076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 -- 172.21.15.19:0/353823955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc105e00 msgr2=0x7f62dc076040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc105e00 0x7f62dc076040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 -- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 msgr2=0x7f62dc077450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 0x7f62dc077450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 -- 172.21.15.19:0/353823955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc0767a0 con 0x7f62dc105460 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e1ae1700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 0x7f62dc077450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e22e2700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc105e00 0x7f62dc076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e12e0700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc075b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e9f0 tx=0x7f62cc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00cbd0 con 0x7f62dc105460 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.700+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62dc1a4890 con 0x7f62dc105460 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.701+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62cc00cd70 con 0x7f62dc105460 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.701+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc018680 con 0x7f62dc105460 2024-05-17T09:17:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.701+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62dc1a4dc0 con 0x7f62dc105460 2024-05-17T09:17:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.702+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62dc0512b0 con 0x7f62dc105460 2024-05-17T09:17:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.703+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f62cc010430 con 0x7f62dc105460 2024-05-17T09:17:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.704+0000 7f62d2ffd700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62c80b2910 0x7f62c80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.704+0000 7f62e1ae1700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62c80b2910 0x7f62c80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.704+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62cc0e2980 con 0x7f62dc105460 2024-05-17T09:17:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.705+0000 7f62e1ae1700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62c80b2910 0x7f62c80b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62d8011550 tx=0x7f62d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:08.709+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62cc0aa130 con 0x7f62dc105460 2024-05-17T09:17:09.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.044+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f62dc02e5e0 con 0x7f62dc105460 2024-05-17T09:17:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.052+0000 7f62d2ffd700 1 -- 172.21.15.19:0/353823955 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f62cc0b21b0 con 0x7f62dc105460 2024-05-17T09:17:09.051 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-05-17T09:17:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCVIEdmMZ61AhAAH/qGzTJCwj9k+4h1jPuS8w== 2024-05-17T09:17:09.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62c80b2910 msgr2=0x7f62c80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62c80b2910 0x7f62c80b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62d8011550 tx=0x7f62d8009260 comp rx=0 tx=0).stop 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 msgr2=0x7f62dc075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc075b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e9f0 tx=0x7f62cc00ed00 comp rx=0 tx=0).stop 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 shutdown_connections 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62dc105e00 0x7f62dc076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62dc104330 0x7f62dc077450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62c80b2910 0x7f62c80b4e20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 --2- 172.21.15.19:0/353823955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62dc105460 0x7f62dc075b00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 >> 172.21.15.19:0/353823955 conn(0x7f62dc0ff9b0 msgr2=0x7f62dc108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 shutdown_connections 2024-05-17T09:17:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:09.054+0000 7f62e3d45700 1 -- 172.21.15.19:0/353823955 wait complete. 2024-05-17T09:17:09.881 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:17:09.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-17T09:17:09.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-17T09:17:09.922 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-17T09:17:09.922 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-17T09:17:09.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-17T09:17:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:09 smithi019 ceph-mon[31620]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:09 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/353823955' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T09:17:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:09 smithi019 ceph-mon[31620]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T09:17:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:09 smithi019 ceph-mon[31620]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T09:17:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:10.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:09 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:09 smithi079 ceph-mon[35502]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:09 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/353823955' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T09:17:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:09 smithi079 ceph-mon[35502]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T09:17:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:09 smithi079 ceph-mon[35502]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T09:17:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:09 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:10.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:09 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:09 smithi069 ceph-mon[35081]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:09 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/353823955' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T09:17:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:09 smithi069 ceph-mon[35081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T09:17:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:09 smithi069 ceph-mon[35081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T09:17:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:09 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:10.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:09 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:17:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:17:11.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:10 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:11.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:17:11.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:17:11.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:10 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:17:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:17:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:10 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:17:11.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 -- 172.21.15.19:0/2805136046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944107de0 msgr2=0x7fd9441081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:11.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 --2- 172.21.15.19:0/2805136046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944107de0 0x7fd9441081f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0099c0 tx=0x7fd92c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:11.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 -- 172.21.15.19:0/2805136046 shutdown_connections 2024-05-17T09:17:11.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 --2- 172.21.15.19:0/2805136046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9441034b0 0x7fd9441038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:11.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 --2- 172.21.15.19:0/2805136046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944108730 0x7fd944110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:11.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 --2- 172.21.15.19:0/2805136046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944107de0 0x7fd9441081f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.955+0000 7fd948827700 1 -- 172.21.15.19:0/2805136046 >> 172.21.15.19:0/2805136046 conn(0x7fd9440fede0 msgr2=0x7fd944101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.956+0000 7fd948827700 1 -- 172.21.15.19:0/2805136046 shutdown_connections 2024-05-17T09:17:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.956+0000 7fd948827700 1 -- 172.21.15.19:0/2805136046 wait complete. 2024-05-17T09:17:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 Processor -- start 2024-05-17T09:17:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 -- start start 2024-05-17T09:17:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 0x7fd94419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944107de0 0x7fd94419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd944108730 0x7fd94419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9441131f0 con 0x7fd9441034b0 2024-05-17T09:17:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd944113070 con 0x7fd944107de0 2024-05-17T09:17:11.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd948827700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd944113370 con 0x7fd944108730 2024-05-17T09:17:11.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd94259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 0x7fd94419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:11.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd94259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 0x7fd94419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43676/0 (socket says 172.21.15.19:43676) 2024-05-17T09:17:11.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd94259c700 1 -- 172.21.15.19:0/3327513661 learned_addr learned my addr 172.21.15.19:0/3327513661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:11.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd941d9b700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944107de0 0x7fd94419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:11.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.957+0000 7fd942d9d700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd944108730 0x7fd94419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:11.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd94259c700 1 -- 172.21.15.19:0/3327513661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd944108730 msgr2=0x7fd94419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:11.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd94259c700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd944108730 0x7fd94419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:11.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd94259c700 1 -- 172.21.15.19:0/3327513661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944107de0 msgr2=0x7fd94419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:11.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd94259c700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944107de0 0x7fd94419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:11.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd94259c700 1 -- 172.21.15.19:0/3327513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441a6ee0 con 0x7fd9441034b0 2024-05-17T09:17:11.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd941d9b700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944107de0 0x7fd94419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:11.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd94259c700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 0x7fd94419bb90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd93400d950 tx=0x7fd93400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:11.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934016070 con 0x7fd9441034b0 2024-05-17T09:17:11.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9441a71d0 con 0x7fd9441034b0 2024-05-17T09:17:11.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd934010460 con 0x7fd9441034b0 2024-05-17T09:17:11.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93400f620 con 0x7fd9441034b0 2024-05-17T09:17:11.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.958+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441a7730 con 0x7fd9441034b0 2024-05-17T09:17:11.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.959+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9440512b0 con 0x7fd9441034b0 2024-05-17T09:17:11.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.961+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd934019070 con 0x7fd9441034b0 2024-05-17T09:17:11.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.962+0000 7fd93b7fe700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd9300b2910 0x7fd9300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:11.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.962+0000 7fd941d9b700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd9300b2910 0x7fd9300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:11.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.962+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd9340e2390 con 0x7fd9441034b0 2024-05-17T09:17:11.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.962+0000 7fd941d9b700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd9300b2910 0x7fd9300b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd92c000c00 tx=0x7fd92c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:11.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:11.966+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9340a9ba0 con 0x7fd9441034b0 2024-05-17T09:17:12.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.244+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd94402e5e0 con 0x7fd9441034b0 2024-05-17T09:17:12.244 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:12 smithi019 ceph-mon[31620]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:12.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.250+0000 7fd93b7fe700 1 -- 172.21.15.19:0/3327513661 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288644 (secure 0 0 0) 0x7fd9340a94a0 con 0x7fd9441034b0 2024-05-17T09:17:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:17:12.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.253+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd9300b2910 msgr2=0x7fd9300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:12.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.253+0000 7fd948827700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd9300b2910 0x7fd9300b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd92c000c00 tx=0x7fd92c01a040 comp rx=0 tx=0).stop 2024-05-17T09:17:12.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.253+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 msgr2=0x7fd94419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:12.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.253+0000 7fd948827700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 0x7fd94419bb90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd93400d950 tx=0x7fd93400dd10 comp rx=0 tx=0).stop 2024-05-17T09:17:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 shutdown_connections 2024-05-17T09:17:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd9300b2910 0x7fd9300b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd944108730 0x7fd94419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9441034b0 0x7fd94419bb90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 --2- 172.21.15.19:0/3327513661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944107de0 0x7fd94419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:12.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 >> 172.21.15.19:0/3327513661 conn(0x7fd9440fede0 msgr2=0x7fd944101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:12.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 shutdown_connections 2024-05-17T09:17:12.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:12.254+0000 7fd948827700 1 -- 172.21.15.19:0/3327513661 wait complete. 2024-05-17T09:17:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:12 smithi069 ceph-mon[35081]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:12 smithi079 ceph-mon[35502]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:12.980 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3017687651},{"type":"v1","addr":"172.21.15.19:6801","nonce":3017687651}]},"active_addr":"172.21.15.19:6801/3017687651","active_change":"2024-05-17T09:07:46.771161+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":377361485}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1294883711}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":732951443}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3537872081}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":382523988}]}]} 2024-05-17T09:17:12.986 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-17T09:17:12.986 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-17T09:17:12.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:17:13.239 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:13.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:13 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3327513661' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T09:17:13.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:13 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3327513661' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T09:17:13.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:13 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3327513661' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T09:17:14.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:14 smithi019 ceph-mon[31620]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:14 smithi069 ceph-mon[35081]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:14 smithi079 ceph-mon[35502]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 -- 172.21.15.19:0/3135383356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 msgr2=0x7fddf0109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 --2- 172.21.15.19:0/3135383356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf0109300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdde00099c0 tx=0x7fdde0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 -- 172.21.15.19:0/3135383356 shutdown_connections 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 --2- 172.21.15.19:0/3135383356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0109840 0x7fddf01111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 --2- 172.21.15.19:0/3135383356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0102ad0 0x7fddf0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 --2- 172.21.15.19:0/3135383356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf0109300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 -- 172.21.15.19:0/3135383356 >> 172.21.15.19:0/3135383356 conn(0x7fddf00fe530 msgr2=0x7fddf0100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 -- 172.21.15.19:0/3135383356 shutdown_connections 2024-05-17T09:17:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.952+0000 7fddf84d8700 1 -- 172.21.15.19:0/3135383356 wait complete. 2024-05-17T09:17:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 Processor -- start 2024-05-17T09:17:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 -- start start 2024-05-17T09:17:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0102ad0 0x7fddf019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0109840 0x7fddf019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0114300 con 0x7fddf0108ef0 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0114180 con 0x7fddf0109840 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.953+0000 7fddf84d8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0114480 con 0x7fddf0102ad0 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43100/0 (socket says 172.21.15.19:43100) 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 -- 172.21.15.19:0/1586306262 learned_addr learned my addr 172.21.15.19:0/1586306262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf6274700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0102ad0 0x7fddf019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf6a75700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0109840 0x7fddf019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 -- 172.21.15.19:0/1586306262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0102ad0 msgr2=0x7fddf019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0102ad0 0x7fddf019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 -- 172.21.15.19:0/1586306262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0109840 msgr2=0x7fddf019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0109840 0x7fddf019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 -- 172.21.15.19:0/1586306262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf019fb30 con 0x7fddf0108ef0 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf6274700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0102ad0 0x7fddf019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf6a75700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0109840 0x7fddf019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf5a73700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf019b2b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdde000b580 tx=0x7fdde000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0027070 con 0x7fddf0108ef0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf019fdc0 con 0x7fddf0108ef0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.954+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdde000fc20 con 0x7fddf0108ef0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.955+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0017410 con 0x7fddf0108ef0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.955+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf01a02f0 con 0x7fddf0108ef0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.955+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf00512b0 con 0x7fddf0108ef0 2024-05-17T09:17:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.957+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdde002a070 con 0x7fddf0108ef0 2024-05-17T09:17:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.958+0000 7fdde77fe700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdddc0b2910 0x7fdddc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.958+0000 7fddf6274700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdddc0b2910 0x7fdddc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.958+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdde00e3c50 con 0x7fddf0108ef0 2024-05-17T09:17:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.958+0000 7fddf6274700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdddc0b2910 0x7fdddc0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fddec005fd0 tx=0x7fddec005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:14.962+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdde0031020 con 0x7fddf0108ef0 2024-05-17T09:17:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.179+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddf002e5e0 con 0x7fddf0108ef0 2024-05-17T09:17:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.181+0000 7fdde77fe700 1 -- 172.21.15.19:0/1586306262 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7fdde0013070 con 0x7fddf0108ef0 2024-05-17T09:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdddc0b2910 msgr2=0x7fdddc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdddc0b2910 0x7fdddc0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fddec005fd0 tx=0x7fddec005e30 comp rx=0 tx=0).stop 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 msgr2=0x7fddf019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf019b2b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdde000b580 tx=0x7fdde000f740 comp rx=0 tx=0).stop 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 shutdown_connections 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdddc0b2910 0x7fdddc0b4e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf0102ad0 0x7fddf019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddf0109840 0x7fddf019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.183+0000 7fddf84d8700 1 --2- 172.21.15.19:0/1586306262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddf0108ef0 0x7fddf019b2b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.184+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 >> 172.21.15.19:0/1586306262 conn(0x7fddf00fe530 msgr2=0x7fddf0107190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.184+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 shutdown_connections 2024-05-17T09:17:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:15.184+0000 7fddf84d8700 1 -- 172.21.15.19:0/1586306262 wait complete. 2024-05-17T09:17:15.859 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-17T09:17:15.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:17:16.121 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:16.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:16 smithi019 ceph-mon[31620]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:16.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:16 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1586306262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:17:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:16 smithi069 ceph-mon[35081]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:16 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1586306262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:17:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:16 smithi079 ceph-mon[35502]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:16 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1586306262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 -- 172.21.15.19:0/4221230610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4108760 msgr2=0x7fbad41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 --2- 172.21.15.19:0/4221230610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4108760 0x7fbad41100c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbac40099c0 tx=0x7fbac4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 -- 172.21.15.19:0/4221230610 shutdown_connections 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 --2- 172.21.15.19:0/4221230610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4103530 0x7fbad4103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 --2- 172.21.15.19:0/4221230610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4108760 0x7fbad41100c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 --2- 172.21.15.19:0/4221230610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad4107e10 0x7fbad4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 -- 172.21.15.19:0/4221230610 >> 172.21.15.19:0/4221230610 conn(0x7fbad4078e40 msgr2=0x7fbad4079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 -- 172.21.15.19:0/4221230610 shutdown_connections 2024-05-17T09:17:17.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.940+0000 7fbad97d9700 1 -- 172.21.15.19:0/4221230610 wait complete. 2024-05-17T09:17:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.941+0000 7fbad97d9700 1 Processor -- start 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.941+0000 7fbad97d9700 1 -- start start 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad97d9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4103530 0x7fbad419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad97d9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 0x7fbad419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad97d9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad4108760 0x7fbad419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad97d9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4113240 con 0x7fbad4107e10 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad97d9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41130c0 con 0x7fbad4108760 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad97d9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41133c0 con 0x7fbad4103530 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 0x7fbad419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 0x7fbad419c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43106/0 (socket says 172.21.15.19:43106) 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4103530 0x7fbad419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 -- 172.21.15.19:0/227988385 learned_addr learned my addr 172.21.15.19:0/227988385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad37fe700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad4108760 0x7fbad419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 -- 172.21.15.19:0/227988385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4103530 msgr2=0x7fbad419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4103530 0x7fbad419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 -- 172.21.15.19:0/227988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad4108760 msgr2=0x7fbad419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad4108760 0x7fbad419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.942+0000 7fbad27fc700 1 -- 172.21.15.19:0/227988385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41a6ed0 con 0x7fbad4107e10 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.943+0000 7fbad27fc700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 0x7fbad419c0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e970 tx=0x7fbabc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.943+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc004a30 con 0x7fbad4107e10 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.943+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41a71c0 con 0x7fbad4107e10 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.943+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbabc00cc70 con 0x7fbad4107e10 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.943+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc005520 con 0x7fbad4107e10 2024-05-17T09:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.943+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a7750 con 0x7fbad4107e10 2024-05-17T09:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.944+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad40512b0 con 0x7fbad4107e10 2024-05-17T09:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.946+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbabc00c7e0 con 0x7fbad4107e10 2024-05-17T09:17:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.946+0000 7fbacbfff700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbac00b2910 0x7fbac00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.947+0000 7fbad2ffd700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbac00b2910 0x7fbac00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.947+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbabc0e2850 con 0x7fbad4107e10 2024-05-17T09:17:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.947+0000 7fbad2ffd700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbac00b2910 0x7fbac00b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009550 tx=0x7fbacc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:17.951+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbabc00ce20 con 0x7fbad4107e10 2024-05-17T09:17:18.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.174+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad4075ff0 con 0x7fbad4107e10 2024-05-17T09:17:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.175+0000 7fbacbfff700 1 -- 172.21.15.19:0/227988385 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7fbabc0aa060 con 0x7fbad4107e10 2024-05-17T09:17:18.175 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:17:18.175 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbac00b2910 msgr2=0x7fbac00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbac00b2910 0x7fbac00b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009550 tx=0x7fbacc0091b0 comp rx=0 tx=0).stop 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 msgr2=0x7fbad419c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 0x7fbad419c0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e970 tx=0x7fbabc00ec80 comp rx=0 tx=0).stop 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 shutdown_connections 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbac00b2910 0x7fbac00b4e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4103530 0x7fbad419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad4108760 0x7fbad419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 --2- 172.21.15.19:0/227988385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbad4107e10 0x7fbad419c0a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 >> 172.21.15.19:0/227988385 conn(0x7fbad4078e40 msgr2=0x7fbad4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 shutdown_connections 2024-05-17T09:17:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:18.178+0000 7fbad97d9700 1 -- 172.21.15.19:0/227988385 wait complete. 2024-05-17T09:17:18.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:18 smithi019 ceph-mon[31620]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:18 smithi069 ceph-mon[35081]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:18 smithi079 ceph-mon[35502]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:18.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-17T09:17:18.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-17T09:17:18.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-17T09:17:18.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-17T09:17:18.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-17T09:17:18.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-17T09:17:18.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-17T09:17:18.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-17T09:17:18.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-17T09:17:18.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-17T09:17:18.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-17T09:17:18.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-17T09:17:19.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:19 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/227988385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:17:19.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:19 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/227988385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:17:19.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:19 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/227988385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:17:19.527 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.353 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.384 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:20.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:20 smithi069 ceph-mon[35081]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:20.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:20 smithi019 ceph-mon[31620]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:20 smithi079 ceph-mon[35502]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:20.828 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:22.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:22 smithi019 ceph-mon[31620]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:22.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:22 smithi069 ceph-mon[35081]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:22 smithi079 ceph-mon[35502]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 -- 172.21.15.19:0/1478024254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb01034d0 msgr2=0x7fdcb01038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 --2- 172.21.15.19:0/1478024254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb01034d0 0x7fdcb01038e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdca00099c0 tx=0x7fdca0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 -- 172.21.15.19:0/1478024254 shutdown_connections 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 --2- 172.21.15.19:0/1478024254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb0108750 0x7fdcb01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 --2- 172.21.15.19:0/1478024254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb01034d0 0x7fdcb01038e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 --2- 172.21.15.19:0/1478024254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0107e00 0x7fdcb0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 -- 172.21.15.19:0/1478024254 >> 172.21.15.19:0/1478024254 conn(0x7fdcb00fef20 msgr2=0x7fdcb0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 -- 172.21.15.19:0/1478024254 shutdown_connections 2024-05-17T09:17:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.963+0000 7fdcb6634700 1 -- 172.21.15.19:0/1478024254 wait complete. 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 Processor -- start 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 -- start start 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb01034d0 0x7fdcb00ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb0107e00 0x7fdcb00ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb0108750 0x7fdcb00ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0113230 con 0x7fdcb0107e00 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01130b0 con 0x7fdcb0108750 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb6634700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01133b0 con 0x7fdcb01034d0 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb0107e00 0x7fdcb00ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb0107e00 0x7fdcb00ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43134/0 (socket says 172.21.15.19:43134) 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdca7fff700 1 -- 172.21.15.19:0/787066456 learned_addr learned my addr 172.21.15.19:0/787066456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.964+0000 7fdcb4bd1700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb0108750 0x7fdcb00ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcaffff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb01034d0 0x7fdcb00ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdca7fff700 1 -- 172.21.15.19:0/787066456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb01034d0 msgr2=0x7fdcb00ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdca7fff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb01034d0 0x7fdcb00ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdca7fff700 1 -- 172.21.15.19:0/787066456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb0108750 msgr2=0x7fdcb00ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdca7fff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb0108750 0x7fdcb00ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdca7fff700 1 -- 172.21.15.19:0/787066456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb01a9000 con 0x7fdcb0107e00 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcaffff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb01034d0 0x7fdcb00ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdca7fff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb0107e00 0x7fdcb00ff940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d980 tx=0x7fdc9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98017070 con 0x7fdcb0107e00 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb01a92f0 con 0x7fdcb0107e00 2024-05-17T09:17:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc98009dc0 con 0x7fdcb0107e00 2024-05-17T09:17:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800f3c0 con 0x7fdcb0107e00 2024-05-17T09:17:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.965+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb01a9830 con 0x7fdcb0107e00 2024-05-17T09:17:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.966+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcb0063ec0 con 0x7fdcb0107e00 2024-05-17T09:17:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.968+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdc98009930 con 0x7fdcb0107e00 2024-05-17T09:17:22.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcadffb700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc9c0b2910 0x7fdc9c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:22.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcaffff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc9c0b2910 0x7fdc9c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:22.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdc980e2670 con 0x7fdcb0107e00 2024-05-17T09:17:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcadffb700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] conn(0x7fdc9c0bdc40 0x7fdc9c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 --> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc9800d130 con 0x7fdc9c0bdc40 2024-05-17T09:17:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fdc980e2ac0 con 0x7fdcb0107e00 2024-05-17T09:17:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcb4bd1700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] conn(0x7fdc9c0bdc40 0x7fdc9c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.969+0000 7fdcaffff700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc9c0b2910 0x7fdc9c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.970+0000 7fdcb4bd1700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] conn(0x7fdc9c0bdc40 0x7fdc9c0c0130 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:22.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.971+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== osd.0 v2:172.21.15.19:6802/3945774453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc9800d130 con 0x7fdc9c0bdc40 2024-05-17T09:17:22.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.988+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 --> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcb00642c0 con 0x7fdc9c0bdc40 2024-05-17T09:17:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcadffb700 1 -- 172.21.15.19:0/787066456 <== osd.0 v2:172.21.15.19:6802/3945774453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdcb00642c0 con 0x7fdc9c0bdc40 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] conn(0x7fdc9c0bdc40 msgr2=0x7fdc9c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] conn(0x7fdc9c0bdc40 0x7fdc9c0c0130 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc9c0b2910 msgr2=0x7fdc9c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc9c0b2910 0x7fdc9c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0005940 comp rx=0 tx=0).stop 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb0107e00 msgr2=0x7fdcb00ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 --2- 172.21.15.19:0/787066456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcb0107e00 0x7fdcb00ff940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d980 tx=0x7fdc9800dc90 comp rx=0 tx=0).stop 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb4bd1700 1 -- 172.21.15.19:0/787066456 reap_dead start 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 shutdown_connections 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 >> 172.21.15.19:0/787066456 conn(0x7fdcb00fef20 msgr2=0x7fdcb006c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 shutdown_connections 2024-05-17T09:17:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:22.989+0000 7fdcb6634700 1 -- 172.21.15.19:0/787066456 wait complete. 2024-05-17T09:17:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:24 smithi069 ceph-mon[35081]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:24.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:24 smithi019 ceph-mon[31620]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:24 smithi079 ceph-mon[35502]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:26.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:26 smithi019 ceph-mon[31620]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:26.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:26 smithi069 ceph-mon[35081]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:26 smithi079 ceph-mon[35502]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:28.186 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:28 smithi019 ceph-mon[31620]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 -- 172.21.15.19:0/175359100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810a980 msgr2=0x7f6c4810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 --2- 172.21.15.19:0/175359100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810a980 0x7f6c4810ad90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009990 tx=0x7f6c3c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 -- 172.21.15.19:0/175359100 shutdown_connections 2024-05-17T09:17:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 --2- 172.21.15.19:0/175359100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c48110960 0x7f6c48110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 --2- 172.21.15.19:0/175359100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c4810b2d0 0x7f6c48117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 --2- 172.21.15.19:0/175359100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810a980 0x7f6c4810ad90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 -- 172.21.15.19:0/175359100 >> 172.21.15.19:0/175359100 conn(0x7f6c48077c70 msgr2=0x7f6c480780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 -- 172.21.15.19:0/175359100 shutdown_connections 2024-05-17T09:17:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 -- 172.21.15.19:0/175359100 wait complete. 2024-05-17T09:17:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.217+0000 7f6c4d577700 1 Processor -- start 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 -- start start 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c4810a980 0x7f6c481a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810b2d0 0x7f6c481a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c48110960 0x7f6c481a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4811a980 con 0x7f6c4810b2d0 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4811a800 con 0x7f6c4810a980 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c4d577700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4811ab00 con 0x7f6c48110960 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c467fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810b2d0 0x7f6c481a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c467fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810b2d0 0x7f6c481a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47274/0 (socket says 172.21.15.19:47274) 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c467fc700 1 -- 172.21.15.19:0/2569571239 learned_addr learned my addr 172.21.15.19:0/2569571239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c477fe700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c48110960 0x7f6c481a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c46ffd700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c4810a980 0x7f6c481a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.218+0000 7f6c467fc700 1 -- 172.21.15.19:0/2569571239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c48110960 msgr2=0x7f6c481a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c467fc700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c48110960 0x7f6c481a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c467fc700 1 -- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c4810a980 msgr2=0x7f6c481a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c467fc700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c4810a980 0x7f6c481a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c467fc700 1 -- 172.21.15.19:0/2569571239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c481a6280 con 0x7f6c4810b2d0 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c467fc700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810b2d0 0x7f6c481a1950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0047b0 tx=0x7f6c3c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c026070 con 0x7f6c4810b2d0 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c481a6510 con 0x7f6c4810b2d0 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c3c021410 con 0x7f6c4810b2d0 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c017650 con 0x7f6c4810b2d0 2024-05-17T09:17:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.219+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c481a69f0 con 0x7f6c4810b2d0 2024-05-17T09:17:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.220+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c48063f40 con 0x7f6c4810b2d0 2024-05-17T09:17:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.222+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6c3c029070 con 0x7f6c4810b2d0 2024-05-17T09:17:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.222+0000 7f6c2ffff700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c300b2910 0x7f6c300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c46ffd700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c300b2910 0x7f6c300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6c3c0e4200 con 0x7f6c4810b2d0 2024-05-17T09:17:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c2ffff700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] conn(0x7f6c300bdc40 0x7f6c300c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 --> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c300c0800 con 0x7f6c300bdc40 2024-05-17T09:17:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c0e4650 con 0x7f6c4810b2d0 2024-05-17T09:17:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c46ffd700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c300b2910 0x7f6c300b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c4807d0e0 tx=0x7f6c38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.223+0000 7f6c477fe700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] conn(0x7f6c300bdc40 0x7f6c300c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.224+0000 7f6c477fe700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] conn(0x7f6c300bdc40 0x7f6c300c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.225+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== osd.10 v2:172.21.15.69:6824/3630021153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c300c0800 con 0x7f6c300bdc40 2024-05-17T09:17:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.241+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 --> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c48064340 con 0x7f6c300bdc40 2024-05-17T09:17:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c2ffff700 1 -- 172.21.15.19:0/2569571239 <== osd.10 v2:172.21.15.69:6824/3630021153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6c48064340 con 0x7f6c300bdc40 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] conn(0x7f6c300bdc40 msgr2=0x7f6c300c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] conn(0x7f6c300bdc40 0x7f6c300c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c300b2910 msgr2=0x7f6c300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6c300b2910 0x7f6c300b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c4807d0e0 tx=0x7f6c38009260 comp rx=0 tx=0).stop 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810b2d0 msgr2=0x7f6c481a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 --2- 172.21.15.19:0/2569571239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4810b2d0 0x7f6c481a1950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0047b0 tx=0x7f6c3c009510 comp rx=0 tx=0).stop 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c477fe700 1 -- 172.21.15.19:0/2569571239 reap_dead start 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 shutdown_connections 2024-05-17T09:17:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 >> 172.21.15.19:0/2569571239 conn(0x7f6c48077c70 msgr2=0x7f6c4807b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 shutdown_connections 2024-05-17T09:17:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:28.242+0000 7f6c4d577700 1 -- 172.21.15.19:0/2569571239 wait complete. 2024-05-17T09:17:28.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:28 smithi069 ceph-mon[35081]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:28 smithi079 ceph-mon[35502]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:28.947 INFO:teuthology.orchestra.run.smithi019.stdout:274877906992 2024-05-17T09:17:28.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-17T09:17:29.506 INFO:teuthology.orchestra.run.smithi019.stdout:34359738467 2024-05-17T09:17:29.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-17T09:17:30.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:30 smithi019 ceph-mon[31620]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 -- 172.21.15.19:0/2738703066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb320108730 msgr2=0x7fb320110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 --2- 172.21.15.19:0/2738703066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb320108730 0x7fb320110090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb3180099c0 tx=0x7fb318009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 -- 172.21.15.19:0/2738703066 shutdown_connections 2024-05-17T09:17:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 --2- 172.21.15.19:0/2738703066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3201034b0 0x7fb3201038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 --2- 172.21.15.19:0/2738703066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb320108730 0x7fb320110090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 --2- 172.21.15.19:0/2738703066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb320107de0 0x7fb3201081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 -- 172.21.15.19:0/2738703066 >> 172.21.15.19:0/2738703066 conn(0x7fb3200fede0 msgr2=0x7fb320101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 -- 172.21.15.19:0/2738703066 shutdown_connections 2024-05-17T09:17:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.331+0000 7fb327448700 1 -- 172.21.15.19:0/2738703066 wait complete. 2024-05-17T09:17:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 Processor -- start 2024-05-17T09:17:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 -- start start 2024-05-17T09:17:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3201034b0 0x7fb3201a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb320107de0 0x7fb3201a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb320108730 0x7fb32019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb320113210 con 0x7fb3201034b0 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb320113090 con 0x7fb320107de0 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb327448700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb320113390 con 0x7fb320108730 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb3251e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3201034b0 0x7fb3201a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb3251e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3201034b0 0x7fb3201a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47302/0 (socket says 172.21.15.19:47302) 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.332+0000 7fb3251e4700 1 -- 172.21.15.19:0/2823166591 learned_addr learned my addr 172.21.15.19:0/2823166591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3249e3700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb320107de0 0x7fb3201a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3259e5700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb320108730 0x7fb32019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3251e4700 1 -- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb320108730 msgr2=0x7fb32019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3251e4700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb320108730 0x7fb32019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3251e4700 1 -- 172.21.15.19:0/2823166591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb320107de0 msgr2=0x7fb3201a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3251e4700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb320107de0 0x7fb3201a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3251e4700 1 -- 172.21.15.19:0/2823166591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32019b0e0 con 0x7fb3201034b0 2024-05-17T09:17:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3259e5700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb320108730 0x7fb32019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3249e3700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb320107de0 0x7fb3201a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3251e4700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3201034b0 0x7fb3201a0880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00ef10 tx=0x7fb31c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c00cad0 con 0x7fb3201034b0 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb31c019070 con 0x7fb3201034b0 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32019b370 con 0x7fb3201034b0 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0146e0 con 0x7fb3201034b0 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.333+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3201a7710 con 0x7fb3201034b0 2024-05-17T09:17:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.334+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb320063f40 con 0x7fb3201034b0 2024-05-17T09:17:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.336+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb31c012050 con 0x7fb3201034b0 2024-05-17T09:17:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.337+0000 7fb3167fc700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb30c0b2990 0x7fb30c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.337+0000 7fb3249e3700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb30c0b2990 0x7fb30c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.337+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb31c0e31c0 con 0x7fb3201034b0 2024-05-17T09:17:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.337+0000 7fb3167fc700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] conn(0x7fb30c0bdcc0 0x7fb30c0c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.337+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 --> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb31c021e50 con 0x7fb30c0bdcc0 2024-05-17T09:17:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.337+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb31c0e3610 con 0x7fb3201034b0 2024-05-17T09:17:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.338+0000 7fb3249e3700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb30c0b2990 0x7fb30c0b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb310005fd0 tx=0x7fb31000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.338+0000 7fb3259e5700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] conn(0x7fb30c0bdcc0 0x7fb30c0c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.338+0000 7fb3259e5700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] conn(0x7fb30c0bdcc0 0x7fb30c0c01b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.339+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== osd.2 v2:172.21.15.79:6800/3984657664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb31c021e50 con 0x7fb30c0bdcc0 2024-05-17T09:17:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.356+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 --> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb320064340 con 0x7fb30c0bdcc0 2024-05-17T09:17:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.356+0000 7fb3167fc700 1 -- 172.21.15.19:0/2823166591 <== osd.2 v2:172.21.15.79:6800/3984657664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb320064340 con 0x7fb30c0bdcc0 2024-05-17T09:17:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] conn(0x7fb30c0bdcc0 msgr2=0x7fb30c0c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] conn(0x7fb30c0bdcc0 0x7fb30c0c01b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb30c0b2990 msgr2=0x7fb30c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb30c0b2990 0x7fb30c0b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb310005fd0 tx=0x7fb31000a5e0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3201034b0 msgr2=0x7fb3201a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 --2- 172.21.15.19:0/2823166591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3201034b0 0x7fb3201a0880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00ef10 tx=0x7fb31c00c3a0 comp rx=0 tx=0).stop 2024-05-17T09:17:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb3259e5700 1 -- 172.21.15.19:0/2823166591 reap_dead start 2024-05-17T09:17:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 shutdown_connections 2024-05-17T09:17:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 >> 172.21.15.19:0/2823166591 conn(0x7fb3200fede0 msgr2=0x7fb320075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 shutdown_connections 2024-05-17T09:17:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:30.357+0000 7fb327448700 1 -- 172.21.15.19:0/2823166591 wait complete. 2024-05-17T09:17:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:30 smithi069 ceph-mon[35081]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:30 smithi079 ceph-mon[35502]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:77309411418 2024-05-17T09:17:31.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-17T09:17:32.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:32 smithi019 ceph-mon[31620]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:32 smithi069 ceph-mon[35081]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:32 smithi079 ceph-mon[35502]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:17:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 -- 172.21.15.19:0/2293773861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0107de0 msgr2=0x7fb4e01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 --2- 172.21.15.19:0/2293773861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0107de0 0x7fb4e01081f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00099c0 tx=0x7fb4d0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 -- 172.21.15.19:0/2293773861 shutdown_connections 2024-05-17T09:17:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 --2- 172.21.15.19:0/2293773861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4e01034b0 0x7fb4e01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 --2- 172.21.15.19:0/2293773861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e0108730 0x7fb4e0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 --2- 172.21.15.19:0/2293773861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0107de0 0x7fb4e01081f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.339+0000 7fb4e792a700 1 -- 172.21.15.19:0/2293773861 >> 172.21.15.19:0/2293773861 conn(0x7fb4e00fedb0 msgr2=0x7fb4e01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.340+0000 7fb4e792a700 1 -- 172.21.15.19:0/2293773861 shutdown_connections 2024-05-17T09:17:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.340+0000 7fb4e792a700 1 -- 172.21.15.19:0/2293773861 wait complete. 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 Processor -- start 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 -- start start 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4e01034b0 0x7fb4e019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e0107de0 0x7fb4e019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0108730 0x7fb4e019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01131f0 con 0x7fb4e0108730 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0113070 con 0x7fb4e0107de0 2024-05-17T09:17:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e792a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0113370 con 0x7fb4e01034b0 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e5ec7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0108730 0x7fb4e019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e5ec7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0108730 0x7fb4e019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47318/0 (socket says 172.21.15.19:47318) 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e5ec7700 1 -- 172.21.15.19:0/4282540035 learned_addr learned my addr 172.21.15.19:0/4282540035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e56c6700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4e01034b0 0x7fb4e019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.341+0000 7fb4e4ec5700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e0107de0 0x7fb4e019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e5ec7700 1 -- 172.21.15.19:0/4282540035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4e01034b0 msgr2=0x7fb4e019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e5ec7700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4e01034b0 0x7fb4e019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e5ec7700 1 -- 172.21.15.19:0/4282540035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e0107de0 msgr2=0x7fb4e019c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e5ec7700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e0107de0 0x7fb4e019c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e5ec7700 1 -- 172.21.15.19:0/4282540035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e01a6e50 con 0x7fb4e0108730 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e5ec7700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0108730 0x7fb4e019c630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8007d70 tx=0x7fb4d800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8016020 con 0x7fb4e0108730 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e01a7140 con 0x7fb4e0108730 2024-05-17T09:17:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4d800ac90 con 0x7fb4e0108730 2024-05-17T09:17:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e01a7650 con 0x7fb4e0108730 2024-05-17T09:17:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.342+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80115e0 con 0x7fb4e0108730 2024-05-17T09:17:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.343+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4e0063e80 con 0x7fb4e0108730 2024-05-17T09:17:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.345+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb4d8019020 con 0x7fb4e0108730 2024-05-17T09:17:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4d67fc700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4e56c6700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4d80e2b00 con 0x7fb4e0108730 2024-05-17T09:17:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4d67fc700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] conn(0x7fb4cc0bdc40 0x7fb4cc0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 --> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4cc0c0800 con 0x7fb4cc0bdc40 2024-05-17T09:17:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb4d80e2eb0 con 0x7fb4e0108730 2024-05-17T09:17:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.346+0000 7fb4e4ec5700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] conn(0x7fb4cc0bdc40 0x7fb4cc0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.347+0000 7fb4e56c6700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005950 tx=0x7fb4dc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.347+0000 7fb4e4ec5700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] conn(0x7fb4cc0bdc40 0x7fb4cc0c0130 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.348+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== osd.3 v2:172.21.15.19:6810/494922873 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4cc0c0800 con 0x7fb4cc0bdc40 2024-05-17T09:17:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.364+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 --> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4e00642b0 con 0x7fb4cc0bdc40 2024-05-17T09:17:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4d67fc700 1 -- 172.21.15.19:0/4282540035 <== osd.3 v2:172.21.15.19:6810/494922873 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4e00642b0 con 0x7fb4cc0bdc40 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] conn(0x7fb4cc0bdc40 msgr2=0x7fb4cc0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] conn(0x7fb4cc0bdc40 0x7fb4cc0c0130 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4cc0b2910 msgr2=0x7fb4cc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005950 tx=0x7fb4dc00a300 comp rx=0 tx=0).stop 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0108730 msgr2=0x7fb4e019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 --2- 172.21.15.19:0/4282540035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e0108730 0x7fb4e019c630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8007d70 tx=0x7fb4d800a340 comp rx=0 tx=0).stop 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e5ec7700 1 -- 172.21.15.19:0/4282540035 reap_dead start 2024-05-17T09:17:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 shutdown_connections 2024-05-17T09:17:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 >> 172.21.15.19:0/4282540035 conn(0x7fb4e00fedb0 msgr2=0x7fb4e01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.365+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 shutdown_connections 2024-05-17T09:17:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:33.366+0000 7fb4e792a700 1 -- 172.21.15.19:0/4282540035 wait complete. 2024-05-17T09:17:34.128 INFO:teuthology.orchestra.run.smithi019.stdout:107374182486 2024-05-17T09:17:34.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-17T09:17:34.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:34 smithi019 ceph-mon[31620]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:34.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:34 smithi069 ceph-mon[35081]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:34 smithi079 ceph-mon[35502]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 -- 172.21.15.19:0/275625784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541034b0 msgr2=0x7fe1541038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 --2- 172.21.15.19:0/275625784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541034b0 0x7fe1541038c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1500099c0 tx=0x7fe150009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 -- 172.21.15.19:0/275625784 shutdown_connections 2024-05-17T09:17:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 --2- 172.21.15.19:0/275625784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe154108730 0x7fe154110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 --2- 172.21.15.19:0/275625784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1541034b0 0x7fe1541038c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 --2- 172.21.15.19:0/275625784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154107de0 0x7fe1541081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.971+0000 7fe15bac1700 1 -- 172.21.15.19:0/275625784 >> 172.21.15.19:0/275625784 conn(0x7fe1540fedb0 msgr2=0x7fe1541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.972+0000 7fe15bac1700 1 -- 172.21.15.19:0/275625784 shutdown_connections 2024-05-17T09:17:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.972+0000 7fe15bac1700 1 -- 172.21.15.19:0/275625784 wait complete. 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.972+0000 7fe15bac1700 1 Processor -- start 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 -- start start 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1541034b0 0x7fe15419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe154107de0 0x7fe15419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154108730 0x7fe15419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe154113210 con 0x7fe154108730 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe154113090 con 0x7fe154107de0 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15bac1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe154113390 con 0x7fe1541034b0 2024-05-17T09:17:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154108730 0x7fe15419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154108730 0x7fe15419f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42724/0 (socket says 172.21.15.19:42724) 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 -- 172.21.15.19:0/1173711534 learned_addr learned my addr 172.21.15.19:0/1173711534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15985d700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1541034b0 0x7fe15419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15905c700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe154107de0 0x7fe15419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 -- 172.21.15.19:0/1173711534 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1541034b0 msgr2=0x7fe15419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1541034b0 0x7fe15419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 -- 172.21.15.19:0/1173711534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe154107de0 msgr2=0x7fe15419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe154107de0 0x7fe15419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15a05e700 1 -- 172.21.15.19:0/1173711534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15419fc40 con 0x7fe154108730 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.973+0000 7fe15985d700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1541034b0 0x7fe15419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.974+0000 7fe15a05e700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154108730 0x7fe15419f5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe14c007ee0 tx=0x7fe14c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.974+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00ddd0 con 0x7fe154108730 2024-05-17T09:17:34.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.974+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe14c004020 con 0x7fe154108730 2024-05-17T09:17:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.974+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15419ff30 con 0x7fe154108730 2024-05-17T09:17:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.974+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c015660 con 0x7fe154108730 2024-05-17T09:17:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.974+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a7850 con 0x7fe154108730 2024-05-17T09:17:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.975+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe154063ec0 con 0x7fe154108730 2024-05-17T09:17:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.977+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe14c0041d0 con 0x7fe154108730 2024-05-17T09:17:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.977+0000 7fe14affd700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2910 0x7fe1400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe15985d700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2910 0x7fe1400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe14c0e2ca0 con 0x7fe154108730 2024-05-17T09:17:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe14affd700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] conn(0x7fe1400bdc40 0x7fe1400c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 --> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1400c0800 con 0x7fe1400bdc40 2024-05-17T09:17:34.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe14c0e30f0 con 0x7fe154108730 2024-05-17T09:17:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe15905c700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] conn(0x7fe1400bdc40 0x7fe1400c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe15985d700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2910 0x7fe1400b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe150005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.978+0000 7fe15905c700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] conn(0x7fe1400bdc40 0x7fe1400c0130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.979+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== osd.9 v2:172.21.15.19:6826/2798428087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1400c0800 con 0x7fe1400bdc40 2024-05-17T09:17:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.996+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 --> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe154064340 con 0x7fe1400bdc40 2024-05-17T09:17:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.996+0000 7fe14affd700 1 -- 172.21.15.19:0/1173711534 <== osd.9 v2:172.21.15.19:6826/2798428087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe154064340 con 0x7fe1400bdc40 2024-05-17T09:17:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] conn(0x7fe1400bdc40 msgr2=0x7fe1400c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] conn(0x7fe1400bdc40 0x7fe1400c0130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2910 msgr2=0x7fe1400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2910 0x7fe1400b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe150005fb0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154108730 msgr2=0x7fe15419f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 --2- 172.21.15.19:0/1173711534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154108730 0x7fe15419f5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe14c007ee0 tx=0x7fe14c00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15a05e700 1 -- 172.21.15.19:0/1173711534 reap_dead start 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 shutdown_connections 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 >> 172.21.15.19:0/1173711534 conn(0x7fe1540fedb0 msgr2=0x7fe15406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 shutdown_connections 2024-05-17T09:17:34.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:34.997+0000 7fe15bac1700 1 -- 172.21.15.19:0/1173711534 wait complete. 2024-05-17T09:17:36.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:36 smithi019 ceph-mon[31620]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:36.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:36 smithi069 ceph-mon[35081]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:36 smithi079 ceph-mon[35502]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.010+0000 7f8af055d700 1 -- 172.21.15.19:0/2869222006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8108f10 msgr2=0x7f8ae8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.010+0000 7f8af055d700 1 --2- 172.21.15.19:0/2869222006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8108f10 0x7f8ae8109320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80099c0 tx=0x7f8ad8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 -- 172.21.15.19:0/2869222006 shutdown_connections 2024-05-17T09:17:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 --2- 172.21.15.19:0/2869222006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae8109860 0x7f8ae81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 --2- 172.21.15.19:0/2869222006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ae81045e0 0x7f8ae81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 --2- 172.21.15.19:0/2869222006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8108f10 0x7f8ae8109320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 -- 172.21.15.19:0/2869222006 >> 172.21.15.19:0/2869222006 conn(0x7f8ae80fffd0 msgr2=0x7f8ae81023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 -- 172.21.15.19:0/2869222006 shutdown_connections 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 -- 172.21.15.19:0/2869222006 wait complete. 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 Processor -- start 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.011+0000 7f8af055d700 1 -- start start 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8af055d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae81045e0 0x7f8ae819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8af055d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ae8108f10 0x7f8ae819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8af055d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8109860 0x7f8ae819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8af055d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8114320 con 0x7f8ae8109860 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8af055d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae81141a0 con 0x7f8ae8108f10 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8af055d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae81144a0 con 0x7f8ae81045e0 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae81045e0 0x7f8ae819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae81045e0 0x7f8ae819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:36194/0 (socket says 172.21.15.19:36194) 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 -- 172.21.15.19:0/2058641671 learned_addr learned my addr 172.21.15.19:0/2058641671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aeeafa700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8109860 0x7f8ae819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aedaf8700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ae8108f10 0x7f8ae819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 -- 172.21.15.19:0/2058641671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ae8108f10 msgr2=0x7f8ae819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ae8108f10 0x7f8ae819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 -- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8109860 msgr2=0x7f8ae819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8109860 0x7f8ae819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aee2f9700 1 -- 172.21.15.19:0/2058641671 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae819fb70 con 0x7f8ae81045e0 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.012+0000 7f8aeeafa700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8109860 0x7f8ae819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.013+0000 7f8aee2f9700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae81045e0 0x7f8ae819ad50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c9b0 tx=0x7f8ae400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.013+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4007b00 con 0x7f8ae81045e0 2024-05-17T09:17:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.013+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae819fe60 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.013+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ae400ce50 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.013+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4018660 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.013+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae81a0370 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.014+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ae8063f00 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.016+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8ae400f450 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.016+0000 7f8adf7fe700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ad40b2910 0x7f8ad40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8aedaf8700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ad40b2910 0x7f8ad40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8ae40e2cf0 con 0x7f8ae81045e0 2024-05-17T09:17:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8adf7fe700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] conn(0x7f8ad40bdc40 0x7f8ad40c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 --> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ae4014ec0 con 0x7f8ad40bdc40 2024-05-17T09:17:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8ae40e30a0 con 0x7f8ae81045e0 2024-05-17T09:17:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8aedaf8700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ad40b2910 0x7f8ad40b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009990 tx=0x7f8ad8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.017+0000 7f8aeeafa700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] conn(0x7f8ad40bdc40 0x7f8ad40c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.018+0000 7f8aeeafa700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] conn(0x7f8ad40bdc40 0x7f8ad40c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.018+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== osd.8 v2:172.21.15.79:6816/2695186498 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ae4014ec0 con 0x7f8ad40bdc40 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.035+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 --> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ae8064330 con 0x7f8ad40bdc40 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8adf7fe700 1 -- 172.21.15.19:0/2058641671 <== osd.8 v2:172.21.15.79:6816/2695186498 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8ae8064330 con 0x7f8ad40bdc40 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] conn(0x7f8ad40bdc40 msgr2=0x7f8ad40c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] conn(0x7f8ad40bdc40 0x7f8ad40c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ad40b2910 msgr2=0x7f8ad40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ad40b2910 0x7f8ad40b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009990 tx=0x7f8ad8005fb0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae81045e0 msgr2=0x7f8ae819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 --2- 172.21.15.19:0/2058641671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ae81045e0 0x7f8ae819ad50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c9b0 tx=0x7f8ae400ccc0 comp rx=0 tx=0).stop 2024-05-17T09:17:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8aeeafa700 1 -- 172.21.15.19:0/2058641671 reap_dead start 2024-05-17T09:17:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 shutdown_connections 2024-05-17T09:17:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 >> 172.21.15.19:0/2058641671 conn(0x7f8ae80fffd0 msgr2=0x7f8ae81070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 shutdown_connections 2024-05-17T09:17:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:37.036+0000 7f8af055d700 1 -- 172.21.15.19:0/2058641671 wait complete. 2024-05-17T09:17:38.408 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:38 smithi019 ceph-mon[31620]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:38 smithi069 ceph-mon[35081]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:38 smithi079 ceph-mon[35502]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:39.507 INFO:teuthology.orchestra.run.smithi019.stdout:227633266749 2024-05-17T09:17:39.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-17T09:17:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 -- 172.21.15.19:0/3114959985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c108f10 msgr2=0x7fbb7c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 --2- 172.21.15.19:0/3114959985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c108f10 0x7fbb7c109320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0099c0 tx=0x7fbb6c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 -- 172.21.15.19:0/3114959985 shutdown_connections 2024-05-17T09:17:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 --2- 172.21.15.19:0/3114959985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb7c109860 0x7fbb7c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 --2- 172.21.15.19:0/3114959985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb7c1045e0 0x7fbb7c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 --2- 172.21.15.19:0/3114959985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c108f10 0x7fbb7c109320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 -- 172.21.15.19:0/3114959985 >> 172.21.15.19:0/3114959985 conn(0x7fbb7c100030 msgr2=0x7fbb7c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.745+0000 7fbb8328d700 1 -- 172.21.15.19:0/3114959985 shutdown_connections 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.746+0000 7fbb8328d700 1 -- 172.21.15.19:0/3114959985 wait complete. 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.746+0000 7fbb8328d700 1 Processor -- start 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.746+0000 7fbb8328d700 1 -- start start 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8328d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1045e0 0x7fbb7c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8328d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb7c108f10 0x7fbb7c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8328d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb7c109860 0x7fbb7c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8328d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c114340 con 0x7fbb7c1045e0 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8328d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1141c0 con 0x7fbb7c109860 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8328d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1144c0 con 0x7fbb7c108f10 2024-05-17T09:17:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1045e0 0x7fbb7c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1045e0 0x7fbb7c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42772/0 (socket says 172.21.15.19:42772) 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 -- 172.21.15.19:0/98179284 learned_addr learned my addr 172.21.15.19:0/98179284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb80828700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb7c108f10 0x7fbb7c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb8182a700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb7c109860 0x7fbb7c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 -- 172.21.15.19:0/98179284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb7c108f10 msgr2=0x7fbb7c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb7c108f10 0x7fbb7c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 -- 172.21.15.19:0/98179284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb7c109860 msgr2=0x7fbb7c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb7c109860 0x7fbb7c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.747+0000 7fbb81029700 1 -- 172.21.15.19:0/98179284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c19fc80 con 0x7fbb7c1045e0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.748+0000 7fbb81029700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1045e0 0x7fbb7c19adb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007f10 tx=0x7fbb7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.748+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78004db0 con 0x7fbb7c1045e0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.748+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb78003a00 con 0x7fbb7c1045e0 2024-05-17T09:17:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.748+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c19ff70 con 0x7fbb7c1045e0 2024-05-17T09:17:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.748+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1a7890 con 0x7fbb7c1045e0 2024-05-17T09:17:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.748+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78004570 con 0x7fbb7c1045e0 2024-05-17T09:17:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.749+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb60001030 con 0x7fbb7c1045e0 2024-05-17T09:17:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.751+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbb7801c020 con 0x7fbb7c1045e0 2024-05-17T09:17:39.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.752+0000 7fbb727fc700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbb680b2990 0x7fbb680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.752+0000 7fbb80828700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbb680b2990 0x7fbb680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.752+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbb780e2bc0 con 0x7fbb7c1045e0 2024-05-17T09:17:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.752+0000 7fbb80828700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbb680b2990 0x7fbb680b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c000c00 tx=0x7fbb6c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.753+0000 7fbb727fc700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] conn(0x7fbb680bdcc0 0x7fbb680c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.753+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 --> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb680c0880 con 0x7fbb680bdcc0 2024-05-17T09:17:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.753+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbb780e3010 con 0x7fbb7c1045e0 2024-05-17T09:17:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.753+0000 7fbb8182a700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] conn(0x7fbb680bdcc0 0x7fbb680c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.753+0000 7fbb8182a700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] conn(0x7fbb680bdcc0 0x7fbb680c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:39.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.754+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== osd.6 v2:172.21.15.19:6818/1984910928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb680c0880 con 0x7fbb680bdcc0 2024-05-17T09:17:39.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.771+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 --> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb60002e60 con 0x7fbb680bdcc0 2024-05-17T09:17:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.771+0000 7fbb727fc700 1 -- 172.21.15.19:0/98179284 <== osd.6 v2:172.21.15.19:6818/1984910928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb60002e60 con 0x7fbb680bdcc0 2024-05-17T09:17:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] conn(0x7fbb680bdcc0 msgr2=0x7fbb680c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] conn(0x7fbb680bdcc0 0x7fbb680c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbb680b2990 msgr2=0x7fbb680b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbb680b2990 0x7fbb680b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c000c00 tx=0x7fbb6c01a040 comp rx=0 tx=0).stop 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1045e0 msgr2=0x7fbb7c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 --2- 172.21.15.19:0/98179284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1045e0 0x7fbb7c19adb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007f10 tx=0x7fbb7800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8182a700 1 -- 172.21.15.19:0/98179284 reap_dead start 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 shutdown_connections 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 >> 172.21.15.19:0/98179284 conn(0x7fbb7c100030 msgr2=0x7fbb7c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 shutdown_connections 2024-05-17T09:17:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.772+0000 7fbb8328d700 1 -- 172.21.15.19:0/98179284 wait complete. 2024-05-17T09:17:39.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 -- 172.21.15.19:0/2917382715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec1045d0 msgr2=0x7f67ec1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 --2- 172.21.15.19:0/2917382715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec1045d0 0x7f67ec1049e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0099c0 tx=0x7f67dc009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 -- 172.21.15.19:0/2917382715 shutdown_connections 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 --2- 172.21.15.19:0/2917382715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67ec109850 0x7f67ec1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 --2- 172.21.15.19:0/2917382715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec1045d0 0x7f67ec1049e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 --2- 172.21.15.19:0/2917382715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67ec108f00 0x7f67ec109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 -- 172.21.15.19:0/2917382715 >> 172.21.15.19:0/2917382715 conn(0x7f67ec0ffe60 msgr2=0x7f67ec1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 -- 172.21.15.19:0/2917382715 shutdown_connections 2024-05-17T09:17:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.976+0000 7f67f21a9700 1 -- 172.21.15.19:0/2917382715 wait complete. 2024-05-17T09:17:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 Processor -- start 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 -- start start 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67ec1045d0 0x7f67ec075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec108f00 0x7f67ec075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67ec109850 0x7f67ec076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec114330 con 0x7f67ec108f00 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1141b0 con 0x7f67ec1045d0 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67f21a9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1144b0 con 0x7f67ec109850 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67eaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec108f00 0x7f67ec075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67eaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec108f00 0x7f67ec075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42790/0 (socket says 172.21.15.19:42790) 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.977+0000 7f67eaffd700 1 -- 172.21.15.19:0/3769523762 learned_addr learned my addr 172.21.15.19:0/3769523762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eb7fe700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67ec1045d0 0x7f67ec075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67ebfff700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67ec109850 0x7f67ec076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eaffd700 1 -- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67ec109850 msgr2=0x7f67ec076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eaffd700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67ec109850 0x7f67ec076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eaffd700 1 -- 172.21.15.19:0/3769523762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67ec1045d0 msgr2=0x7f67ec075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eaffd700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67ec1045d0 0x7f67ec075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eaffd700 1 -- 172.21.15.19:0/3769523762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec1a90a0 con 0x7f67ec108f00 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67eaffd700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec108f00 0x7f67ec075ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e970 tx=0x7f67e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0004a30 con 0x7f67ec108f00 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec1a9390 con 0x7f67ec108f00 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67e000cc70 con 0x7f67ec108f00 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0005520 con 0x7f67ec108f00 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.978+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec1a9920 con 0x7f67ec108f00 2024-05-17T09:17:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.979+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67ec063ec0 con 0x7f67ec108f00 2024-05-17T09:17:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.981+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f67e000c7e0 con 0x7f67ec108f00 2024-05-17T09:17:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.982+0000 7f67e8ff9700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f67d40b2910 0x7f67d40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.982+0000 7f67eb7fe700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f67d40b2910 0x7f67d40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.982+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f67e00e2d30 con 0x7f67ec108f00 2024-05-17T09:17:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.982+0000 7f67e8ff9700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] conn(0x7f67d40bdc40 0x7f67d40c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.982+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 --> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67e0004cc0 con 0x7f67d40bdc40 2024-05-17T09:17:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.982+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f67e00e3180 con 0x7f67ec108f00 2024-05-17T09:17:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.983+0000 7f67eb7fe700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f67d40b2910 0x7f67d40b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000c00 tx=0x7f67dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.983+0000 7f67ebfff700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] conn(0x7f67d40bdc40 0x7f67d40c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.983+0000 7f67ebfff700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] conn(0x7f67d40bdc40 0x7f67d40c0130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:39.984+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== osd.11 v2:172.21.15.79:6824/2698518199 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67e0004cc0 con 0x7f67d40bdc40 2024-05-17T09:17:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.001+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 --> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67ec0642c0 con 0x7f67d40bdc40 2024-05-17T09:17:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.001+0000 7f67e8ff9700 1 -- 172.21.15.19:0/3769523762 <== osd.11 v2:172.21.15.79:6824/2698518199 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f67ec0642c0 con 0x7f67d40bdc40 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] conn(0x7f67d40bdc40 msgr2=0x7f67d40c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] conn(0x7f67d40bdc40 0x7f67d40c0130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f67d40b2910 msgr2=0x7f67d40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f67d40b2910 0x7f67d40b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000c00 tx=0x7f67dc01a040 comp rx=0 tx=0).stop 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec108f00 msgr2=0x7f67ec075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 --2- 172.21.15.19:0/3769523762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ec108f00 0x7f67ec075ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e970 tx=0x7f67e000ec80 comp rx=0 tx=0).stop 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67ebfff700 1 -- 172.21.15.19:0/3769523762 reap_dead start 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 shutdown_connections 2024-05-17T09:17:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 >> 172.21.15.19:0/3769523762 conn(0x7f67ec0ffe60 msgr2=0x7f67ec1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 shutdown_connections 2024-05-17T09:17:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.002+0000 7f67f21a9700 1 -- 172.21.15.19:0/3769523762 wait complete. 2024-05-17T09:17:40.283 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:40 smithi019 ceph-mon[31620]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:40 smithi069 ceph-mon[35081]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:40 smithi079 ceph-mon[35502]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:17:41.053 INFO:teuthology.orchestra.run.smithi019.stdout:180388626505 2024-05-17T09:17:41.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-17T09:17:41.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.272+0000 7f342485f700 1 -- 172.21.15.19:0/1681600545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34201034b0 msgr2=0x7f34201038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.272+0000 7f342485f700 1 --2- 172.21.15.19:0/1681600545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34201034b0 0x7f34201038c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f34100099c0 tx=0x7f3410009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 -- 172.21.15.19:0/1681600545 shutdown_connections 2024-05-17T09:17:41.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 --2- 172.21.15.19:0/1681600545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3420108730 0x7f3420110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 --2- 172.21.15.19:0/1681600545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34201034b0 0x7f34201038c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 --2- 172.21.15.19:0/1681600545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3420107de0 0x7f34201081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 -- 172.21.15.19:0/1681600545 >> 172.21.15.19:0/1681600545 conn(0x7f34200fedb0 msgr2=0x7f34201011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:41.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 -- 172.21.15.19:0/1681600545 shutdown_connections 2024-05-17T09:17:41.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.273+0000 7f342485f700 1 -- 172.21.15.19:0/1681600545 wait complete. 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 Processor -- start 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 -- start start 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34201034b0 0x7f34200ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3420107de0 0x7f34200ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3420108730 0x7f34200ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113210 con 0x7f3420108730 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113090 con 0x7f3420107de0 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f342485f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113390 con 0x7f34201034b0 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f341ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3420108730 0x7f34200ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f341ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3420108730 0x7f34200ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42828/0 (socket says 172.21.15.19:42828) 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f341ed9d700 1 -- 172.21.15.19:0/2211036203 learned_addr learned my addr 172.21.15.19:0/2211036203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f341dd9b700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3420107de0 0x7f34200ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.274+0000 7f341e59c700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34201034b0 0x7f34200ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341ed9d700 1 -- 172.21.15.19:0/2211036203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34201034b0 msgr2=0x7f34200ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341ed9d700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34201034b0 0x7f34200ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341ed9d700 1 -- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3420107de0 msgr2=0x7f34200ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341ed9d700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3420107de0 0x7f34200ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341ed9d700 1 -- 172.21.15.19:0/2211036203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34201a90e0 con 0x7f3420108730 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341dd9b700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3420107de0 0x7f34200ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f341ed9d700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3420108730 0x7f34200ffd10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3418007ee0 tx=0x7f341800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800dd20 con 0x7f3420108730 2024-05-17T09:17:41.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201a93d0 con 0x7f3420108730 2024-05-17T09:17:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3418004020 con 0x7f3420108730 2024-05-17T09:17:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a98c0 con 0x7f3420108730 2024-05-17T09:17:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.275+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418023060 con 0x7f3420108730 2024-05-17T09:17:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.276+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3420063ec0 con 0x7f3420108730 2024-05-17T09:17:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.278+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f34180041d0 con 0x7f3420108730 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.279+0000 7f34177fe700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f340c0b2910 0x7f340c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.279+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f34180e2f40 con 0x7f3420108730 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.279+0000 7f341e59c700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f340c0b2910 0x7f340c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.279+0000 7f34177fe700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] conn(0x7f340c0bdc40 0x7f340c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.279+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 --> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f340c0c0800 con 0x7f340c0bdc40 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.279+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f34180e3390 con 0x7f3420108730 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.280+0000 7f341dd9b700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] conn(0x7f340c0bdc40 0x7f340c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.280+0000 7f341e59c700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f340c0b2910 0x7f340c0b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3410000c00 tx=0x7f3410005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:41.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.280+0000 7f341dd9b700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] conn(0x7f340c0bdc40 0x7f340c0c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:41.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.281+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== osd.4 v2:172.21.15.69:6808/3507791356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f340c0c0800 con 0x7f340c0bdc40 2024-05-17T09:17:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.298+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 --> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34200642f0 con 0x7f340c0bdc40 2024-05-17T09:17:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f34177fe700 1 -- 172.21.15.19:0/2211036203 <== osd.4 v2:172.21.15.69:6808/3507791356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34200642f0 con 0x7f340c0bdc40 2024-05-17T09:17:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] conn(0x7f340c0bdc40 msgr2=0x7f340c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] conn(0x7f340c0bdc40 0x7f340c0c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f340c0b2910 msgr2=0x7f340c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f340c0b2910 0x7f340c0b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3410000c00 tx=0x7f3410005fb0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3420108730 msgr2=0x7f34200ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 --2- 172.21.15.19:0/2211036203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3420108730 0x7f34200ffd10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3418007ee0 tx=0x7f341800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f341ed9d700 1 -- 172.21.15.19:0/2211036203 reap_dead start 2024-05-17T09:17:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 shutdown_connections 2024-05-17T09:17:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.299+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 >> 172.21.15.19:0/2211036203 conn(0x7f34200fedb0 msgr2=0x7f342006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.300+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 shutdown_connections 2024-05-17T09:17:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:41.300+0000 7f342485f700 1 -- 172.21.15.19:0/2211036203 wait complete. 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 -- 172.21.15.19:0/1714898585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 msgr2=0x7fc350110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 --2- 172.21.15.19:0/1714898585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc350110dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc34800b0a0 tx=0x7fc34800b3b0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 -- 172.21.15.19:0/1714898585 shutdown_connections 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 --2- 172.21.15.19:0/1714898585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc350105510 0x7fc350105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 --2- 172.21.15.19:0/1714898585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc350110dc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 --2- 172.21.15.19:0/1714898585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc350105e60 0x7fc35010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 -- 172.21.15.19:0/1714898585 >> 172.21.15.19:0/1714898585 conn(0x7fc3500fede0 msgr2=0x7fc350101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 -- 172.21.15.19:0/1714898585 shutdown_connections 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.526+0000 7fc3549b0700 1 -- 172.21.15.19:0/1714898585 wait complete. 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.527+0000 7fc3549b0700 1 Processor -- start 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.527+0000 7fc3549b0700 1 -- start start 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.527+0000 7fc3549b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc350105510 0x7fc3501a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.527+0000 7fc3549b0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc350105e60 0x7fc3501a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.527+0000 7fc3549b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc35019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.527+0000 7fc3549b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501142f0 con 0x7fc35010e8f0 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc3549b0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc350114170 con 0x7fc350105e60 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc3549b0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc350114470 con 0x7fc350105510 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc35019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc35019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42814/0 (socket says 172.21.15.19:42814) 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 -- 172.21.15.19:0/3353283529 learned_addr learned my addr 172.21.15.19:0/3353283529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34e59c700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc350105510 0x7fc3501a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34dd9b700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc350105e60 0x7fc3501a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 -- 172.21.15.19:0/3353283529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc350105510 msgr2=0x7fc3501a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc350105510 0x7fc3501a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 -- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc350105e60 msgr2=0x7fc3501a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc350105e60 0x7fc3501a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 -- 172.21.15.19:0/3353283529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35019b1d0 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc34ed9d700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc35019aaf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc34800b830 tx=0x7fc348004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348021070 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35019b460 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc348007b80 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.528+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3480052a0 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.529+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501a7760 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.529+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc350063f40 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.531+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc348024070 con 0x7fc35010e8f0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc3477fe700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc33c0b2910 0x7fc33c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc34e59c700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc33c0b2910 0x7fc33c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc34800f020 con 0x7fc35010e8f0 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc3477fe700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] conn(0x7fc33c0bdb10 0x7fc33c0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 --> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc348027e50 con 0x7fc33c0bdb10 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc3480e43d0 con 0x7fc35010e8f0 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc34e59c700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc33c0b2910 0x7fc33c0b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc35019bc30 tx=0x7fc33800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.532+0000 7fc34dd9b700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] conn(0x7fc33c0bdb10 0x7fc33c0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.533+0000 7fc34dd9b700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] conn(0x7fc33c0bdb10 0x7fc33c0c0000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.534+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== osd.7 v2:172.21.15.69:6816/2368256164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc348027e50 con 0x7fc33c0bdb10 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.551+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 --> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc350064340 con 0x7fc33c0bdb10 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.551+0000 7fc3477fe700 1 -- 172.21.15.19:0/3353283529 <== osd.7 v2:172.21.15.69:6816/2368256164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc350064340 con 0x7fc33c0bdb10 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] conn(0x7fc33c0bdb10 msgr2=0x7fc33c0c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] conn(0x7fc33c0bdb10 0x7fc33c0c0000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc33c0b2910 msgr2=0x7fc33c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc33c0b2910 0x7fc33c0b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc35019bc30 tx=0x7fc33800b410 comp rx=0 tx=0).stop 2024-05-17T09:17:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 msgr2=0x7fc35019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 --2- 172.21.15.19:0/3353283529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc35010e8f0 0x7fc35019aaf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc34800b830 tx=0x7fc348004d00 comp rx=0 tx=0).stop 2024-05-17T09:17:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc34ed9d700 1 -- 172.21.15.19:0/3353283529 reap_dead start 2024-05-17T09:17:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 shutdown_connections 2024-05-17T09:17:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 >> 172.21.15.19:0/3353283529 conn(0x7fc3500fede0 msgr2=0x7fc3500ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 shutdown_connections 2024-05-17T09:17:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:40.552+0000 7fc3549b0700 1 -- 172.21.15.19:0/3353283529 wait complete. 2024-05-17T09:17:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:42 smithi069 ceph-mon[35081]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-17T09:17:42.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:42 smithi019 ceph-mon[31620]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-17T09:17:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:42 smithi079 ceph-mon[35502]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-17T09:17:42.779 INFO:teuthology.orchestra.run.smithi019.stdout:253403070520 2024-05-17T09:17:42.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-17T09:17:43.444 INFO:teuthology.orchestra.run.smithi019.stdout:300647710767 2024-05-17T09:17:43.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-17T09:17:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 -- 172.21.15.19:0/1924944242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b0105470 msgr2=0x7fc1b0105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 --2- 172.21.15.19:0/1924944242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b0105470 0x7fc1b0105880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b0a0 tx=0x7fc1a000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 -- 172.21.15.19:0/1924944242 shutdown_connections 2024-05-17T09:17:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 --2- 172.21.15.19:0/1924944242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b010e930 0x7fc1b0110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 --2- 172.21.15.19:0/1924944242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b0105470 0x7fc1b0105880 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 --2- 172.21.15.19:0/1924944242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b0105e50 0x7fc1b010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 -- 172.21.15.19:0/1924944242 >> 172.21.15.19:0/1924944242 conn(0x7fc1b00fef40 msgr2=0x7fc1b0101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 -- 172.21.15.19:0/1924944242 shutdown_connections 2024-05-17T09:17:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.947+0000 7fc1b515c700 1 -- 172.21.15.19:0/1924944242 wait complete. 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.948+0000 7fc1b515c700 1 Processor -- start 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.948+0000 7fc1b515c700 1 -- start start 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1b515c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b0105470 0x7fc1b019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1b515c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b0105e50 0x7fc1b019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1b515c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b010e930 0x7fc1b019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1b515c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0114340 con 0x7fc1b010e930 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1b515c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01141c0 con 0x7fc1b0105470 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1b515c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01144c0 con 0x7fc1b0105e50 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b010e930 0x7fc1b019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b010e930 0x7fc1b019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33176/0 (socket says 172.21.15.19:33176) 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 -- 172.21.15.19:0/4116872284 learned_addr learned my addr 172.21.15.19:0/4116872284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1ae59c700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b0105e50 0x7fc1b019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1aed9d700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b0105470 0x7fc1b019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 -- 172.21.15.19:0/4116872284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b0105e50 msgr2=0x7fc1b019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b0105e50 0x7fc1b019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 -- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b0105470 msgr2=0x7fc1b019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b0105470 0x7fc1b019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.949+0000 7fc1af59e700 1 -- 172.21.15.19:0/4116872284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b019fc00 con 0x7fc1b010e930 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.950+0000 7fc1af59e700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b010e930 0x7fc1b019f560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800c970 tx=0x7fc1a800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.950+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004a30 con 0x7fc1b010e930 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.950+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1a80054b0 con 0x7fc1b010e930 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.950+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b019fe90 con 0x7fc1b010e930 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.950+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800eac0 con 0x7fc1b010e930 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.950+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b01a0380 con 0x7fc1b010e930 2024-05-17T09:17:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.951+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1b0063f00 con 0x7fc1b010e930 2024-05-17T09:17:43.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.953+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc1a800ec60 con 0x7fc1b010e930 2024-05-17T09:17:43.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.953+0000 7fc1a7fff700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc19c0b2910 0x7fc19c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:43.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1aed9d700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc19c0b2910 0x7fc19c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:43.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc1a80e2750 con 0x7fc1b010e930 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1a7fff700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] conn(0x7fc19c0bdc40 0x7fc19c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 --> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1a8012ec0 con 0x7fc19c0bdc40 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc1a80e2ba0 con 0x7fc1b010e930 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1aed9d700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc19c0b2910 0x7fc19c0b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1ae59c700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] conn(0x7fc19c0bdc40 0x7fc19c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.954+0000 7fc1ae59c700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] conn(0x7fc19c0bdc40 0x7fc19c0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.955+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== osd.1 v2:172.21.15.69:6800/3851861092 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1a8012ec0 con 0x7fc19c0bdc40 2024-05-17T09:17:43.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.972+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 --> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1b0064300 con 0x7fc19c0bdc40 2024-05-17T09:17:43.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1a7fff700 1 -- 172.21.15.19:0/4116872284 <== osd.1 v2:172.21.15.69:6800/3851861092 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc1b0064300 con 0x7fc19c0bdc40 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] conn(0x7fc19c0bdc40 msgr2=0x7fc19c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] conn(0x7fc19c0bdc40 0x7fc19c0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc19c0b2910 msgr2=0x7fc19c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc19c0b2910 0x7fc19c0b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a000b7b0 comp rx=0 tx=0).stop 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b010e930 msgr2=0x7fc1b019f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 --2- 172.21.15.19:0/4116872284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1b010e930 0x7fc1b019f560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800c970 tx=0x7fc1a800cc80 comp rx=0 tx=0).stop 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1af59e700 1 -- 172.21.15.19:0/4116872284 reap_dead start 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 shutdown_connections 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 >> 172.21.15.19:0/4116872284 conn(0x7fc1b00fef40 msgr2=0x7fc1b0100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 shutdown_connections 2024-05-17T09:17:43.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:43.973+0000 7fc1b515c700 1 -- 172.21.15.19:0/4116872284 wait complete. 2024-05-17T09:17:44.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:44 smithi019 ceph-mon[31620]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:44.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:44 smithi069 ceph-mon[35081]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:44 smithi079 ceph-mon[35502]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:17:44.737 INFO:teuthology.orchestra.run.smithi019.stdout:133143986258 2024-05-17T09:17:44.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-17T09:17:45.553 INFO:teuthology.orchestra.run.smithi019.stdout:55834574947 2024-05-17T09:17:45.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-17T09:17:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:46.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:46 smithi019 ceph-mon[31620]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:46 smithi069 ceph-mon[35081]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 -- 172.21.15.19:0/2553696748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 msgr2=0x7f699c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 --2- 172.21.15.19:0/2553696748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c110090 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f699400b0d0 tx=0x7f699400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 -- 172.21.15.19:0/2553696748 shutdown_connections 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 --2- 172.21.15.19:0/2553696748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c1034b0 0x7f699c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 --2- 172.21.15.19:0/2553696748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c110090 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 --2- 172.21.15.19:0/2553696748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f699c107de0 0x7f699c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 -- 172.21.15.19:0/2553696748 >> 172.21.15.19:0/2553696748 conn(0x7f699c0fedb0 msgr2=0x7f699c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 -- 172.21.15.19:0/2553696748 shutdown_connections 2024-05-17T09:17:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.466+0000 7f69a311a700 1 -- 172.21.15.19:0/2553696748 wait complete. 2024-05-17T09:17:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.467+0000 7f69a311a700 1 Processor -- start 2024-05-17T09:17:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.467+0000 7f69a311a700 1 -- start start 2024-05-17T09:17:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.467+0000 7f69a311a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c1034b0 0x7f699c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.467+0000 7f69a311a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f699c107de0 0x7f699c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a311a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a311a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113210 con 0x7f699c108730 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a311a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113090 con 0x7f699c107de0 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a311a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113390 con 0x7f699c1034b0 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33188/0 (socket says 172.21.15.19:33188) 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 -- 172.21.15.19:0/47456731 learned_addr learned my addr 172.21.15.19:0/47456731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f6993fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f699c107de0 0x7f699c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a0eb6700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c1034b0 0x7f699c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 -- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c1034b0 msgr2=0x7f699c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c1034b0 0x7f699c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 -- 172.21.15.19:0/47456731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f699c107de0 msgr2=0x7f699c19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f699c107de0 0x7f699c19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f69a16b7700 1 -- 172.21.15.19:0/47456731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a6e60 con 0x7f699c108730 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.468+0000 7f6993fff700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f699c107de0 0x7f699c19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.469+0000 7f69a16b7700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c19c5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6994009850 tx=0x7f6994007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.469+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994004c50 con 0x7f699c108730 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.469+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c199900 con 0x7f699c108730 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.469+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6994004df0 con 0x7f699c108730 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.469+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940056e0 con 0x7f699c108730 2024-05-17T09:17:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.469+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c199e30 con 0x7f699c108730 2024-05-17T09:17:46.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.470+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f699c063ec0 con 0x7f699c108730 2024-05-17T09:17:46.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.472+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6994005880 con 0x7f699c108730 2024-05-17T09:17:46.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f6991ffb700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69840b2910 0x7f69840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:46.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f69a0eb6700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69840b2910 0x7f69840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:46.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f699400f030 con 0x7f699c108730 2024-05-17T09:17:46.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f6991ffb700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] conn(0x7f69840bdc40 0x7f69840c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 --> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6994022ec0 con 0x7f69840bdc40 2024-05-17T09:17:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f69940e80d0 con 0x7f699c108730 2024-05-17T09:17:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f69a0eb6700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69840b2910 0x7f69840b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6998007be0 tx=0x7f69980073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.473+0000 7f6993fff700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] conn(0x7f69840bdc40 0x7f69840c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.474+0000 7f6993fff700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] conn(0x7f69840bdc40 0x7f69840c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.475+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== osd.5 v2:172.21.15.79:6808/3720398663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6994022ec0 con 0x7f69840bdc40 2024-05-17T09:17:46.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.492+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 --> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f699c0642f0 con 0x7f69840bdc40 2024-05-17T09:17:46.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f6991ffb700 1 -- 172.21.15.19:0/47456731 <== osd.5 v2:172.21.15.79:6808/3720398663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f699c0642f0 con 0x7f69840bdc40 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] conn(0x7f69840bdc40 msgr2=0x7f69840c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a311a700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] conn(0x7f69840bdc40 0x7f69840c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69840b2910 msgr2=0x7f69840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a311a700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69840b2910 0x7f69840b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6998007be0 tx=0x7f69980073d0 comp rx=0 tx=0).stop 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 msgr2=0x7f699c19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a311a700 1 --2- 172.21.15.19:0/47456731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108730 0x7f699c19c5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6994009850 tx=0x7f6994007a20 comp rx=0 tx=0).stop 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.493+0000 7f69a16b7700 1 -- 172.21.15.19:0/47456731 reap_dead start 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.494+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 shutdown_connections 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.494+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 >> 172.21.15.19:0/47456731 conn(0x7f699c0fedb0 msgr2=0x7f699c107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.494+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 shutdown_connections 2024-05-17T09:17:46.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:46.494+0000 7f69a311a700 1 -- 172.21.15.19:0/47456731 wait complete. 2024-05-17T09:17:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:46 smithi079 ceph-mon[35502]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:17:48.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:48 smithi019 ceph-mon[31620]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:48.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:48 smithi019 ceph-mon[31620]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T09:17:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:48 smithi069 ceph-mon[35081]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:48 smithi069 ceph-mon[35081]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T09:17:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:48 smithi079 ceph-mon[35502]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:17:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:48 smithi079 ceph-mon[35502]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T09:17:48.568 INFO:teuthology.orchestra.run.smithi019.stdout:154618822735 2024-05-17T09:17:48.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-17T09:17:48.994 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:49.459 INFO:teuthology.orchestra.run.smithi019.stdout:201863462979 2024-05-17T09:17:49.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-17T09:17:50.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:50 smithi019 ceph-mon[31620]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-17T09:17:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:50 smithi069 ceph-mon[35081]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-17T09:17:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:50 smithi079 ceph-mon[35502]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-17T09:17:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 -- 172.21.15.19:0/281696487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 msgr2=0x7fe300110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 --2- 172.21.15.19:0/281696487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe300110090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40099c0 tx=0x7fe2f4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 -- 172.21.15.19:0/281696487 shutdown_connections 2024-05-17T09:17:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 --2- 172.21.15.19:0/281696487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3001034b0 0x7fe3001038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 --2- 172.21.15.19:0/281696487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe300110090 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 --2- 172.21.15.19:0/281696487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 0x7fe3001081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:50.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.906+0000 7fe3057c3700 1 -- 172.21.15.19:0/281696487 >> 172.21.15.19:0/281696487 conn(0x7fe3000fedb0 msgr2=0x7fe3001011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:50.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.907+0000 7fe3057c3700 1 -- 172.21.15.19:0/281696487 shutdown_connections 2024-05-17T09:17:50.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.907+0000 7fe3057c3700 1 -- 172.21.15.19:0/281696487 wait complete. 2024-05-17T09:17:50.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.907+0000 7fe3057c3700 1 Processor -- start 2024-05-17T09:17:50.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.907+0000 7fe3057c3700 1 -- start start 2024-05-17T09:17:50.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe3057c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3001034b0 0x7fe30019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:50.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe3057c3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 0x7fe30019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe3057c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe30019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe3057c3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe300113210 con 0x7fe300108730 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe3057c3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe300113090 con 0x7fe300107de0 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe3057c3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe300113390 con 0x7fe3001034b0 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe30019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe30019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33196/0 (socket says 172.21.15.19:33196) 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2feffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3001034b0 0x7fe30019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 -- 172.21.15.19:0/664932558 learned_addr learned my addr 172.21.15.19:0/664932558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2fe7fc700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 0x7fe30019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 -- 172.21.15.19:0/664932558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3001034b0 msgr2=0x7fe30019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3001034b0 0x7fe30019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 -- 172.21.15.19:0/664932558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 msgr2=0x7fe30019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 0x7fe30019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2ff7fe700 1 -- 172.21.15.19:0/664932558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30019fbe0 con 0x7fe300108730 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.908+0000 7fe2fe7fc700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 0x7fe30019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.909+0000 7fe2ff7fe700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe30019f540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009990 tx=0x7fe2f40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.909+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400f8a0 con 0x7fe300108730 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.909+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2f4004830 con 0x7fe300108730 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.909+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30019fe70 con 0x7fe300108730 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.909+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4005350 con 0x7fe300108730 2024-05-17T09:17:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.909+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3001a02a0 con 0x7fe300108730 2024-05-17T09:17:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.910+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3000512b0 con 0x7fe300108730 2024-05-17T09:17:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.912+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe2f4005510 con 0x7fe300108730 2024-05-17T09:17:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.912+0000 7fe2e7fff700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2e80b2880 0x7fe2e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.912+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe2f40e3960 con 0x7fe300108730 2024-05-17T09:17:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.912+0000 7fe2feffd700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2e80b2880 0x7fe2e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.913+0000 7fe2feffd700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2e80b2880 0x7fe2e80b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005950 tx=0x7fe2f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:50.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:50.916+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2f4013070 con 0x7fe300108730 2024-05-17T09:17:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.139+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe30002e5e0 con 0x7fe300108730 2024-05-17T09:17:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.139+0000 7fe2e7fff700 1 -- 172.21.15.19:0/664932558 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe2f40ab360 con 0x7fe300108730 2024-05-17T09:17:51.140 INFO:teuthology.orchestra.run.smithi019.stdout:34359738473 2024-05-17T09:17:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2e80b2880 msgr2=0x7fe2e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2e80b2880 0x7fe2e80b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005950 tx=0x7fe2f000b410 comp rx=0 tx=0).stop 2024-05-17T09:17:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 msgr2=0x7fe30019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe30019f540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009990 tx=0x7fe2f40047b0 comp rx=0 tx=0).stop 2024-05-17T09:17:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 shutdown_connections 2024-05-17T09:17:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2e80b2880 0x7fe2e80b4d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.141+0000 7fe3057c3700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3001034b0 0x7fe30019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.142+0000 7fe3057c3700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe300108730 0x7fe30019f540 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.142+0000 7fe3057c3700 1 --2- 172.21.15.19:0/664932558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe300107de0 0x7fe30019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.142+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 >> 172.21.15.19:0/664932558 conn(0x7fe3000fedb0 msgr2=0x7fe30006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.142+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 shutdown_connections 2024-05-17T09:17:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:51.142+0000 7fe3057c3700 1 -- 172.21.15.19:0/664932558 wait complete. 2024-05-17T09:17:52.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:52 smithi019 ceph-mon[31620]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-17T09:17:52.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:52 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/664932558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T09:17:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:52 smithi069 ceph-mon[35081]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-17T09:17:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:52 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/664932558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T09:17:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:52 smithi079 ceph-mon[35502]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-17T09:17:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:52 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/664932558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T09:17:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 -- 172.21.15.19:0/2269646193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0108760 msgr2=0x7f13f01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 --2- 172.21.15.19:0/2269646193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0108760 0x7f13f01100c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b0d0 tx=0x7f13e800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 -- 172.21.15.19:0/2269646193 shutdown_connections 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 --2- 172.21.15.19:0/2269646193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0108760 0x7f13f01100c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 --2- 172.21.15.19:0/2269646193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0103c10 0x7f13f0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 --2- 172.21.15.19:0/2269646193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f0107e10 0x7f13f0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 -- 172.21.15.19:0/2269646193 >> 172.21.15.19:0/2269646193 conn(0x7f13f00fe6e0 msgr2=0x7f13f0100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 -- 172.21.15.19:0/2269646193 shutdown_connections 2024-05-17T09:17:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.210+0000 7f13f4b07700 1 -- 172.21.15.19:0/2269646193 wait complete. 2024-05-17T09:17:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 Processor -- start 2024-05-17T09:17:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 -- start start 2024-05-17T09:17:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f0103c10 0x7f13f00febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 0x7f13f00ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0108760 0x7f13f00ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13f01132b0 con 0x7f13f0107e10 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0113130 con 0x7f13f0108760 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13f4b07700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0113430 con 0x7f13f0103c10 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 0x7f13f00ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 0x7f13f00ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55344/0 (socket says 172.21.15.19:55344) 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13edd9b700 1 -- 172.21.15.19:0/3428276699 learned_addr learned my addr 172.21.15.19:0/3428276699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13eed9d700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0108760 0x7f13f00ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.211+0000 7f13ee59c700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f0103c10 0x7f13f00febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13eed9d700 1 -- 172.21.15.19:0/3428276699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f0103c10 msgr2=0x7f13f00febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13eed9d700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f0103c10 0x7f13f00febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13eed9d700 1 -- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 msgr2=0x7f13f00ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13eed9d700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 0x7f13f00ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13eed9d700 1 -- 172.21.15.19:0/3428276699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f01a9150 con 0x7f13f0108760 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13edd9b700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 0x7f13f00ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13eed9d700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0108760 0x7f13f00ff640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b770 tx=0x7f13e8007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e801f070 con 0x7f13f0108760 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13e8007dc0 con 0x7f13f0108760 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e801b430 con 0x7f13f0108760 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f01a93e0 con 0x7f13f0108760 2024-05-17T09:17:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.212+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f01a9830 con 0x7f13f0108760 2024-05-17T09:17:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.213+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f00512b0 con 0x7f13f0108760 2024-05-17T09:17:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.215+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f13e8022070 con 0x7f13f0108760 2024-05-17T09:17:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.216+0000 7f13df7fe700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13d80b2880 0x7f13d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.216+0000 7f13ee59c700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13d80b2880 0x7f13d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.216+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f13e800f020 con 0x7f13f0108760 2024-05-17T09:17:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.217+0000 7f13ee59c700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13d80b2880 0x7f13d80b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13f006b100 tx=0x7f13e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.220+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f13e8029020 con 0x7f13f0108760 2024-05-17T09:17:54.332 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:54.333 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:54.336 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:17:54.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:54 smithi019 ceph-mon[31620]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:17:54.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738467 got 34359738473 for osd.0 2024-05-17T09:17:54.367 DEBUG:teuthology.parallel:result is None 2024-05-17T09:17:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:54 smithi069 ceph-mon[35081]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:17:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.460+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f13f002e840 con 0x7f13f0108760 2024-05-17T09:17:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.461+0000 7f13df7fe700 1 -- 172.21.15.19:0/3428276699 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13e80ab380 con 0x7f13f0108760 2024-05-17T09:17:54.461 INFO:teuthology.orchestra.run.smithi019.stdout:77309411423 2024-05-17T09:17:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.463+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13d80b2880 msgr2=0x7f13d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.463+0000 7f13f4b07700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13d80b2880 0x7f13d80b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13f006b100 tx=0x7f13e0009260 comp rx=0 tx=0).stop 2024-05-17T09:17:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.463+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0108760 msgr2=0x7f13f00ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.463+0000 7f13f4b07700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0108760 0x7f13f00ff640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b770 tx=0x7f13e8007980 comp rx=0 tx=0).stop 2024-05-17T09:17:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 shutdown_connections 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f13d80b2880 0x7f13d80b4d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f0103c10 0x7f13f00febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f0108760 0x7f13f00ff640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 --2- 172.21.15.19:0/3428276699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13f0107e10 0x7f13f00ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 >> 172.21.15.19:0/3428276699 conn(0x7f13f00fe6e0 msgr2=0x7f13f0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 shutdown_connections 2024-05-17T09:17:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:54.464+0000 7f13f4b07700 1 -- 172.21.15.19:0/3428276699 wait complete. 2024-05-17T09:17:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:54 smithi079 ceph-mon[35502]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:17:55.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:55 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3428276699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T09:17:55.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:55 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3428276699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T09:17:55.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:55 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3428276699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T09:17:56.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:56 smithi019 ceph-mon[31620]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:17:56.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:56 smithi069 ceph-mon[35081]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:17:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:56 smithi079 ceph-mon[35502]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 -- 172.21.15.19:0/129799855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0107e20 msgr2=0x7f84e0108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 --2- 172.21.15.19:0/129799855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0107e20 0x7f84e0108230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f84d40099c0 tx=0x7f84d4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 -- 172.21.15.19:0/129799855 shutdown_connections 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 --2- 172.21.15.19:0/129799855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0103540 0x7f84e0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 --2- 172.21.15.19:0/129799855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84e0108770 0x7f84e01100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 --2- 172.21.15.19:0/129799855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0107e20 0x7f84e0108230 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 -- 172.21.15.19:0/129799855 >> 172.21.15.19:0/129799855 conn(0x7f84e0078df0 msgr2=0x7f84e0079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 -- 172.21.15.19:0/129799855 shutdown_connections 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.995+0000 7f84e6434700 1 -- 172.21.15.19:0/129799855 wait complete. 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 Processor -- start 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 -- start start 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 0x7f84e0070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0107e20 0x7f84e0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84e0108770 0x7f84e0071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0113230 con 0x7f84e0103540 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01130b0 con 0x7f84e0108770 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84e6434700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01133b0 con 0x7f84e0107e20 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.996+0000 7f84dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 0x7f84e0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 0x7f84e0070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55370/0 (socket says 172.21.15.19:55370) 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 -- 172.21.15.19:0/3226676381 learned_addr learned my addr 172.21.15.19:0/3226676381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84e49d1700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84e0108770 0x7f84e0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84df7fe700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0107e20 0x7f84e0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 -- 172.21.15.19:0/3226676381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0107e20 msgr2=0x7f84e0070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0107e20 0x7f84e0070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 -- 172.21.15.19:0/3226676381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84e0108770 msgr2=0x7f84e0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84e0108770 0x7f84e0071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 -- 172.21.15.19:0/3226676381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01af7c0 con 0x7f84e0103540 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84df7fe700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0107e20 0x7f84e0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dffff700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 0x7f84e0070810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c950 tx=0x7f84d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0007ac0 con 0x7f84e0103540 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01afab0 con 0x7f84e0103540 2024-05-17T09:17:58.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84d0004cc0 con 0x7f84e0103540 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0005420 con 0x7f84e0103540 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.997+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01b0010 con 0x7f84e0103540 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:57.999+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e01049a0 con 0x7f84e0103540 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.000+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f84d0004750 con 0x7f84e0103540 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.001+0000 7f84dd7fa700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f84c80b2880 0x7f84c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.001+0000 7f84df7fe700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f84c80b2880 0x7f84c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.001+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f84d00e25a0 con 0x7f84e0103540 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.002+0000 7f84df7fe700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f84c80b2880 0x7f84c80b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009990 tx=0x7f84d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:17:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.005+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84d0014030 con 0x7f84e0103540 2024-05-17T09:17:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.242+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f84e0071d40 con 0x7f84e0103540 2024-05-17T09:17:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.242+0000 7f84dd7fa700 1 -- 172.21.15.19:0/3226676381 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f84d00a9f10 con 0x7f84e0103540 2024-05-17T09:17:58.243 INFO:teuthology.orchestra.run.smithi019.stdout:253403070524 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.244+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f84c80b2880 msgr2=0x7f84c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.244+0000 7f84e6434700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f84c80b2880 0x7f84c80b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009990 tx=0x7f84d4005fd0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.244+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 msgr2=0x7f84e0070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.244+0000 7f84e6434700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 0x7f84e0070810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c950 tx=0x7f84d000cc60 comp rx=0 tx=0).stop 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 shutdown_connections 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84e0108770 0x7f84e0071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e0103540 0x7f84e0070810 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f84c80b2880 0x7f84c80b4d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 --2- 172.21.15.19:0/3226676381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84e0107e20 0x7f84e0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:17:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 >> 172.21.15.19:0/3226676381 conn(0x7f84e0078df0 msgr2=0x7f84e0101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:17:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 shutdown_connections 2024-05-17T09:17:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:17:58.245+0000 7f84e6434700 1 -- 172.21.15.19:0/3226676381 wait complete. 2024-05-17T09:17:58.404 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:58 smithi019 ceph-mon[31620]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T09:17:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:58 smithi069 ceph-mon[35081]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T09:17:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:58 smithi079 ceph-mon[35502]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T09:17:59.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411423 for osd.2 2024-05-17T09:17:59.204 DEBUG:teuthology.parallel:result is None 2024-05-17T09:17:59.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:17:59 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3226676381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T09:17:59.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:17:59 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3226676381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T09:17:59.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:17:59 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3226676381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T09:18:00.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:00 smithi019 ceph-mon[31620]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:18:00.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070520 got 253403070524 for osd.9 2024-05-17T09:18:00.423 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:00.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:00 smithi079 ceph-mon[35502]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:18:00.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:00 smithi069 ceph-mon[35081]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-17T09:18:00.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 -- 172.21.15.19:0/3093343191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 msgr2=0x7febe41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 --2- 172.21.15.19:0/3093343191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe41038c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7febd40099c0 tx=0x7febd4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 -- 172.21.15.19:0/3093343191 shutdown_connections 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 --2- 172.21.15.19:0/3093343191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febe4108730 0x7febe4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 --2- 172.21.15.19:0/3093343191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe41038c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 --2- 172.21.15.19:0/3093343191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe4107de0 0x7febe41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 -- 172.21.15.19:0/3093343191 >> 172.21.15.19:0/3093343191 conn(0x7febe40fedb0 msgr2=0x7febe41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 -- 172.21.15.19:0/3093343191 shutdown_connections 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.406+0000 7febe8ec5700 1 -- 172.21.15.19:0/3093343191 wait complete. 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.407+0000 7febe8ec5700 1 Processor -- start 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.407+0000 7febe8ec5700 1 -- start start 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe8ec5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe8ec5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febe4107de0 0x7febe419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe8ec5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe4108730 0x7febe419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe8ec5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febe4113210 con 0x7febe41034b0 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe8ec5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7febe4113090 con 0x7febe4107de0 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe8ec5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febe4113390 con 0x7febe4108730 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55386/0 (socket says 172.21.15.19:55386) 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 -- 172.21.15.19:0/346362006 learned_addr learned my addr 172.21.15.19:0/346362006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe2d9d700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe4108730 0x7febe419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe1d9b700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febe4107de0 0x7febe419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 -- 172.21.15.19:0/346362006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe4108730 msgr2=0x7febe419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe4108730 0x7febe419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 -- 172.21.15.19:0/346362006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febe4107de0 msgr2=0x7febe419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febe4107de0 0x7febe419c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 -- 172.21.15.19:0/346362006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe41a6ea0 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.408+0000 7febe259c700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe419bb50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7febd4000c00 tx=0x7febd40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.409+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd400f8a0 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.409+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febd4004830 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.409+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe41a7130 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.409+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4005410 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.409+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe41a7580 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.410+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe40512b0 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.413+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7febd4029060 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.414+0000 7febd37fe700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7febcc0b2900 0x7febcc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.414+0000 7febe1d9b700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7febcc0b2900 0x7febcc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.414+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7febd40e32b0 con 0x7febe41034b0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.414+0000 7febe1d9b700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7febcc0b2900 0x7febcc0b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febd8005950 tx=0x7febd80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.418+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febd4013070 con 0x7febe41034b0 2024-05-17T09:18:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.638+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7febe402e5e0 con 0x7febe41034b0 2024-05-17T09:18:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.639+0000 7febd37fe700 1 -- 172.21.15.19:0/346362006 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7febd40aacb0 con 0x7febe41034b0 2024-05-17T09:18:00.639 INFO:teuthology.orchestra.run.smithi019.stdout:274877906999 2024-05-17T09:18:00.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7febcc0b2900 msgr2=0x7febcc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7febcc0b2900 0x7febcc0b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febd8005950 tx=0x7febd80058e0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 msgr2=0x7febe419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe419bb50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7febd4000c00 tx=0x7febd40047b0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 shutdown_connections 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7febcc0b2900 0x7febcc0b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febe4108730 0x7febe419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe41034b0 0x7febe419bb50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 --2- 172.21.15.19:0/346362006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febe4107de0 0x7febe419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 >> 172.21.15.19:0/346362006 conn(0x7febe40fedb0 msgr2=0x7febe4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 shutdown_connections 2024-05-17T09:18:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:00.641+0000 7febe8ec5700 1 -- 172.21.15.19:0/346362006 wait complete. 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 -- 172.21.15.19:0/2703349297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 msgr2=0x7efca4114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 --2- 172.21.15.19:0/2703349297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca4114e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc90009990 tx=0x7efc90009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 -- 172.21.15.19:0/2703349297 shutdown_connections 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 --2- 172.21.15.19:0/2703349297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 0x7efca4077070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 --2- 172.21.15.19:0/2703349297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca4115360 0x7efca4117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 --2- 172.21.15.19:0/2703349297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca4114e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 -- 172.21.15.19:0/2703349297 >> 172.21.15.19:0/2703349297 conn(0x7efca40ffeb0 msgr2=0x7efca4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 -- 172.21.15.19:0/2703349297 shutdown_connections 2024-05-17T09:18:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.214+0000 7efcaae64700 1 -- 172.21.15.19:0/2703349297 wait complete. 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.215+0000 7efcaae64700 1 Processor -- start 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.215+0000 7efcaae64700 1 -- start start 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efcaae64700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 0x7efca41a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efcaae64700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca41a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efcaae64700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca4115360 0x7efca41a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efcaae64700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efca411a910 con 0x7efca40775b0 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efcaae64700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efca411a790 con 0x7efca4115360 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efcaae64700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efca411aa90 con 0x7efca4076c60 2024-05-17T09:18:01.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca41a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca41a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55416/0 (socket says 172.21.15.19:55416) 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca3fff700 1 -- 172.21.15.19:0/544909012 learned_addr learned my addr 172.21.15.19:0/544909012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca8c00700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 0x7efca41a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca9401700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca4115360 0x7efca41a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca8c00700 1 -- 172.21.15.19:0/544909012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca4115360 msgr2=0x7efca41a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca8c00700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca4115360 0x7efca41a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca8c00700 1 -- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 msgr2=0x7efca41a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca8c00700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca41a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca8c00700 1 -- 172.21.15.19:0/544909012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca41a61a0 con 0x7efca4076c60 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.216+0000 7efca3fff700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.217+0000 7efca8c00700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 0x7efca41a1380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e9f0 tx=0x7efc9800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.217+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800c9c0 con 0x7efca4076c60 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.217+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc98010430 con 0x7efca4076c60 2024-05-17T09:18:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.217+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98018680 con 0x7efca4076c60 2024-05-17T09:18:01.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.217+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca41a6490 con 0x7efca4076c60 2024-05-17T09:18:01.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.217+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41a68c0 con 0x7efca4076c60 2024-05-17T09:18:01.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.218+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca40512b0 con 0x7efca4076c60 2024-05-17T09:18:01.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.220+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efc98018820 con 0x7efca4076c60 2024-05-17T09:18:01.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.221+0000 7efca1ffb700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc940b2900 0x7efc940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:01.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.221+0000 7efca3fff700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc940b2900 0x7efc940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:01.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.221+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efc980e2700 con 0x7efca4076c60 2024-05-17T09:18:01.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.221+0000 7efca3fff700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc940b2900 0x7efc940b4e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efca4103500 tx=0x7efc90005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:01.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.226+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc980aa010 con 0x7efca4076c60 2024-05-17T09:18:01.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:01 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/346362006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T09:18:01.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:01 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/346362006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T09:18:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.449+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efca402e5e0 con 0x7efca4076c60 2024-05-17T09:18:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.449+0000 7efca1ffb700 1 -- 172.21.15.19:0/544909012 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc980b2090 con 0x7efca4076c60 2024-05-17T09:18:01.450 INFO:teuthology.orchestra.run.smithi019.stdout:55834574950 2024-05-17T09:18:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc940b2900 msgr2=0x7efc940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc940b2900 0x7efc940b4e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efca4103500 tx=0x7efc90005960 comp rx=0 tx=0).stop 2024-05-17T09:18:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 msgr2=0x7efca41a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 0x7efca41a1380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e9f0 tx=0x7efc9800ed00 comp rx=0 tx=0).stop 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 shutdown_connections 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efc940b2900 0x7efc940b4e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca4076c60 0x7efca41a1380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca4115360 0x7efca41a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 --2- 172.21.15.19:0/544909012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca40775b0 0x7efca41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 >> 172.21.15.19:0/544909012 conn(0x7efca40ffeb0 msgr2=0x7efca4100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 shutdown_connections 2024-05-17T09:18:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:01.452+0000 7efcaae64700 1 -- 172.21.15.19:0/544909012 wait complete. 2024-05-17T09:18:01.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:01 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/346362006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T09:18:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:02 smithi069 ceph-mon[35081]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-17T09:18:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/544909012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T09:18:02.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:02 smithi019 ceph-mon[31620]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-17T09:18:02.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/544909012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 -- 172.21.15.19:0/2901161393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc201034d0 msgr2=0x7fcc201038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 --2- 172.21.15.19:0/2901161393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc201034d0 0x7fcc201038e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc140099c0 tx=0x7fcc14009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 -- 172.21.15.19:0/2901161393 shutdown_connections 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 --2- 172.21.15.19:0/2901161393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc20108750 0x7fcc201100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 --2- 172.21.15.19:0/2901161393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc201034d0 0x7fcc201038e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 --2- 172.21.15.19:0/2901161393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc20107e00 0x7fcc20108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 -- 172.21.15.19:0/2901161393 >> 172.21.15.19:0/2901161393 conn(0x7fcc200fef20 msgr2=0x7fcc20101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 -- 172.21.15.19:0/2901161393 shutdown_connections 2024-05-17T09:18:02.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.488+0000 7fcc270ff700 1 -- 172.21.15.19:0/2901161393 wait complete. 2024-05-17T09:18:02.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 Processor -- start 2024-05-17T09:18:02.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 -- start start 2024-05-17T09:18:02.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc201034d0 0x7fcc20110890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:02.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 0x7fcc20110dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc20108750 0x7fcc20111310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20114380 con 0x7fcc20107e00 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20114200 con 0x7fcc201034d0 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.489+0000 7fcc270ff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20114500 con 0x7fcc20108750 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 0x7fcc20110dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 0x7fcc20110dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55440/0 (socket says 172.21.15.19:55440) 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 -- 172.21.15.19:0/3787037133 learned_addr learned my addr 172.21.15.19:0/3787037133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc24e9b700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc201034d0 0x7fcc20110890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc2569c700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc20108750 0x7fcc20111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 -- 172.21.15.19:0/3787037133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc20108750 msgr2=0x7fcc20111310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc20108750 0x7fcc20111310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 -- 172.21.15.19:0/3787037133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc201034d0 msgr2=0x7fcc20110890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc201034d0 0x7fcc20110890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 -- 172.21.15.19:0/3787037133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc201a91b0 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1ffff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 0x7fcc20110dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00d980 tx=0x7fcc0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c017070 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc0c009dc0 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00f3c0 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.490+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc201a94a0 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.491+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201a9a30 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.492+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc200512b0 con 0x7fcc20107e00 2024-05-17T09:18:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.493+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcc0c009930 con 0x7fcc20107e00 2024-05-17T09:18:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.494+0000 7fcc1dffb700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc100b2900 0x7fcc100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.494+0000 7fcc24e9b700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc100b2900 0x7fcc100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.494+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcc0c0e2080 con 0x7fcc20107e00 2024-05-17T09:18:02.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.495+0000 7fcc24e9b700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc100b2900 0x7fcc100b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc14000c00 tx=0x7fcc1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.497+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc0c0a9a10 con 0x7fcc20107e00 2024-05-17T09:18:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:02 smithi079 ceph-mon[35502]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-17T09:18:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/544909012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T09:18:02.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.715+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcc2002e5e0 con 0x7fcc20107e00 2024-05-17T09:18:02.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.715+0000 7fcc1dffb700 1 -- 172.21.15.19:0/3787037133 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc0c0a9310 con 0x7fcc20107e00 2024-05-17T09:18:02.716 INFO:teuthology.orchestra.run.smithi019.stdout:107374182492 2024-05-17T09:18:02.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.717+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc100b2900 msgr2=0x7fcc100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:02.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.717+0000 7fcc270ff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc100b2900 0x7fcc100b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc14000c00 tx=0x7fcc1401a040 comp rx=0 tx=0).stop 2024-05-17T09:18:02.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.717+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 msgr2=0x7fcc20110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:02.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.717+0000 7fcc270ff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 0x7fcc20110dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00d980 tx=0x7fcc0c00dc90 comp rx=0 tx=0).stop 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 shutdown_connections 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc100b2900 0x7fcc100b4e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc20108750 0x7fcc20111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc201034d0 0x7fcc20110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 --2- 172.21.15.19:0/3787037133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20107e00 0x7fcc20110dd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 >> 172.21.15.19:0/3787037133 conn(0x7fcc200fef20 msgr2=0x7fcc20107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 shutdown_connections 2024-05-17T09:18:02.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:02.718+0000 7fcc270ff700 1 -- 172.21.15.19:0/3787037133 wait complete. 2024-05-17T09:18:03.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:03 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3787037133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T09:18:03.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:03 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3787037133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T09:18:03.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:03 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3787037133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T09:18:03.637 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182486 got 107374182492 for osd.3 2024-05-17T09:18:03.638 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:04.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574947 got 55834574950 for osd.1 2024-05-17T09:18:04.109 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:04 smithi069 ceph-mon[35081]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T09:18:04.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:04 smithi019 ceph-mon[31620]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T09:18:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:04 smithi079 ceph-mon[35502]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T09:18:04.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906992 got 274877906999 for osd.10 2024-05-17T09:18:04.831 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:06.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:06 smithi019 ceph-mon[31620]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:06 smithi079 ceph-mon[35502]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.633+0000 7f3557e29700 1 -- 172.21.15.19:0/4177129992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3550105e30 msgr2=0x7f355010e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.633+0000 7f3557e29700 1 --2- 172.21.15.19:0/4177129992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3550105e30 0x7f355010e380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3540009990 tx=0x7f3540009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 -- 172.21.15.19:0/4177129992 shutdown_connections 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 --2- 172.21.15.19:0/4177129992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35501054e0 0x7f35501058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 --2- 172.21.15.19:0/4177129992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 0x7f3550110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 --2- 172.21.15.19:0/4177129992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3550105e30 0x7f355010e380 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 -- 172.21.15.19:0/4177129992 >> 172.21.15.19:0/4177129992 conn(0x7f3550078df0 msgr2=0x7f3550079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 -- 172.21.15.19:0/4177129992 shutdown_connections 2024-05-17T09:18:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 -- 172.21.15.19:0/4177129992 wait complete. 2024-05-17T09:18:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.634+0000 7f3557e29700 1 Processor -- start 2024-05-17T09:18:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 -- start start 2024-05-17T09:18:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 0x7f355019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3550105e30 0x7f355019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 0x7f355019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 0x7f355019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3550114320 con 0x7f35501054e0 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35501141a0 con 0x7f355010e910 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3557e29700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35501144a0 con 0x7f3550105e30 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 0x7f355019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46392/0 (socket says 172.21.15.19:46392) 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 -- 172.21.15.19:0/2408653605 learned_addr learned my addr 172.21.15.19:0/2408653605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f35553c4700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3550105e30 0x7f355019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f35563c6700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 0x7f355019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 -- 172.21.15.19:0/2408653605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3550105e30 msgr2=0x7f355019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3550105e30 0x7f355019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 -- 172.21.15.19:0/2408653605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 msgr2=0x7f355019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 0x7f355019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f3555bc5700 1 -- 172.21.15.19:0/2408653605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355019fbe0 con 0x7f35501054e0 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.635+0000 7f35563c6700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 0x7f355019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.636+0000 7f3555bc5700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 0x7f355019ad10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f354c0060b0 tx=0x7f354c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.636+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c01a070 con 0x7f35501054e0 2024-05-17T09:18:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.636+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355019fed0 con 0x7f35501054e0 2024-05-17T09:18:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.636+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f354c00ddd0 con 0x7f35501054e0 2024-05-17T09:18:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.636+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c014690 con 0x7f35501054e0 2024-05-17T09:18:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.636+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35501a77f0 con 0x7f35501054e0 2024-05-17T09:18:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.637+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35500512b0 con 0x7f35501054e0 2024-05-17T09:18:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.638+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f354c01d070 con 0x7f35501054e0 2024-05-17T09:18:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.639+0000 7f3546ffd700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f353c0b2880 0x7f353c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.639+0000 7f35553c4700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f353c0b2880 0x7f353c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.639+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f354c0e1e10 con 0x7f35501054e0 2024-05-17T09:18:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.640+0000 7f35553c4700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f353c0b2880 0x7f353c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f354000b5c0 tx=0x7f3540005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:06.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.644+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f354c010070 con 0x7f35501054e0 2024-05-17T09:18:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:06 smithi069 ceph-mon[35081]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:06.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.861+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f355019c070 con 0x7f35501054e0 2024-05-17T09:18:06.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.862+0000 7f3546ffd700 1 -- 172.21.15.19:0/2408653605 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f354c0a9780 con 0x7f35501054e0 2024-05-17T09:18:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:180388626510 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f353c0b2880 msgr2=0x7f353c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f353c0b2880 0x7f353c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f354000b5c0 tx=0x7f3540005960 comp rx=0 tx=0).stop 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 msgr2=0x7f355019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 0x7f355019ad10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f354c0060b0 tx=0x7f354c00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 shutdown_connections 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355010e910 0x7f355019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35501054e0 0x7f355019ad10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f353c0b2880 0x7f353c0b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 --2- 172.21.15.19:0/2408653605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3550105e30 0x7f355019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 >> 172.21.15.19:0/2408653605 conn(0x7f3550078df0 msgr2=0x7f3550100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 shutdown_connections 2024-05-17T09:18:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.864+0000 7f3557e29700 1 -- 172.21.15.19:0/2408653605 wait complete. 2024-05-17T09:18:06.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/1945579065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38103c10 msgr2=0x7f1c38104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/1945579065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38103c10 0x7f1c38104020 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0099c0 tx=0x7f1c2c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/1945579065 shutdown_connections 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/1945579065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38108760 0x7f1c381100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/1945579065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38103c10 0x7f1c38104020 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/1945579065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38107e10 0x7f1c38108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/1945579065 >> 172.21.15.19:0/1945579065 conn(0x7f1c380fe6e0 msgr2=0x7f1c38100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/1945579065 shutdown_connections 2024-05-17T09:18:06.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.874+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/1945579065 wait complete. 2024-05-17T09:18:06.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 Processor -- start 2024-05-17T09:18:06.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 -- start start 2024-05-17T09:18:06.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38103c10 0x7f1c3819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38107e10 0x7f1c3819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38108760 0x7f1c3819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113240 con 0x7f1c38108760 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381130c0 con 0x7f1c38107e10 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3eeb3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381133c0 con 0x7f1c38103c10 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c37fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38107e10 0x7f1c3819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c3d450700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38108760 0x7f1c3819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c37fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38107e10 0x7f1c3819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45516/0 (socket says 172.21.15.19:45516) 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.875+0000 7f1c37fff700 1 -- 172.21.15.19:0/3622293787 learned_addr learned my addr 172.21.15.19:0/3622293787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3cc4f700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38103c10 0x7f1c3819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3d450700 1 -- 172.21.15.19:0/3622293787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38103c10 msgr2=0x7f1c3819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3d450700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38103c10 0x7f1c3819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3d450700 1 -- 172.21.15.19:0/3622293787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38107e10 msgr2=0x7f1c3819c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3d450700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38107e10 0x7f1c3819c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3d450700 1 -- 172.21.15.19:0/3622293787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c381a6f30 con 0x7f1c38108760 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3cc4f700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38103c10 0x7f1c3819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3d450700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38108760 0x7f1c3819c680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c30007f10 tx=0x7f1c3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000ddd0 con 0x7f1c38108760 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c30009e20 con 0x7f1c38108760 2024-05-17T09:18:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c381a7220 con 0x7f1c38108760 2024-05-17T09:18:06.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.876+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30015400 con 0x7f1c38108760 2024-05-17T09:18:06.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.877+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c381a7760 con 0x7f1c38108760 2024-05-17T09:18:06.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.878+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c380512b0 con 0x7f1c38108760 2024-05-17T09:18:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.880+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1c30005000 con 0x7f1c38108760 2024-05-17T09:18:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.880+0000 7f1c35ffb700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1c280b2900 0x7f1c280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.880+0000 7f1c3cc4f700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1c280b2900 0x7f1c280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.881+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1c300e2720 con 0x7f1c38108760 2024-05-17T09:18:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.881+0000 7f1c3cc4f700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1c280b2900 0x7f1c280b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009990 tx=0x7f1c2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:06.884+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c30010070 con 0x7f1c38108760 2024-05-17T09:18:07.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.103+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1c3802e7a0 con 0x7f1c38108760 2024-05-17T09:18:07.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.104+0000 7f1c35ffb700 1 -- 172.21.15.19:0/3622293787 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1c300aa090 con 0x7f1c38108760 2024-05-17T09:18:07.105 INFO:teuthology.orchestra.run.smithi019.stdout:133143986263 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1c280b2900 msgr2=0x7f1c280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1c280b2900 0x7f1c280b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009990 tx=0x7f1c2c005fd0 comp rx=0 tx=0).stop 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38108760 msgr2=0x7f1c3819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38108760 0x7f1c3819c680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c30007f10 tx=0x7f1c3000d3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 shutdown_connections 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1c280b2900 0x7f1c280b4e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c38103c10 0x7f1c3819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c38108760 0x7f1c3819c680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 --2- 172.21.15.19:0/3622293787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38107e10 0x7f1c3819c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 >> 172.21.15.19:0/3622293787 conn(0x7f1c380fe6e0 msgr2=0x7f1c381015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 shutdown_connections 2024-05-17T09:18:07.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:07.106+0000 7f1c3eeb3700 1 -- 172.21.15.19:0/3622293787 wait complete. 2024-05-17T09:18:07.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:07 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2408653605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T09:18:07.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:07 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3622293787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T09:18:07.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:07 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2408653605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T09:18:07.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:07 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3622293787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T09:18:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:07.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:07 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2408653605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T09:18:07.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:07 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3622293787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T09:18:08.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:08 smithi019 ceph-mon[31620]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:08 smithi079 ceph-mon[35502]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:08 smithi069 ceph-mon[35081]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:09.580 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986258 got 133143986263 for osd.4 2024-05-17T09:18:09.581 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:10.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:10 smithi019 ceph-mon[31620]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:10 smithi079 ceph-mon[35502]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:10 smithi069 ceph-mon[35081]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 -- 172.21.15.19:0/317076886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0104600 msgr2=0x7f0ef0104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/317076886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0104600 0x7f0ef0104a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b0d0 tx=0x7f0ee000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 -- 172.21.15.19:0/317076886 shutdown_connections 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/317076886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0109880 0x7f0ef01111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/317076886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0104600 0x7f0ef0104a10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/317076886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ef0108f30 0x7f0ef0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 -- 172.21.15.19:0/317076886 >> 172.21.15.19:0/317076886 conn(0x7f0ef007af60 msgr2=0x7f0ef007b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 -- 172.21.15.19:0/317076886 shutdown_connections 2024-05-17T09:18:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.246+0000 7f0ef7c59700 1 -- 172.21.15.19:0/317076886 wait complete. 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 Processor -- start 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 -- start start 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0104600 0x7f0ef019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ef0108f30 0x7f0ef019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 0x7f0ef019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0114360 con 0x7f0ef0109880 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01141e0 con 0x7f0ef0104600 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef7c59700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01144e0 con 0x7f0ef0108f30 2024-05-17T09:18:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef61f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 0x7f0ef019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef61f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 0x7f0ef019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46416/0 (socket says 172.21.15.19:46416) 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef61f6700 1 -- 172.21.15.19:0/2426842650 learned_addr learned my addr 172.21.15.19:0/2426842650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.247+0000 7f0ef59f5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0104600 0x7f0ef019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef51f4700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ef0108f30 0x7f0ef019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef59f5700 1 -- 172.21.15.19:0/2426842650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ef0108f30 msgr2=0x7f0ef019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef59f5700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ef0108f30 0x7f0ef019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef59f5700 1 -- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 msgr2=0x7f0ef019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef59f5700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 0x7f0ef019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef59f5700 1 -- 172.21.15.19:0/2426842650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019fc00 con 0x7f0ef0104600 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef61f6700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 0x7f0ef019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef59f5700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0104600 0x7f0ef019ad30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b770 tx=0x7f0ee0007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee001f070 con 0x7f0ef0104600 2024-05-17T09:18:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef019fe90 con 0x7f0ef0104600 2024-05-17T09:18:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ee0004c50 con 0x7f0ef0104600 2024-05-17T09:18:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee001b690 con 0x7f0ef0104600 2024-05-17T09:18:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.248+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a7810 con 0x7f0ef0104600 2024-05-17T09:18:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.249+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef00512b0 con 0x7f0ef0104600 2024-05-17T09:18:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.251+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0ee0022070 con 0x7f0ef0104600 2024-05-17T09:18:11.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.252+0000 7f0eeeffd700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0ed80b2880 0x7f0ed80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.252+0000 7f0ef51f4700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0ed80b2880 0x7f0ed80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.252+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0ee000f020 con 0x7f0ef0104600 2024-05-17T09:18:11.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.252+0000 7f0ef51f4700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0ed80b2880 0x7f0ed80b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005fd0 tx=0x7f0ee4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.255+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ee0029020 con 0x7f0ef0104600 2024-05-17T09:18:11.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.479+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0ef002e5e0 con 0x7f0ef0104600 2024-05-17T09:18:11.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:18:11.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:11.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:11 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:11.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.480+0000 7f0eeeffd700 1 -- 172.21.15.19:0/2426842650 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ee00ab4e0 con 0x7f0ef0104600 2024-05-17T09:18:11.481 INFO:teuthology.orchestra.run.smithi019.stdout:227633266755 2024-05-17T09:18:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0ed80b2880 msgr2=0x7f0ed80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0ed80b2880 0x7f0ed80b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005fd0 tx=0x7f0ee4005f60 comp rx=0 tx=0).stop 2024-05-17T09:18:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0104600 msgr2=0x7f0ef019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0104600 0x7f0ef019ad30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b770 tx=0x7f0ee0007a20 comp rx=0 tx=0).stop 2024-05-17T09:18:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 shutdown_connections 2024-05-17T09:18:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ef0109880 0x7f0ef019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ef0104600 0x7f0ef019ad30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0ed80b2880 0x7f0ed80b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 --2- 172.21.15.19:0/2426842650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ef0108f30 0x7f0ef019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 >> 172.21.15.19:0/2426842650 conn(0x7f0ef007af60 msgr2=0x7f0ef0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 shutdown_connections 2024-05-17T09:18:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.483+0000 7f0ef7c59700 1 -- 172.21.15.19:0/2426842650 wait complete. 2024-05-17T09:18:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:18:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:11 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 -- 172.21.15.19:0/1341745857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 msgr2=0x7f32401038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 --2- 172.21.15.19:0/1341745857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32401038c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f323000b0d0 tx=0x7f323000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 -- 172.21.15.19:0/1341745857 shutdown_connections 2024-05-17T09:18:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 --2- 172.21.15.19:0/1341745857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 0x7f3240110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 --2- 172.21.15.19:0/1341745857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32401038c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 --2- 172.21.15.19:0/1341745857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3240107de0 0x7f32401081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 -- 172.21.15.19:0/1341745857 >> 172.21.15.19:0/1341745857 conn(0x7f32400fedb0 msgr2=0x7f32401011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.609+0000 7f3245de6700 1 -- 172.21.15.19:0/1341745857 shutdown_connections 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.610+0000 7f3245de6700 1 -- 172.21.15.19:0/1341745857 wait complete. 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.610+0000 7f3245de6700 1 Processor -- start 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 -- start start 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32400ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3240107de0 0x7f32400ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 0x7f32400ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3240113210 con 0x7f32401034b0 2024-05-17T09:18:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3240113090 con 0x7f3240108730 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f3245de6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3240113390 con 0x7f3240107de0 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32400ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32400ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46430/0 (socket says 172.21.15.19:46430) 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 -- 172.21.15.19:0/152740493 learned_addr learned my addr 172.21.15.19:0/152740493 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323effd700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3240107de0 0x7f32400ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323ffff700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 0x7f32400ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 -- 172.21.15.19:0/152740493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3240107de0 msgr2=0x7f32400ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3240107de0 0x7f32400ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 -- 172.21.15.19:0/152740493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 msgr2=0x7f32400ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 0x7f32400ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 -- 172.21.15.19:0/152740493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32401a9170 con 0x7f32401034b0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323ffff700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 0x7f32400ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.611+0000 7f323f7fe700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32400ff290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3230007d90 tx=0x7f3230007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.612+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323001f030 con 0x7f32401034b0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.612+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32401a9400 con 0x7f32401034b0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.612+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f323001bad0 con 0x7f32401034b0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.612+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230012b50 con 0x7f32401034b0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.612+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401a9900 con 0x7f32401034b0 2024-05-17T09:18:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.613+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32400512b0 con 0x7f32401034b0 2024-05-17T09:18:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.616+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3230022030 con 0x7f32401034b0 2024-05-17T09:18:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.616+0000 7f323cff9700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f322c0b2880 0x7f322c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.616+0000 7f323effd700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f322c0b2880 0x7f322c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.616+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f323000f020 con 0x7f32401034b0 2024-05-17T09:18:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.617+0000 7f323effd700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f322c0b2880 0x7f322c0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3228005fd0 tx=0x7f3228005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.620+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3230029020 con 0x7f32401034b0 2024-05-17T09:18:11.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:18:11.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:11.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:11 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:11.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.847+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f324002e5e0 con 0x7f32401034b0 2024-05-17T09:18:11.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.847+0000 7f323cff9700 1 -- 172.21.15.19:0/152740493 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f32300ab960 con 0x7f32401034b0 2024-05-17T09:18:11.848 INFO:teuthology.orchestra.run.smithi019.stdout:154618822740 2024-05-17T09:18:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f322c0b2880 msgr2=0x7f322c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f322c0b2880 0x7f322c0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3228005fd0 tx=0x7f3228005f60 comp rx=0 tx=0).stop 2024-05-17T09:18:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 msgr2=0x7f32400ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32400ff290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3230007d90 tx=0x7f3230007e70 comp rx=0 tx=0).stop 2024-05-17T09:18:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 shutdown_connections 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3240108730 0x7f32400ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32401034b0 0x7f32400ff290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f322c0b2880 0x7f322c0b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 --2- 172.21.15.19:0/152740493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3240107de0 0x7f32400ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 >> 172.21.15.19:0/152740493 conn(0x7f32400fedb0 msgr2=0x7f324006ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 shutdown_connections 2024-05-17T09:18:11.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:11.850+0000 7f3245de6700 1 -- 172.21.15.19:0/152740493 wait complete. 2024-05-17T09:18:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2426842650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T09:18:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/152740493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T09:18:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:18:12.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:18:12.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2426842650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/152740493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:18:12.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:18:12.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2426842650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/152740493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:18:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:18:12.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:18:12.691 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626505 got 180388626510 for osd.6 2024-05-17T09:18:12.691 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:12.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266749 got 227633266755 for osd.8 2024-05-17T09:18:12.835 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 -- 172.21.15.19:0/3470085424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c105e20 msgr2=0x7f095c10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 --2- 172.21.15.19:0/3470085424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c105e20 0x7f095c10e370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0950009990 tx=0x7f0950009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 -- 172.21.15.19:0/3470085424 shutdown_connections 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 --2- 172.21.15.19:0/3470085424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f095c10e900 0x7f095c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 --2- 172.21.15.19:0/3470085424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c1054d0 0x7f095c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 --2- 172.21.15.19:0/3470085424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c105e20 0x7f095c10e370 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 -- 172.21.15.19:0/3470085424 >> 172.21.15.19:0/3470085424 conn(0x7f095c0fef20 msgr2=0x7f095c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 -- 172.21.15.19:0/3470085424 shutdown_connections 2024-05-17T09:18:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.528+0000 7f09622e0700 1 -- 172.21.15.19:0/3470085424 wait complete. 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 Processor -- start 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 -- start start 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 0x7f095c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f095c105e20 0x7f095c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c10e900 0x7f095c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f095c114310 con 0x7f095c1054d0 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f095c114190 con 0x7f095c10e900 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.529+0000 7f09622e0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f095c114490 con 0x7f095c105e20 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 0x7f095c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 0x7f095c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46448/0 (socket says 172.21.15.19:46448) 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 -- 172.21.15.19:0/3294561921 learned_addr learned my addr 172.21.15.19:0/3294561921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f096087d700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c10e900 0x7f095c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095b7fe700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f095c105e20 0x7f095c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 -- 172.21.15.19:0/3294561921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f095c105e20 msgr2=0x7f095c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f095c105e20 0x7f095c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 -- 172.21.15.19:0/3294561921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c10e900 msgr2=0x7f095c19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c10e900 0x7f095c19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 -- 172.21.15.19:0/3294561921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c19fc10 con 0x7f095c1054d0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f096087d700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c10e900 0x7f095c19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f095bfff700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 0x7f095c19ad40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f094c00d950 tx=0x7f094c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c016070 con 0x7f095c1054d0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c19ff00 con 0x7f095c1054d0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f094c010460 con 0x7f095c1054d0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.530+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c00f620 con 0x7f095c1054d0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.531+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a7820 con 0x7f095c1054d0 2024-05-17T09:18:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.531+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c0512b0 con 0x7f095c1054d0 2024-05-17T09:18:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.534+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f094c019070 con 0x7f095c1054d0 2024-05-17T09:18:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.535+0000 7f09597fa700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f09440b2880 0x7f09440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.535+0000 7f095b7fe700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f09440b2880 0x7f09440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.535+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f094c0e2230 con 0x7f095c1054d0 2024-05-17T09:18:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.535+0000 7f095b7fe700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f09440b2880 0x7f09440b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0950006010 tx=0x7f095000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.538+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f094c0a9b40 con 0x7f095c1054d0 2024-05-17T09:18:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.755+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f095c02e5e0 con 0x7f095c1054d0 2024-05-17T09:18:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.755+0000 7f09597fa700 1 -- 172.21.15.19:0/3294561921 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f094c0a9440 con 0x7f095c1054d0 2024-05-17T09:18:13.756 INFO:teuthology.orchestra.run.smithi019.stdout:300647710773 2024-05-17T09:18:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f09440b2880 msgr2=0x7f09440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f09440b2880 0x7f09440b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0950006010 tx=0x7f095000b540 comp rx=0 tx=0).stop 2024-05-17T09:18:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 msgr2=0x7f095c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 0x7f095c19ad40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f094c00d950 tx=0x7f094c00dd10 comp rx=0 tx=0).stop 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 shutdown_connections 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c10e900 0x7f095c19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1054d0 0x7f095c19ad40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f09440b2880 0x7f09440b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 --2- 172.21.15.19:0/3294561921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f095c105e20 0x7f095c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 >> 172.21.15.19:0/3294561921 conn(0x7f095c0fef20 msgr2=0x7f095c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 shutdown_connections 2024-05-17T09:18:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:13.758+0000 7f09622e0700 1 -- 172.21.15.19:0/3294561921 wait complete. 2024-05-17T09:18:14.422 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710767 got 300647710773 for osd.11 2024-05-17T09:18:14.422 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:14 smithi079 ceph-mon[35502]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:14 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3294561921' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T09:18:14.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:14 smithi019 ceph-mon[31620]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:14.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:14 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3294561921' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T09:18:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:14 smithi069 ceph-mon[35081]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:14 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3294561921' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T09:18:14.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822735 got 154618822740 for osd.5 2024-05-17T09:18:14.899 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 -- 172.21.15.19:0/2569022248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 msgr2=0x7fcab8103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 --2- 172.21.15.19:0/2569022248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8103950 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80099c0 tx=0x7fcaa8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 -- 172.21.15.19:0/2569022248 shutdown_connections 2024-05-17T09:18:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 --2- 172.21.15.19:0/2569022248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 0x7fcab81100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 --2- 172.21.15.19:0/2569022248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8103950 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 --2- 172.21.15.19:0/2569022248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcab8107e20 0x7fcab8108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 -- 172.21.15.19:0/2569022248 >> 172.21.15.19:0/2569022248 conn(0x7fcab8078df0 msgr2=0x7fcab8079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:15.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 -- 172.21.15.19:0/2569022248 shutdown_connections 2024-05-17T09:18:15.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.081+0000 7fcabd035700 1 -- 172.21.15.19:0/2569022248 wait complete. 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.082+0000 7fcabd035700 1 Processor -- start 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.082+0000 7fcabd035700 1 -- start start 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcabd035700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcabd035700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcab8107e20 0x7fcab8110ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcabd035700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 0x7fcab8111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcabd035700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81143a0 con 0x7fcab8103540 2024-05-17T09:18:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcabd035700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8114220 con 0x7fcab8108770 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcabd035700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8114520 con 0x7fcab8107e20 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8110ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52898/0 (socket says 172.21.15.19:52898) 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 -- 172.21.15.19:0/685173905 learned_addr learned my addr 172.21.15.19:0/685173905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcaae59c700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcab8107e20 0x7fcab8110ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab759e700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 0x7fcab8111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 -- 172.21.15.19:0/685173905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcab8107e20 msgr2=0x7fcab8110ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcab8107e20 0x7fcab8110ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 -- 172.21.15.19:0/685173905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 msgr2=0x7fcab8111530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 0x7fcab8111530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab6d9d700 1 -- 172.21.15.19:0/685173905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab81a90d0 con 0x7fcab8103540 2024-05-17T09:18:15.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.083+0000 7fcab759e700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 0x7fcab8111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.084+0000 7fcab6d9d700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8110ab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005f50 tx=0x7fcaa80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.084+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8027070 con 0x7fcab8103540 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.084+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaa800fd60 con 0x7fcab8103540 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.084+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8022e30 con 0x7fcab8103540 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.084+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab81a9360 con 0x7fcab8103540 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.084+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81a97b0 con 0x7fcab8103540 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.085+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab80512b0 con 0x7fcab8103540 2024-05-17T09:18:15.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.087+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcaa802a070 con 0x7fcab8103540 2024-05-17T09:18:15.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.088+0000 7fcab4d99700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa40b2880 0x7fcaa40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:15.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.088+0000 7fcaae59c700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa40b2880 0x7fcaa40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:15.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.088+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcaa80e3500 con 0x7fcab8103540 2024-05-17T09:18:15.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.089+0000 7fcaae59c700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa40b2880 0x7fcaa40b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcab8079490 tx=0x7fcaa000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:15.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.092+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcaa8013070 con 0x7fcab8103540 2024-05-17T09:18:15.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.314+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcab802e7a0 con 0x7fcab8103540 2024-05-17T09:18:15.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.314+0000 7fcab4d99700 1 -- 172.21.15.19:0/685173905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcaa80aaf00 con 0x7fcab8103540 2024-05-17T09:18:15.315 INFO:teuthology.orchestra.run.smithi019.stdout:201863462986 2024-05-17T09:18:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa40b2880 msgr2=0x7fcaa40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa40b2880 0x7fcaa40b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcab8079490 tx=0x7fcaa000b410 comp rx=0 tx=0).stop 2024-05-17T09:18:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 msgr2=0x7fcab8110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8110ab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005f50 tx=0x7fcaa80057c0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 shutdown_connections 2024-05-17T09:18:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8108770 0x7fcab8111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcab8103540 0x7fcab8110ab0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcaa40b2880 0x7fcaa40b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 --2- 172.21.15.19:0/685173905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcab8107e20 0x7fcab8110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 >> 172.21.15.19:0/685173905 conn(0x7fcab8078df0 msgr2=0x7fcab8101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 shutdown_connections 2024-05-17T09:18:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:15.317+0000 7fcabd035700 1 -- 172.21.15.19:0/685173905 wait complete. 2024-05-17T09:18:16.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462979 got 201863462986 for osd.7 2024-05-17T09:18:16.139 DEBUG:teuthology.parallel:result is None 2024-05-17T09:18:16.140 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T09:18:16.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:18:16.400 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:16.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:16 smithi019 ceph-mon[31620]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:16.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:16 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/685173905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T09:18:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:16 smithi079 ceph-mon[35502]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:16 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/685173905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T09:18:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:16 smithi069 ceph-mon[35081]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:16 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/685173905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 -- 172.21.15.19:0/2758813455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84077530 msgr2=0x7f1a841106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2758813455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84077530 0x7f1a841106c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009990 tx=0x7f1a78009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 -- 172.21.15.19:0/2758813455 shutdown_connections 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2758813455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84076be0 0x7f1a84076ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2758813455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a84110c00 0x7f1a84113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2758813455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84077530 0x7f1a841106c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 -- 172.21.15.19:0/2758813455 >> 172.21.15.19:0/2758813455 conn(0x7f1a840ffec0 msgr2=0x7f1a84102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 -- 172.21.15.19:0/2758813455 shutdown_connections 2024-05-17T09:18:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.167+0000 7f1a8b853700 1 -- 172.21.15.19:0/2758813455 wait complete. 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 Processor -- start 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 -- start start 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a84076be0 0x7f1a8419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84077530 0x7f1a8419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 0x7f1a841a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84116510 con 0x7f1a84110c00 2024-05-17T09:18:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84116390 con 0x7f1a84076be0 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a8b853700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84116690 con 0x7f1a84077530 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a89df0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 0x7f1a841a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a89df0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 0x7f1a841a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52916/0 (socket says 172.21.15.19:52916) 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.168+0000 7f1a89df0700 1 -- 172.21.15.19:0/2115732204 learned_addr learned my addr 172.21.15.19:0/2115732204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a895ef700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a84076be0 0x7f1a8419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a88dee700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84077530 0x7f1a8419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a89df0700 1 -- 172.21.15.19:0/2115732204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84077530 msgr2=0x7f1a8419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a89df0700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84077530 0x7f1a8419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a89df0700 1 -- 172.21.15.19:0/2115732204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a84076be0 msgr2=0x7f1a8419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a89df0700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a84076be0 0x7f1a8419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a89df0700 1 -- 172.21.15.19:0/2115732204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a841a1e70 con 0x7f1a84110c00 2024-05-17T09:18:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.169+0000 7f1a88dee700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84077530 0x7f1a8419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.170+0000 7f1a89df0700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 0x7f1a841a17d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00ef30 tx=0x7f1a7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.170+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0092e0 con 0x7f1a84110c00 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.170+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a7c004bb0 con 0x7f1a84110c00 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.170+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a841a2100 con 0x7f1a84110c00 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.170+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c004020 con 0x7f1a84110c00 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.170+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a841a9a80 con 0x7f1a84110c00 2024-05-17T09:18:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.171+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a840512b0 con 0x7f1a84110c00 2024-05-17T09:18:18.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.173+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1a7c01b030 con 0x7f1a84110c00 2024-05-17T09:18:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.173+0000 7f1a767fc700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1a700b2900 0x7f1a700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.174+0000 7f1a895ef700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1a700b2900 0x7f1a700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.174+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1a7c0e2a10 con 0x7f1a84110c00 2024-05-17T09:18:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.174+0000 7f1a895ef700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1a700b2900 0x7f1a700b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009c60 tx=0x7f1a80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.177+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a7c022020 con 0x7f1a84110c00 2024-05-17T09:18:18.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.385+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a8419e300 con 0x7f1a700b2900 2024-05-17T09:18:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.395+0000 7f1a767fc700 1 -- 172.21.15.19:0/2115732204 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340725 (secure 0 0 0) 0x7f1a8419e300 con 0x7f1a700b2900 2024-05-17T09:18:18.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1a700b2900 msgr2=0x7f1a700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1a700b2900 0x7f1a700b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009c60 tx=0x7f1a80009260 comp rx=0 tx=0).stop 2024-05-17T09:18:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 msgr2=0x7f1a841a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 0x7f1a841a17d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00ef30 tx=0x7f1a7c00c3a0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 shutdown_connections 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a84110c00 0x7f1a841a17d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a84076be0 0x7f1a8419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1a700b2900 0x7f1a700b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 --2- 172.21.15.19:0/2115732204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a84077530 0x7f1a8419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 >> 172.21.15.19:0/2115732204 conn(0x7f1a840ffec0 msgr2=0x7f1a84100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 shutdown_connections 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:18.398+0000 7f1a8b853700 1 -- 172.21.15.19:0/2115732204 wait complete. 2024-05-17T09:18:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:18:18.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:18 smithi019 ceph-mon[31620]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:18 smithi079 ceph-mon[35502]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:18 smithi069 ceph-mon[35081]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:19.209 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-05-17T09:18:16.924308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659661,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":803,"num_read_kb":498,"num_write":302,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348644,"kb_used_data":11972,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593180,"statfs":{"total":1151940427776,"available":1151583416320,"internally_reserved":0,"allocated":12259328,"data_stored":8136029,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003821"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906682+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906682+0000","last_peered":"2024-05-17T09:17:46.906682+0000","last_clean":"2024-05-17T09:17:46.906682+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906682+0000","last_undegraded":"2024-05-17T09:17:46.906682+0000","last_fullsized":"2024-05-17T09:17:46.906682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897617+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.897617+0000","last_peered":"2024-05-17T09:17:46.897617+0000","last_clean":"2024-05-17T09:17:46.897617+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.897617+0000","last_undegraded":"2024-05-17T09:17:46.897617+0000","last_fullsized":"2024-05-17T09:17:46.897617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893315+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.893315+0000","last_peered":"2024-05-17T09:17:46.893315+0000","last_clean":"2024-05-17T09:17:46.893315+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.893315+0000","last_undegraded":"2024-05-17T09:17:46.893315+0000","last_fullsized":"2024-05-17T09:17:46.893315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.884948+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.884948+0000","last_peered":"2024-05-17T09:17:46.884948+0000","last_clean":"2024-05-17T09:17:46.884948+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.884948+0000","last_undegraded":"2024-05-17T09:17:46.884948+0000","last_fullsized":"2024-05-17T09:17:46.884948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897920+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.897920+0000","last_peered":"2024-05-17T09:17:46.897920+0000","last_clean":"2024-05-17T09:17:46.897920+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.897920+0000","last_undegraded":"2024-05-17T09:17:46.897920+0000","last_fullsized":"2024-05-17T09:17:46.897920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893468+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893468+0000","last_peered":"2024-05-17T09:17:46.893468+0000","last_clean":"2024-05-17T09:17:46.893468+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893468+0000","last_undegraded":"2024-05-17T09:17:46.893468+0000","last_fullsized":"2024-05-17T09:17:46.893468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908147+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908147+0000","last_peered":"2024-05-17T09:17:46.908147+0000","last_clean":"2024-05-17T09:17:46.908147+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908147+0000","last_undegraded":"2024-05-17T09:17:46.908147+0000","last_fullsized":"2024-05-17T09:17:46.908147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889231+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889231+0000","last_peered":"2024-05-17T09:17:46.889231+0000","last_clean":"2024-05-17T09:17:46.889231+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889231+0000","last_undegraded":"2024-05-17T09:17:46.889231+0000","last_fullsized":"2024-05-17T09:17:46.889231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317769+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:48.317769+0000","last_peered":"2024-05-17T09:16:48.317769+0000","last_clean":"2024-05-17T09:16:48.317769+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:48.317769+0000","last_undegraded":"2024-05-17T09:16:48.317769+0000","last_fullsized":"2024-05-17T09:16:48.317769+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898546+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898546+0000","last_peered":"2024-05-17T09:17:46.898546+0000","last_clean":"2024-05-17T09:17:46.898546+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898546+0000","last_undegraded":"2024-05-17T09:17:46.898546+0000","last_fullsized":"2024-05-17T09:17:46.898546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755662+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:16:48.755662+0000","last_peered":"2024-05-17T09:16:48.755662+0000","last_clean":"2024-05-17T09:16:48.755662+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:16:48.755662+0000","last_undegraded":"2024-05-17T09:16:48.755662+0000","last_fullsized":"2024-05-17T09:16:48.755662+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905045+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905045+0000","last_peered":"2024-05-17T09:17:46.905045+0000","last_clean":"2024-05-17T09:17:46.905045+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905045+0000","last_undegraded":"2024-05-17T09:17:46.905045+0000","last_fullsized":"2024-05-17T09:17:46.905045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:15.414120+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:18:15.414120+0000","last_peered":"2024-05-17T09:18:15.414120+0000","last_clean":"2024-05-17T09:18:15.414120+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:18:15.414120+0000","last_undegraded":"2024-05-17T09:18:15.414120+0000","last_fullsized":"2024-05-17T09:18:15.414120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322317+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:16:48.322317+0000","last_peered":"2024-05-17T09:16:48.322317+0000","last_clean":"2024-05-17T09:16:48.322317+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:16:48.322317+0000","last_undegraded":"2024-05-17T09:16:48.322317+0000","last_fullsized":"2024-05-17T09:16:48.322317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860181+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:16:48.860181+0000","last_peered":"2024-05-17T09:16:48.860181+0000","last_clean":"2024-05-17T09:16:48.860181+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:16:48.860181+0000","last_undegraded":"2024-05-17T09:16:48.860181+0000","last_fullsized":"2024-05-17T09:16:48.860181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:12.408554+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:18:12.408554+0000","last_peered":"2024-05-17T09:18:12.408554+0000","last_clean":"2024-05-17T09:18:12.408554+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:18:12.408554+0000","last_undegraded":"2024-05-17T09:18:12.408554+0000","last_fullsized":"2024-05-17T09:18:12.408554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:13.410079+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:18:13.410079+0000","last_peered":"2024-05-17T09:18:13.410079+0000","last_clean":"2024-05-17T09:18:13.410079+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:18:13.410079+0000","last_undegraded":"2024-05-17T09:18:13.410079+0000","last_fullsized":"2024-05-17T09:18:13.410079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":240,"num_read_kb":183,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903452+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903452+0000","last_peered":"2024-05-17T09:17:46.903452+0000","last_clean":"2024-05-17T09:17:46.903452+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903452+0000","last_undegraded":"2024-05-17T09:17:46.903452+0000","last_fullsized":"2024-05-17T09:17:46.903452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423686+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423686+0000","last_peered":"2024-05-17T09:16:48.423686+0000","last_clean":"2024-05-17T09:16:48.423686+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423686+0000","last_undegraded":"2024-05-17T09:16:48.423686+0000","last_fullsized":"2024-05-17T09:16:48.423686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904979+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.904979+0000","last_peered":"2024-05-17T09:17:46.904979+0000","last_clean":"2024-05-17T09:17:46.904979+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.904979+0000","last_undegraded":"2024-05-17T09:17:46.904979+0000","last_fullsized":"2024-05-17T09:17:46.904979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:15.412322+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:18:15.412322+0000","last_peered":"2024-05-17T09:18:15.412322+0000","last_clean":"2024-05-17T09:18:15.412322+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:18:15.412322+0000","last_undegraded":"2024-05-17T09:18:15.412322+0000","last_fullsized":"2024-05-17T09:18:15.412322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900338+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.900338+0000","last_peered":"2024-05-17T09:17:46.900338+0000","last_clean":"2024-05-17T09:17:46.900338+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.900338+0000","last_undegraded":"2024-05-17T09:17:46.900338+0000","last_fullsized":"2024-05-17T09:17:46.900338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829668+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829668+0000","last_peered":"2024-05-17T09:16:48.829668+0000","last_clean":"2024-05-17T09:16:48.829668+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829668+0000","last_undegraded":"2024-05-17T09:16:48.829668+0000","last_fullsized":"2024-05-17T09:16:48.829668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423264+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423264+0000","last_peered":"2024-05-17T09:16:48.423264+0000","last_clean":"2024-05-17T09:16:48.423264+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423264+0000","last_undegraded":"2024-05-17T09:16:48.423264+0000","last_fullsized":"2024-05-17T09:16:48.423264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897951+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.897951+0000","last_peered":"2024-05-17T09:17:46.897951+0000","last_clean":"2024-05-17T09:17:46.897951+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.897951+0000","last_undegraded":"2024-05-17T09:17:46.897951+0000","last_fullsized":"2024-05-17T09:17:46.897951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829653+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:16:48.829653+0000","last_peered":"2024-05-17T09:16:48.829653+0000","last_clean":"2024-05-17T09:16:48.829653+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:16:48.829653+0000","last_undegraded":"2024-05-17T09:16:48.829653+0000","last_fullsized":"2024-05-17T09:16:48.829653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317692+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:16:48.317692+0000","last_peered":"2024-05-17T09:16:48.317692+0000","last_clean":"2024-05-17T09:16:48.317692+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:16:48.317692+0000","last_undegraded":"2024-05-17T09:16:48.317692+0000","last_fullsized":"2024-05-17T09:16:48.317692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904346+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904346+0000","last_peered":"2024-05-17T09:17:46.904346+0000","last_clean":"2024-05-17T09:17:46.904346+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904346+0000","last_undegraded":"2024-05-17T09:17:46.904346+0000","last_fullsized":"2024-05-17T09:17:46.904346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320205+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:16:48.320205+0000","last_peered":"2024-05-17T09:16:48.320205+0000","last_clean":"2024-05-17T09:16:48.320205+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:16:48.320205+0000","last_undegraded":"2024-05-17T09:16:48.320205+0000","last_fullsized":"2024-05-17T09:16:48.320205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595761+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:16:48.595761+0000","last_peered":"2024-05-17T09:16:48.595761+0000","last_clean":"2024-05-17T09:16:48.595761+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:16:48.595761+0000","last_undegraded":"2024-05-17T09:16:48.595761+0000","last_fullsized":"2024-05-17T09:16:48.595761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897522+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.897522+0000","last_peered":"2024-05-17T09:17:46.897522+0000","last_clean":"2024-05-17T09:17:46.897522+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.897522+0000","last_undegraded":"2024-05-17T09:17:46.897522+0000","last_fullsized":"2024-05-17T09:17:46.897522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320360+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320360+0000","last_peered":"2024-05-17T09:16:48.320360+0000","last_clean":"2024-05-17T09:16:48.320360+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320360+0000","last_undegraded":"2024-05-17T09:16:48.320360+0000","last_fullsized":"2024-05-17T09:16:48.320360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901408+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901408+0000","last_peered":"2024-05-17T09:17:46.901408+0000","last_clean":"2024-05-17T09:17:46.901408+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901408+0000","last_undegraded":"2024-05-17T09:17:46.901408+0000","last_fullsized":"2024-05-17T09:17:46.901408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:11.405493+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:18:11.405493+0000","last_peered":"2024-05-17T09:18:11.405493+0000","last_clean":"2024-05-17T09:18:11.405493+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:18:11.405493+0000","last_undegraded":"2024-05-17T09:18:11.405493+0000","last_fullsized":"2024-05-17T09:18:11.405493+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595643+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:16:48.595643+0000","last_peered":"2024-05-17T09:16:48.595643+0000","last_clean":"2024-05-17T09:16:48.595643+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:16:48.595643+0000","last_undegraded":"2024-05-17T09:16:48.595643+0000","last_fullsized":"2024-05-17T09:16:48.595643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903335+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.903335+0000","last_peered":"2024-05-17T09:17:46.903335+0000","last_clean":"2024-05-17T09:17:46.903335+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.903335+0000","last_undegraded":"2024-05-17T09:17:46.903335+0000","last_fullsized":"2024-05-17T09:17:46.903335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731214+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:48.731214+0000","last_peered":"2024-05-17T09:16:48.731214+0000","last_clean":"2024-05-17T09:16:48.731214+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:48.731214+0000","last_undegraded":"2024-05-17T09:16:48.731214+0000","last_fullsized":"2024-05-17T09:16:48.731214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595149+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595149+0000","last_peered":"2024-05-17T09:16:48.595149+0000","last_clean":"2024-05-17T09:16:48.595149+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595149+0000","last_undegraded":"2024-05-17T09:16:48.595149+0000","last_fullsized":"2024-05-17T09:16:48.595149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423844+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423844+0000","last_peered":"2024-05-17T09:16:48.423844+0000","last_clean":"2024-05-17T09:16:48.423844+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423844+0000","last_undegraded":"2024-05-17T09:16:48.423844+0000","last_fullsized":"2024-05-17T09:16:48.423844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903435+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903435+0000","last_peered":"2024-05-17T09:17:46.903435+0000","last_clean":"2024-05-17T09:17:46.903435+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903435+0000","last_undegraded":"2024-05-17T09:17:46.903435+0000","last_fullsized":"2024-05-17T09:17:46.903435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595374+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:48.595374+0000","last_peered":"2024-05-17T09:16:48.595374+0000","last_clean":"2024-05-17T09:16:48.595374+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:48.595374+0000","last_undegraded":"2024-05-17T09:16:48.595374+0000","last_fullsized":"2024-05-17T09:16:48.595374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321909+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:16:48.321909+0000","last_peered":"2024-05-17T09:16:48.321909+0000","last_clean":"2024-05-17T09:16:48.321909+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:16:48.321909+0000","last_undegraded":"2024-05-17T09:16:48.321909+0000","last_fullsized":"2024-05-17T09:16:48.321909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830308+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:48.830308+0000","last_peered":"2024-05-17T09:16:48.830308+0000","last_clean":"2024-05-17T09:16:48.830308+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:48.830308+0000","last_undegraded":"2024-05-17T09:16:48.830308+0000","last_fullsized":"2024-05-17T09:16:48.830308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903731+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903731+0000","last_peered":"2024-05-17T09:17:46.903731+0000","last_clean":"2024-05-17T09:17:46.903731+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903731+0000","last_undegraded":"2024-05-17T09:17:46.903731+0000","last_fullsized":"2024-05-17T09:17:46.903731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755018+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:16:48.755018+0000","last_peered":"2024-05-17T09:16:48.755018+0000","last_clean":"2024-05-17T09:16:48.755018+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:16:48.755018+0000","last_undegraded":"2024-05-17T09:16:48.755018+0000","last_fullsized":"2024-05-17T09:16:48.755018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.935621+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:17:16.935621+0000","last_peered":"2024-05-17T09:17:16.935621+0000","last_clean":"2024-05-17T09:17:16.935621+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:17:16.935621+0000","last_undegraded":"2024-05-17T09:17:16.935621+0000","last_fullsized":"2024-05-17T09:17:16.935621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889844+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889844+0000","last_peered":"2024-05-17T09:17:46.889844+0000","last_clean":"2024-05-17T09:17:46.889844+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889844+0000","last_undegraded":"2024-05-17T09:17:46.889844+0000","last_fullsized":"2024-05-17T09:17:46.889844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318709+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:16:48.318709+0000","last_peered":"2024-05-17T09:16:48.318709+0000","last_clean":"2024-05-17T09:16:48.318709+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:16:48.318709+0000","last_undegraded":"2024-05-17T09:16:48.318709+0000","last_fullsized":"2024-05-17T09:16:48.318709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731656+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:48.731656+0000","last_peered":"2024-05-17T09:16:48.731656+0000","last_clean":"2024-05-17T09:16:48.731656+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:48.731656+0000","last_undegraded":"2024-05-17T09:16:48.731656+0000","last_fullsized":"2024-05-17T09:16:48.731656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898653+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898653+0000","last_peered":"2024-05-17T09:17:46.898653+0000","last_clean":"2024-05-17T09:17:46.898653+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898653+0000","last_undegraded":"2024-05-17T09:17:46.898653+0000","last_fullsized":"2024-05-17T09:17:46.898653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859544+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:48.859544+0000","last_peered":"2024-05-17T09:16:48.859544+0000","last_clean":"2024-05-17T09:16:48.859544+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:48.859544+0000","last_undegraded":"2024-05-17T09:16:48.859544+0000","last_fullsized":"2024-05-17T09:16:48.859544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.868521+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.868521+0000","last_peered":"2024-05-17T09:17:46.868521+0000","last_clean":"2024-05-17T09:17:46.868521+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.868521+0000","last_undegraded":"2024-05-17T09:17:46.868521+0000","last_fullsized":"2024-05-17T09:17:46.868521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755202+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:48.755202+0000","last_peered":"2024-05-17T09:16:48.755202+0000","last_clean":"2024-05-17T09:16:48.755202+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:48.755202+0000","last_undegraded":"2024-05-17T09:16:48.755202+0000","last_fullsized":"2024-05-17T09:16:48.755202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900516+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.900516+0000","last_peered":"2024-05-17T09:17:46.900516+0000","last_clean":"2024-05-17T09:17:46.900516+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.900516+0000","last_undegraded":"2024-05-17T09:17:46.900516+0000","last_fullsized":"2024-05-17T09:17:46.900516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318835+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:16:48.318835+0000","last_peered":"2024-05-17T09:16:48.318835+0000","last_clean":"2024-05-17T09:16:48.318835+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:16:48.318835+0000","last_undegraded":"2024-05-17T09:16:48.318835+0000","last_fullsized":"2024-05-17T09:16:48.318835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900941+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.900941+0000","last_peered":"2024-05-17T09:17:46.900941+0000","last_clean":"2024-05-17T09:17:46.900941+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.900941+0000","last_undegraded":"2024-05-17T09:17:46.900941+0000","last_fullsized":"2024-05-17T09:17:46.900941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829950+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:48.829950+0000","last_peered":"2024-05-17T09:16:48.829950+0000","last_clean":"2024-05-17T09:16:48.829950+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:48.829950+0000","last_undegraded":"2024-05-17T09:16:48.829950+0000","last_fullsized":"2024-05-17T09:16:48.829950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908173+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908173+0000","last_peered":"2024-05-17T09:17:46.908173+0000","last_clean":"2024-05-17T09:17:46.908173+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908173+0000","last_undegraded":"2024-05-17T09:17:46.908173+0000","last_fullsized":"2024-05-17T09:17:46.908173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859520+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859520+0000","last_peered":"2024-05-17T09:16:48.859520+0000","last_clean":"2024-05-17T09:16:48.859520+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859520+0000","last_undegraded":"2024-05-17T09:16:48.859520+0000","last_fullsized":"2024-05-17T09:16:48.859520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319435+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319435+0000","last_peered":"2024-05-17T09:16:48.319435+0000","last_clean":"2024-05-17T09:16:48.319435+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319435+0000","last_undegraded":"2024-05-17T09:16:48.319435+0000","last_fullsized":"2024-05-17T09:16:48.319435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908319+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908319+0000","last_peered":"2024-05-17T09:17:46.908319+0000","last_clean":"2024-05-17T09:17:46.908319+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908319+0000","last_undegraded":"2024-05-17T09:17:46.908319+0000","last_fullsized":"2024-05-17T09:17:46.908319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423786+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423786+0000","last_peered":"2024-05-17T09:16:48.423786+0000","last_clean":"2024-05-17T09:16:48.423786+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423786+0000","last_undegraded":"2024-05-17T09:16:48.423786+0000","last_fullsized":"2024-05-17T09:16:48.423786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901697+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901697+0000","last_peered":"2024-05-17T09:17:46.901697+0000","last_clean":"2024-05-17T09:17:46.901697+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901697+0000","last_undegraded":"2024-05-17T09:17:46.901697+0000","last_fullsized":"2024-05-17T09:17:46.901697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423060+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.423060+0000","last_peered":"2024-05-17T09:16:48.423060+0000","last_clean":"2024-05-17T09:16:48.423060+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.423060+0000","last_undegraded":"2024-05-17T09:16:48.423060+0000","last_fullsized":"2024-05-17T09:16:48.423060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320546+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320546+0000","last_peered":"2024-05-17T09:16:48.320546+0000","last_clean":"2024-05-17T09:16:48.320546+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320546+0000","last_undegraded":"2024-05-17T09:16:48.320546+0000","last_fullsized":"2024-05-17T09:16:48.320546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897876+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897876+0000","last_peered":"2024-05-17T09:17:46.897876+0000","last_clean":"2024-05-17T09:17:46.897876+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897876+0000","last_undegraded":"2024-05-17T09:17:46.897876+0000","last_fullsized":"2024-05-17T09:17:46.897876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898957+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.898957+0000","last_peered":"2024-05-17T09:17:46.898957+0000","last_clean":"2024-05-17T09:17:46.898957+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.898957+0000","last_undegraded":"2024-05-17T09:17:46.898957+0000","last_fullsized":"2024-05-17T09:17:46.898957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888154+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888154+0000","last_peered":"2024-05-17T09:17:46.888154+0000","last_clean":"2024-05-17T09:17:46.888154+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888154+0000","last_undegraded":"2024-05-17T09:17:46.888154+0000","last_fullsized":"2024-05-17T09:17:46.888154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896799+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896799+0000","last_peered":"2024-05-17T09:17:46.896799+0000","last_clean":"2024-05-17T09:17:46.896799+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896799+0000","last_undegraded":"2024-05-17T09:17:46.896799+0000","last_fullsized":"2024-05-17T09:17:46.896799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904713+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904713+0000","last_peered":"2024-05-17T09:17:46.904713+0000","last_clean":"2024-05-17T09:17:46.904713+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904713+0000","last_undegraded":"2024-05-17T09:17:46.904713+0000","last_fullsized":"2024-05-17T09:17:46.904713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904287+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904287+0000","last_peered":"2024-05-17T09:17:46.904287+0000","last_clean":"2024-05-17T09:17:46.904287+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904287+0000","last_undegraded":"2024-05-17T09:17:46.904287+0000","last_fullsized":"2024-05-17T09:17:46.904287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899050+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899050+0000","last_peered":"2024-05-17T09:17:46.899050+0000","last_clean":"2024-05-17T09:17:46.899050+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899050+0000","last_undegraded":"2024-05-17T09:17:46.899050+0000","last_fullsized":"2024-05-17T09:17:46.899050+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891869+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891869+0000","last_peered":"2024-05-17T09:17:46.891869+0000","last_clean":"2024-05-17T09:17:46.891869+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891869+0000","last_undegraded":"2024-05-17T09:17:46.891869+0000","last_fullsized":"2024-05-17T09:17:46.891869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.885493+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.885493+0000","last_peered":"2024-05-17T09:17:46.885493+0000","last_clean":"2024-05-17T09:17:46.885493+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.885493+0000","last_undegraded":"2024-05-17T09:17:46.885493+0000","last_fullsized":"2024-05-17T09:17:46.885493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899844+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899844+0000","last_peered":"2024-05-17T09:17:46.899844+0000","last_clean":"2024-05-17T09:17:46.899844+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899844+0000","last_undegraded":"2024-05-17T09:17:46.899844+0000","last_fullsized":"2024-05-17T09:17:46.899844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894550+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894550+0000","last_peered":"2024-05-17T09:17:46.894550+0000","last_clean":"2024-05-17T09:17:46.894550+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894550+0000","last_undegraded":"2024-05-17T09:17:46.894550+0000","last_fullsized":"2024-05-17T09:17:46.894550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908241+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908241+0000","last_peered":"2024-05-17T09:17:46.908241+0000","last_clean":"2024-05-17T09:17:46.908241+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908241+0000","last_undegraded":"2024-05-17T09:17:46.908241+0000","last_fullsized":"2024-05-17T09:17:46.908241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904498+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904498+0000","last_peered":"2024-05-17T09:17:46.904498+0000","last_clean":"2024-05-17T09:17:46.904498+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904498+0000","last_undegraded":"2024-05-17T09:17:46.904498+0000","last_fullsized":"2024-05-17T09:17:46.904498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899413+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899413+0000","last_peered":"2024-05-17T09:17:46.899413+0000","last_clean":"2024-05-17T09:17:46.899413+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899413+0000","last_undegraded":"2024-05-17T09:17:46.899413+0000","last_fullsized":"2024-05-17T09:17:46.899413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69059,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":727,"num_read_kb":434,"num_write":177,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":1171374,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070528,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266757,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462986,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1111831,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626512,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":574800,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":576648,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":515687,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822742,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1102791,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:18:19.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:18:19.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:19 smithi019 ceph-mon[31620]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:18:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:19.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:19 smithi079 ceph-mon[35502]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:18:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:19 smithi069 ceph-mon[35081]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:18:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:20 smithi079 ceph-mon[35502]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:20.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:20 smithi019 ceph-mon[31620]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:20 smithi069 ceph-mon[35081]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 -- 172.21.15.19:0/2186265904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494110bb0 msgr2=0x7f6494113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2186265904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494110bb0 0x7f6494113010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f648c00b0a0 tx=0x7f648c00b3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 -- 172.21.15.19:0/2186265904 shutdown_connections 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2186265904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494075d20 0x7f6494076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2186265904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494110bb0 0x7f6494113010 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2186265904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6494076670 0x7f6494110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 -- 172.21.15.19:0/2186265904 >> 172.21.15.19:0/2186265904 conn(0x7f649407af60 msgr2=0x7f649407b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 -- 172.21.15.19:0/2186265904 shutdown_connections 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.247+0000 7f649bd4c700 1 -- 172.21.15.19:0/2186265904 wait complete. 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.248+0000 7f649bd4c700 1 Processor -- start 2024-05-17T09:18:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.248+0000 7f649bd4c700 1 -- start start 2024-05-17T09:18:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.248+0000 7f649bd4c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 0x7f649419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f649bd4c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494076670 0x7f649419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f649bd4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6494110bb0 0x7f64941a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f649bd4c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6494116570 con 0x7f6494075d20 2024-05-17T09:18:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f649bd4c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64941163f0 con 0x7f6494110bb0 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f649bd4c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f64941166f0 con 0x7f6494076670 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 0x7f649419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 0x7f649419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52932/0 (socket says 172.21.15.19:52932) 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 -- 172.21.15.19:0/2614477746 learned_addr learned my addr 172.21.15.19:0/2614477746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f649a2e9700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6494110bb0 0x7f64941a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f64992e7700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494076670 0x7f649419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 -- 172.21.15.19:0/2614477746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494076670 msgr2=0x7f649419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494076670 0x7f649419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 -- 172.21.15.19:0/2614477746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6494110bb0 msgr2=0x7f64941a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6494110bb0 0x7f64941a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 -- 172.21.15.19:0/2614477746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64941a1d20 con 0x7f6494075d20 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f64992e7700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494076670 0x7f649419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:18:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.249+0000 7f6499ae8700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 0x7f649419cf00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f649000d980 tx=0x7f649000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.250+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490017070 con 0x7f6494075d20 2024-05-17T09:18:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.250+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6490005020 con 0x7f6494075d20 2024-05-17T09:18:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.250+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64941a1fb0 con 0x7f6494075d20 2024-05-17T09:18:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.250+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64941a24a0 con 0x7f6494075d20 2024-05-17T09:18:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.250+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490009ab0 con 0x7f6494075d20 2024-05-17T09:18:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.251+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64940512b0 con 0x7f6494075d20 2024-05-17T09:18:21.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.252+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f64900051d0 con 0x7f6494075d20 2024-05-17T09:18:21.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.253+0000 7f648affd700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f64800b2880 0x7f64800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:21.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.253+0000 7f64992e7700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f64800b2880 0x7f64800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:21.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.253+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f64900e2520 con 0x7f6494075d20 2024-05-17T09:18:21.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.254+0000 7f64992e7700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f64800b2880 0x7f64800b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6484005fd0 tx=0x7f648400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.258+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6490020020 con 0x7f6494075d20 2024-05-17T09:18:21.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.465+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f649419e260 con 0x7f64800b2880 2024-05-17T09:18:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.475+0000 7f648affd700 1 -- 172.21.15.19:0/2614477746 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340725 (secure 0 0 0) 0x7f649419e260 con 0x7f64800b2880 2024-05-17T09:18:21.479 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f64800b2880 msgr2=0x7f64800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f64800b2880 0x7f64800b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6484005fd0 tx=0x7f648400a5e0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 msgr2=0x7f649419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 0x7f649419cf00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f649000d980 tx=0x7f649000dc90 comp rx=0 tx=0).stop 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 shutdown_connections 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6494110bb0 0x7f64941a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6494075d20 0x7f649419cf00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f64800b2880 0x7f64800b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 --2- 172.21.15.19:0/2614477746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6494076670 0x7f649419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 >> 172.21.15.19:0/2614477746 conn(0x7f649407af60 msgr2=0x7f6494101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 shutdown_connections 2024-05-17T09:18:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:21.479+0000 7f649bd4c700 1 -- 172.21.15.19:0/2614477746 wait complete. 2024-05-17T09:18:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:18:22.205 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-05-17T09:18:20.925414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659661,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":823,"num_read_kb":513,"num_write":302,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348644,"kb_used_data":11972,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593180,"statfs":{"total":1151940427776,"available":1151583416320,"internally_reserved":0,"allocated":12259328,"data_stored":8136029,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003837"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906682+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906682+0000","last_peered":"2024-05-17T09:17:46.906682+0000","last_clean":"2024-05-17T09:17:46.906682+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906682+0000","last_undegraded":"2024-05-17T09:17:46.906682+0000","last_fullsized":"2024-05-17T09:17:46.906682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897617+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.897617+0000","last_peered":"2024-05-17T09:17:46.897617+0000","last_clean":"2024-05-17T09:17:46.897617+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.897617+0000","last_undegraded":"2024-05-17T09:17:46.897617+0000","last_fullsized":"2024-05-17T09:17:46.897617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893315+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.893315+0000","last_peered":"2024-05-17T09:17:46.893315+0000","last_clean":"2024-05-17T09:17:46.893315+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.893315+0000","last_undegraded":"2024-05-17T09:17:46.893315+0000","last_fullsized":"2024-05-17T09:17:46.893315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.884948+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.884948+0000","last_peered":"2024-05-17T09:17:46.884948+0000","last_clean":"2024-05-17T09:17:46.884948+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.884948+0000","last_undegraded":"2024-05-17T09:17:46.884948+0000","last_fullsized":"2024-05-17T09:17:46.884948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897920+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.897920+0000","last_peered":"2024-05-17T09:17:46.897920+0000","last_clean":"2024-05-17T09:17:46.897920+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.897920+0000","last_undegraded":"2024-05-17T09:17:46.897920+0000","last_fullsized":"2024-05-17T09:17:46.897920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893468+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893468+0000","last_peered":"2024-05-17T09:17:46.893468+0000","last_clean":"2024-05-17T09:17:46.893468+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893468+0000","last_undegraded":"2024-05-17T09:17:46.893468+0000","last_fullsized":"2024-05-17T09:17:46.893468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908147+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908147+0000","last_peered":"2024-05-17T09:17:46.908147+0000","last_clean":"2024-05-17T09:17:46.908147+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908147+0000","last_undegraded":"2024-05-17T09:17:46.908147+0000","last_fullsized":"2024-05-17T09:17:46.908147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889231+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889231+0000","last_peered":"2024-05-17T09:17:46.889231+0000","last_clean":"2024-05-17T09:17:46.889231+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889231+0000","last_undegraded":"2024-05-17T09:17:46.889231+0000","last_fullsized":"2024-05-17T09:17:46.889231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317769+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:16:48.317769+0000","last_peered":"2024-05-17T09:16:48.317769+0000","last_clean":"2024-05-17T09:16:48.317769+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:16:48.317769+0000","last_undegraded":"2024-05-17T09:16:48.317769+0000","last_fullsized":"2024-05-17T09:16:48.317769+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898546+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898546+0000","last_peered":"2024-05-17T09:17:46.898546+0000","last_clean":"2024-05-17T09:17:46.898546+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898546+0000","last_undegraded":"2024-05-17T09:17:46.898546+0000","last_fullsized":"2024-05-17T09:17:46.898546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755662+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:16:48.755662+0000","last_peered":"2024-05-17T09:16:48.755662+0000","last_clean":"2024-05-17T09:16:48.755662+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:16:48.755662+0000","last_undegraded":"2024-05-17T09:16:48.755662+0000","last_fullsized":"2024-05-17T09:16:48.755662+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905045+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905045+0000","last_peered":"2024-05-17T09:17:46.905045+0000","last_clean":"2024-05-17T09:17:46.905045+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905045+0000","last_undegraded":"2024-05-17T09:17:46.905045+0000","last_fullsized":"2024-05-17T09:17:46.905045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:15.414120+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:18:15.414120+0000","last_peered":"2024-05-17T09:18:15.414120+0000","last_clean":"2024-05-17T09:18:15.414120+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:18:15.414120+0000","last_undegraded":"2024-05-17T09:18:15.414120+0000","last_fullsized":"2024-05-17T09:18:15.414120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322317+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:16:48.322317+0000","last_peered":"2024-05-17T09:16:48.322317+0000","last_clean":"2024-05-17T09:16:48.322317+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:16:48.322317+0000","last_undegraded":"2024-05-17T09:16:48.322317+0000","last_fullsized":"2024-05-17T09:16:48.322317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.860181+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:16:48.860181+0000","last_peered":"2024-05-17T09:16:48.860181+0000","last_clean":"2024-05-17T09:16:48.860181+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:16:48.860181+0000","last_undegraded":"2024-05-17T09:16:48.860181+0000","last_fullsized":"2024-05-17T09:16:48.860181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:17.416175+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:18:17.416175+0000","last_peered":"2024-05-17T09:18:17.416175+0000","last_clean":"2024-05-17T09:18:17.416175+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:18:17.416175+0000","last_undegraded":"2024-05-17T09:18:17.416175+0000","last_fullsized":"2024-05-17T09:18:17.416175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":331,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:18.417378+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:18:18.417378+0000","last_peered":"2024-05-17T09:18:18.417378+0000","last_clean":"2024-05-17T09:18:18.417378+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:18:18.417378+0000","last_undegraded":"2024-05-17T09:18:18.417378+0000","last_fullsized":"2024-05-17T09:18:18.417378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":188,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903452+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903452+0000","last_peered":"2024-05-17T09:17:46.903452+0000","last_clean":"2024-05-17T09:17:46.903452+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903452+0000","last_undegraded":"2024-05-17T09:17:46.903452+0000","last_fullsized":"2024-05-17T09:17:46.903452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423686+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423686+0000","last_peered":"2024-05-17T09:16:48.423686+0000","last_clean":"2024-05-17T09:16:48.423686+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423686+0000","last_undegraded":"2024-05-17T09:16:48.423686+0000","last_fullsized":"2024-05-17T09:16:48.423686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904979+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.904979+0000","last_peered":"2024-05-17T09:17:46.904979+0000","last_clean":"2024-05-17T09:17:46.904979+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.904979+0000","last_undegraded":"2024-05-17T09:17:46.904979+0000","last_fullsized":"2024-05-17T09:17:46.904979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:15.412322+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:18:15.412322+0000","last_peered":"2024-05-17T09:18:15.412322+0000","last_clean":"2024-05-17T09:18:15.412322+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:18:15.412322+0000","last_undegraded":"2024-05-17T09:18:15.412322+0000","last_fullsized":"2024-05-17T09:18:15.412322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900338+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.900338+0000","last_peered":"2024-05-17T09:17:46.900338+0000","last_clean":"2024-05-17T09:17:46.900338+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.900338+0000","last_undegraded":"2024-05-17T09:17:46.900338+0000","last_fullsized":"2024-05-17T09:17:46.900338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829668+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829668+0000","last_peered":"2024-05-17T09:16:48.829668+0000","last_clean":"2024-05-17T09:16:48.829668+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829668+0000","last_undegraded":"2024-05-17T09:16:48.829668+0000","last_fullsized":"2024-05-17T09:16:48.829668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423264+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423264+0000","last_peered":"2024-05-17T09:16:48.423264+0000","last_clean":"2024-05-17T09:16:48.423264+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423264+0000","last_undegraded":"2024-05-17T09:16:48.423264+0000","last_fullsized":"2024-05-17T09:16:48.423264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897951+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.897951+0000","last_peered":"2024-05-17T09:17:46.897951+0000","last_clean":"2024-05-17T09:17:46.897951+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.897951+0000","last_undegraded":"2024-05-17T09:17:46.897951+0000","last_fullsized":"2024-05-17T09:17:46.897951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829653+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:16:48.829653+0000","last_peered":"2024-05-17T09:16:48.829653+0000","last_clean":"2024-05-17T09:16:48.829653+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:16:48.829653+0000","last_undegraded":"2024-05-17T09:16:48.829653+0000","last_fullsized":"2024-05-17T09:16:48.829653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317692+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:16:48.317692+0000","last_peered":"2024-05-17T09:16:48.317692+0000","last_clean":"2024-05-17T09:16:48.317692+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:16:48.317692+0000","last_undegraded":"2024-05-17T09:16:48.317692+0000","last_fullsized":"2024-05-17T09:16:48.317692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904346+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904346+0000","last_peered":"2024-05-17T09:17:46.904346+0000","last_clean":"2024-05-17T09:17:46.904346+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904346+0000","last_undegraded":"2024-05-17T09:17:46.904346+0000","last_fullsized":"2024-05-17T09:17:46.904346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320205+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:16:48.320205+0000","last_peered":"2024-05-17T09:16:48.320205+0000","last_clean":"2024-05-17T09:16:48.320205+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:16:48.320205+0000","last_undegraded":"2024-05-17T09:16:48.320205+0000","last_fullsized":"2024-05-17T09:16:48.320205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595761+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:16:48.595761+0000","last_peered":"2024-05-17T09:16:48.595761+0000","last_clean":"2024-05-17T09:16:48.595761+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:16:48.595761+0000","last_undegraded":"2024-05-17T09:16:48.595761+0000","last_fullsized":"2024-05-17T09:16:48.595761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897522+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.897522+0000","last_peered":"2024-05-17T09:17:46.897522+0000","last_clean":"2024-05-17T09:17:46.897522+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.897522+0000","last_undegraded":"2024-05-17T09:17:46.897522+0000","last_fullsized":"2024-05-17T09:17:46.897522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320360+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320360+0000","last_peered":"2024-05-17T09:16:48.320360+0000","last_clean":"2024-05-17T09:16:48.320360+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320360+0000","last_undegraded":"2024-05-17T09:16:48.320360+0000","last_fullsized":"2024-05-17T09:16:48.320360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901408+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901408+0000","last_peered":"2024-05-17T09:17:46.901408+0000","last_clean":"2024-05-17T09:17:46.901408+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901408+0000","last_undegraded":"2024-05-17T09:17:46.901408+0000","last_fullsized":"2024-05-17T09:17:46.901408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:18:16.413017+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:18:16.413017+0000","last_peered":"2024-05-17T09:18:16.413017+0000","last_clean":"2024-05-17T09:18:16.413017+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:18:16.413017+0000","last_undegraded":"2024-05-17T09:18:16.413017+0000","last_fullsized":"2024-05-17T09:18:16.413017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595643+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:16:48.595643+0000","last_peered":"2024-05-17T09:16:48.595643+0000","last_clean":"2024-05-17T09:16:48.595643+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:16:48.595643+0000","last_undegraded":"2024-05-17T09:16:48.595643+0000","last_fullsized":"2024-05-17T09:16:48.595643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903335+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.903335+0000","last_peered":"2024-05-17T09:17:46.903335+0000","last_clean":"2024-05-17T09:17:46.903335+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.903335+0000","last_undegraded":"2024-05-17T09:17:46.903335+0000","last_fullsized":"2024-05-17T09:17:46.903335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731214+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:16:48.731214+0000","last_peered":"2024-05-17T09:16:48.731214+0000","last_clean":"2024-05-17T09:16:48.731214+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:16:48.731214+0000","last_undegraded":"2024-05-17T09:16:48.731214+0000","last_fullsized":"2024-05-17T09:16:48.731214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595149+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595149+0000","last_peered":"2024-05-17T09:16:48.595149+0000","last_clean":"2024-05-17T09:16:48.595149+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595149+0000","last_undegraded":"2024-05-17T09:16:48.595149+0000","last_fullsized":"2024-05-17T09:16:48.595149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423844+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423844+0000","last_peered":"2024-05-17T09:16:48.423844+0000","last_clean":"2024-05-17T09:16:48.423844+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423844+0000","last_undegraded":"2024-05-17T09:16:48.423844+0000","last_fullsized":"2024-05-17T09:16:48.423844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903435+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903435+0000","last_peered":"2024-05-17T09:17:46.903435+0000","last_clean":"2024-05-17T09:17:46.903435+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903435+0000","last_undegraded":"2024-05-17T09:17:46.903435+0000","last_fullsized":"2024-05-17T09:17:46.903435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595374+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:16:48.595374+0000","last_peered":"2024-05-17T09:16:48.595374+0000","last_clean":"2024-05-17T09:16:48.595374+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:16:48.595374+0000","last_undegraded":"2024-05-17T09:16:48.595374+0000","last_fullsized":"2024-05-17T09:16:48.595374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321909+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:16:48.321909+0000","last_peered":"2024-05-17T09:16:48.321909+0000","last_clean":"2024-05-17T09:16:48.321909+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:16:48.321909+0000","last_undegraded":"2024-05-17T09:16:48.321909+0000","last_fullsized":"2024-05-17T09:16:48.321909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830308+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:16:48.830308+0000","last_peered":"2024-05-17T09:16:48.830308+0000","last_clean":"2024-05-17T09:16:48.830308+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:16:48.830308+0000","last_undegraded":"2024-05-17T09:16:48.830308+0000","last_fullsized":"2024-05-17T09:16:48.830308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903731+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903731+0000","last_peered":"2024-05-17T09:17:46.903731+0000","last_clean":"2024-05-17T09:17:46.903731+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903731+0000","last_undegraded":"2024-05-17T09:17:46.903731+0000","last_fullsized":"2024-05-17T09:17:46.903731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755018+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:16:48.755018+0000","last_peered":"2024-05-17T09:16:48.755018+0000","last_clean":"2024-05-17T09:16:48.755018+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:16:48.755018+0000","last_undegraded":"2024-05-17T09:16:48.755018+0000","last_fullsized":"2024-05-17T09:16:48.755018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.935621+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:17:16.935621+0000","last_peered":"2024-05-17T09:17:16.935621+0000","last_clean":"2024-05-17T09:17:16.935621+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:17:16.935621+0000","last_undegraded":"2024-05-17T09:17:16.935621+0000","last_fullsized":"2024-05-17T09:17:16.935621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889844+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889844+0000","last_peered":"2024-05-17T09:17:46.889844+0000","last_clean":"2024-05-17T09:17:46.889844+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889844+0000","last_undegraded":"2024-05-17T09:17:46.889844+0000","last_fullsized":"2024-05-17T09:17:46.889844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318709+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:16:48.318709+0000","last_peered":"2024-05-17T09:16:48.318709+0000","last_clean":"2024-05-17T09:16:48.318709+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:16:48.318709+0000","last_undegraded":"2024-05-17T09:16:48.318709+0000","last_fullsized":"2024-05-17T09:16:48.318709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731656+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:16:48.731656+0000","last_peered":"2024-05-17T09:16:48.731656+0000","last_clean":"2024-05-17T09:16:48.731656+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:16:48.731656+0000","last_undegraded":"2024-05-17T09:16:48.731656+0000","last_fullsized":"2024-05-17T09:16:48.731656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898653+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898653+0000","last_peered":"2024-05-17T09:17:46.898653+0000","last_clean":"2024-05-17T09:17:46.898653+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898653+0000","last_undegraded":"2024-05-17T09:17:46.898653+0000","last_fullsized":"2024-05-17T09:17:46.898653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859544+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:16:48.859544+0000","last_peered":"2024-05-17T09:16:48.859544+0000","last_clean":"2024-05-17T09:16:48.859544+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:16:48.859544+0000","last_undegraded":"2024-05-17T09:16:48.859544+0000","last_fullsized":"2024-05-17T09:16:48.859544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.868521+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.868521+0000","last_peered":"2024-05-17T09:17:46.868521+0000","last_clean":"2024-05-17T09:17:46.868521+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.868521+0000","last_undegraded":"2024-05-17T09:17:46.868521+0000","last_fullsized":"2024-05-17T09:17:46.868521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755202+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:16:48.755202+0000","last_peered":"2024-05-17T09:16:48.755202+0000","last_clean":"2024-05-17T09:16:48.755202+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:16:48.755202+0000","last_undegraded":"2024-05-17T09:16:48.755202+0000","last_fullsized":"2024-05-17T09:16:48.755202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900516+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.900516+0000","last_peered":"2024-05-17T09:17:46.900516+0000","last_clean":"2024-05-17T09:17:46.900516+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.900516+0000","last_undegraded":"2024-05-17T09:17:46.900516+0000","last_fullsized":"2024-05-17T09:17:46.900516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318835+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:16:48.318835+0000","last_peered":"2024-05-17T09:16:48.318835+0000","last_clean":"2024-05-17T09:16:48.318835+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:16:48.318835+0000","last_undegraded":"2024-05-17T09:16:48.318835+0000","last_fullsized":"2024-05-17T09:16:48.318835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900941+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.900941+0000","last_peered":"2024-05-17T09:17:46.900941+0000","last_clean":"2024-05-17T09:17:46.900941+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.900941+0000","last_undegraded":"2024-05-17T09:17:46.900941+0000","last_fullsized":"2024-05-17T09:17:46.900941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829950+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:16:48.829950+0000","last_peered":"2024-05-17T09:16:48.829950+0000","last_clean":"2024-05-17T09:16:48.829950+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:16:48.829950+0000","last_undegraded":"2024-05-17T09:16:48.829950+0000","last_fullsized":"2024-05-17T09:16:48.829950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908173+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908173+0000","last_peered":"2024-05-17T09:17:46.908173+0000","last_clean":"2024-05-17T09:17:46.908173+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908173+0000","last_undegraded":"2024-05-17T09:17:46.908173+0000","last_fullsized":"2024-05-17T09:17:46.908173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859520+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859520+0000","last_peered":"2024-05-17T09:16:48.859520+0000","last_clean":"2024-05-17T09:16:48.859520+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859520+0000","last_undegraded":"2024-05-17T09:16:48.859520+0000","last_fullsized":"2024-05-17T09:16:48.859520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319435+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319435+0000","last_peered":"2024-05-17T09:16:48.319435+0000","last_clean":"2024-05-17T09:16:48.319435+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319435+0000","last_undegraded":"2024-05-17T09:16:48.319435+0000","last_fullsized":"2024-05-17T09:16:48.319435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908319+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908319+0000","last_peered":"2024-05-17T09:17:46.908319+0000","last_clean":"2024-05-17T09:17:46.908319+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908319+0000","last_undegraded":"2024-05-17T09:17:46.908319+0000","last_fullsized":"2024-05-17T09:17:46.908319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423786+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423786+0000","last_peered":"2024-05-17T09:16:48.423786+0000","last_clean":"2024-05-17T09:16:48.423786+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423786+0000","last_undegraded":"2024-05-17T09:16:48.423786+0000","last_fullsized":"2024-05-17T09:16:48.423786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901697+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901697+0000","last_peered":"2024-05-17T09:17:46.901697+0000","last_clean":"2024-05-17T09:17:46.901697+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901697+0000","last_undegraded":"2024-05-17T09:17:46.901697+0000","last_fullsized":"2024-05-17T09:17:46.901697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423060+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.423060+0000","last_peered":"2024-05-17T09:16:48.423060+0000","last_clean":"2024-05-17T09:16:48.423060+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.423060+0000","last_undegraded":"2024-05-17T09:16:48.423060+0000","last_fullsized":"2024-05-17T09:16:48.423060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320546+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320546+0000","last_peered":"2024-05-17T09:16:48.320546+0000","last_clean":"2024-05-17T09:16:48.320546+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320546+0000","last_undegraded":"2024-05-17T09:16:48.320546+0000","last_fullsized":"2024-05-17T09:16:48.320546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897876+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897876+0000","last_peered":"2024-05-17T09:17:46.897876+0000","last_clean":"2024-05-17T09:17:46.897876+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897876+0000","last_undegraded":"2024-05-17T09:17:46.897876+0000","last_fullsized":"2024-05-17T09:17:46.897876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898957+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.898957+0000","last_peered":"2024-05-17T09:17:46.898957+0000","last_clean":"2024-05-17T09:17:46.898957+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.898957+0000","last_undegraded":"2024-05-17T09:17:46.898957+0000","last_fullsized":"2024-05-17T09:17:46.898957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888154+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888154+0000","last_peered":"2024-05-17T09:17:46.888154+0000","last_clean":"2024-05-17T09:17:46.888154+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888154+0000","last_undegraded":"2024-05-17T09:17:46.888154+0000","last_fullsized":"2024-05-17T09:17:46.888154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896799+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896799+0000","last_peered":"2024-05-17T09:17:46.896799+0000","last_clean":"2024-05-17T09:17:46.896799+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896799+0000","last_undegraded":"2024-05-17T09:17:46.896799+0000","last_fullsized":"2024-05-17T09:17:46.896799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904713+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904713+0000","last_peered":"2024-05-17T09:17:46.904713+0000","last_clean":"2024-05-17T09:17:46.904713+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904713+0000","last_undegraded":"2024-05-17T09:17:46.904713+0000","last_fullsized":"2024-05-17T09:17:46.904713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904287+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904287+0000","last_peered":"2024-05-17T09:17:46.904287+0000","last_clean":"2024-05-17T09:17:46.904287+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904287+0000","last_undegraded":"2024-05-17T09:17:46.904287+0000","last_fullsized":"2024-05-17T09:17:46.904287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899050+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899050+0000","last_peered":"2024-05-17T09:17:46.899050+0000","last_clean":"2024-05-17T09:17:46.899050+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899050+0000","last_undegraded":"2024-05-17T09:17:46.899050+0000","last_fullsized":"2024-05-17T09:17:46.899050+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891869+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891869+0000","last_peered":"2024-05-17T09:17:46.891869+0000","last_clean":"2024-05-17T09:17:46.891869+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891869+0000","last_undegraded":"2024-05-17T09:17:46.891869+0000","last_fullsized":"2024-05-17T09:17:46.891869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.885493+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.885493+0000","last_peered":"2024-05-17T09:17:46.885493+0000","last_clean":"2024-05-17T09:17:46.885493+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.885493+0000","last_undegraded":"2024-05-17T09:17:46.885493+0000","last_fullsized":"2024-05-17T09:17:46.885493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899844+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899844+0000","last_peered":"2024-05-17T09:17:46.899844+0000","last_clean":"2024-05-17T09:17:46.899844+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899844+0000","last_undegraded":"2024-05-17T09:17:46.899844+0000","last_fullsized":"2024-05-17T09:17:46.899844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894550+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894550+0000","last_peered":"2024-05-17T09:17:46.894550+0000","last_clean":"2024-05-17T09:17:46.894550+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894550+0000","last_undegraded":"2024-05-17T09:17:46.894550+0000","last_fullsized":"2024-05-17T09:17:46.894550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908241+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908241+0000","last_peered":"2024-05-17T09:17:46.908241+0000","last_clean":"2024-05-17T09:17:46.908241+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908241+0000","last_undegraded":"2024-05-17T09:17:46.908241+0000","last_fullsized":"2024-05-17T09:17:46.908241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904498+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904498+0000","last_peered":"2024-05-17T09:17:46.904498+0000","last_clean":"2024-05-17T09:17:46.904498+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904498+0000","last_undegraded":"2024-05-17T09:17:46.904498+0000","last_fullsized":"2024-05-17T09:17:46.904498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899413+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899413+0000","last_peered":"2024-05-17T09:17:46.899413+0000","last_clean":"2024-05-17T09:17:46.899413+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899413+0000","last_undegraded":"2024-05-17T09:17:46.899413+0000","last_fullsized":"2024-05-17T09:17:46.899413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69059,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":747,"num_read_kb":449,"num_write":177,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710775,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907003,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":1171374,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070529,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462987,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1111831,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":574800,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":576648,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":515687,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822742,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1102791,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:18:22.210 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T09:18:22.210 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-17T09:18:22.210 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-17T09:18:22.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-17T09:18:22.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:22 smithi019 ceph-mon[31620]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:22 smithi079 ceph-mon[35502]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:22 smithi069 ceph-mon[35081]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:23.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:23 smithi079 ceph-mon[35502]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:18:23.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:23 smithi019 ceph-mon[31620]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:18:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:23 smithi069 ceph-mon[35081]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 -- 172.21.15.19:0/996659417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701092f0 msgr2=0x7f5470109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 --2- 172.21.15.19:0/996659417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701092f0 0x7f5470109700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5460009990 tx=0x7f5460009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 -- 172.21.15.19:0/996659417 shutdown_connections 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 --2- 172.21.15.19:0/996659417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54701030d0 0x7f5470111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 --2- 172.21.15.19:0/996659417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701092f0 0x7f5470109700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 --2- 172.21.15.19:0/996659417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5470102780 0x7f5470102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 -- 172.21.15.19:0/996659417 >> 172.21.15.19:0/996659417 conn(0x7f54700fe4d0 msgr2=0x7f5470100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 -- 172.21.15.19:0/996659417 shutdown_connections 2024-05-17T09:18:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.150+0000 7f5476093700 1 -- 172.21.15.19:0/996659417 wait complete. 2024-05-17T09:18:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.151+0000 7f5476093700 1 Processor -- start 2024-05-17T09:18:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.151+0000 7f5476093700 1 -- start start 2024-05-17T09:18:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f5476093700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5470102780 0x7f547019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f5476093700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 0x7f547019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f5476093700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54701092f0 0x7f547019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f5476093700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5470114430 con 0x7f54701030d0 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f5476093700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54701142b0 con 0x7f5470102780 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f5476093700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f54701145b0 con 0x7f54701092f0 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 0x7f547019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 0x7f547019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59990/0 (socket says 172.21.15.19:59990) 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 -- 172.21.15.19:0/3376009607 learned_addr learned my addr 172.21.15.19:0/3376009607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546f7fe700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5470102780 0x7f547019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546ffff700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54701092f0 0x7f547019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 -- 172.21.15.19:0/3376009607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54701092f0 msgr2=0x7f547019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54701092f0 0x7f547019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 -- 172.21.15.19:0/3376009607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5470102780 msgr2=0x7f547019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5470102780 0x7f547019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 -- 172.21.15.19:0/3376009607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547019fcc0 con 0x7f54701030d0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.152+0000 7f546effd700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 0x7f547019b3e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f546400e970 tx=0x7f546400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.153+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546400cbd0 con 0x7f54701030d0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.153+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f546400cd70 con 0x7f54701030d0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.153+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464005570 con 0x7f54701030d0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.153+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547019ffb0 con 0x7f54701030d0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.153+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54701a78d0 con 0x7f54701030d0 2024-05-17T09:18:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.154+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54700512b0 con 0x7f54701030d0 2024-05-17T09:18:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.155+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5464004750 con 0x7f54701030d0 2024-05-17T09:18:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.156+0000 7f546cff9700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f54580b2900 0x7f54580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.156+0000 7f546f7fe700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f54580b2900 0x7f54580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.156+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f54640e27d0 con 0x7f54701030d0 2024-05-17T09:18:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.157+0000 7f546f7fe700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f54580b2900 0x7f54580b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f546000b650 tx=0x7f5460005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.161+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54640aa140 con 0x7f54701030d0 2024-05-17T09:18:24.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.427+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f547019c150 con 0x7f54701030d0 2024-05-17T09:18:24.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:24 smithi019 ceph-mon[31620]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:24.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.428+0000 7f546cff9700 1 -- 172.21.15.19:0/3376009607 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f54640a9a40 con 0x7f54701030d0 2024-05-17T09:18:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f54580b2900 msgr2=0x7f54580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f54580b2900 0x7f54580b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f546000b650 tx=0x7f5460005fb0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 msgr2=0x7f547019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 0x7f547019b3e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f546400e970 tx=0x7f546400ec80 comp rx=0 tx=0).stop 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 shutdown_connections 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f54580b2900 0x7f54580b4e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54701092f0 0x7f547019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5470102780 0x7f547019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 --2- 172.21.15.19:0/3376009607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54701030d0 0x7f547019b3e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 >> 172.21.15.19:0/3376009607 conn(0x7f54700fe4d0 msgr2=0x7f54701072a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 shutdown_connections 2024-05-17T09:18:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:24.430+0000 7f5476093700 1 -- 172.21.15.19:0/3376009607 wait complete. 2024-05-17T09:18:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:24 smithi079 ceph-mon[35502]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:24 smithi069 ceph-mon[35081]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:25.128 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-17T09:18:25.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:18:25.387 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:25.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:25 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3376009607' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T09:18:25.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:25 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3376009607' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T09:18:25.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:25 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3376009607' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T09:18:26.519 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:26 smithi019 ceph-mon[31620]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:26 smithi079 ceph-mon[35502]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:26 smithi069 ceph-mon[35081]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:27.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 -- 172.21.15.19:0/2824105192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 msgr2=0x7f87c8117780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:27.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 --2- 172.21.15.19:0/2824105192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c8117780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87c000b0a0 tx=0x7f87c000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 -- 172.21.15.19:0/2824105192 shutdown_connections 2024-05-17T09:18:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 --2- 172.21.15.19:0/2824105192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c8117780 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 --2- 172.21.15.19:0/2824105192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87c8075b50 0x7f87c8075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 --2- 172.21.15.19:0/2824105192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87c80764a0 0x7f87c8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 -- 172.21.15.19:0/2824105192 >> 172.21.15.19:0/2824105192 conn(0x7f87c8100030 msgr2=0x7f87c8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 -- 172.21.15.19:0/2824105192 shutdown_connections 2024-05-17T09:18:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 -- 172.21.15.19:0/2824105192 wait complete. 2024-05-17T09:18:27.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.065+0000 7f87cf733700 1 Processor -- start 2024-05-17T09:18:27.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 -- start start 2024-05-17T09:18:27.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87c8075b50 0x7f87c81a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:27.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87c80764a0 0x7f87c81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87c811a910 con 0x7f87c8115360 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87c811a790 con 0x7f87c8075b50 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cf733700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87c811aa90 con 0x7f87c80764a0 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c81a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59998/0 (socket says 172.21.15.19:59998) 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 -- 172.21.15.19:0/434151691 learned_addr learned my addr 172.21.15.19:0/434151691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cccce700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87c80764a0 0x7f87c81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cd4cf700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87c8075b50 0x7f87c81a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 -- 172.21.15.19:0/434151691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87c80764a0 msgr2=0x7f87c81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87c80764a0 0x7f87c81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 -- 172.21.15.19:0/434151691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87c8075b50 msgr2=0x7f87c81a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87c8075b50 0x7f87c81a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.066+0000 7f87cdcd0700 1 -- 172.21.15.19:0/434151691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c81a61b0 con 0x7f87c8115360 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.067+0000 7f87cdcd0700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c81a5b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87c000b830 tx=0x7f87c0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.067+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0021070 con 0x7f87c8115360 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.067+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c0007b80 con 0x7f87c8115360 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.067+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c81a6440 con 0x7f87c8115360 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.067+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00052a0 con 0x7f87c8115360 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.067+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c81a6920 con 0x7f87c8115360 2024-05-17T09:18:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.068+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c80512b0 con 0x7f87c8115360 2024-05-17T09:18:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.070+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f87c0024070 con 0x7f87c8115360 2024-05-17T09:18:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.070+0000 7f87be7fc700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87b40b2880 0x7f87b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.071+0000 7f87cd4cf700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87b40b2880 0x7f87b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.071+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f87c000f020 con 0x7f87c8115360 2024-05-17T09:18:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.071+0000 7f87cd4cf700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87b40b2880 0x7f87b40b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f87c4005950 tx=0x7f87c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:27.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.074+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87c002b020 con 0x7f87c8115360 2024-05-17T09:18:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.340+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87c802e5e0 con 0x7f87c8115360 2024-05-17T09:18:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.341+0000 7f87be7fc700 1 -- 172.21.15.19:0/434151691 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f87c00aba40 con 0x7f87c8115360 2024-05-17T09:18:27.341 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:27.341 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:16:22.157327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:18:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87b40b2880 msgr2=0x7f87b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87b40b2880 0x7f87b40b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f87c4005950 tx=0x7f87c400b410 comp rx=0 tx=0).stop 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 msgr2=0x7f87c81a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c81a5b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87c000b830 tx=0x7f87c0004d00 comp rx=0 tx=0).stop 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 shutdown_connections 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87c8115360 0x7f87c81a5b10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87c8075b50 0x7f87c81a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87b40b2880 0x7f87b40b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 --2- 172.21.15.19:0/434151691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87c80764a0 0x7f87c81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 >> 172.21.15.19:0/434151691 conn(0x7f87c8100030 msgr2=0x7f87c81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 shutdown_connections 2024-05-17T09:18:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:27.343+0000 7f87cf733700 1 -- 172.21.15.19:0/434151691 wait complete. 2024-05-17T09:18:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:18:28.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:18:28.275 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:28.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:28 smithi019 ceph-mon[31620]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:28.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:28 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/434151691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:28 smithi079 ceph-mon[35502]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:28 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/434151691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:28 smithi069 ceph-mon[35081]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:28 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/434151691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 -- 172.21.15.19:0/1523035393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc107de0 msgr2=0x7f88fc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 --2- 172.21.15.19:0/1523035393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc107de0 0x7f88fc1081f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88e40099c0 tx=0x7f88e4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 -- 172.21.15.19:0/1523035393 shutdown_connections 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 --2- 172.21.15.19:0/1523035393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88fc108730 0x7f88fc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 --2- 172.21.15.19:0/1523035393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88fc1034b0 0x7f88fc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 --2- 172.21.15.19:0/1523035393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc107de0 0x7f88fc1081f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 -- 172.21.15.19:0/1523035393 >> 172.21.15.19:0/1523035393 conn(0x7f88fc0fede0 msgr2=0x7f88fc101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 -- 172.21.15.19:0/1523035393 shutdown_connections 2024-05-17T09:18:30.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.006+0000 7f890218f700 1 -- 172.21.15.19:0/1523035393 wait complete. 2024-05-17T09:18:30.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 Processor -- start 2024-05-17T09:18:30.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 -- start start 2024-05-17T09:18:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88fc1034b0 0x7f88fc19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88fc107de0 0x7f88fc19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 0x7f88fc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc113210 con 0x7f88fc108730 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc113090 con 0x7f88fc1034b0 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.007+0000 7f890218f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc113390 con 0x7f88fc107de0 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 0x7f88fc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 0x7f88fc19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60024/0 (socket says 172.21.15.19:60024) 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88f2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88fc107de0 0x7f88fc19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 -- 172.21.15.19:0/2189004968 learned_addr learned my addr 172.21.15.19:0/2189004968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fb7fe700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88fc1034b0 0x7f88fc19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 -- 172.21.15.19:0/2189004968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88fc107de0 msgr2=0x7f88fc19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88fc107de0 0x7f88fc19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 -- 172.21.15.19:0/2189004968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88fc1034b0 msgr2=0x7f88fc19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88fc1034b0 0x7f88fc19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 -- 172.21.15.19:0/2189004968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc1a6ee0 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88fbfff700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 0x7f88fc19c630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007ee0 tx=0x7f88f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4004bb0 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc1a71d0 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88f4004d50 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f40058b0 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.008+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc1a7730 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.010+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc0512b0 con 0x7f88fc108730 2024-05-17T09:18:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.011+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f88f401a020 con 0x7f88fc108730 2024-05-17T09:18:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.012+0000 7f88f97fa700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88e80b2880 0x7f88e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.012+0000 7f88fb7fe700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88e80b2880 0x7f88e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.012+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f88f40e1bc0 con 0x7f88fc108730 2024-05-17T09:18:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.012+0000 7f88fb7fe700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88e80b2880 0x7f88e80b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88ec005950 tx=0x7f88ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.017+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88f4010070 con 0x7f88fc108730 2024-05-17T09:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.277+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88fc02e7a0 con 0x7f88fc108730 2024-05-17T09:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.278+0000 7f88f97fa700 1 -- 172.21.15.19:0/2189004968 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f88f40a9530 con 0x7f88fc108730 2024-05-17T09:18:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:30.279 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:16:22.157327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.280+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88e80b2880 msgr2=0x7f88e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.280+0000 7f890218f700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88e80b2880 0x7f88e80b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88ec005950 tx=0x7f88ec00b410 comp rx=0 tx=0).stop 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.280+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 msgr2=0x7f88fc19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.280+0000 7f890218f700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 0x7f88fc19c630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007ee0 tx=0x7f88f400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 shutdown_connections 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc108730 0x7f88fc19c630 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88fc1034b0 0x7f88fc19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88e80b2880 0x7f88e80b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 --2- 172.21.15.19:0/2189004968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88fc107de0 0x7f88fc19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 >> 172.21.15.19:0/2189004968 conn(0x7f88fc0fede0 msgr2=0x7f88fc109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 shutdown_connections 2024-05-17T09:18:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:30.281+0000 7f890218f700 1 -- 172.21.15.19:0/2189004968 wait complete. 2024-05-17T09:18:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:18:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:30 smithi079 ceph-mon[35502]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:30.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:30 smithi019 ceph-mon[31620]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:30 smithi069 ceph-mon[35081]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:30.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:18:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:31.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2189004968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:31.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2189004968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:31.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2189004968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:32.508 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:32 smithi019 ceph-mon[31620]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:32 smithi079 ceph-mon[35502]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:32 smithi069 ceph-mon[35081]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 -- 172.21.15.19:0/3248659763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a481045e0 msgr2=0x7f4a481049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 --2- 172.21.15.19:0/3248659763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a481045e0 0x7f4a481049f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4a440099c0 tx=0x7f4a44009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 -- 172.21.15.19:0/3248659763 shutdown_connections 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 --2- 172.21.15.19:0/3248659763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a48109860 0x7f4a481111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 --2- 172.21.15.19:0/3248659763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a481045e0 0x7f4a481049f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 --2- 172.21.15.19:0/3248659763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a48108f10 0x7f4a48109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 -- 172.21.15.19:0/3248659763 >> 172.21.15.19:0/3248659763 conn(0x7f4a480ffeb0 msgr2=0x7f4a48102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 -- 172.21.15.19:0/3248659763 shutdown_connections 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.874+0000 7f4a50429700 1 -- 172.21.15.19:0/3248659763 wait complete. 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 Processor -- start 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 -- start start 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a481045e0 0x7f4a4819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 0x7f4a4819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a48109860 0x7f4a4819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48114340 con 0x7f4a48108f10 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481141c0 con 0x7f4a48109860 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a50429700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481144c0 con 0x7f4a481045e0 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a4d9c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 0x7f4a4819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a4d9c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 0x7f4a4819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60044/0 (socket says 172.21.15.19:60044) 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a4d9c4700 1 -- 172.21.15.19:0/4216464946 learned_addr learned my addr 172.21.15.19:0/4216464946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a4e1c5700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a481045e0 0x7f4a4819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.875+0000 7f4a4e9c6700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a48109860 0x7f4a4819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a4d9c4700 1 -- 172.21.15.19:0/4216464946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a481045e0 msgr2=0x7f4a4819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a4d9c4700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a481045e0 0x7f4a4819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a4d9c4700 1 -- 172.21.15.19:0/4216464946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a48109860 msgr2=0x7f4a4819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a4d9c4700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a48109860 0x7f4a4819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a4d9c4700 1 -- 172.21.15.19:0/4216464946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4819fb30 con 0x7f4a48108f10 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a4d9c4700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 0x7f4a4819b2b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800d980 tx=0x7f4a3800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38017070 con 0x7f4a48108f10 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4819fe20 con 0x7f4a48108f10 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a38010460 con 0x7f4a48108f10 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800de60 con 0x7f4a48108f10 2024-05-17T09:18:32.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.876+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a481a77f0 con 0x7f4a48108f10 2024-05-17T09:18:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.877+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a480512b0 con 0x7f4a48108f10 2024-05-17T09:18:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.879+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4a38010610 con 0x7f4a48108f10 2024-05-17T09:18:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.879+0000 7f4a3f7fe700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a340b2880 0x7f4a340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.879+0000 7f4a4e1c5700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a340b2880 0x7f4a340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.879+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4a380e20e0 con 0x7f4a48108f10 2024-05-17T09:18:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.880+0000 7f4a4e1c5700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a340b2880 0x7f4a340b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a440058e0 tx=0x7f4a44005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:32.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:32.883+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a380a9a50 con 0x7f4a48108f10 2024-05-17T09:18:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.098+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a4802e5e0 con 0x7f4a48108f10 2024-05-17T09:18:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.099+0000 7f4a3f7fe700 1 -- 172.21.15.19:0/4216464946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f4a380a9350 con 0x7f4a48108f10 2024-05-17T09:18:33.100 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:33.100 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:18:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a340b2880 msgr2=0x7f4a340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a340b2880 0x7f4a340b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a440058e0 tx=0x7f4a44005940 comp rx=0 tx=0).stop 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 msgr2=0x7f4a4819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 0x7f4a4819b2b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800d980 tx=0x7f4a3800dd40 comp rx=0 tx=0).stop 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 shutdown_connections 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a340b2880 0x7f4a340b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a481045e0 0x7f4a4819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a48109860 0x7f4a4819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 --2- 172.21.15.19:0/4216464946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a48108f10 0x7f4a4819b2b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 >> 172.21.15.19:0/4216464946 conn(0x7f4a480ffeb0 msgr2=0x7f4a481022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 shutdown_connections 2024-05-17T09:18:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:33.103+0000 7f4a50429700 1 -- 172.21.15.19:0/4216464946 wait complete. 2024-05-17T09:18:33.413 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:33 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4216464946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:18:33.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:33 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4216464946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:18:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:33 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4216464946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:18:33.782 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:18:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:34.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:34 smithi019 ceph-mon[31620]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:34 smithi079 ceph-mon[35502]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:34 smithi069 ceph-mon[35081]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 -- 172.21.15.19:0/393257601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 msgr2=0x7f72e81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 --2- 172.21.15.19:0/393257601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e81081f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f72d80099c0 tx=0x7f72d8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 -- 172.21.15.19:0/393257601 shutdown_connections 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 --2- 172.21.15.19:0/393257601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e81034b0 0x7f72e81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 --2- 172.21.15.19:0/393257601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72e8108730 0x7f72e8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 --2- 172.21.15.19:0/393257601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e81081f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 -- 172.21.15.19:0/393257601 >> 172.21.15.19:0/393257601 conn(0x7f72e80fede0 msgr2=0x7f72e8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 -- 172.21.15.19:0/393257601 shutdown_connections 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.742+0000 7f72f0720700 1 -- 172.21.15.19:0/393257601 wait complete. 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 Processor -- start 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 -- start start 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e81034b0 0x7f72e819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72e8108730 0x7f72e819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81131f0 con 0x7f72e8107de0 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113070 con 0x7f72e8108730 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72f0720700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113370 con 0x7f72e81034b0 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72edcbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72edcbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36728/0 (socket says 172.21.15.19:36728) 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72edcbb700 1 -- 172.21.15.19:0/3716813835 learned_addr learned my addr 172.21.15.19:0/3716813835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72eecbd700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72e8108730 0x7f72e819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.743+0000 7f72ee4bc700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e81034b0 0x7f72e819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72edcbb700 1 -- 172.21.15.19:0/3716813835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e81034b0 msgr2=0x7f72e819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72edcbb700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e81034b0 0x7f72e819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72edcbb700 1 -- 172.21.15.19:0/3716813835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72e8108730 msgr2=0x7f72e819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72edcbb700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72e8108730 0x7f72e819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72edcbb700 1 -- 172.21.15.19:0/3716813835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e819fcc0 con 0x7f72e8107de0 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72edcbb700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e819b390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f72d800b580 tx=0x7f72d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8027070 con 0x7f72e8107de0 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e819ff50 con 0x7f72e8107de0 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72d8022470 con 0x7f72e8107de0 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8017650 con 0x7f72e8107de0 2024-05-17T09:18:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.744+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a0430 con 0x7f72e8107de0 2024-05-17T09:18:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.745+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e80512b0 con 0x7f72e8107de0 2024-05-17T09:18:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.747+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f72d802a070 con 0x7f72e8107de0 2024-05-17T09:18:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.748+0000 7f72df7fe700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72d40b2880 0x7f72d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.748+0000 7f72ee4bc700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72d40b2880 0x7f72d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.748+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f72d80e3c80 con 0x7f72e8107de0 2024-05-17T09:18:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.748+0000 7f72ee4bc700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72d40b2880 0x7f72d40b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72e40060b0 tx=0x7f72e4007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:35.751+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72d8013070 con 0x7f72e8107de0 2024-05-17T09:18:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.012+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e802e5e0 con 0x7f72e8107de0 2024-05-17T09:18:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.013+0000 7f72df7fe700 1 -- 172.21.15.19:0/3716813835 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f72d80ab680 con 0x7f72e8107de0 2024-05-17T09:18:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:18:36.013 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:16:22.157327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:18:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72d40b2880 msgr2=0x7f72d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72d40b2880 0x7f72d40b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72e40060b0 tx=0x7f72e4007b80 comp rx=0 tx=0).stop 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 msgr2=0x7f72e819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e819b390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f72d800b580 tx=0x7f72d800f690 comp rx=0 tx=0).stop 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 shutdown_connections 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72d40b2880 0x7f72d40b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e81034b0 0x7f72e819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72e8108730 0x7f72e819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 --2- 172.21.15.19:0/3716813835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72e8107de0 0x7f72e819b390 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 >> 172.21.15.19:0/3716813835 conn(0x7f72e80fede0 msgr2=0x7f72e8075ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 shutdown_connections 2024-05-17T09:18:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:36.015+0000 7f72f0720700 1 -- 172.21.15.19:0/3716813835 wait complete. 2024-05-17T09:18:36.017 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:18:36.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:36 smithi019 ceph-mon[31620]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:36.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:36 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3716813835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:36 smithi079 ceph-mon[35502]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:36 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3716813835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:36.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T09:15:09.950009+0000', 'modified': '2024-05-17T09:16:22.157327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24487, 'mds_2': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6835/437548660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 437548660}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 437548660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/1132484996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1132484996}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 1132484996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/720193094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 720193094}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 720193094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.69:6837/1860504493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 1860504493}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 1860504493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.79:6839/2520420584', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2520420584}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2520420584}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.69:6839/814844032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 814844032}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 814844032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24487, 24497, 24470]} 2024-05-17T09:18:36.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T09:18:36.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-17T09:18:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:36 smithi069 ceph-mon[35081]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:36 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3716813835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:18:36.907 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:38.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:38 smithi019 ceph-mon[31620]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:38 smithi079 ceph-mon[35502]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:38 smithi069 ceph-mon[35081]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/2305060254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4108730 msgr2=0x7fd1f4110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/2305060254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4108730 0x7fd1f4110090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b0d0 tx=0x7fd1ec00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/2305060254 shutdown_connections 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/2305060254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1f41034b0 0x7fd1f41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/2305060254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4108730 0x7fd1f4110090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/2305060254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f4107de0 0x7fd1f41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/2305060254 >> 172.21.15.19:0/2305060254 conn(0x7fd1f40fedb0 msgr2=0x7fd1f41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/2305060254 shutdown_connections 2024-05-17T09:18:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.771+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/2305060254 wait complete. 2024-05-17T09:18:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 Processor -- start 2024-05-17T09:18:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 -- start start 2024-05-17T09:18:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1f41034b0 0x7fd1f4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 0x7fd1f4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f4108730 0x7fd1f419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4113210 con 0x7fd1f4107de0 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4113090 con 0x7fd1f4108730 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1fb4e2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4113390 con 0x7fd1f41034b0 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1f8a7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 0x7fd1f4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1f8a7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 0x7fd1f4199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36752/0 (socket says 172.21.15.19:36752) 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.772+0000 7fd1f8a7d700 1 -- 172.21.15.19:0/3464745551 learned_addr learned my addr 172.21.15.19:0/3464745551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1f41034b0 0x7fd1f4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f4108730 0x7fd1f419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f8a7d700 1 -- 172.21.15.19:0/3464745551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1f41034b0 msgr2=0x7fd1f4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1f41034b0 0x7fd1f4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f8a7d700 1 -- 172.21.15.19:0/3464745551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f4108730 msgr2=0x7fd1f419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f4108730 0x7fd1f419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f8a7d700 1 -- 172.21.15.19:0/3464745551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41a9040 con 0x7fd1f4107de0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 0x7fd1f4199e80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400d980 tx=0x7fd1e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1ea7fc700 1 -- 172.21.15.19:0/3464745551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4017070 con 0x7fd1f4107de0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a9330 con 0x7fd1f4107de0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1ea7fc700 1 -- 172.21.15.19:0/3464745551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1e400fd20 con 0x7fd1f4107de0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1ea7fc700 1 -- 172.21.15.19:0/3464745551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400e3b0 con 0x7fd1f4107de0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.773+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a9870 con 0x7fd1f4107de0 2024-05-17T09:18:38.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.776+0000 7fd1ea7fc700 1 -- 172.21.15.19:0/3464745551 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd1e400e550 con 0x7fd1f4107de0 2024-05-17T09:18:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.777+0000 7fd1ea7fc700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1e00b2880 0x7fd1e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.777+0000 7fd209ef7b80 1 Processor -- start 2024-05-17T09:18:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.777+0000 7fd209ef7b80 1 -- start start 2024-05-17T09:18:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.777+0000 7fd1ea7fc700 1 -- 172.21.15.19:0/3464745551 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd1e40e1fb0 con 0x7fd1f4107de0 2024-05-17T09:18:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.777+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1e00b2880 0x7fd1e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557193ff6c40 con 0x5571940c8370 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557193fe67e0 con 0x557193f49df0 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557193fea820 con 0x557193fc7380 2024-05-17T09:18:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36754/0 (socket says 172.21.15.19:36754) 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/2082928117 learned_addr learned my addr 172.21.15.19:0/2082928117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1e00b2880 0x7fd1e00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005950 tx=0x7fd1f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f927e700 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/2082928117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 msgr2=0x5571940cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/2082928117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 msgr2=0x5571940ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/2082928117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557193f36880 con 0x5571940c8370 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.778+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940c8780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009c30 tx=0x7fd1ec009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c0fa1131c2ad69 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.779+0000 7fd1df7fe700 1 -- 172.21.15.19:0/2082928117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0090d0 con 0x5571940c8370 2024-05-17T09:18:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.779+0000 7fd1df7fe700 1 -- 172.21.15.19:0/2082928117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1ec020070 con 0x5571940c8370 2024-05-17T09:18:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.779+0000 7fd1df7fe700 1 -- 172.21.15.19:0/2082928117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0056e0 con 0x5571940c8370 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 -- 172.21.15.19:0/2082928117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 msgr2=0x5571940c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940c8780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009c30 tx=0x7fd1ec009c60 comp rx=0 tx=0).stop 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 -- 172.21.15.19:0/2082928117 shutdown_connections 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940c8780 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/2082928117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 -- 172.21.15.19:0/2082928117 >> 172.21.15.19:0/2082928117 conn(0x557193f88c30 msgr2=0x5571940c2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 -- 172.21.15.19:0/2082928117 shutdown_connections 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 -- 172.21.15.19:0/2082928117 wait complete. 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 -- start start 2024-05-17T09:18:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.780+0000 7fd209ef7b80 1 Processor -- start 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557193eecf50 con 0x5571940c8370 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557193f88770 con 0x557193f49df0 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5571940c9140 con 0x557193fc7380 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd209ef7b80 10 client.0 _ll_register_callbacks cb 0x557193f818d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36756/0 (socket says 172.21.15.19:36756) 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/4064866420 learned_addr learned my addr 172.21.15.19:0/4064866420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f927e700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/4064866420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 msgr2=0x5571940d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 msgr2=0x5571940d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/4064866420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557193f85640 con 0x5571940c8370 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d26d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b820 tx=0x7fd1ec003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ef6a829ab75270 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.781+0000 7fd1dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd1dcff9700 1 -- 172.21.15.19:0/4064866420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec005520 con 0x5571940c8370 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec020070 con 0x5571940c8370 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd1dcff9700 1 -- 172.21.15.19:0/4064866420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd1ec0056c0 con 0x5571940c8370 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd1dcff9700 1 -- 172.21.15.19:0/4064866420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1ec037050 con 0x5571940c8370 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 10 client.15060 fetch_fsmap learned FSMap version 26 2024-05-17T09:18:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571940df4f0 con 0x5571940c8370 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd1dcff9700 1 -- 172.21.15.19:0/4064866420 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1ec020070 con 0x5571940c8370 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd1dcff9700 1 -- 172.21.15.19:0/4064866420 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd1ec011070 con 0x5571940c8370 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 10 client.15060 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 20 client.15060 populate_metadata read hostname 'smithi019' 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571940e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 4 client.15060 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.782+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557193ff8b90 con 0x5571940e0840 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.783+0000 7fd1f927e700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571940e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.783+0000 7fd1dcff9700 10 client.15060 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:18:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.783+0000 7fd1f927e700 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571940e29f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e67b74b3cc6b54f0 server_cookie=d7400399fa5c45cc in_seq=0 out_seq=0 2024-05-17T09:18:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd1dcff9700 1 -- 172.21.15.19:0/4064866420 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557193ff8b90 con 0x5571940e0840 2024-05-17T09:18:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd1dcff9700 10 client.15060 handle_command_reply: tid=0 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 client.15060 shutdown 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 msgr2=0x5571940d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d26d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b820 tx=0x7fd1ec003a00 comp rx=0 tx=0).stop 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 shutdown_connections 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571940e29f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd1dcff9700 0 client.15060 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d26d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193f49df0 0x5571940d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/4064866420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193fc7380 0x5571940d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 >> 172.21.15.19:0/4064866420 conn(0x557193f88c30 msgr2=0x5571940c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 shutdown_connections 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.784+0000 7fd209ef7b80 1 -- 172.21.15.19:0/4064866420 wait complete. 2024-05-17T09:18:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.785+0000 7fd209ef7b80 20 client.15060 trim_cache size 0 max 16384 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 Processor -- start 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 -- start start 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193f49df0 0x5571940bc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571940e0d30 con 0x5571940c8370 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5571940e1400 con 0x557193fc7380 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.806+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5571940e1ad0 con 0x557193f49df0 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36766/0 (socket says 172.21.15.19:36766) 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/3548099796 learned_addr learned my addr 172.21.15.19:0/3548099796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:38.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193f49df0 0x5571940bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/3548099796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193f49df0 msgr2=0x5571940bc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193f49df0 0x5571940bc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/3548099796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 msgr2=0x5571940ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 -- 172.21.15.19:0/3548099796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571940d01e0 con 0x5571940c8370 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193f49df0 0x5571940bc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d1440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b420 tx=0x7fd1ec0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3553401003d96e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1deffd700 1 -- 172.21.15.19:0/3548099796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec005c60 con 0x5571940c8370 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1deffd700 1 -- 172.21.15.19:0/3548099796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1ec020040 con 0x5571940c8370 2024-05-17T09:18:38.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.807+0000 7fd1deffd700 1 -- 172.21.15.19:0/3548099796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0075c0 con 0x5571940c8370 2024-05-17T09:18:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3548099796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 msgr2=0x5571940d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d1440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b420 tx=0x7fd1ec0051b0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3548099796 shutdown_connections 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557193f49df0 0x5571940bc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571940c8370 0x5571940d1440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3548099796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3548099796 >> 172.21.15.19:0/3548099796 conn(0x557193f88c30 msgr2=0x5571940cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3548099796 shutdown_connections 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3548099796 wait complete. 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.808+0000 7fd209ef7b80 1 -- start start 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 Processor -- start 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 0x5571940ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571940c8370 0x5571940e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571940e2870 con 0x557193f49df0 2024-05-17T09:18:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5571940e21a0 con 0x557193fc7380 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5571940e2f40 con 0x5571940c8370 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd209ef7b80 10 client.0 _ll_register_callbacks cb 0x557193f818d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd1f927e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 0x5571940ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd1f927e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 0x5571940ebe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36774/0 (socket says 172.21.15.19:36774) 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd1f927e700 1 -- 172.21.15.19:0/3107465771 learned_addr learned my addr 172.21.15.19:0/3107465771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.809+0000 7fd1f9a7f700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571940c8370 0x5571940e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1f927e700 1 -- 172.21.15.19:0/3107465771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571940c8370 msgr2=0x5571940e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571940c8370 0x5571940e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1f927e700 1 -- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 msgr2=0x5571940ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1f927e700 1 -- 172.21.15.19:0/3107465771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571940d18e0 con 0x557193f49df0 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1f927e700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 0x5571940ebe70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5571940c3e30 tx=0x7fd1f0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de71c15e82dc0af server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1dcff9700 1 -- 172.21.15.19:0/3107465771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002e070 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1dcff9700 1 -- 172.21.15.19:0/3107465771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd1f002a5b0 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec020040 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1dcff9700 1 -- 172.21.15.19:0/3107465771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1f003a050 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd209ef7b80 10 client.15066 fetch_fsmap learned FSMap version 26 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55719414d0b0 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.810+0000 7fd1dcff9700 1 -- 172.21.15.19:0/3107465771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1ec020040 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd1dcff9700 1 -- 172.21.15.19:0/3107465771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd1f001ea70 con 0x557193f49df0 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd209ef7b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd209ef7b80 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd209ef7b80 20 client.15066 populate_metadata read hostname 'smithi019' 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571941509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd209ef7b80 4 client.15066 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "status"}) -- 0x557193ff2370 con 0x5571940e0840 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.811+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571941509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd1dcff9700 10 client.15066 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd1f8a7d700 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571941509e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea440e760b16528b server_cookie=31351cc3671310c9 in_seq=0 out_seq=0 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd1dcff9700 1 -- 172.21.15.19:0/3107465771 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557193ff2370 con 0x5571940e0840 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd1dcff9700 10 client.15066 handle_command_reply: tid=0 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 client.15066 shutdown 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 msgr2=0x5571940ebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 0x5571940ebe70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5571940c3e30 tx=0x7fd1f0013070 comp rx=0 tx=0).stop 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 shutdown_connections 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5571940e0840 0x5571941509e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571940c8370 0x5571940e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557193f49df0 0x5571940ebe70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 --2- 172.21.15.19:0/3107465771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557193fc7380 0x5571940ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 >> 172.21.15.19:0/3107465771 conn(0x557193f88c30 msgr2=0x5571940bcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd1dcff9700 0 client.15066 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 shutdown_connections 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 1 -- 172.21.15.19:0/3107465771 wait complete. 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.812+0000 7fd209ef7b80 20 client.15066 trim_cache size 0 max 16384 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24470, 2024-05-17T09:18:38.818 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 141.60261861199999, 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 239.380904853 2024-05-17T09:18:38.819 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1e00b2880 msgr2=0x7fd1e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1e00b2880 0x7fd1e00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005950 tx=0x7fd1f000b410 comp rx=0 tx=0).stop 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 msgr2=0x7fd1f4199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 0x7fd1f4199e80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400d980 tx=0x7fd1e400dc90 comp rx=0 tx=0).stop 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 shutdown_connections 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1e00b2880 0x7fd1e00b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1f41034b0 0x7fd1f4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f4108730 0x7fd1f419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 --2- 172.21.15.19:0/3464745551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f4107de0 0x7fd1f4199e80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:38.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 >> 172.21.15.19:0/3464745551 conn(0x7fd1f40fedb0 msgr2=0x7fd1f41075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:38.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 shutdown_connections 2024-05-17T09:18:38.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:38.813+0000 7fd1fb4e2700 1 -- 172.21.15.19:0/3464745551 wait complete. 2024-05-17T09:18:39.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T09:18:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:40 smithi079 ceph-mon[35502]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:40 smithi069 ceph-mon[35081]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:40 smithi019 ceph-mon[31620]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 -- 172.21.15.19:0/1634309446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 msgr2=0x7f77f01038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 --2- 172.21.15.19:0/1634309446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f01038c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f77e00099c0 tx=0x7f77e0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 -- 172.21.15.19:0/1634309446 shutdown_connections 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 --2- 172.21.15.19:0/1634309446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77f0108730 0x7f77f0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 --2- 172.21.15.19:0/1634309446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f01038c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 --2- 172.21.15.19:0/1634309446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77f0107de0 0x7f77f01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 -- 172.21.15.19:0/1634309446 >> 172.21.15.19:0/1634309446 conn(0x7f77f00fede0 msgr2=0x7f77f0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 -- 172.21.15.19:0/1634309446 shutdown_connections 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.592+0000 7f77f63cf700 1 -- 172.21.15.19:0/1634309446 wait complete. 2024-05-17T09:18:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 Processor -- start 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 -- start start 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f00ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77f0107de0 0x7f77f00ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77f0108730 0x7f77f00ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77f0113260 con 0x7f77f01034b0 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77f01130e0 con 0x7f77f0107de0 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f63cf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77f01133e0 con 0x7f77f0108730 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f00ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f00ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36786/0 (socket says 172.21.15.19:36786) 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77effff700 1 -- 172.21.15.19:0/428546927 learned_addr learned my addr 172.21.15.19:0/428546927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77f496c700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77f0108730 0x7f77f00ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.593+0000 7f77e7fff700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77f0107de0 0x7f77f00ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77effff700 1 -- 172.21.15.19:0/428546927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77f0108730 msgr2=0x7f77f00ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77effff700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77f0108730 0x7f77f00ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77effff700 1 -- 172.21.15.19:0/428546927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77f0107de0 msgr2=0x7f77f00ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77effff700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77f0107de0 0x7f77f00ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77effff700 1 -- 172.21.15.19:0/428546927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f01a90b0 con 0x7f77f01034b0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77effff700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f00ff2c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000c00 tx=0x7f77e000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77edffb700 1 -- 172.21.15.19:0/428546927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0027070 con 0x7f77f01034b0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77edffb700 1 -- 172.21.15.19:0/428546927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77e000fcf0 con 0x7f77f01034b0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f01a9340 con 0x7f77f01034b0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77edffb700 1 -- 172.21.15.19:0/428546927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0022dc0 con 0x7f77f01034b0 2024-05-17T09:18:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.594+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f01a97f0 con 0x7f77f01034b0 2024-05-17T09:18:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.598+0000 7f77edffb700 1 -- 172.21.15.19:0/428546927 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f77e002a070 con 0x7f77f01034b0 2024-05-17T09:18:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 Processor -- start 2024-05-17T09:18:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 -- start start 2024-05-17T09:18:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f77edffb700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77dc0b2900 0x7f77dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f77f496c700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77dc0b2900 0x7f77dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f77edffb700 1 -- 172.21.15.19:0/428546927 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f77e00e30e0 con 0x7f77f01034b0 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f4bf380 0x55d81f5c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 0x55d81f5c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d81f4eec40 con 0x55d81f5c0370 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d81f4de7e0 con 0x55d81f441df0 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f7804de4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d81f4e2820 con 0x55d81f4bf380 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f77f496c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 0x55d81f5c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f77f496c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 0x55d81f5c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36792/0 (socket says 172.21.15.19:36792) 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.599+0000 7f77f496c700 1 -- 172.21.15.19:0/3196759848 learned_addr learned my addr 172.21.15.19:0/3196759848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77effff700 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f4bf380 0x55d81f5c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77dc0b2900 0x7f77dc0b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009cd0 tx=0x7f77e8004320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 -- 172.21.15.19:0/3196759848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f4bf380 msgr2=0x55d81f5c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f4bf380 0x55d81f5c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 -- 172.21.15.19:0/3196759848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 msgr2=0x55d81f5c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 -- 172.21.15.19:0/3196759848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81f42e880 con 0x55d81f5c0370 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77f496c700 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 0x55d81f5c0780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f77e8003d00 tx=0x7f77e801c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba980419ba1eba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77e67fc700 1 -- 172.21.15.19:0/3196759848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8004db0 con 0x55d81f5c0370 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77e67fc700 1 -- 172.21.15.19:0/3196759848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77e801a470 con 0x55d81f5c0370 2024-05-17T09:18:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.600+0000 7f77e67fc700 1 -- 172.21.15.19:0/3196759848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e80246c0 con 0x55d81f5c0370 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 -- 172.21.15.19:0/3196759848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 msgr2=0x55d81f5c0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 0x55d81f5c0780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f77e8003d00 tx=0x7f77e801c030 comp rx=0 tx=0).stop 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 -- 172.21.15.19:0/3196759848 shutdown_connections 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f4bf380 0x55d81f5c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f5c0370 0x55d81f5c0780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3196759848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 -- 172.21.15.19:0/3196759848 >> 172.21.15.19:0/3196759848 conn(0x55d81f480c30 msgr2=0x55d81f5bae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 -- 172.21.15.19:0/3196759848 shutdown_connections 2024-05-17T09:18:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.601+0000 7f7804de4b80 1 -- 172.21.15.19:0/3196759848 wait complete. 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 -- start start 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 Processor -- start 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 0x55d81f5c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f4bf380 0x55d81f5ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d81f3e4f50 con 0x55d81f441df0 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d81f480770 con 0x55d81f4bf380 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5c11a0 con 0x55d81f5c0370 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f7804de4b80 10 client.0 _ll_register_callbacks cb 0x55d81f4798d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f77effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 0x55d81f5c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.602+0000 7f77effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 0x55d81f5c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36794/0 (socket says 172.21.15.19:36794) 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 -- 172.21.15.19:0/3519614449 learned_addr learned my addr 172.21.15.19:0/3519614449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77f496c700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f4bf380 0x55d81f5ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 -- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 msgr2=0x55d81f5ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 -- 172.21.15.19:0/3519614449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f4bf380 msgr2=0x55d81f5ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f4bf380 0x55d81f5ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 -- 172.21.15.19:0/3519614449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81f47d640 con 0x55d81f441df0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77effff700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 0x55d81f5c9c70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77e00a7650 tx=0x7f77e0022870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dcbe25dd1e7b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77cf7fe700 1 -- 172.21.15.19:0/3519614449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e00aa3e0 con 0x55d81f441df0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77cf7fe700 1 -- 172.21.15.19:0/3519614449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f77e00b2f30 con 0x55d81f441df0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e801a470 con 0x55d81f441df0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f77cf7fe700 1 -- 172.21.15.19:0/3519614449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77e00ae790 con 0x55d81f441df0 2024-05-17T09:18:41.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f7804de4b80 10 client.15078 fetch_fsmap learned FSMap version 26 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.603+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d81f5d7590 con 0x55d81f441df0 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f77cf7fe700 1 -- 172.21.15.19:0/3519614449 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77e801a470 con 0x55d81f441df0 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f77cf7fe700 1 -- 172.21.15.19:0/3519614449 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f77e0005140 con 0x55d81f441df0 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f7804de4b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f7804de4b80 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f7804de4b80 20 client.15078 populate_metadata read hostname 'smithi019' 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f5d9c30 0x55d81f5db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f7804de4b80 4 client.15078 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d81f4f0b90 con 0x55d81f5d9c30 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.604+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f5d9c30 0x55d81f5db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.605+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f5d9c30 0x55d81f5db0a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7de907be245481f7 server_cookie=b231953761b34c84 in_seq=0 out_seq=0 2024-05-17T09:18:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.605+0000 7f77cf7fe700 10 client.15078 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:18:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f77cf7fe700 1 -- 172.21.15.19:0/3519614449 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d81f4f0b90 con 0x55d81f5d9c30 2024-05-17T09:18:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f77cf7fe700 10 client.15078 handle_command_reply: tid=0 2024-05-17T09:18:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 client.15078 shutdown 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 msgr2=0x55d81f5c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 0x55d81f5c9c70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77e00a7650 tx=0x7f77e0022870 comp rx=0 tx=0).stop 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 shutdown_connections 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f5d9c30 0x55d81f5db0a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f441df0 0x55d81f5c9c70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f77cf7fe700 0 client.15078 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3519614449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f4bf380 0x55d81f5ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 >> 172.21.15.19:0/3519614449 conn(0x55d81f480c30 msgr2=0x55d81f5bae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 shutdown_connections 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 1 -- 172.21.15.19:0/3519614449 wait complete. 2024-05-17T09:18:41.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.606+0000 7f7804de4b80 20 client.15078 trim_cache size 0 max 16384 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.627+0000 7f7804de4b80 1 Processor -- start 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 -- start start 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f441df0 0x55d81f5d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f5c0370 0x55d81f5bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5d93e0 con 0x55d81f4bf380 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5d9ab0 con 0x55d81f5c0370 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f7804de4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5da180 con 0x55d81f441df0 2024-05-17T09:18:41.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36804/0 (socket says 172.21.15.19:36804) 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 -- 172.21.15.19:0/3589895758 learned_addr learned my addr 172.21.15.19:0/3589895758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77effff700 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f441df0 0x55d81f5d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77f496c700 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f5c0370 0x55d81f5bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 -- 172.21.15.19:0/3589895758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f441df0 msgr2=0x55d81f5d5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f441df0 0x55d81f5d5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 -- 172.21.15.19:0/3589895758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f5c0370 msgr2=0x55d81f5bf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f5c0370 0x55d81f5bf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.628+0000 7f77e7fff700 1 -- 172.21.15.19:0/3589895758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81f5c5480 con 0x55d81f4bf380 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.629+0000 7f77e7fff700 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5b5620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77f0109b50 tx=0x7f77d8010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c0209b10397736 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.629+0000 7f77e5ffb700 1 -- 172.21.15.19:0/3589895758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8010b70 con 0x55d81f4bf380 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.629+0000 7f77e5ffb700 1 -- 172.21.15.19:0/3589895758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d8010d10 con 0x55d81f4bf380 2024-05-17T09:18:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.629+0000 7f77e5ffb700 1 -- 172.21.15.19:0/3589895758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d801e410 con 0x55d81f4bf380 2024-05-17T09:18:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 -- 172.21.15.19:0/3589895758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 msgr2=0x55d81f5b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5b5620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77f0109b50 tx=0x7f77d8010430 comp rx=0 tx=0).stop 2024-05-17T09:18:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 -- 172.21.15.19:0/3589895758 shutdown_connections 2024-05-17T09:18:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f441df0 0x55d81f5d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f5c0370 0x55d81f5bf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 --2- 172.21.15.19:0/3589895758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5b5620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 -- 172.21.15.19:0/3589895758 >> 172.21.15.19:0/3589895758 conn(0x55d81f480c30 msgr2=0x55d81f5b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 -- 172.21.15.19:0/3589895758 shutdown_connections 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 -- 172.21.15.19:0/3589895758 wait complete. 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 -- start start 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.630+0000 7f7804de4b80 1 Processor -- start 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5e4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5daf20 con 0x55d81f4bf380 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5da850 con 0x55d81f441df0 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d81f5db5f0 con 0x55d81f5c0370 2024-05-17T09:18:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f7804de4b80 10 client.0 _ll_register_callbacks cb 0x55d81f4798d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5e4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36814/0 (socket says 172.21.15.19:36814) 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 -- 172.21.15.19:0/2822050815 learned_addr learned my addr 172.21.15.19:0/2822050815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77f496c700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77effff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 -- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 msgr2=0x55d81f5df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 -- 172.21.15.19:0/2822050815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 msgr2=0x55d81f5e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 -- 172.21.15.19:0/2822050815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81f5e5050 con 0x55d81f4bf380 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77effff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77e7fff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5e4a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77d800a780 tx=0x7f77d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f215084ebd7b3ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.631+0000 7f77cf7fe700 1 -- 172.21.15.19:0/2822050815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004bf0 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f77cf7fe700 1 -- 172.21.15.19:0/2822050815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f77d8004d90 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d8010d10 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f77cf7fe700 1 -- 172.21.15.19:0/2822050815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77d8039050 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 10 client.15084 fetch_fsmap learned FSMap version 26 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d81f6458d0 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f77cf7fe700 1 -- 172.21.15.19:0/2822050815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d8010d10 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f77cf7fe700 1 -- 172.21.15.19:0/2822050815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f77d80268e0 con 0x55d81f4bf380 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 20 client.15084 populate_metadata read hostname 'smithi019' 2024-05-17T09:18:41.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f647980 0x55d81f648df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 4 client.15084 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.632+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "status"}) -- 0x55d81f4ea370 con 0x55d81f647980 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f77effff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f647980 0x55d81f648df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f77effff700 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f647980 0x55d81f648df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5470e10f2bcd188c server_cookie=2078dce03bf71d5b in_seq=0 out_seq=0 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f77cf7fe700 10 client.15084 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f77cf7fe700 1 -- 172.21.15.19:0/2822050815 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d81f4ea370 con 0x55d81f647980 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f77cf7fe700 10 client.15084 handle_command_reply: tid=0 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 client.15084 shutdown 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 msgr2=0x55d81f5e4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5e4a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77d800a780 tx=0x7f77d80047c0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 shutdown_connections 2024-05-17T09:18:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55d81f647980 0x55d81f648df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d81f5c0370 0x55d81f5df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.633+0000 7f7804de4b80 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d81f441df0 0x55d81f5e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f7804de4b80 1 --2- 172.21.15.19:0/2822050815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d81f4bf380 0x55d81f5e4a60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77cf7fe700 0 client.15084 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 >> 172.21.15.19:0/2822050815 conn(0x55d81f480c30 msgr2=0x55d81f5c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 shutdown_connections 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f7804de4b80 1 -- 172.21.15.19:0/2822050815 wait complete. 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f7804de4b80 20 client.15084 trim_cache size 0 max 16384 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:18:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 145.43912280699999, 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 239.44634297100001 2024-05-17T09:18:41.640 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77dc0b2900 msgr2=0x7f77dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77dc0b2900 0x7f77dc0b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009cd0 tx=0x7f77e8004320 comp rx=0 tx=0).stop 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 msgr2=0x7f77f00ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f00ff2c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000c00 tx=0x7f77e000f460 comp rx=0 tx=0).stop 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 shutdown_connections 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77dc0b2900 0x7f77dc0b4e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77f0108730 0x7f77f00ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f01034b0 0x7f77f00ff2c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 --2- 172.21.15.19:0/428546927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77f0107de0 0x7f77f00ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 >> 172.21.15.19:0/428546927 conn(0x7f77f00fede0 msgr2=0x7f77f0075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.634+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 shutdown_connections 2024-05-17T09:18:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:41.635+0000 7f77f63cf700 1 -- 172.21.15.19:0/428546927 wait complete. 2024-05-17T09:18:42.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:42 smithi019 ceph-mon[31620]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:42.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T09:18:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:42 smithi079 ceph-mon[35502]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:42 smithi069 ceph-mon[35081]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 -- 172.21.15.19:0/94400558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 msgr2=0x7feea41100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/94400558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea41100b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00b0d0 tx=0x7fee9c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 -- 172.21.15.19:0/94400558 shutdown_connections 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/94400558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea41100b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/94400558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea41034d0 0x7feea41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/94400558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea4107e00 0x7feea4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 -- 172.21.15.19:0/94400558 >> 172.21.15.19:0/94400558 conn(0x7feea40fef20 msgr2=0x7feea4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 -- 172.21.15.19:0/94400558 shutdown_connections 2024-05-17T09:18:44.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.316+0000 7feeaaa6f700 1 -- 172.21.15.19:0/94400558 wait complete. 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.317+0000 7feeaaa6f700 1 Processor -- start 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.317+0000 7feeaaa6f700 1 -- start start 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feeaaa6f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea41034d0 0x7feea41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feeaaa6f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea4107e00 0x7feea41a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feeaaa6f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feeaaa6f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feea4113230 con 0x7feea4108750 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feeaaa6f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feea41130b0 con 0x7feea41034d0 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feeaaa6f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feea41133b0 con 0x7feea4107e00 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea4107e00 0x7feea41a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42462/0 (socket says 172.21.15.19:42462) 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 -- 172.21.15.19:0/2668319360 learned_addr learned my addr 172.21.15.19:0/2668319360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea880b700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea41034d0 0x7feea41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 -- 172.21.15.19:0/2668319360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea4107e00 msgr2=0x7feea41a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea4107e00 0x7feea41a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 -- 172.21.15.19:0/2668319360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea41034d0 msgr2=0x7feea41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea41034d0 0x7feea41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 -- 172.21.15.19:0/2668319360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea419b1e0 con 0x7feea4108750 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.318+0000 7feea900c700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea419ab00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009850 tx=0x7fee9c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.319+0000 7feea1ffb700 1 -- 172.21.15.19:0/2668319360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01f070 con 0x7feea4108750 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.319+0000 7feea1ffb700 1 -- 172.21.15.19:0/2668319360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee9c005500 con 0x7feea4108750 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.319+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea419b470 con 0x7feea4108750 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.319+0000 7feea1ffb700 1 -- 172.21.15.19:0/2668319360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01b8f0 con 0x7feea4108750 2024-05-17T09:18:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.319+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41a7770 con 0x7feea4108750 2024-05-17T09:18:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.321+0000 7feea1ffb700 1 -- 172.21.15.19:0/2668319360 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fee9c022070 con 0x7feea4108750 2024-05-17T09:18:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.322+0000 7feea1ffb700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee940b2880 0x7fee940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.322+0000 7feea880b700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee940b2880 0x7fee940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.322+0000 7feea1ffb700 1 -- 172.21.15.19:0/2668319360 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fee9c00f020 con 0x7feea4108750 2024-05-17T09:18:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feea880b700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee940b2880 0x7fee940b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feea419bc40 tx=0x7fee9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 Processor -- start 2024-05-17T09:18:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 -- start start 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14301edf0 0x55d1431a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 0x55d14319e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1430cbd20 con 0x55d14319dc80 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1430bb8c0 con 0x55d14301edf0 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feeb9484b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d1430bf900 con 0x55d14310a3e0 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feea900c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 0x55d14319e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.323+0000 7feea3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14301edf0 0x55d1431a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 0x55d14319e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42470/0 (socket says 172.21.15.19:42470) 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 -- 172.21.15.19:0/2782507371 learned_addr learned my addr 172.21.15.19:0/2782507371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea880b700 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 -- 172.21.15.19:0/2782507371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 msgr2=0x55d1431a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 -- 172.21.15.19:0/2782507371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14301edf0 msgr2=0x55d1431a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14301edf0 0x55d1431a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 -- 172.21.15.19:0/2782507371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d143064f60 con 0x55d14319dc80 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea3fff700 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14301edf0 0x55d1431a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7feea900c700 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 0x55d14319e150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0090a0 tx=0x7fee9c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba563615fab65c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7fee8effd700 1 -- 172.21.15.19:0/2782507371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0aad50 con 0x55d14319dc80 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7fee8effd700 1 -- 172.21.15.19:0/2782507371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee9c093070 con 0x55d14319dc80 2024-05-17T09:18:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.324+0000 7fee8effd700 1 -- 172.21.15.19:0/2782507371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0aafb0 con 0x55d14319dc80 2024-05-17T09:18:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 -- 172.21.15.19:0/2782507371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 msgr2=0x55d14319e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 0x55d14319e150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0090a0 tx=0x7fee9c0099e0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 -- 172.21.15.19:0/2782507371 shutdown_connections 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14319dc80 0x55d14319e150 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2782507371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14301edf0 0x55d1431a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 -- 172.21.15.19:0/2782507371 >> 172.21.15.19:0/2782507371 conn(0x55d14307f170 msgr2=0x55d143198780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 -- 172.21.15.19:0/2782507371 shutdown_connections 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.325+0000 7feeb9484b80 1 -- 172.21.15.19:0/2782507371 wait complete. 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 -- start start 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 Processor -- start 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d142fc1f50 con 0x55d14301edf0 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d143023d50 con 0x55d14310a3e0 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d14319e990 con 0x55d14319dc80 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.326+0000 7feeb9484b80 10 client.0 _ll_register_callbacks cb 0x55d143056900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42472/0 (socket says 172.21.15.19:42472) 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 -- 172.21.15.19:0/2235268321 learned_addr learned my addr 172.21.15.19:0/2235268321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea900c700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea3fff700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 -- 172.21.15.19:0/2235268321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 msgr2=0x55d1431a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 -- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 msgr2=0x55d1431ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 -- 172.21.15.19:0/2235268321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d14300b880 con 0x55d14301edf0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea3fff700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feea880b700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431ad030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee980116f0 tx=0x7fee980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab33305caeae9e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7fee83fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7fee83fff700 1 -- 172.21.15.19:0/2235268321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9801f840 con 0x55d14301edf0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c093070 con 0x55d14301edf0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.327+0000 7fee83fff700 1 -- 172.21.15.19:0/2235268321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fee9801f9e0 con 0x55d14301edf0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7fee83fff700 1 -- 172.21.15.19:0/2235268321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee98031050 con 0x55d14301edf0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 10 client.15096 fetch_fsmap learned FSMap version 26 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d143075710 con 0x55d14301edf0 2024-05-17T09:18:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7fee83fff700 1 -- 172.21.15.19:0/2235268321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee9c093070 con 0x55d14301edf0 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7fee83fff700 1 -- 172.21.15.19:0/2235268321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fee9801ebf0 con 0x55d14301edf0 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 20 client.15096 populate_metadata read hostname 'smithi019' 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1431b7150 0x55d1431b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 4 client.15096 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.328+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d143169e80 con 0x55d1431b7150 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.329+0000 7feea3fff700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1431b7150 0x55d1431b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.329+0000 7fee83fff700 10 client.15096 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:18:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.329+0000 7feea3fff700 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1431b7150 0x55d1431b85c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8cf85479d45a05 server_cookie=bb478bbbbf3265a3 in_seq=0 out_seq=0 2024-05-17T09:18:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7fee83fff700 1 -- 172.21.15.19:0/2235268321 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d143169e80 con 0x55d1431b7150 2024-05-17T09:18:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7fee83fff700 10 client.15096 handle_command_reply: tid=0 2024-05-17T09:18:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 client.15096 shutdown 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 msgr2=0x55d1431ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431ad030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee980116f0 tx=0x7fee980047c0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 shutdown_connections 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1431b7150 0x55d1431b85c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431ad030 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2235268321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 >> 172.21.15.19:0/2235268321 conn(0x55d14307f170 msgr2=0x55d143198740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7fee83fff700 0 client.15096 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 shutdown_connections 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 1 -- 172.21.15.19:0/2235268321 wait complete. 2024-05-17T09:18:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.330+0000 7feeb9484b80 20 client.15096 trim_cache size 0 max 16384 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 Processor -- start 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 -- start start 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14319dc80 0x55d1431c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1431b6900 con 0x55d14301edf0 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1431b6fd0 con 0x55d14319dc80 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feeb9484b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d1431b76a0 con 0x55d14310a3e0 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42484/0 (socket says 172.21.15.19:42484) 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 -- 172.21.15.19:0/3419478383 learned_addr learned my addr 172.21.15.19:0/3419478383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea3fff700 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 -- 172.21.15.19:0/3419478383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 msgr2=0x55d1431c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea900c700 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14319dc80 0x55d1431c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 -- 172.21.15.19:0/3419478383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14319dc80 msgr2=0x55d1431c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14319dc80 0x55d1431c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 -- 172.21.15.19:0/3419478383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1431a59a0 con 0x55d14301edf0 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.352+0000 7feea880b700 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a3c30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee98005d90 tx=0x7fee9800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cd899e3ad9b196 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.353+0000 7fee8e7fc700 1 -- 172.21.15.19:0/3419478383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98039020 con 0x55d14301edf0 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.353+0000 7fee8e7fc700 1 -- 172.21.15.19:0/3419478383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee98023070 con 0x55d14301edf0 2024-05-17T09:18:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.353+0000 7fee8e7fc700 1 -- 172.21.15.19:0/3419478383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9801fc90 con 0x55d14301edf0 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.353+0000 7feeb9484b80 1 -- 172.21.15.19:0/3419478383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 msgr2=0x55d1431a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.353+0000 7feeb9484b80 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a3c30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee98005d90 tx=0x7fee9800b480 comp rx=0 tx=0).stop 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 -- 172.21.15.19:0/3419478383 shutdown_connections 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14319dc80 0x55d1431c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a3c30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 --2- 172.21.15.19:0/3419478383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14310a3e0 0x55d1431c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 -- 172.21.15.19:0/3419478383 >> 172.21.15.19:0/3419478383 conn(0x55d14307f170 msgr2=0x55d1431ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 -- 172.21.15.19:0/3419478383 shutdown_connections 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 -- 172.21.15.19:0/3419478383 wait complete. 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 -- start start 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.354+0000 7feeb9484b80 1 Processor -- start 2024-05-17T09:18:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1431b8440 con 0x55d14301edf0 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1431b7d70 con 0x55d14310a3e0 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d1431b8b10 con 0x55d14319dc80 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feeb9484b80 10 client.0 _ll_register_callbacks cb 0x55d143056900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42500/0 (socket says 172.21.15.19:42500) 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 -- 172.21.15.19:0/2702398513 learned_addr learned my addr 172.21.15.19:0/2702398513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea3fff700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea900c700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 -- 172.21.15.19:0/2702398513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 msgr2=0x55d1431b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 -- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 msgr2=0x55d1431afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:18:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 -- 172.21.15.19:0/2702398513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1431b0950 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7feea880b700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a4ef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee98005810 tx=0x7fee98010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef734ff9b945925f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7fee83fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7fee83fff700 1 -- 172.21.15.19:0/2702398513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98039030 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.355+0000 7fee83fff700 1 -- 172.21.15.19:0/2702398513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fee98005220 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee98023070 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7fee83fff700 1 -- 172.21.15.19:0/2702398513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee98050050 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 10 client.15102 fetch_fsmap learned FSMap version 26 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d143226bc0 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7fee83fff700 1 -- 172.21.15.19:0/2702398513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee98023070 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7fee83fff700 1 -- 172.21.15.19:0/2702398513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fee98034070 con 0x55d14301edf0 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:18:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 20 client.15102 populate_metadata read hostname 'smithi019' 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d143228c40 0x55d14322a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 4 client.15102 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "status"}) -- 0x55d1430cdc70 con 0x55d143228c40 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.356+0000 7feea3fff700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d143228c40 0x55d14322a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7fee83fff700 10 client.15102 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feea3fff700 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d143228c40 0x55d14322a0b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2d8ea181299457 server_cookie=c54cdbba5cab0ca2 in_seq=0 out_seq=0 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7fee83fff700 1 -- 172.21.15.19:0/2702398513 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d1430cdc70 con 0x55d143228c40 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7fee83fff700 10 client.15102 handle_command_reply: tid=0 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 client.15102 shutdown 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 msgr2=0x55d1431a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a4ef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee98005810 tx=0x7fee98010a40 comp rx=0 tx=0).stop 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 shutdown_connections 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d14319dc80 0x55d1431b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d143228c40 0x55d14322a0b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7fee83fff700 0 client.15102 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d14301edf0 0x55d1431a4ef0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 --2- 172.21.15.19:0/2702398513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d14310a3e0 0x55d1431afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.357+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 >> 172.21.15.19:0/2702398513 conn(0x55d14307f170 msgr2=0x55d1431a11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 shutdown_connections 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeb9484b80 1 -- 172.21.15.19:0/2702398513 wait complete. 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeb9484b80 20 client.15102 trim_cache size 0 max 16384 2024-05-17T09:18:44.362 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24497, 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 214.39998601900001, 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 231.75170665499999 2024-05-17T09:18:44.363 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee940b2880 msgr2=0x7fee940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee940b2880 0x7fee940b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feea419bc40 tx=0x7fee9800b410 comp rx=0 tx=0).stop 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 msgr2=0x7feea419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea419ab00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009850 tx=0x7fee9c007d90 comp rx=0 tx=0).stop 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 shutdown_connections 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea4108750 0x7feea419ab00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea41034d0 0x7feea41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee940b2880 0x7fee940b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 --2- 172.21.15.19:0/2668319360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea4107e00 0x7feea41a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.358+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 >> 172.21.15.19:0/2668319360 conn(0x7feea40fef20 msgr2=0x7feea406c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.359+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 shutdown_connections 2024-05-17T09:18:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:44.359+0000 7feeaaa6f700 1 -- 172.21.15.19:0/2668319360 wait complete. 2024-05-17T09:18:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:44 smithi079 ceph-mon[35502]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:44 smithi069 ceph-mon[35081]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:44 smithi019 ceph-mon[31620]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:45.109 INFO:tasks.cephadm:Setup complete, yielding 2024-05-17T09:18:45.109 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T09:18:45.120 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:18:45.120 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-17T09:18:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:46.498 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:46 smithi019 ceph-mon[31620]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:46 smithi069 ceph-mon[35081]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:46 smithi079 ceph-mon[35502]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:47.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/3821247367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc1074c0 msgr2=0x7f3ecc109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:47.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/3821247367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc1074c0 0x7f3ecc109960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009990 tx=0x7f3ec0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/3821247367 shutdown_connections 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/3821247367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc109ff0 0x7f3ecc10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/3821247367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ecc076660 0x7f3ecc106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/3821247367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc1074c0 0x7f3ecc109960 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/3821247367 >> 172.21.15.19:0/3821247367 conn(0x7f3ecc0fe880 msgr2=0x7f3ecc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/3821247367 shutdown_connections 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.039+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/3821247367 wait complete. 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.040+0000 7f3ed3ab8700 1 Processor -- start 2024-05-17T09:18:47.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.040+0000 7f3ed3ab8700 1 -- start start 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed3ab8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 0x7f3ecc104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed3ab8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ecc1074c0 0x7f3ecc102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed3ab8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc109ff0 0x7f3ecc103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed3ab8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10fc50 con 0x7f3ecc076660 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed3ab8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10fad0 con 0x7f3ecc1074c0 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed3ab8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10fdd0 con 0x7f3ecc109ff0 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 0x7f3ecc104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 0x7f3ecc104740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42516/0 (socket says 172.21.15.19:42516) 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 -- 172.21.15.19:0/1700610944 learned_addr learned my addr 172.21.15.19:0/1700610944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed2055700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc109ff0 0x7f3ecc103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1053700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ecc1074c0 0x7f3ecc102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 -- 172.21.15.19:0/1700610944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc109ff0 msgr2=0x7f3ecc103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc109ff0 0x7f3ecc103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 -- 172.21.15.19:0/1700610944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ecc1074c0 msgr2=0x7f3ecc102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ecc1074c0 0x7f3ecc102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 -- 172.21.15.19:0/1700610944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc103a00 con 0x7f3ecc076660 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.041+0000 7f3ed1854700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 0x7f3ecc104740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc007f10 tx=0x7f3ebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:47.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.042+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00dd20 con 0x7f3ecc076660 2024-05-17T09:18:47.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.042+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc1a4920 con 0x7f3ecc076660 2024-05-17T09:18:47.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.042+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ebc018070 con 0x7f3ecc076660 2024-05-17T09:18:47.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.042+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0125b0 con 0x7f3ecc076660 2024-05-17T09:18:47.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.042+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc1a4dd0 con 0x7f3ecc076660 2024-05-17T09:18:47.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.043+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc0512b0 con 0x7f3ecc076660 2024-05-17T09:18:47.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.044+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3ebc0133f0 con 0x7f3ecc076660 2024-05-17T09:18:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.045+0000 7f3ecaffd700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3eb40b2900 0x7f3eb40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.045+0000 7f3ed1053700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3eb40b2900 0x7f3eb40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.045+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3ebc0e26b0 con 0x7f3ecc076660 2024-05-17T09:18:47.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.046+0000 7f3ed1053700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3eb40b2900 0x7f3eb40b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b5c0 tx=0x7f3ec001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.049+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ebc010070 con 0x7f3ecc076660 2024-05-17T09:18:47.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.283+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3ecc063bf0 con 0x7f3eb40b2900 2024-05-17T09:18:47.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.284+0000 7f3ecaffd700 1 -- 172.21.15.19:0/1700610944 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3ecc063bf0 con 0x7f3eb40b2900 2024-05-17T09:18:47.283 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-05-17T09:18:47.283 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-05-17T09:18:47.283 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3eb40b2900 msgr2=0x7f3eb40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3eb40b2900 0x7f3eb40b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b5c0 tx=0x7f3ec001a040 comp rx=0 tx=0).stop 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 msgr2=0x7f3ecc104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 0x7f3ecc104740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc007f10 tx=0x7f3ebc00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 shutdown_connections 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3eb40b2900 0x7f3eb40b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc109ff0 0x7f3ecc103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ecc076660 0x7f3ecc104740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 --2- 172.21.15.19:0/1700610944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ecc1074c0 0x7f3ecc102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 >> 172.21.15.19:0/1700610944 conn(0x7f3ecc0fe880 msgr2=0x7f3ecc1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 shutdown_connections 2024-05-17T09:18:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:47.286+0000 7f3ed3ab8700 1 -- 172.21.15.19:0/1700610944 wait complete. 2024-05-17T09:18:47.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-17T09:18:48.212 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:48.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:48 smithi019 ceph-mon[31620]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:48.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:48 smithi019 ceph-mon[31620]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:48 smithi069 ceph-mon[35081]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:48 smithi069 ceph-mon[35081]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:48 smithi079 ceph-mon[35502]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:48 smithi079 ceph-mon[35502]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:49.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 -- 172.21.15.19:0/3115399665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201026f0 msgr2=0x7fea20102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 --2- 172.21.15.19:0/3115399665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201026f0 0x7fea20102b00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fea10009990 tx=0x7fea10009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 -- 172.21.15.19:0/3115399665 shutdown_connections 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 --2- 172.21.15.19:0/3115399665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea2010a180 0x7fea2010c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 --2- 172.21.15.19:0/3115399665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201026f0 0x7fea20102b00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 --2- 172.21.15.19:0/3115399665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea20107820 0x7fea20109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 -- 172.21.15.19:0/3115399665 >> 172.21.15.19:0/3115399665 conn(0x7fea200fe010 msgr2=0x7fea201004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 -- 172.21.15.19:0/3115399665 shutdown_connections 2024-05-17T09:18:49.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.968+0000 7fea24dc5700 1 -- 172.21.15.19:0/3115399665 wait complete. 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 Processor -- start 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 -- start start 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea201026f0 0x7fea20104fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea20107820 0x7fea20103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 0x7fea20103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea2010fc40 con 0x7fea2010a180 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea2010fac0 con 0x7fea20107820 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.969+0000 7fea24dc5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea2010fdc0 con 0x7fea201026f0 2024-05-17T09:18:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 0x7fea20103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 0x7fea20103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42538/0 (socket says 172.21.15.19:42538) 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 -- 172.21.15.19:0/2276652806 learned_addr learned my addr 172.21.15.19:0/2276652806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea201026f0 0x7fea20104fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1dd9b700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea20107820 0x7fea20103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 -- 172.21.15.19:0/2276652806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea201026f0 msgr2=0x7fea20104fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea201026f0 0x7fea20104fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 -- 172.21.15.19:0/2276652806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea20107820 msgr2=0x7fea20103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea20107820 0x7fea20103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 -- 172.21.15.19:0/2276652806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea20104330 con 0x7fea2010a180 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1e59c700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea201026f0 0x7fea20104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea1ed9d700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 0x7fea20103bd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea1800ada0 tx=0x7fea1800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18017020 con 0x7fea2010a180 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.970+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201a4910 con 0x7fea2010a180 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.971+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201a4df0 con 0x7fea2010a180 2024-05-17T09:18:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.971+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea1800cd60 con 0x7fea2010a180 2024-05-17T09:18:49.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.971+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1801c9a0 con 0x7fea2010a180 2024-05-17T09:18:49.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.972+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea00005360 con 0x7fea2010a180 2024-05-17T09:18:49.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.974+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea1800a0d0 con 0x7fea2010a180 2024-05-17T09:18:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.975+0000 7fea0f7fe700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea080b2880 0x7fea080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.975+0000 7fea1e59c700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea080b2880 0x7fea080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.975+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fea180e1f30 con 0x7fea2010a180 2024-05-17T09:18:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.976+0000 7fea1e59c700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea080b2880 0x7fea080b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b2e0 tx=0x7fea1001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:49.979+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea18021020 con 0x7fea2010a180 2024-05-17T09:18:50.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.202+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fea00000bf0 con 0x7fea080b2880 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.216+0000 7fea0f7fe700 1 -- 172.21.15.19:0/2276652806 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fea00000bf0 con 0x7fea080b2880 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (4m) 101s ago 4m 82.1M - 19.0.0-3778-g3790c0fe 1a138b128aca 7b974af6d095 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi069 running (3m) 102s ago 3m 93.1M - 19.0.0-3778-g3790c0fe 1a138b128aca b5bad18d1aeb 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi079 running (4m) 102s ago 4m 85.7M - 19.0.0-3778-g3790c0fe 1a138b128aca 5fb9fd294d80 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (4m) 101s ago 4m 82.1M - 19.0.0-3778-g3790c0fe 1a138b128aca 9b7cbf1ff599 2024-05-17T09:18:50.215 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi069 running (4m) 102s ago 4m 86.9M - 19.0.0-3778-g3790c0fe 1a138b128aca d569de8c19bf 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi079 running (4m) 102s ago 4m 84.7M - 19.0.0-3778-g3790c0fe 1a138b128aca 75d43ba0bff5 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (4m) 101s ago 4m 82.2M - 19.0.0-3778-g3790c0fe 1a138b128aca 9153c36085c5 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi069 running (4m) 102s ago 4m 85.6M - 19.0.0-3778-g3790c0fe 1a138b128aca fcc5974a4e12 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi079 running (4m) 102s ago 4m 88.0M - 19.0.0-3778-g3790c0fe 1a138b128aca 8ed450be6521 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (4m) 101s ago 4m 82.0M - 19.0.0-3778-g3790c0fe 1a138b128aca 8c824afc6b5e 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi069 running (4m) 102s ago 4m 84.5M - 19.0.0-3778-g3790c0fe 1a138b128aca 113eecb51671 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi079 running (4m) 102s ago 4m 82.1M - 19.0.0-3778-g3790c0fe 1a138b128aca 04e9b62046ad 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (12m) 101s ago 12m 598M - 19.0.0-3778-g3790c0fe 1a138b128aca 02efc6d99c29 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi069 *:8443,8765 running (9m) 102s ago 9m 536M - 19.0.0-3778-g3790c0fe 1a138b128aca 843efbb5ae80 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi079 *:8443,8765 running (9m) 102s ago 9m 534M - 19.0.0-3778-g3790c0fe 1a138b128aca 76c49bc365b0 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (12m) 101s ago 12m 118M 2048M 19.0.0-3778-g3790c0fe 1a138b128aca 05794409a601 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi069 running (10m) 102s ago 10m 110M 2048M 19.0.0-3778-g3790c0fe 1a138b128aca c59ff213553a 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi079 running (10m) 102s ago 10m 108M 2048M 19.0.0-3778-g3790c0fe 1a138b128aca feca069ebc75 2024-05-17T09:18:50.216 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (9m) 101s ago 9m 112M 4273M 19.0.0-3778-g3790c0fe 1a138b128aca 30ddb289d49e 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi069 running (9m) 102s ago 9m 113M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 07e7050c4530 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi079 running (8m) 102s ago 8m 115M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 5b3221df2e58 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (8m) 101s ago 8m 111M 4273M 19.0.0-3778-g3790c0fe 1a138b128aca a5ee99d07711 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi069 running (7m) 102s ago 7m 111M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 05d855392ef1 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi079 running (7m) 102s ago 7m 122M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 9c70af9ae649 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (7m) 101s ago 7m 111M 4273M 19.0.0-3778-g3790c0fe 1a138b128aca 5dd2c9ec430f 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi069 running (6m) 102s ago 6m 111M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 22712f785ab1 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi079 running (6m) 102s ago 6m 110M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 7e4ec2b5b519 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (5m) 101s ago 5m 120M 4273M 19.0.0-3778-g3790c0fe 1a138b128aca 48370521cbe2 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi069 running (5m) 102s ago 5m 111M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca ece9aa6fc9a4 2024-05-17T09:18:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi079 running (4m) 102s ago 4m 121M 4272M 19.0.0-3778-g3790c0fe 1a138b128aca 53f06deaa3f9 2024-05-17T09:18:50.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea080b2880 msgr2=0x7fea080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:50.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea080b2880 0x7fea080b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b2e0 tx=0x7fea1001a040 comp rx=0 tx=0).stop 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 msgr2=0x7fea20103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 0x7fea20103bd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea1800ada0 tx=0x7fea1800c330 comp rx=0 tx=0).stop 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 shutdown_connections 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea080b2880 0x7fea080b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea201026f0 0x7fea20104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2010a180 0x7fea20103bd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 --2- 172.21.15.19:0/2276652806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea20107820 0x7fea20103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 >> 172.21.15.19:0/2276652806 conn(0x7fea200fe010 msgr2=0x7fea20108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.218+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 shutdown_connections 2024-05-17T09:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:50.219+0000 7fea24dc5700 1 -- 172.21.15.19:0/2276652806 wait complete. 2024-05-17T09:18:50.525 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:50 smithi019 ceph-mon[31620]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:50 smithi069 ceph-mon[35081]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:50 smithi079 ceph-mon[35502]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:18:50.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-17T09:18:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:51.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:51 smithi019 ceph-mon[31620]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:51.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:51 smithi069 ceph-mon[35081]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:51.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:51 smithi079 ceph-mon[35502]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:52.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:52 smithi019 ceph-mon[31620]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:52 smithi069 ceph-mon[35081]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:52 smithi079 ceph-mon[35502]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 -- 172.21.15.19:0/2738958435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c1049d0 msgr2=0x7f5d4c10c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 --2- 172.21.15.19:0/2738958435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c1049d0 0x7f5d4c10c6e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400b0a0 tx=0x7f5d4400b3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 -- 172.21.15.19:0/2738958435 shutdown_connections 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 --2- 172.21.15.19:0/2738958435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d4c1059a0 0x7f5d4c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 --2- 172.21.15.19:0/2738958435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c1049d0 0x7f5d4c10c6e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 --2- 172.21.15.19:0/2738958435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d4c103f50 0x7f5d4c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 -- 172.21.15.19:0/2738958435 >> 172.21.15.19:0/2738958435 conn(0x7f5d4c0ff9c0 msgr2=0x7f5d4c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 -- 172.21.15.19:0/2738958435 shutdown_connections 2024-05-17T09:18:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.994+0000 7f5d54405700 1 -- 172.21.15.19:0/2738958435 wait complete. 2024-05-17T09:18:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 Processor -- start 2024-05-17T09:18:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 -- start start 2024-05-17T09:18:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 0x7f5d4c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d4c1049d0 0x7f5d4c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d4c1059a0 0x7f5d4c1a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10fd90 con 0x7f5d4c103f50 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10fc10 con 0x7f5d4c1059a0 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.995+0000 7f5d54405700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10ff10 con 0x7f5d4c1049d0 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 0x7f5d4c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 0x7f5d4c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42546/0 (socket says 172.21.15.19:42546) 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 -- 172.21.15.19:0/908168989 learned_addr learned my addr 172.21.15.19:0/908168989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d529a2700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d4c1059a0 0x7f5d4c1a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d519a0700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d4c1049d0 0x7f5d4c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 -- 172.21.15.19:0/908168989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d4c1049d0 msgr2=0x7f5d4c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d4c1049d0 0x7f5d4c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 -- 172.21.15.19:0/908168989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d4c1059a0 msgr2=0x7f5d4c1a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d4c1059a0 0x7f5d4c1a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 -- 172.21.15.19:0/908168989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c1a4640 con 0x7f5d4c103f50 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d521a1700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 0x7f5d4c19cae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d950 tx=0x7f5d4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48017070 con 0x7f5d4c103f50 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d48009730 con 0x7f5d4c103f50 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48003a00 con 0x7f5d4c103f50 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.996+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c1a4930 con 0x7f5d4c103f50 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.997+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1a4e70 con 0x7f5d4c103f50 2024-05-17T09:18:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.997+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c0512b0 con 0x7f5d4c103f50 2024-05-17T09:18:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:52.999+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d48003ba0 con 0x7f5d4c103f50 2024-05-17T09:18:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.000+0000 7f5d437fe700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d380b2880 0x7f5d380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.000+0000 7f5d519a0700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d380b2880 0x7f5d380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.000+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5d480e2830 con 0x7f5d4c103f50 2024-05-17T09:18:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.000+0000 7f5d519a0700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d380b2880 0x7f5d380b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c19db90 tx=0x7f5d3c00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:53.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.003+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d48020030 con 0x7f5d4c103f50 2024-05-17T09:18:53.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.231+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5d4c078210 con 0x7f5d380b2880 2024-05-17T09:18:53.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.237+0000 7f5d437fe700 1 -- 172.21.15.19:0/908168989 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5d4c078210 con 0x7f5d380b2880 2024-05-17T09:18:53.261 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-17T09:18:53.261 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 105s ago 4m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-17T09:18:53.261 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 105s ago 10m smithi019=x;smithi069=y;smithi079=z;count:3 2024-05-17T09:18:53.261 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 105s ago 10m smithi019:172.21.15.19=a;smithi069:172.21.15.69=b;smithi079:172.21.15.79=c;count:3 2024-05-17T09:18:53.261 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 105s ago - 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d380b2880 msgr2=0x7f5d380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d380b2880 0x7f5d380b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c19db90 tx=0x7f5d3c00b5d0 comp rx=0 tx=0).stop 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 msgr2=0x7f5d4c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 0x7f5d4c19cae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d950 tx=0x7f5d4800dc60 comp rx=0 tx=0).stop 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 shutdown_connections 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d4c1059a0 0x7f5d4c1a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4c103f50 0x7f5d4c19cae0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5d380b2880 0x7f5d380b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:53.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 --2- 172.21.15.19:0/908168989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d4c1049d0 0x7f5d4c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:53.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 >> 172.21.15.19:0/908168989 conn(0x7f5d4c0ff9c0 msgr2=0x7f5d4c076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:53.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 shutdown_connections 2024-05-17T09:18:53.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:53.239+0000 7f5d54405700 1 -- 172.21.15.19:0/908168989 wait complete. 2024-05-17T09:18:53.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-17T09:18:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:54.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:54 smithi019 ceph-mon[31620]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:54.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:54 smithi019 ceph-mon[31620]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:54 smithi069 ceph-mon[35081]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:54 smithi069 ceph-mon[35081]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:54 smithi079 ceph-mon[35502]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:18:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:54 smithi079 ceph-mon[35502]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 -- 172.21.15.19:0/3667494206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 msgr2=0x7f570c103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 --2- 172.21.15.19:0/3667494206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c103970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009990 tx=0x7f56f4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 -- 172.21.15.19:0/3667494206 shutdown_connections 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 --2- 172.21.15.19:0/3667494206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f570c102c10 0x7f570c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 --2- 172.21.15.19:0/3667494206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570c103eb0 0x7f570c1042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 --2- 172.21.15.19:0/3667494206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c103970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 -- 172.21.15.19:0/3667494206 >> 172.21.15.19:0/3667494206 conn(0x7f570c0fe860 msgr2=0x7f570c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 -- 172.21.15.19:0/3667494206 shutdown_connections 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.768+0000 7f57122dc700 1 -- 172.21.15.19:0/3667494206 wait complete. 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 Processor -- start 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 -- start start 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f570c102c10 0x7f570c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570c103eb0 0x7f570c1a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f570c111e30 con 0x7f570c103560 2024-05-17T09:18:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f570c111cb0 con 0x7f570c103eb0 2024-05-17T09:18:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f57122dc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f570c111fb0 con 0x7f570c102c10 2024-05-17T09:18:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f5703fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f5703fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39314/0 (socket says 172.21.15.19:39314) 2024-05-17T09:18:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f5703fff700 1 -- 172.21.15.19:0/4045451841 learned_addr learned my addr 172.21.15.19:0/4045451841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f5710879700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570c103eb0 0x7f570c1a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.769+0000 7f570bfff700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f570c102c10 0x7f570c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5703fff700 1 -- 172.21.15.19:0/4045451841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f570c102c10 msgr2=0x7f570c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5703fff700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f570c102c10 0x7f570c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5703fff700 1 -- 172.21.15.19:0/4045451841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570c103eb0 msgr2=0x7f570c1a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5703fff700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570c103eb0 0x7f570c1a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5703fff700 1 -- 172.21.15.19:0/4045451841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c1a4540 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5703fff700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c19cf20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f56f40098e0 tx=0x7f56f40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4003c10 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570c1a47d0 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56f4004830 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4005350 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.770+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570c1a4c20 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.771+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570c0512b0 con 0x7f570c103560 2024-05-17T09:18:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.773+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f56f40054f0 con 0x7f570c103560 2024-05-17T09:18:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.774+0000 7f5709ffb700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f56f80b2880 0x7f56f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.774+0000 7f570bfff700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f56f80b2880 0x7f56f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.774+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f56f40e3b10 con 0x7f570c103560 2024-05-17T09:18:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.775+0000 7f570bfff700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f56f80b2880 0x7f56f80b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f570c06ca50 tx=0x7f56fc00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:55.778+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56f4013070 con 0x7f570c103560 2024-05-17T09:18:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.007+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f570c063bf0 con 0x7f56f80b2880 2024-05-17T09:18:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.008+0000 7f5709ffb700 1 -- 172.21.15.19:0/4045451841 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f570c063bf0 con 0x7f56f80b2880 2024-05-17T09:18:56.008 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-05-17T09:18:56.008 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-05-17T09:18:56.008 INFO:teuthology.orchestra.run.smithi019.stdout:smithi069 172.21.15.69 2024-05-17T09:18:56.008 INFO:teuthology.orchestra.run.smithi019.stdout:smithi079 172.21.15.79 2024-05-17T09:18:56.008 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-05-17T09:18:56.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f56f80b2880 msgr2=0x7f56f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:56.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f56f80b2880 0x7f56f80b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f570c06ca50 tx=0x7f56fc00b5b0 comp rx=0 tx=0).stop 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 msgr2=0x7f570c19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c19cf20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f56f40098e0 tx=0x7f56f40047b0 comp rx=0 tx=0).stop 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 shutdown_connections 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f56f80b2880 0x7f56f80b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f570c102c10 0x7f570c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570c103eb0 0x7f570c1a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 --2- 172.21.15.19:0/4045451841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c103560 0x7f570c19cf20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 >> 172.21.15.19:0/4045451841 conn(0x7f570c0fe860 msgr2=0x7f570c10b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 shutdown_connections 2024-05-17T09:18:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:56.011+0000 7f57122dc700 1 -- 172.21.15.19:0/4045451841 wait complete. 2024-05-17T09:18:56.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:56 smithi019 ceph-mon[31620]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:56 smithi069 ceph-mon[35081]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:56.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-17T09:18:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:56 smithi079 ceph-mon[35502]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:18:56.958 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:18:57.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:57 smithi019 ceph-mon[31620]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:57.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:57 smithi069 ceph-mon[35081]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:57.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:57 smithi079 ceph-mon[35502]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:18:58.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:18:58 smithi019 ceph-mon[31620]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:18:58 smithi069 ceph-mon[35081]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:58.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 -- 172.21.15.19:0/486314424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8104d00 msgr2=0x7f2be810b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:58.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/486314424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8104d00 0x7f2be810b5b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b0a0 tx=0x7f2be000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 -- 172.21.15.19:0/486314424 shutdown_connections 2024-05-17T09:18:58.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/486314424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8104d00 0x7f2be810b5b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/486314424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be8103230 0x7f2be8103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/486314424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8104360 0x7f2be8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 -- 172.21.15.19:0/486314424 >> 172.21.15.19:0/486314424 conn(0x7f2be80fe870 msgr2=0x7f2be8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:58.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 -- 172.21.15.19:0/486314424 shutdown_connections 2024-05-17T09:18:58.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.694+0000 7f2bf06a0700 1 -- 172.21.15.19:0/486314424 wait complete. 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 Processor -- start 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 -- start start 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 0x7f2be819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8104360 0x7f2be819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be8104d00 0x7f2be81a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810eb40 con 0x7f2be8103230 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e9c0 con 0x7f2be8104d00 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bf06a0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810ecc0 con 0x7f2be8104360 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 0x7f2be819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 0x7f2be819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39342/0 (socket says 172.21.15.19:39342) 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 -- 172.21.15.19:0/2698496575 learned_addr learned my addr 172.21.15.19:0/2698496575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2beec3d700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be8104d00 0x7f2be81a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bedc3b700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8104360 0x7f2be819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 -- 172.21.15.19:0/2698496575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8104360 msgr2=0x7f2be819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8104360 0x7f2be819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 -- 172.21.15.19:0/2698496575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be8104d00 msgr2=0x7f2be81a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be8104d00 0x7f2be81a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.695+0000 7f2bee43c700 1 -- 172.21.15.19:0/2698496575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81a2330 con 0x7f2be8103230 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.696+0000 7f2bee43c700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 0x7f2be819a7d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e940 tx=0x7f2be400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.696+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be400cbd0 con 0x7f2be8103230 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.696+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be400ee90 con 0x7f2be8103230 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.696+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4003d00 con 0x7f2be8103230 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.696+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be81a25c0 con 0x7f2be8103230 2024-05-17T09:18:58.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.696+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a2ab0 con 0x7f2be8103230 2024-05-17T09:18:58.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.697+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be80512b0 con 0x7f2be8103230 2024-05-17T09:18:58.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.700+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2be4003ea0 con 0x7f2be8103230 2024-05-17T09:18:58.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.700+0000 7f2bdb7fe700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2bd40b2880 0x7f2bd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:18:58.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.701+0000 7f2bedc3b700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2bd40b2880 0x7f2bd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:18:58.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.701+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2be40e2910 con 0x7f2be8103230 2024-05-17T09:18:58.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.701+0000 7f2bedc3b700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2bd40b2880 0x7f2bd40b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005950 tx=0x7f2bdc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:18:58.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.704+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2be40aa310 con 0x7f2be8103230 2024-05-17T09:18:58.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:18:58 smithi079 ceph-mon[35502]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:18:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.929+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2be81a2d70 con 0x7f2bd40b2880 2024-05-17T09:18:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.931+0000 7f2bdb7fe700 1 -- 172.21.15.19:0/2698496575 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2be81a2d70 con 0x7f2bd40b2880 2024-05-17T09:18:58.930 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-17T09:18:58.931 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-17T09:18:58.931 INFO:teuthology.orchestra.run.smithi019.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-17T09:18:58.931 INFO:teuthology.orchestra.run.smithi019.stdout:smithi079 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013G400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-17T09:18:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2bd40b2880 msgr2=0x7f2bd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2bd40b2880 0x7f2bd40b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005950 tx=0x7f2bdc0058e0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 msgr2=0x7f2be819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 0x7f2be819a7d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e940 tx=0x7f2be400ec50 comp rx=0 tx=0).stop 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 shutdown_connections 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be8104d00 0x7f2be81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8103230 0x7f2be819a7d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2bd40b2880 0x7f2bd40b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 --2- 172.21.15.19:0/2698496575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8104360 0x7f2be819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 >> 172.21.15.19:0/2698496575 conn(0x7f2be80fe870 msgr2=0x7f2be81001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 shutdown_connections 2024-05-17T09:18:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:18:58.934+0000 7f2bf06a0700 1 -- 172.21.15.19:0/2698496575 wait complete. 2024-05-17T09:18:59.592 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T09:18:59.606 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:18:59.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-17T09:18:59.865 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:00.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:00 smithi079 ceph-mon[35502]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:19:00.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:00 smithi079 ceph-mon[35502]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:00 smithi069 ceph-mon[35081]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:19:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:00 smithi069 ceph-mon[35081]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:00 smithi019 ceph-mon[31620]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T09:19:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:00 smithi019 ceph-mon[31620]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 -- 172.21.15.19:0/2325579928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24104350 msgr2=0x7fac24104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 --2- 172.21.15.19:0/2325579928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24104350 0x7fac24104760 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fac14009990 tx=0x7fac14009ca0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 -- 172.21.15.19:0/2325579928 shutdown_connections 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 --2- 172.21.15.19:0/2325579928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104cf0 0x7fac2410b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 --2- 172.21.15.19:0/2325579928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac24103220 0x7fac24103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 --2- 172.21.15.19:0/2325579928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24104350 0x7fac24104760 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 -- 172.21.15.19:0/2325579928 >> 172.21.15.19:0/2325579928 conn(0x7fac240fe860 msgr2=0x7fac24100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 -- 172.21.15.19:0/2325579928 shutdown_connections 2024-05-17T09:19:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.608+0000 7fac2b026700 1 -- 172.21.15.19:0/2325579928 wait complete. 2024-05-17T09:19:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 Processor -- start 2024-05-17T09:19:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 -- start start 2024-05-17T09:19:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 0x7fac24106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104350 0x7fac24105690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac24104cf0 0x7fac24105bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410eb30 con 0x7fac24103220 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410e9b0 con 0x7fac24104cf0 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.609+0000 7fac2b026700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410ecb0 con 0x7fac24104350 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac28dc2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 0x7fac24106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac28dc2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 0x7fac24106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39356/0 (socket says 172.21.15.19:39356) 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac28dc2700 1 -- 172.21.15.19:0/1020234330 learned_addr learned my addr 172.21.15.19:0/1020234330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104350 0x7fac24105690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac295c3700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac24104cf0 0x7fac24105bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 -- 172.21.15.19:0/1020234330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac24104cf0 msgr2=0x7fac24105bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac24104cf0 0x7fac24105bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 -- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 msgr2=0x7fac24106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 0x7fac24106fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 -- 172.21.15.19:0/1020234330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac241062a0 con 0x7fac24104350 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac28dc2700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 0x7fac24106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.610+0000 7fac1bfff700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104350 0x7fac24105690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac14000c00 tx=0x7fac140057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.611+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14027070 con 0x7fac24104350 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.611+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac241a4940 con 0x7fac24104350 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.611+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac14004020 con 0x7fac24104350 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.611+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14022c10 con 0x7fac24104350 2024-05-17T09:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.611+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac241a4ce0 con 0x7fac24104350 2024-05-17T09:19:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.612+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac240512b0 con 0x7fac24104350 2024-05-17T09:19:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.614+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fac1402a070 con 0x7fac24104350 2024-05-17T09:19:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.614+0000 7fac19ffb700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fac0c0b2880 0x7fac0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:01.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.615+0000 7fac28dc2700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fac0c0b2880 0x7fac0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:01.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.615+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fac140e3e20 con 0x7fac24104350 2024-05-17T09:19:01.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.615+0000 7fac28dc2700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fac0c0b2880 0x7fac0c0b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fac2000afd0 tx=0x7fac2000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:01.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.618+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac14031080 con 0x7fac24104350 2024-05-17T09:19:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.879+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fac241a5070 con 0x7fac24104350 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.879+0000 7fac19ffb700 1 -- 172.21.15.19:0/1020234330 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3038 (secure 0 0 0) 0x7fac241a5070 con 0x7fac24104350 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stdout:e26 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:01.879 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-17T09:15:09.950009+0000 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-17T09:16:22.157327+0000 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-05-17T09:19:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 3 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24497,1=24487,2=24470} 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3,4] 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-05-17T09:19:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24497 members: 24487,24497,24470 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24497} state up:active seq 6 addr [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{0:24469} state up:standby-replay seq 1 addr [v2:172.21.15.79:6836/720193094,v1:172.21.15.79:6837/720193094] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{1:24487} state up:active seq 25 addr [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{1:24463} state up:standby-replay seq 1 addr [v2:172.21.15.79:6834/1132484996,v1:172.21.15.79:6835/1132484996] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{2:24470} state up:active seq 26 addr [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{2:24449} state up:standby-replay seq 1 addr [v2:172.21.15.69:6834/437548660,v1:172.21.15.69:6835/437548660] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:01.882 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.19:6834/4138238464,v1:172.21.15.19:6835/4138238464] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.883 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.19:6836/2308476934,v1:172.21.15.19:6837/2308476934] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.883 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{-1:14577} state up:standby seq 1 addr [v2:172.21.15.19:6838/1964664313,v1:172.21.15.19:6839/1964664313] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.883 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{-1:14589} state up:standby seq 1 addr [v2:172.21.15.19:6840/2849498318,v1:172.21.15.19:6841/2849498318] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.883 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{-1:24424} state up:standby seq 1 addr [v2:172.21.15.79:6832/1681609094,v1:172.21.15.79:6833/1681609094] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.883 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.69:6832/700922121,v1:172.21.15.69:6833/700922121] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fac0c0b2880 msgr2=0x7fac0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fac0c0b2880 0x7fac0c0b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fac2000afd0 tx=0x7fac2000a380 comp rx=0 tx=0).stop 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104350 msgr2=0x7fac24105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104350 0x7fac24105690 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac14000c00 tx=0x7fac140057c0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 shutdown_connections 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac24104cf0 0x7fac24105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac24103220 0x7fac24106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fac0c0b2880 0x7fac0c0b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 --2- 172.21.15.19:0/1020234330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac24104350 0x7fac24105690 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 >> 172.21.15.19:0/1020234330 conn(0x7fac240fe860 msgr2=0x7fac24100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:01.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 shutdown_connections 2024-05-17T09:19:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:01.882+0000 7fac2b026700 1 -- 172.21.15.19:0/1020234330 wait complete. 2024-05-17T09:19:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:19:02.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:02 smithi019 ceph-mon[31620]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:02.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1020234330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T09:19:02.618 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-17T09:19:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:02 smithi069 ceph-mon[35081]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1020234330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T09:19:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:02 smithi079 ceph-mon[35502]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1020234330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T09:19:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 -- 172.21.15.19:0/3603632796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788104370 msgr2=0x7ff788104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 --2- 172.21.15.19:0/3603632796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788104370 0x7ff788104780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff778009990 tx=0x7ff778009ca0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 -- 172.21.15.19:0/3603632796 shutdown_connections 2024-05-17T09:19:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 --2- 172.21.15.19:0/3603632796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788103240 0x7ff788103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 --2- 172.21.15.19:0/3603632796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff788104d10 0x7ff78810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 --2- 172.21.15.19:0/3603632796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788104370 0x7ff788104780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 -- 172.21.15.19:0/3603632796 >> 172.21.15.19:0/3603632796 conn(0x7ff7880fe880 msgr2=0x7ff788100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:04.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:04 smithi019 ceph-mon[31620]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 -- 172.21.15.19:0/3603632796 shutdown_connections 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.588+0000 7ff7900ce700 1 -- 172.21.15.19:0/3603632796 wait complete. 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 Processor -- start 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 -- start start 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 0x7ff788076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff788104370 0x7ff78819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788104d10 0x7ff7881a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff78810ebd0 con 0x7ff788103240 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff78810ea50 con 0x7ff788104370 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff7900ce700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff78810ed50 con 0x7ff788104d10 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff78de6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 0x7ff788076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff78de6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 0x7ff788076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41026/0 (socket says 172.21.15.19:41026) 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.589+0000 7ff78de6a700 1 -- 172.21.15.19:0/1100309551 learned_addr learned my addr 172.21.15.19:0/1100309551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78d669700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff788104370 0x7ff78819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78e66b700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788104d10 0x7ff7881a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78de6a700 1 -- 172.21.15.19:0/1100309551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788104d10 msgr2=0x7ff7881a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78de6a700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788104d10 0x7ff7881a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78de6a700 1 -- 172.21.15.19:0/1100309551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff788104370 msgr2=0x7ff78819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78de6a700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff788104370 0x7ff78819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78de6a700 1 -- 172.21.15.19:0/1100309551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7881a23f0 con 0x7ff788103240 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78e66b700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788104d10 0x7ff7881a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff78de6a700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 0x7ff788076450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff78400d950 tx=0x7ff78400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784017070 con 0x7ff788103240 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7881a26e0 con 0x7ff788103240 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff784009730 con 0x7ff788103240 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784003a00 con 0x7ff788103240 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.590+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7881a2ca0 con 0x7ff788103240 2024-05-17T09:19:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.591+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7880512b0 con 0x7ff788103240 2024-05-17T09:19:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.593+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff784003ba0 con 0x7ff788103240 2024-05-17T09:19:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.594+0000 7ff77effd700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7740b2880 0x7ff7740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.594+0000 7ff78d669700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7740b2880 0x7ff7740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.594+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff7840e28d0 con 0x7ff788103240 2024-05-17T09:19:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.595+0000 7ff78d669700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7740b2880 0x7ff7740b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff778006010 tx=0x7ff77801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.598+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff784020030 con 0x7ff788103240 2024-05-17T09:19:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:04 smithi069 ceph-mon[35081]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:04 smithi079 ceph-mon[35502]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:04.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.813+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff78802e5e0 con 0x7ff788103240 2024-05-17T09:19:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.814+0000 7ff77effd700 1 -- 172.21.15.19:0/1100309551 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5655 (secure 0 0 0) 0x7ff7840aa240 con 0x7ff788103240 2024-05-17T09:19:04.813 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 89 2024-05-17T09:19:04.813 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:19:04.813 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-17T09:06:43.021173+0000 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-17T09:16:48.308262+0000 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 27 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-17T09:19:04.814 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/3945774453,v1:172.21.15.19:6803/3945774453] [v2:172.21.15.19:6804/3945774453,v1:172.21.15.19:6805/3945774453] exists,up 7de60f57-fe91-4722-8fac-00c122296aac 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6800/3851861092,v1:172.21.15.69:6801/3851861092] [v2:172.21.15.69:6802/3851861092,v1:172.21.15.69:6803/3851861092] exists,up 840241e0-3ae2-4033-8bbb-01a3935c3123 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/3984657664,v1:172.21.15.79:6801/3984657664] [v2:172.21.15.79:6802/3984657664,v1:172.21.15.79:6803/3984657664] exists,up afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/494922873,v1:172.21.15.19:6811/494922873] [v2:172.21.15.19:6812/494922873,v1:172.21.15.19:6813/494922873] exists,up 774aea07-241e-4f94-aa94-5e8823cb5b48 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6808/3507791356,v1:172.21.15.69:6809/3507791356] [v2:172.21.15.69:6810/3507791356,v1:172.21.15.69:6811/3507791356] exists,up 8760fee7-c08a-4a4e-bc91-754119cb9d3c 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/3720398663,v1:172.21.15.79:6809/3720398663] [v2:172.21.15.79:6810/3720398663,v1:172.21.15.79:6811/3720398663] exists,up 7609df91-0a4b-4f8f-9a41-9d644e9a3ee3 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 42 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/1984910928,v1:172.21.15.19:6819/1984910928] [v2:172.21.15.19:6820/1984910928,v1:172.21.15.19:6821/1984910928] exists,up 37a6a6d7-3e20-4503-afce-96f03af0e774 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6816/2368256164,v1:172.21.15.69:6817/2368256164] [v2:172.21.15.69:6818/2368256164,v1:172.21.15.69:6819/2368256164] exists,up 829c24b4-70f4-4d39-8126-1766fe7920b4 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/2695186498,v1:172.21.15.79:6817/2695186498] [v2:172.21.15.79:6818/2695186498,v1:172.21.15.79:6819/2695186498] exists,up 106603bf-b7f4-438c-abc4-b660029e1128 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 59 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/2798428087,v1:172.21.15.19:6827/2798428087] [v2:172.21.15.19:6828/2798428087,v1:172.21.15.19:6829/2798428087] exists,up e132748b-b8b9-4b25-b233-d786e59d9330 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6824/3630021153,v1:172.21.15.69:6825/3630021153] [v2:172.21.15.69:6826/3630021153,v1:172.21.15.69:6827/3630021153] exists,up 9a14a50a-2686-4e06-9473-0cf82a95e252 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 70 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6824/2698518199,v1:172.21.15.79:6825/2698518199] [v2:172.21.15.79:6826/2698518199,v1:172.21.15.79:6827/2698518199] exists,up 2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68 2024-05-17T09:19:04.815 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.0 [8,1] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.f [8,11] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.13 [2,11] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1d [2,4] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1 [4,9] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.12 [3,9] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1b [4,6] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.25 [5,9] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.31 [4,8] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.5 [5,10] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.12 [9,10] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.17 [9,8] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.1f [9,8] 2024-05-17T09:19:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/21803007 expires 2024-05-18T09:07:46.771034+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2577177111 expires 2024-05-18T09:07:46.771034+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3426431385 expires 2024-05-18T09:07:46.771034+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/4126868109 expires 2024-05-18T09:07:20.018567+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3686497433 expires 2024-05-18T09:07:20.018567+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/747673746 expires 2024-05-18T09:07:20.018567+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3120865986 expires 2024-05-18T09:07:46.771034+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/747673746 expires 2024-05-18T09:07:20.018567+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2577177111 expires 2024-05-18T09:07:46.771034+0000 2024-05-17T09:19:04.817 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3378222227 expires 2024-05-18T09:07:20.018567+0000 2024-05-17T09:19:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7740b2880 msgr2=0x7ff7740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7740b2880 0x7ff7740b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff778006010 tx=0x7ff77801a040 comp rx=0 tx=0).stop 2024-05-17T09:19:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 msgr2=0x7ff788076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 0x7ff788076450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff78400d950 tx=0x7ff78400dc60 comp rx=0 tx=0).stop 2024-05-17T09:19:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 shutdown_connections 2024-05-17T09:19:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff7740b2880 0x7ff7740b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff788104d10 0x7ff7881a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff788103240 0x7ff788076450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 --2- 172.21.15.19:0/1100309551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff788104370 0x7ff78819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:04.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 >> 172.21.15.19:0/1100309551 conn(0x7ff7880fe880 msgr2=0x7ff7881075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:04.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 shutdown_connections 2024-05-17T09:19:04.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:04.817+0000 7ff7900ce700 1 -- 172.21.15.19:0/1100309551 wait complete. 2024-05-17T09:19:05.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:05 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1100309551' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T09:19:05.617 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-17T09:19:05.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:19:05.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:05 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1100309551' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T09:19:05.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:05 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1100309551' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T09:19:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:06 smithi069 ceph-mon[35081]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:06.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:06 smithi019 ceph-mon[31620]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:06 smithi079 ceph-mon[35502]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 -- 172.21.15.19:0/2247509812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 msgr2=0x7f76901100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2247509812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f76901100b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f768800b0d0 tx=0x7f768800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 -- 172.21.15.19:0/2247509812 shutdown_connections 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2247509812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f76901100b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2247509812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76901034d0 0x7f76901038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2247509812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690107e00 0x7f7690108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 -- 172.21.15.19:0/2247509812 >> 172.21.15.19:0/2247509812 conn(0x7f76900fef20 msgr2=0x7f7690101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 -- 172.21.15.19:0/2247509812 shutdown_connections 2024-05-17T09:19:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.582+0000 7f7694a8e700 1 -- 172.21.15.19:0/2247509812 wait complete. 2024-05-17T09:19:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 Processor -- start 2024-05-17T09:19:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 -- start start 2024-05-17T09:19:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76901034d0 0x7f76901a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7690107e00 0x7f76901a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f769019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7690114380 con 0x7f7690108750 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7690114200 con 0x7f7690107e00 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f7694a8e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7690114500 con 0x7f76901034d0 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f768ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f769019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f768ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f769019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41054/0 (socket says 172.21.15.19:41054) 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f768ed9d700 1 -- 172.21.15.19:0/2360356459 learned_addr learned my addr 172.21.15.19:0/2360356459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.583+0000 7f768dd9b700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7690107e00 0x7f76901a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768e59c700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76901034d0 0x7f76901a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768ed9d700 1 -- 172.21.15.19:0/2360356459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76901034d0 msgr2=0x7f76901a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768ed9d700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76901034d0 0x7f76901a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768ed9d700 1 -- 172.21.15.19:0/2360356459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7690107e00 msgr2=0x7f76901a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768ed9d700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7690107e00 0x7f76901a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768ed9d700 1 -- 172.21.15.19:0/2360356459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769019b100 con 0x7f7690108750 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768dd9b700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7690107e00 0x7f76901a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f768ed9d700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f769019aac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7688007d90 tx=0x7f7688007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768801f030 con 0x7f7690108750 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f768801bad0 con 0x7f7690108750 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769019b390 con 0x7f7690108750 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688012b70 con 0x7f7690108750 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.584+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769019b8d0 con 0x7f7690108750 2024-05-17T09:19:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.585+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76900512b0 con 0x7f7690108750 2024-05-17T09:19:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.588+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7688012d10 con 0x7f7690108750 2024-05-17T09:19:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.589+0000 7f767f7fe700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f76780b2880 0x7f76780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.589+0000 7f768e59c700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f76780b2880 0x7f76780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.589+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f768800f020 con 0x7f7690108750 2024-05-17T09:19:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.589+0000 7f768e59c700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f76780b2880 0x7f76780b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f769019bc00 tx=0x7f768000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.592+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7688029020 con 0x7f7690108750 2024-05-17T09:19:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.852+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f769002e5e0 con 0x7f7690108750 2024-05-17T09:19:07.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.853+0000 7f767f7fe700 1 -- 172.21.15.19:0/2360356459 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f76880ab950 con 0x7f7690108750 2024-05-17T09:19:07.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:07.853 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:16:22.157327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:19:07.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f76780b2880 msgr2=0x7f76780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:07.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f76780b2880 0x7f76780b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f769019bc00 tx=0x7f768000a380 comp rx=0 tx=0).stop 2024-05-17T09:19:07.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 msgr2=0x7f769019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:07.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f769019aac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7688007d90 tx=0x7f7688007dc0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 shutdown_connections 2024-05-17T09:19:07.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f76780b2880 0x7f76780b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76901034d0 0x7f76901a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7690108750 0x7f769019aac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 --2- 172.21.15.19:0/2360356459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7690107e00 0x7f76901a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 >> 172.21.15.19:0/2360356459 conn(0x7f76900fef20 msgr2=0x7f7690107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 shutdown_connections 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:07.855+0000 7f7694a8e700 1 -- 172.21.15.19:0/2360356459 wait complete. 2024-05-17T09:19:07.856 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:19:08.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:08 smithi019 ceph-mon[31620]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:08.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:08 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2360356459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:08.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:19:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:08 smithi069 ceph-mon[35081]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:08 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2360356459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:08 smithi079 ceph-mon[35502]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:08.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:08 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2360356459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:10.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 -- 172.21.15.19:0/2341688213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c108010 msgr2=0x7f388c108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:10.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 --2- 172.21.15.19:0/2341688213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c108010 0x7f388c108420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f387c0099c0 tx=0x7f387c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 -- 172.21.15.19:0/2341688213 shutdown_connections 2024-05-17T09:19:10.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 --2- 172.21.15.19:0/2341688213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388c108960 0x7f388c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 --2- 172.21.15.19:0/2341688213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c103c00 0x7f388c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 --2- 172.21.15.19:0/2341688213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c108010 0x7f388c108420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 -- 172.21.15.19:0/2341688213 >> 172.21.15.19:0/2341688213 conn(0x7f388c0fe530 msgr2=0x7f388c100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 -- 172.21.15.19:0/2341688213 shutdown_connections 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.525+0000 7f3893ab6700 1 -- 172.21.15.19:0/2341688213 wait complete. 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 Processor -- start 2024-05-17T09:19:10.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 -- start start 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 0x7f388c1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c108010 0x7f388c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388c108960 0x7f388c1a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113420 con 0x7f388c103c00 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1132a0 con 0x7f388c108010 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3893ab6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1135a0 con 0x7f388c108960 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.526+0000 7f3891852700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 0x7f388c1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 0x7f388c1a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41076/0 (socket says 172.21.15.19:41076) 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 -- 172.21.15.19:0/708877401 learned_addr learned my addr 172.21.15.19:0/708877401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3892053700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388c108960 0x7f388c1a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891051700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c108010 0x7f388c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:10.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 -- 172.21.15.19:0/708877401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388c108960 msgr2=0x7f388c1a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388c108960 0x7f388c1a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 -- 172.21.15.19:0/708877401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c108010 msgr2=0x7f388c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c108010 0x7f388c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 -- 172.21.15.19:0/708877401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c1a6410 con 0x7f388c103c00 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3891852700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 0x7f388c1a1540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f388800d950 tx=0x7f388800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888016070 con 0x7f388c103c00 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f388800fd20 con 0x7f388c103c00 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388800e3b0 con 0x7f388c103c00 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1a6700 con 0x7f388c103c00 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.527+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1ae020 con 0x7f388c103c00 2024-05-17T09:19:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.528+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c0512b0 con 0x7f388c103c00 2024-05-17T09:19:10.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.530+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3888019070 con 0x7f388c103c00 2024-05-17T09:19:10.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:10 smithi019 ceph-mon[31620]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:10.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.531+0000 7f3882ffd700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38780b2880 0x7f38780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.531+0000 7f3891051700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38780b2880 0x7f38780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.531+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f38880e2090 con 0x7f388c103c00 2024-05-17T09:19:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.532+0000 7f3891051700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38780b2880 0x7f38780b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.535+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38880a9a00 con 0x7f388c103c00 2024-05-17T09:19:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:10 smithi069 ceph-mon[35081]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:10 smithi079 ceph-mon[35502]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.798+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f388c06c150 con 0x7f388c103c00 2024-05-17T09:19:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.798+0000 7f3882ffd700 1 -- 172.21.15.19:0/708877401 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f38880a9300 con 0x7f388c103c00 2024-05-17T09:19:10.798 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:10.798 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:16:22.157327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38780b2880 msgr2=0x7f38780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38780b2880 0x7f38780b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c005fb0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 msgr2=0x7f388c1a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 0x7f388c1a1540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f388800d950 tx=0x7f388800dc60 comp rx=0 tx=0).stop 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 shutdown_connections 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38780b2880 0x7f38780b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388c108960 0x7f388c1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388c103c00 0x7f388c1a1540 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 --2- 172.21.15.19:0/708877401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c108010 0x7f388c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 >> 172.21.15.19:0/708877401 conn(0x7f388c0fe530 msgr2=0x7f388c102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 shutdown_connections 2024-05-17T09:19:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:10.801+0000 7f3893ab6700 1 -- 172.21.15.19:0/708877401 wait complete. 2024-05-17T09:19:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-05-17T09:19:11.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:11 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/708877401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:11.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:19:11.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:11 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/708877401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:11.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:11 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/708877401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:12.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:12 smithi019 ceph-mon[31620]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:12 smithi069 ceph-mon[35081]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:12 smithi079 ceph-mon[35502]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 -- 172.21.15.19:0/1482910547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 msgr2=0x7fe4f01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/1482910547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f01081f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80099c0 tx=0x7fe4d8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 -- 172.21.15.19:0/1482910547 shutdown_connections 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/1482910547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 0x7fe4f01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/1482910547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f0108730 0x7fe4f0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/1482910547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f01081f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 -- 172.21.15.19:0/1482910547 >> 172.21.15.19:0/1482910547 conn(0x7fe4f00fede0 msgr2=0x7fe4f0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 -- 172.21.15.19:0/1482910547 shutdown_connections 2024-05-17T09:19:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.556+0000 7fe4f5a15700 1 -- 172.21.15.19:0/1482910547 wait complete. 2024-05-17T09:19:13.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:19:13.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:13.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:13 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.557+0000 7fe4f5a15700 1 Processor -- start 2024-05-17T09:19:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.557+0000 7fe4f5a15700 1 -- start start 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4f5a15700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 0x7fe4f019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4f5a15700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4f5a15700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f0108730 0x7fe4f019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4f5a15700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01131f0 con 0x7fe4f0107de0 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4f5a15700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0113070 con 0x7fe4f0108730 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4f5a15700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0113370 con 0x7fe4f01034b0 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4eeffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 0x7fe4f019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4eeffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 0x7fe4f019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47684/0 (socket says 172.21.15.19:47684) 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4e67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4eeffd700 1 -- 172.21.15.19:0/994392658 learned_addr learned my addr 172.21.15.19:0/994392658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4ef7fe700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f0108730 0x7fe4f019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4e67fc700 1 -- 172.21.15.19:0/994392658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 msgr2=0x7fe4f019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4e67fc700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 0x7fe4f019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4e67fc700 1 -- 172.21.15.19:0/994392658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f0108730 msgr2=0x7fe4f019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4e67fc700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f0108730 0x7fe4f019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.558+0000 7fe4e67fc700 1 -- 172.21.15.19:0/994392658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f01a6f60 con 0x7fe4f0107de0 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.559+0000 7fe4e67fc700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f019c150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098e0 tx=0x7fe4d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.559+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8026070 con 0x7fe4f0107de0 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.559+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4d800f460 con 0x7fe4f0107de0 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.559+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8009e60 con 0x7fe4f0107de0 2024-05-17T09:19:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.559+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f01a71f0 con 0x7fe4f0107de0 2024-05-17T09:19:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.559+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f01a7590 con 0x7fe4f0107de0 2024-05-17T09:19:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.560+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f00512b0 con 0x7fe4f0107de0 2024-05-17T09:19:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.562+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe4d8029070 con 0x7fe4f0107de0 2024-05-17T09:19:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.563+0000 7fe4f4a13700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4dc0b2900 0x7fe4dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.563+0000 7fe4eeffd700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4dc0b2900 0x7fe4dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.563+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe4d80e3040 con 0x7fe4f0107de0 2024-05-17T09:19:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.563+0000 7fe4eeffd700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4dc0b2900 0x7fe4dc0b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009d20 tx=0x7fe4e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.566+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4d8013070 con 0x7fe4f0107de0 2024-05-17T09:19:13.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:19:13.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:13.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:13 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:13.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:19:13.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:13.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:13 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.784+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4f002e5e0 con 0x7fe4f0107de0 2024-05-17T09:19:13.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.786+0000 7fe4f4a13700 1 -- 172.21.15.19:0/994392658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7fe4d80aaa40 con 0x7fe4f0107de0 2024-05-17T09:19:13.786 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:13.786 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.788+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4dc0b2900 msgr2=0x7fe4dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4dc0b2900 0x7fe4dc0b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009d20 tx=0x7fe4e0009260 comp rx=0 tx=0).stop 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 msgr2=0x7fe4f019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f019c150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098e0 tx=0x7fe4d80045e0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 shutdown_connections 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4dc0b2900 0x7fe4dc0b4e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f01034b0 0x7fe4f019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f0108730 0x7fe4f019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 --2- 172.21.15.19:0/994392658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4f0107de0 0x7fe4f019c150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 >> 172.21.15.19:0/994392658 conn(0x7fe4f00fede0 msgr2=0x7fe4f0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 shutdown_connections 2024-05-17T09:19:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:13.789+0000 7fe4f5a15700 1 -- 172.21.15.19:0/994392658 wait complete. 2024-05-17T09:19:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:14 smithi019 ceph-mon[31620]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:19:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:19:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:14.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:14 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/994392658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:14.615 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T09:15:09.950009+0000', 'modified': '2024-05-17T09:16:22.157327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24487, 'mds_2': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6835/437548660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 437548660}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 437548660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/1132484996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1132484996}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 1132484996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/720193094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 720193094}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 720193094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.69:6837/1860504493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 1860504493}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 1860504493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.79:6839/2520420584', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2520420584}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2520420584}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.69:6839/814844032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 814844032}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 814844032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24487, 24497, 24470]} 2024-05-17T09:19:14.615 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T09:19:14.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-17T09:19:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:14 smithi069 ceph-mon[35081]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:19:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:19:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:14 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/994392658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:14 smithi079 ceph-mon[35502]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:19:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:19:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:19:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:14 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/994392658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 -- 172.21.15.19:0/1379754996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31010e900 msgr2=0x7fe310110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 --2- 172.21.15.19:0/1379754996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31010e900 0x7fe310110de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b0a0 tx=0x7fe30800b3b0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:16 smithi019 ceph-mon[31620]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 -- 172.21.15.19:0/1379754996 shutdown_connections 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 --2- 172.21.15.19:0/1379754996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3101054d0 0x7fe3101058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 --2- 172.21.15.19:0/1379754996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31010e900 0x7fe310110de0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 --2- 172.21.15.19:0/1379754996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 0x7fe31010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 -- 172.21.15.19:0/1379754996 >> 172.21.15.19:0/1379754996 conn(0x7fe3100fef20 msgr2=0x7fe310101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 -- 172.21.15.19:0/1379754996 shutdown_connections 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.622+0000 7fe316ace700 1 -- 172.21.15.19:0/1379754996 wait complete. 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.623+0000 7fe316ace700 1 Processor -- start 2024-05-17T09:19:16.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.623+0000 7fe316ace700 1 -- start start 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.623+0000 7fe316ace700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 0x7fe3101a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.623+0000 7fe316ace700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 0x7fe3101a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe316ace700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31010e900 0x7fe31019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe316ace700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe310114310 con 0x7fe3101054d0 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe316ace700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe310114190 con 0x7fe310105e20 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe316ace700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe310114490 con 0x7fe31010e900 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 0x7fe3101a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 0x7fe3101a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54574/0 (socket says 172.21.15.19:54574) 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 -- 172.21.15.19:0/1426184656 learned_addr learned my addr 172.21.15.19:0/1426184656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31506b700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31010e900 0x7fe31019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe30ffff700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 0x7fe3101a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 -- 172.21.15.19:0/1426184656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31010e900 msgr2=0x7fe31019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31010e900 0x7fe31019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 -- 172.21.15.19:0/1426184656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 msgr2=0x7fe3101a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 0x7fe3101a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 -- 172.21.15.19:0/1426184656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31019b1a0 con 0x7fe3101054d0 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe30ffff700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 0x7fe3101a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe31486a700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 0x7fe3101a0880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c920 tx=0x7fe30000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.624+0000 7fe30dffb700 1 -- 172.21.15.19:0/1426184656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300007b00 con 0x7fe3101054d0 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.625+0000 7fe30dffb700 1 -- 172.21.15.19:0/1426184656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe300004cc0 con 0x7fe3101054d0 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.625+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31019b430 con 0x7fe3101054d0 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.625+0000 7fe30dffb700 1 -- 172.21.15.19:0/1426184656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300005440 con 0x7fe3101054d0 2024-05-17T09:19:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.625+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3101a7710 con 0x7fe3101054d0 2024-05-17T09:19:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.628+0000 7fe30dffb700 1 -- 172.21.15.19:0/1426184656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe300004750 con 0x7fe3101054d0 2024-05-17T09:19:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.628+0000 7fe30dffb700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2f80b2880 0x7fe2f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.628+0000 7fe30ffff700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2f80b2880 0x7fe2f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.628+0000 7fe30dffb700 1 -- 172.21.15.19:0/1426184656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe3000e2150 con 0x7fe3101054d0 2024-05-17T09:19:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe30ffff700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2f80b2880 0x7fe2f80b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe304005e10 tx=0x7fe304005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:16.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe3254e3b80 1 Processor -- start 2024-05-17T09:19:16.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe3254e3b80 1 -- start start 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 0x55b725d08f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.629+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b725c31c40 con 0x55b725c02380 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 0x55b725d08f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b725c217e0 con 0x55b725b84df0 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b725c25820 con 0x55b725d03370 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 0x55b725d08f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54584/0 (socket says 172.21.15.19:54584) 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 -- 172.21.15.19:0/3427141755 learned_addr learned my addr 172.21.15.19:0/3427141755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe30ffff700 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31506b700 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 -- 172.21.15.19:0/3427141755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 msgr2=0x55b725d03780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d03780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 -- 172.21.15.19:0/3427141755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 msgr2=0x55b725d094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 -- 172.21.15.19:0/3427141755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b725b71880 con 0x55b725c02380 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe31486a700 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 0x55b725d08f90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe3000a5440 tx=0x7fe3000a8bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed3b38efe16e544 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe2f6ffd700 1 -- 172.21.15.19:0/3427141755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30001d040 con 0x55b725c02380 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe2f6ffd700 1 -- 172.21.15.19:0/3427141755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3000a93c0 con 0x55b725c02380 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.630+0000 7fe2f6ffd700 1 -- 172.21.15.19:0/3427141755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3000a9620 con 0x55b725c02380 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3427141755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 msgr2=0x55b725d08f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 0x55b725d08f90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe3000a5440 tx=0x7fe3000a8bb0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3427141755 shutdown_connections 2024-05-17T09:19:16.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d03780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725c02380 0x55b725d08f90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3427141755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3427141755 >> 172.21.15.19:0/3427141755 conn(0x55b725bc3c30 msgr2=0x55b725cfde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:16.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3427141755 shutdown_connections 2024-05-17T09:19:16.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.631+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3427141755 wait complete. 2024-05-17T09:19:16.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 -- start start 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 Processor -- start 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d0d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b725b27f50 con 0x55b725b84df0 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b725bc3770 con 0x55b725c02380 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.632+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b725d04200 con 0x55b725d03370 2024-05-17T09:19:16.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe3254e3b80 10 client.0 _ll_register_callbacks cb 0x55b725bbc8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54592/0 (socket says 172.21.15.19:54592) 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 -- 172.21.15.19:0/3662725286 learned_addr learned my addr 172.21.15.19:0/3662725286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31506b700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe30ffff700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d0d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 -- 172.21.15.19:0/3662725286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 msgr2=0x55b725d0d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 -- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 msgr2=0x55b725d0d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d0d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 -- 172.21.15.19:0/3662725286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b725bc0640 con 0x55b725b84df0 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe30ffff700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d0d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:16.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe31486a700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0cc50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe3000a6630 tx=0x7fe3000a9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8a00351c008681 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe2effff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe2effff700 1 -- 172.21.15.19:0/3662725286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30001d050 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe2effff700 1 -- 172.21.15.19:0/3662725286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe300007ca0 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.633+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3000a93c0 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe2effff700 1 -- 172.21.15.19:0/3662725286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3000b2220 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 10 client.15174 fetch_fsmap learned FSMap version 26 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b725d1a5b0 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe2effff700 1 -- 172.21.15.19:0/3662725286 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3000a93c0 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe2effff700 1 -- 172.21.15.19:0/3662725286 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe3000a3ae0 con 0x55b725b84df0 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 10 client.15174 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 20 client.15174 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:16.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d1b900 0x55b725d1dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 4 client.15174 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b725c33b90 con 0x55b725d1b900 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.634+0000 7fe30ffff700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d1b900 0x55b725d1dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.635+0000 7fe30ffff700 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d1b900 0x55b725d1dab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcc239b973b16aee server_cookie=25ce772394a8107 in_seq=0 out_seq=0 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.635+0000 7fe2effff700 10 client.15174 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe2effff700 1 -- 172.21.15.19:0/3662725286 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b725c33b90 con 0x55b725d1b900 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe2effff700 10 client.15174 handle_command_reply: tid=0 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 client.15174 shutdown 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 msgr2=0x55b725d0cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0cc50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe3000a6630 tx=0x7fe3000a9970 comp rx=0 tx=0).stop 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 shutdown_connections 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d1b900 0x55b725d1dab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe2effff700 0 client.15174 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0cc50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3662725286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d0d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 >> 172.21.15.19:0/3662725286 conn(0x55b725bc3c30 msgr2=0x55b725cfde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 shutdown_connections 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3662725286 wait complete. 2024-05-17T09:19:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.636+0000 7fe3254e3b80 20 client.15174 trim_cache size 0 max 16384 2024-05-17T09:19:16.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 Processor -- start 2024-05-17T09:19:16.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 -- start start 2024-05-17T09:19:16.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b725d1bdf0 con 0x55b725b84df0 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b725d1c4c0 con 0x55b725c02380 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b725d1cb90 con 0x55b725d03370 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe31486a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54604/0 (socket says 172.21.15.19:54604) 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe31486a700 1 -- 172.21.15.19:0/3378226417 learned_addr learned my addr 172.21.15.19:0/3378226417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe30ffff700 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.658+0000 7fe31506b700 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31486a700 1 -- 172.21.15.19:0/3378226417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 msgr2=0x55b725d0d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31486a700 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31486a700 1 -- 172.21.15.19:0/3378226417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 msgr2=0x55b725d08430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31486a700 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d08430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31486a700 1 -- 172.21.15.19:0/3378226417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b725cffe50 con 0x55b725b84df0 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31506b700 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe31486a700 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0bf70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe3000ad8b0 tx=0x7fe300004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb37369f3f16b71 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe2f67fc700 1 -- 172.21.15.19:0/3378226417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30001d050 con 0x55b725b84df0 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe2f67fc700 1 -- 172.21.15.19:0/3378226417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3000a8ed0 con 0x55b725b84df0 2024-05-17T09:19:16.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.659+0000 7fe2f67fc700 1 -- 172.21.15.19:0/3378226417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300005720 con 0x55b725b84df0 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3378226417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 msgr2=0x55b725d0bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0bf70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe3000ad8b0 tx=0x7fe300004c50 comp rx=0 tx=0).stop 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3378226417 shutdown_connections 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725d03370 0x55b725d0d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725b84df0 0x55b725d0bf70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/3378226417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725c02380 0x55b725d08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3378226417 >> 172.21.15.19:0/3378226417 conn(0x55b725bc3c30 msgr2=0x55b725cf7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:16.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3378226417 shutdown_connections 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 -- 172.21.15.19:0/3378226417 wait complete. 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 -- start start 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.660+0000 7fe3254e3b80 1 Processor -- start 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d27620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725c02380 0x55b725d09990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 0x55b725d09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b725d1d930 con 0x55b725d03370 2024-05-17T09:19:16.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b725d1d260 con 0x55b725b84df0 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b725d1e000 con 0x55b725c02380 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe3254e3b80 10 client.0 _ll_register_callbacks cb 0x55b725bbc8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 0x55b725d09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 0x55b725d09ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54618/0 (socket says 172.21.15.19:54618) 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 -- 172.21.15.19:0/1522575469 learned_addr learned my addr 172.21.15.19:0/1522575469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31486a700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d27620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe30ffff700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725c02380 0x55b725d09990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 -- 172.21.15.19:0/1522575469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725c02380 msgr2=0x55b725d09990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725c02380 0x55b725d09990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 -- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 msgr2=0x55b725d27620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d27620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31506b700 1 -- 172.21.15.19:0/1522575469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b725d0a670 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe31486a700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d27620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.661+0000 7fe30ffff700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725c02380 0x55b725d09990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe31506b700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 0x55b725d09ed0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b070 tx=0x7fe308003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa39d52fccd1aa07 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe2effff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe2effff700 1 -- 172.21.15.19:0/1522575469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30802d060 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe2effff700 1 -- 172.21.15.19:0/1522575469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe308007a50 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3000a8ed0 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe2effff700 1 -- 172.21.15.19:0/1522575469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe308034050 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 10 client.15180 fetch_fsmap learned FSMap version 26 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b725d87dd0 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe2effff700 1 -- 172.21.15.19:0/1522575469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3000a8ed0 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe2effff700 1 -- 172.21.15.19:0/1522575469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe308027070 con 0x55b725d03370 2024-05-17T09:19:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 20 client.15180 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d8a160 0x55b725d8b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 4 client.15180 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.662+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "status"}) -- 0x55b725c2d370 con 0x55b725d8a160 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe31486a700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d8a160 0x55b725d8b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe31486a700 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d8a160 0x55b725d8b5d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b64fce0789a54ec server_cookie=d0e3afa164b3cff1 in_seq=0 out_seq=0 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe2effff700 10 client.15180 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe2effff700 1 -- 172.21.15.19:0/1522575469 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b725c2d370 con 0x55b725d8a160 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe2effff700 10 client.15180 handle_command_reply: tid=0 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe3254e3b80 1 client.15180 shutdown 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 msgr2=0x55b725d09ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.663+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 0x55b725d09ed0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b070 tx=0x7fe308003e20 comp rx=0 tx=0).stop 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 shutdown_connections 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b725d8a160 0x55b725d8b5d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b725d03370 0x55b725d09ed0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b725b84df0 0x55b725d27620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 --2- 172.21.15.19:0/1522575469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b725c02380 0x55b725d09990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 >> 172.21.15.19:0/1522575469 conn(0x55b725bc3c30 msgr2=0x55b725d0bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe2effff700 0 client.15180 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 shutdown_connections 2024-05-17T09:19:16.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 1 -- 172.21.15.19:0/1522575469 wait complete. 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe3254e3b80 20 client.15180 trim_cache size 0 max 16384 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24470, 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 179.45410430300001, 2024-05-17T09:19:16.671 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-05-17T09:19:16.672 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:19:16.672 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T09:19:16.672 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 277.23238987000002 2024-05-17T09:19:16.672 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2f80b2880 msgr2=0x7fe2f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe316ace700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2f80b2880 0x7fe2f80b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe304005e10 tx=0x7fe304005d60 comp rx=0 tx=0).stop 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 msgr2=0x7fe3101a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.664+0000 7fe316ace700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 0x7fe3101a0880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c920 tx=0x7fe30000cc30 comp rx=0 tx=0).stop 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 shutdown_connections 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2f80b2880 0x7fe2f80b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31010e900 0x7fe31019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3101054d0 0x7fe3101a0880 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 --2- 172.21.15.19:0/1426184656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310105e20 0x7fe3101a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 >> 172.21.15.19:0/1426184656 conn(0x7fe3100fef20 msgr2=0x7fe3100ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:16.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 shutdown_connections 2024-05-17T09:19:16.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:16.665+0000 7fe316ace700 1 -- 172.21.15.19:0/1426184656 wait complete. 2024-05-17T09:19:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:16 smithi069 ceph-mon[35081]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:16 smithi079 ceph-mon[35502]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:17.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T09:19:17.727 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:18 smithi069 ceph-mon[35081]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:18 smithi019 ceph-mon[31620]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:18 smithi079 ceph-mon[35502]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:19.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 -- 172.21.15.19:0/1299400096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081092f0 msgr2=0x7fd108109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 --2- 172.21.15.19:0/1299400096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081092f0 0x7fd108109700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009990 tx=0x7fd0fc009ca0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 -- 172.21.15.19:0/1299400096 shutdown_connections 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 --2- 172.21.15.19:0/1299400096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081030d0 0x7fd108111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 --2- 172.21.15.19:0/1299400096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081092f0 0x7fd108109700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 --2- 172.21.15.19:0/1299400096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd108102780 0x7fd108102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 -- 172.21.15.19:0/1299400096 >> 172.21.15.19:0/1299400096 conn(0x7fd1080fe510 msgr2=0x7fd108100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 -- 172.21.15.19:0/1299400096 shutdown_connections 2024-05-17T09:19:19.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.401+0000 7fd10ec78700 1 -- 172.21.15.19:0/1299400096 wait complete. 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.402+0000 7fd10ec78700 1 Processor -- start 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.402+0000 7fd10ec78700 1 -- start start 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ec78700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd108102780 0x7fd10819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ec78700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 0x7fd10819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ec78700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081092f0 0x7fd10819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ec78700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd108114430 con 0x7fd1081030d0 2024-05-17T09:19:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ec78700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081142b0 con 0x7fd1081092f0 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ec78700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081145b0 con 0x7fd108102780 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 0x7fd10819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 0x7fd10819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54632/0 (socket says 172.21.15.19:54632) 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 -- 172.21.15.19:0/3313822256 learned_addr learned my addr 172.21.15.19:0/3313822256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10ca14700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd108102780 0x7fd10819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10d215700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081092f0 0x7fd10819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 -- 172.21.15.19:0/3313822256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd108102780 msgr2=0x7fd10819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd108102780 0x7fd10819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 -- 172.21.15.19:0/3313822256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081092f0 msgr2=0x7fd10819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081092f0 0x7fd10819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd107fff700 1 -- 172.21.15.19:0/3313822256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10819fc00 con 0x7fd1081030d0 2024-05-17T09:19:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.403+0000 7fd10d215700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081092f0 0x7fd10819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.404+0000 7fd107fff700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 0x7fd10819b380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c9e0 tx=0x7fd0f400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.404+0000 7fd105ffb700 1 -- 172.21.15.19:0/3313822256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4007aa0 con 0x7fd1081030d0 2024-05-17T09:19:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.404+0000 7fd105ffb700 1 -- 172.21.15.19:0/3313822256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f4007c40 con 0x7fd1081030d0 2024-05-17T09:19:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.404+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10819fef0 con 0x7fd1081030d0 2024-05-17T09:19:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.404+0000 7fd105ffb700 1 -- 172.21.15.19:0/3313822256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4018660 con 0x7fd1081030d0 2024-05-17T09:19:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.404+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081a0430 con 0x7fd1081030d0 2024-05-17T09:19:19.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.407+0000 7fd105ffb700 1 -- 172.21.15.19:0/3313822256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd0f400f450 con 0x7fd1081030d0 2024-05-17T09:19:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.408+0000 7fd105ffb700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0f80b2880 0x7fd0f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.408+0000 7fd10ca14700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0f80b2880 0x7fd0f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.408+0000 7fd105ffb700 1 -- 172.21.15.19:0/3313822256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd0f40e25a0 con 0x7fd1081030d0 2024-05-17T09:19:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.408+0000 7fd10ca14700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0f80b2880 0x7fd0f80b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc006010 tx=0x7fd0fc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 Processor -- start 2024-05-17T09:19:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 -- start start 2024-05-17T09:19:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb800e380 0x55ecb81182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb7f90df0 0x55ecb8118800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8113b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ecb803dc40 con 0x55ecb8113780 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ecb802d7e0 con 0x55ecb800e380 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd11d68db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8031820 con 0x55ecb7f90df0 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54644/0 (socket says 172.21.15.19:54644) 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 -- 172.21.15.19:0/3194918806 learned_addr learned my addr 172.21.15.19:0/3194918806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd107fff700 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb7f90df0 0x55ecb8118800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10ca14700 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb800e380 0x55ecb81182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 -- 172.21.15.19:0/3194918806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb7f90df0 msgr2=0x55ecb8118800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb7f90df0 0x55ecb8118800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 -- 172.21.15.19:0/3194918806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb800e380 msgr2=0x55ecb81182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb800e380 0x55ecb81182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd10d215700 1 -- 172.21.15.19:0/3194918806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb7f7d880 con 0x55ecb8113780 2024-05-17T09:19:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.409+0000 7fd107fff700 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb7f90df0 0x55ecb8118800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.410+0000 7fd10d215700 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8113b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd100009e60 tx=0x7fd1000077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5e49a348f1687e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.410+0000 7fd0f2ffd700 1 -- 172.21.15.19:0/3194918806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100014610 con 0x55ecb8113780 2024-05-17T09:19:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.410+0000 7fd0f2ffd700 1 -- 172.21.15.19:0/3194918806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd100019070 con 0x55ecb8113780 2024-05-17T09:19:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.410+0000 7fd0f2ffd700 1 -- 172.21.15.19:0/3194918806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100013a80 con 0x55ecb8113780 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 -- 172.21.15.19:0/3194918806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 msgr2=0x55ecb8113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8113b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd100009e60 tx=0x7fd1000077e0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 -- 172.21.15.19:0/3194918806 shutdown_connections 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8113b90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb800e380 0x55ecb81182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 --2- 172.21.15.19:0/3194918806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb7f90df0 0x55ecb8118800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 -- 172.21.15.19:0/3194918806 >> 172.21.15.19:0/3194918806 conn(0x55ecb7fcfc30 msgr2=0x55ecb8109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 -- 172.21.15.19:0/3194918806 shutdown_connections 2024-05-17T09:19:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 -- 172.21.15.19:0/3194918806 wait complete. 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 -- start start 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.411+0000 7fd11d68db80 1 Processor -- start 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb811d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb811d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb811db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ecb7f33f50 con 0x55ecb800e380 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ecb7fcf770 con 0x55ecb7f90df0 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8114570 con 0x55ecb8113780 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd11d68db80 10 client.0 _ll_register_callbacks cb 0x55ecb7fc88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb811d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb811d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54650/0 (socket says 172.21.15.19:54650) 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 -- 172.21.15.19:0/1085260072 learned_addr learned my addr 172.21.15.19:0/1085260072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd10ca14700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb811d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd10d215700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb811db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 -- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 msgr2=0x55ecb811db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb811db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 -- 172.21.15.19:0/1085260072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 msgr2=0x55ecb811d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb811d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd107fff700 1 -- 172.21.15.19:0/1085260072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb7fcc640 con 0x55ecb800e380 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.412+0000 7fd10d215700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb811db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd107fff700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb811d5e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40a5890 tx=0x7fd0f40a9320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93034e89302730f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd0e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd0e7fff700 1 -- 172.21.15.19:0/1085260072 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40c3060 con 0x55ecb800e380 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd0e7fff700 1 -- 172.21.15.19:0/1085260072 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd0f40b21e0 con 0x55ecb800e380 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd100019070 con 0x55ecb800e380 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd0e7fff700 1 -- 172.21.15.19:0/1085260072 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0f4007df0 con 0x55ecb800e380 2024-05-17T09:19:19.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 10 client.15192 fetch_fsmap learned FSMap version 26 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecb812a9e0 con 0x55ecb800e380 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd0e7fff700 1 -- 172.21.15.19:0/1085260072 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0f40c6070 con 0x55ecb800e380 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd0e7fff700 1 -- 172.21.15.19:0/1085260072 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd0f40aab40 con 0x55ecb800e380 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 20 client.15192 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb812cfe0 0x55ecb812e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.413+0000 7fd11d68db80 4 client.15192 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.414+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb803fb90 con 0x55ecb812cfe0 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.414+0000 7fd10ca14700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb812cfe0 0x55ecb812e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.414+0000 7fd10ca14700 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb812cfe0 0x55ecb812e450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b36675a9c9d6c65a server_cookie=d83aa500836661ee in_seq=0 out_seq=0 2024-05-17T09:19:19.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.414+0000 7fd0e7fff700 10 client.15192 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:19:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd0e7fff700 1 -- 172.21.15.19:0/1085260072 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ecb803fb90 con 0x55ecb812cfe0 2024-05-17T09:19:19.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd0e7fff700 10 client.15192 handle_command_reply: tid=0 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 client.15192 shutdown 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 msgr2=0x55ecb811d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb811d5e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40a5890 tx=0x7fd0f40a9320 comp rx=0 tx=0).stop 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 shutdown_connections 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb812cfe0 0x55ecb812e450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb811db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb811d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1085260072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb811d5e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 >> 172.21.15.19:0/1085260072 conn(0x55ecb7fcfc30 msgr2=0x55ecb8109e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd0e7fff700 0 client.15192 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 shutdown_connections 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.415+0000 7fd11d68db80 1 -- 172.21.15.19:0/1085260072 wait complete. 2024-05-17T09:19:19.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.416+0000 7fd11d68db80 20 client.15192 trim_cache size 0 max 16384 2024-05-17T09:19:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 Processor -- start 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 -- start start 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb81291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb800e380 0x55ecb811d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8117440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ecb812c790 con 0x55ecb8113780 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ecb812ce60 con 0x55ecb7f90df0 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.437+0000 7fd11d68db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ecb812d530 con 0x55ecb800e380 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8117440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8117440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54652/0 (socket says 172.21.15.19:54652) 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 -- 172.21.15.19:0/1317179793 learned_addr learned my addr 172.21.15.19:0/1317179793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd107fff700 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb800e380 0x55ecb811d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10ca14700 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb81291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 -- 172.21.15.19:0/1317179793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb800e380 msgr2=0x55ecb811d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb800e380 0x55ecb811d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 -- 172.21.15.19:0/1317179793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 msgr2=0x55ecb81291e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb81291e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 -- 172.21.15.19:0/1317179793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb811f940 con 0x55ecb8113780 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd107fff700 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb800e380 0x55ecb811d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd10d215700 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8117440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd1000090d0 tx=0x7fd100004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8011d9ba2ad9ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd0f27fc700 1 -- 172.21.15.19:0/1317179793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100027670 con 0x55ecb8113780 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd0f27fc700 1 -- 172.21.15.19:0/1317179793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd100022070 con 0x55ecb8113780 2024-05-17T09:19:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.438+0000 7fd0f27fc700 1 -- 172.21.15.19:0/1317179793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100005280 con 0x55ecb8113780 2024-05-17T09:19:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 -- 172.21.15.19:0/1317179793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 msgr2=0x55ecb8117440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8117440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd1000090d0 tx=0x7fd100004c20 comp rx=0 tx=0).stop 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 -- 172.21.15.19:0/1317179793 shutdown_connections 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb8113780 0x55ecb8117440 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb81291e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 --2- 172.21.15.19:0/1317179793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb800e380 0x55ecb811d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 -- 172.21.15.19:0/1317179793 >> 172.21.15.19:0/1317179793 conn(0x55ecb7fcfc30 msgr2=0x55ecb8116d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 -- 172.21.15.19:0/1317179793 shutdown_connections 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.439+0000 7fd11d68db80 1 -- 172.21.15.19:0/1317179793 wait complete. 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 -- start start 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 Processor -- start 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb8137ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb8138010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb8132880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ecb812e2d0 con 0x55ecb800e380 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ecb812dc00 con 0x55ecb7f90df0 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ecb812e9a0 con 0x55ecb8113780 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd11d68db80 10 client.0 _ll_register_callbacks cb 0x55ecb7fc88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd107fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb8138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd107fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb8138010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54666/0 (socket says 172.21.15.19:54666) 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd107fff700 1 -- 172.21.15.19:0/2363360647 learned_addr learned my addr 172.21.15.19:0/2363360647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.440+0000 7fd10ca14700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb8137ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd10d215700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb8132880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd107fff700 1 -- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 msgr2=0x55ecb8132880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd107fff700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb8132880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd107fff700 1 -- 172.21.15.19:0/2363360647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 msgr2=0x55ecb8137ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd107fff700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb8137ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd107fff700 1 -- 172.21.15.19:0/2363360647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb8132fb0 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd10d215700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb8132880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd107fff700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb8138010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406f190 tx=0x7fd0f400a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f49b5bf238115c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd0e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd0e7fff700 1 -- 172.21.15.19:0/2363360647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40b2670 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd100022070 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd0e7fff700 1 -- 172.21.15.19:0/2363360647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd0f400f950 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd0e7fff700 1 -- 172.21.15.19:0/2363360647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0f406f1c0 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd11d68db80 10 client.15198 fetch_fsmap learned FSMap version 26 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecb81990d0 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.441+0000 7fd0e7fff700 1 -- 172.21.15.19:0/2363360647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd100022070 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd0e7fff700 1 -- 172.21.15.19:0/2363360647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd0f40911d0 con 0x55ecb800e380 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd11d68db80 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:19:19.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd11d68db80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd11d68db80 20 client.15198 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd11d68db80 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb819b320 0x55ecb819c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd11d68db80 4 client.15198 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "status"}) -- 0x55ecb8039370 con 0x55ecb819b320 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd10ca14700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb819b320 0x55ecb819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd0e7fff700 10 client.15198 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.442+0000 7fd10ca14700 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb819b320 0x55ecb819c790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c3bbb0119f7ecad server_cookie=38ff5f67a7a489d4 in_seq=0 out_seq=0 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd0e7fff700 1 -- 172.21.15.19:0/2363360647 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ecb8039370 con 0x55ecb819b320 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd0e7fff700 10 client.15198 handle_command_reply: tid=0 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 client.15198 shutdown 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 msgr2=0x55ecb8138010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb8138010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406f190 tx=0x7fd0f400a6b0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 shutdown_connections 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55ecb819b320 0x55ecb819c790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecb8113780 0x55ecb8132880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecb7f90df0 0x55ecb8137ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 --2- 172.21.15.19:0/2363360647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecb800e380 0x55ecb8138010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 >> 172.21.15.19:0/2363360647 conn(0x55ecb7fcfc30 msgr2=0x55ecb81174c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd0e7fff700 0 client.15198 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 shutdown_connections 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 1 -- 172.21.15.19:0/2363360647 wait complete. 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.443+0000 7fd11d68db80 20 client.15198 trim_cache size 0 max 16384 2024-05-17T09:19:19.449 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 183.24849255999999, 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T09:19:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 277.25571226699998 2024-05-17T09:19:19.451 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:19:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0f80b2880 msgr2=0x7fd0f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0f80b2880 0x7fd0f80b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc006010 tx=0x7fd0fc005960 comp rx=0 tx=0).stop 2024-05-17T09:19:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 msgr2=0x7fd10819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 0x7fd10819b380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c9e0 tx=0x7fd0f400ccf0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 shutdown_connections 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0f80b2880 0x7fd0f80b4d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd108102780 0x7fd10819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1081092f0 0x7fd10819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 --2- 172.21.15.19:0/3313822256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1081030d0 0x7fd10819b380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 >> 172.21.15.19:0/3313822256 conn(0x7fd1080fe510 msgr2=0x7fd108107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 shutdown_connections 2024-05-17T09:19:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:19.444+0000 7fd10ec78700 1 -- 172.21.15.19:0/3313822256 wait complete. 2024-05-17T09:19:20.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T09:19:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:20.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:20 smithi019 ceph-mon[31620]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:20 smithi069 ceph-mon[35081]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:20 smithi079 ceph-mon[35502]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 -- 172.21.15.19:0/2119193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d8108f10 msgr2=0x7fc2d8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 --2- 172.21.15.19:0/2119193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d8108f10 0x7fc2d8109320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0099c0 tx=0x7fc2cc009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 -- 172.21.15.19:0/2119193953 shutdown_connections 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 --2- 172.21.15.19:0/2119193953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d81045e0 0x7fc2d81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 --2- 172.21.15.19:0/2119193953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8109860 0x7fc2d81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 --2- 172.21.15.19:0/2119193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d8108f10 0x7fc2d8109320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 -- 172.21.15.19:0/2119193953 >> 172.21.15.19:0/2119193953 conn(0x7fc2d8100030 msgr2=0x7fc2d8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 -- 172.21.15.19:0/2119193953 shutdown_connections 2024-05-17T09:19:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.370+0000 7fc2dd908700 1 -- 172.21.15.19:0/2119193953 wait complete. 2024-05-17T09:19:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.371+0000 7fc2dd908700 1 Processor -- start 2024-05-17T09:19:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.371+0000 7fc2dd908700 1 -- start start 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2dd908700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 0x7fc2d819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2dd908700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8108f10 0x7fc2d819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2dd908700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d8109860 0x7fc2d819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2dd908700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8114370 con 0x7fc2d81045e0 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2dd908700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81141f0 con 0x7fc2d8108f10 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2dd908700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81144f0 con 0x7fc2d8109860 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 0x7fc2d819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 0x7fc2d819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54690/0 (socket says 172.21.15.19:54690) 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/727293789 learned_addr learned my addr 172.21.15.19:0/727293789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8108f10 0x7fc2d819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d8109860 0x7fc2d819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/727293789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d8109860 msgr2=0x7fc2d819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d8109860 0x7fc2d819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/727293789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8108f10 msgr2=0x7fc2d819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8108f10 0x7fc2d819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/727293789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d819fc90 con 0x7fc2d81045e0 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.372+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8108f10 0x7fc2d819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.373+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 0x7fc2d819adc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800e9a0 tx=0x7fc2c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.373+0000 7fc2dc906700 1 -- 172.21.15.19:0/727293789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800ca10 con 0x7fc2d81045e0 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.373+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d819ff80 con 0x7fc2d81045e0 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.373+0000 7fc2dc906700 1 -- 172.21.15.19:0/727293789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c800cbb0 con 0x7fc2d81045e0 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.373+0000 7fc2dc906700 1 -- 172.21.15.19:0/727293789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8018680 con 0x7fc2d81045e0 2024-05-17T09:19:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.373+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81a78a0 con 0x7fc2d81045e0 2024-05-17T09:19:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.375+0000 7fc2dc906700 1 -- 172.21.15.19:0/727293789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2c8018820 con 0x7fc2d81045e0 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.376+0000 7fc2dc906700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2c00b2880 0x7fc2c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.376+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2c00b2880 0x7fc2c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.376+0000 7fc2dc906700 1 -- 172.21.15.19:0/727293789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2c80e27f0 con 0x7fc2d81045e0 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.376+0000 7fc2ec31db80 1 Processor -- start 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 -- start start 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2c00b2880 0x7fc2c00b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc000c00 tx=0x7fc2cc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 0x56334abc1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abbc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56334aae5d20 con 0x56334ab243e0 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56334aad58c0 con 0x56334aa38df0 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56334aad9900 con 0x56334abbc090 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 0x56334abc1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 0x56334abc1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54700/0 (socket says 172.21.15.19:54700) 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/606830519 learned_addr learned my addr 172.21.15.19:0/606830519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.377+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abbc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/606830519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 msgr2=0x56334abbc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abbc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/606830519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 msgr2=0x56334abc2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/606830519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334aa7ef60 con 0x56334ab243e0 2024-05-17T09:19:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abbc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 0x56334abc1d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80a9760 tx=0x7fc2c80a92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0f5588df36934f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2bdffb700 1 -- 172.21.15.19:0/606830519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80ade10 con 0x56334ab243e0 2024-05-17T09:19:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2bdffb700 1 -- 172.21.15.19:0/606830519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c80a5050 con 0x56334ab243e0 2024-05-17T09:19:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.378+0000 7fc2bdffb700 1 -- 172.21.15.19:0/606830519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80a8cc0 con 0x56334ab243e0 2024-05-17T09:19:22.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 -- 172.21.15.19:0/606830519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 msgr2=0x56334abc1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 0x56334abc1d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80a9760 tx=0x7fc2c80a92e0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 -- 172.21.15.19:0/606830519 shutdown_connections 2024-05-17T09:19:22.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abbc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334ab243e0 0x56334abc1d40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/606830519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 -- 172.21.15.19:0/606830519 >> 172.21.15.19:0/606830519 conn(0x56334aa99170 msgr2=0x56334abb2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 -- 172.21.15.19:0/606830519 shutdown_connections 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 -- 172.21.15.19:0/606830519 wait complete. 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.379+0000 7fc2ec31db80 1 -- start start 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 Processor -- start 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 0x56334abc64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56334a9dbf50 con 0x56334abbc090 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56334aa3dd50 con 0x56334aa38df0 2024-05-17T09:19:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56334abbceb0 con 0x56334ab243e0 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2ec31db80 10 client.0 _ll_register_callbacks cb 0x56334aa70900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 0x56334abc64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 0x56334abc64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54704/0 (socket says 172.21.15.19:54704) 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 -- 172.21.15.19:0/556303022 learned_addr learned my addr 172.21.15.19:0/556303022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 -- 172.21.15.19:0/556303022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 msgr2=0x56334abc5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 -- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 msgr2=0x56334abc5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d77fe700 1 -- 172.21.15.19:0/556303022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334aa25880 con 0x56334abbc090 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.380+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 0x56334abc64b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0007c60 tx=0x7fc2d0007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baed12f75942447 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/556303022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d001a070 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c80a5050 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/556303022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2d0013b30 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/556303022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2d0026050 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2ec31db80 10 client.15210 fetch_fsmap learned FSMap version 26 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56334aa8f710 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/556303022 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c80a5050 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.381+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/556303022 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc2d001e3b0 con 0x56334abbc090 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2ec31db80 10 client.15210 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2ec31db80 10 client.15210 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2ec31db80 20 client.15210 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334abd57c0 0x56334abd6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2ec31db80 4 client.15210 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56334ab83e80 con 0x56334abd57c0 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334abd57c0 0x56334abd6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2b6ffd700 10 client.15210 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.382+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334abd57c0 0x56334abd6c30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a09164b8d0a66dd server_cookie=ac685c023d425c9d in_seq=0 out_seq=0 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/556303022 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56334ab83e80 con 0x56334abd57c0 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2b6ffd700 10 client.15210 handle_command_reply: tid=0 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 client.15210 shutdown 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 msgr2=0x56334abc64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 0x56334abc64b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0007c60 tx=0x7fc2d0007d40 comp rx=0 tx=0).stop 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 shutdown_connections 2024-05-17T09:19:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334abd57c0 0x56334abd6c30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334abbc090 0x56334abc64b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334aa38df0 0x56334abc5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.383+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/556303022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.384+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 >> 172.21.15.19:0/556303022 conn(0x56334aa99170 msgr2=0x56334abb2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.384+0000 7fc2b6ffd700 0 client.15210 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.384+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 shutdown_connections 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.384+0000 7fc2ec31db80 1 -- 172.21.15.19:0/556303022 wait complete. 2024-05-17T09:19:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.384+0000 7fc2ec31db80 20 client.15210 trim_cache size 0 max 16384 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 Processor -- start 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 -- start start 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334abbc090 0x56334abb6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56334abd4f70 con 0x56334aa38df0 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56334abd5640 con 0x56334abbc090 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.405+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56334abd5d10 con 0x56334ab243e0 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54712/0 (socket says 172.21.15.19:54712) 2024-05-17T09:19:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2143999943 learned_addr learned my addr 172.21.15.19:0/2143999943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:22.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334abbc090 0x56334abb6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2143999943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 msgr2=0x56334abc1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2143999943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334abbc090 msgr2=0x56334abb6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334abbc090 0x56334abb6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2143999943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334abd3290 con 0x56334aa38df0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc0b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80a9760 tx=0x7fc2c8009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41e5f5ffb986749 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2bd7fa700 1 -- 172.21.15.19:0/2143999943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c806e070 con 0x56334aa38df0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2bd7fa700 1 -- 172.21.15.19:0/2143999943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c80b28c0 con 0x56334aa38df0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.406+0000 7fc2bd7fa700 1 -- 172.21.15.19:0/2143999943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c808ae00 con 0x56334aa38df0 2024-05-17T09:19:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2143999943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 msgr2=0x56334abc0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc0b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80a9760 tx=0x7fc2c8009790 comp rx=0 tx=0).stop 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2143999943 shutdown_connections 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334abbc090 0x56334abb6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc0b30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2143999943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334ab243e0 0x56334abc1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2143999943 >> 172.21.15.19:0/2143999943 conn(0x56334aa99170 msgr2=0x56334abc4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2143999943 shutdown_connections 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.407+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2143999943 wait complete. 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 -- start start 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 Processor -- start 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334ab243e0 0x56334abc2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abdf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56334abd6ab0 con 0x56334aa38df0 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56334abd63e0 con 0x56334ab243e0 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56334abd7180 con 0x56334abbc090 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2ec31db80 10 client.0 _ll_register_callbacks cb 0x56334aa70900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2d67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334ab243e0 0x56334abc2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54726/0 (socket says 172.21.15.19:54726) 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.408+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2911799191 learned_addr learned my addr 172.21.15.19:0/2911799191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d77fe700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abdf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2911799191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 msgr2=0x56334abdf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abdf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334ab243e0 msgr2=0x56334abc2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:22.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334ab243e0 0x56334abc2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d6ffd700 1 -- 172.21.15.19:0/2911799191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334abdf690 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334ab243e0 0x56334abc2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2d6ffd700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc2500 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009230 tx=0x7fc2c80dffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8816ca2d800e10ca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/2911799191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c806e050 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/2911799191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2c80b2430 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c80b28c0 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/2911799191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2c806f4b0 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2ec31db80 10 client.15216 fetch_fsmap learned FSMap version 26 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56334abe08d0 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.409+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/2911799191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c80b28c0 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/2911799191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc2c8018b10 con 0x56334aa38df0 2024-05-17T09:19:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2ec31db80 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2ec31db80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2ec31db80 20 client.15216 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334ac42d00 0x56334ac43aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2ec31db80 4 client.15216 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "status"}) -- 0x56334aae7c70 con 0x56334ac42d00 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334ac42d00 0x56334ac43aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2b6ffd700 10 client.15216 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.410+0000 7fc2d67fc700 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334ac42d00 0x56334ac43aa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f402f7678d7946d9 server_cookie=4cd9b41d1e1a3ccf in_seq=0 out_seq=0 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2b6ffd700 1 -- 172.21.15.19:0/2911799191 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56334aae7c70 con 0x56334ac42d00 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2b6ffd700 10 client.15216 handle_command_reply: tid=0 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 client.15216 shutdown 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 msgr2=0x56334abc2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc2500 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009230 tx=0x7fc2c80dffd0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 shutdown_connections 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56334abbc090 0x56334abdf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56334ac42d00 0x56334ac43aa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56334aa38df0 0x56334abc2500 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 --2- 172.21.15.19:0/2911799191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56334ab243e0 0x56334abc2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 >> 172.21.15.19:0/2911799191 conn(0x56334aa99170 msgr2=0x56334abc0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2b6ffd700 0 client.15216 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 shutdown_connections 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 1 -- 172.21.15.19:0/2911799191 wait complete. 2024-05-17T09:19:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.411+0000 7fc2ec31db80 20 client.15216 trim_cache size 0 max 16384 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24497, 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 252.453546741, 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T09:19:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 269.80526913099999 2024-05-17T09:19:22.419 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:19:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2c00b2880 msgr2=0x7fc2c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2c00b2880 0x7fc2c00b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc000c00 tx=0x7fc2cc01a040 comp rx=0 tx=0).stop 2024-05-17T09:19:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 msgr2=0x7fc2d819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 0x7fc2d819adc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800e9a0 tx=0x7fc2c800ecb0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 shutdown_connections 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2c00b2880 0x7fc2c00b4d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d8109860 0x7fc2d819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d81045e0 0x7fc2d819adc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 --2- 172.21.15.19:0/727293789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d8108f10 0x7fc2d819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 >> 172.21.15.19:0/727293789 conn(0x7fc2d8100030 msgr2=0x7fc2d8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 shutdown_connections 2024-05-17T09:19:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:22.412+0000 7fc2dd908700 1 -- 172.21.15.19:0/727293789 wait complete. 2024-05-17T09:19:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:22 smithi069 ceph-mon[35081]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:22 smithi019 ceph-mon[31620]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:22 smithi079 ceph-mon[35502]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T09:19:23.181 INFO:teuthology.run_tasks:Running task sequential... 2024-05-17T09:19:23.194 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-17T09:19:23.194 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-17T09:19:23.194 INFO:teuthology.task.print:Enabling mgr modules 2024-05-17T09:19:23.195 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-17T09:19:23.221 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-17T09:19:23.221 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-17T09:19:23.221 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-17T09:19:23.221 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T09:19:23.222 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:23.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:23.222 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-17T09:19:23.241 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-17T09:19:23.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:23.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T09:19:23.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:23.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-17T09:19:23.420 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-17T09:19:23.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T09:19:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:24 smithi069 ceph-mon[35081]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:24 smithi019 ceph-mon[31620]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:24 smithi079 ceph-mon[35502]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-17T09:19:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.414+0000 7f3decbe4700 1 -- 172.21.15.19:0/3877662584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de81045d0 msgr2=0x7f3de81049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.414+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3877662584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de81045d0 0x7f3de81049e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80099c0 tx=0x7f3dd8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.414+0000 7f3decbe4700 1 -- 172.21.15.19:0/3877662584 shutdown_connections 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.415+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3877662584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3de8109850 0x7f3de81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.415+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3877662584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de81045d0 0x7f3de81049e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.415+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3877662584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8108f00 0x7f3de8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.415+0000 7f3decbe4700 1 -- 172.21.15.19:0/3877662584 >> 172.21.15.19:0/3877662584 conn(0x7f3de80ffe60 msgr2=0x7f3de81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.415+0000 7f3decbe4700 1 -- 172.21.15.19:0/3877662584 shutdown_connections 2024-05-17T09:19:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.415+0000 7f3decbe4700 1 -- 172.21.15.19:0/3877662584 wait complete. 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 Processor -- start 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 -- start start 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3de81045d0 0x7f3de8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 0x7f3de8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8109850 0x7f3de8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8114330 con 0x7f3de8108f00 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81141b0 con 0x7f3de81045d0 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3decbe4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81144b0 con 0x7f3de8109850 2024-05-17T09:19:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3de5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 0x7f3de8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3de5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 0x7f3de8075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42676/0 (socket says 172.21.15.19:42676) 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3de5d9b700 1 -- 172.21.15.19:0/3512713001 learned_addr learned my addr 172.21.15.19:0/3512713001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3de659c700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3de81045d0 0x7f3de8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.416+0000 7f3de6d9d700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8109850 0x7f3de8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de5d9b700 1 -- 172.21.15.19:0/3512713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8109850 msgr2=0x7f3de8076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de5d9b700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8109850 0x7f3de8076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de5d9b700 1 -- 172.21.15.19:0/3512713001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3de81045d0 msgr2=0x7f3de8075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de5d9b700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3de81045d0 0x7f3de8075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de5d9b700 1 -- 172.21.15.19:0/3512713001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de81a9090 con 0x7f3de8108f00 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de6d9d700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8109850 0x7f3de8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3de5d9b700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 0x7f3de8075ff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d980 tx=0x7f3ddc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc017070 con 0x7f3de8108f00 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ddc009dc0 con 0x7f3de8108f00 2024-05-17T09:19:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de81a9380 con 0x7f3de8108f00 2024-05-17T09:19:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00f3c0 con 0x7f3de8108f00 2024-05-17T09:19:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.417+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de81a98c0 con 0x7f3de8108f00 2024-05-17T09:19:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.418+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de80512b0 con 0x7f3de8108f00 2024-05-17T09:19:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.420+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3ddc009930 con 0x7f3de8108f00 2024-05-17T09:19:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.420+0000 7f3dd77fe700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3dd00b2880 0x7f3dd00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.420+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3ddc0e20d0 con 0x7f3de8108f00 2024-05-17T09:19:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.420+0000 7f3de659c700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3dd00b2880 0x7f3dd00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.421+0000 7f3de659c700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3dd00b2880 0x7f3dd00b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8000c00 tx=0x7f3dd8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.424+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ddc0a9a40 con 0x7f3de8108f00 2024-05-17T09:19:25.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.685+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3de802e5e0 con 0x7f3de8108f00 2024-05-17T09:19:25.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.686+0000 7f3dd77fe700 1 -- 172.21.15.19:0/3512713001 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f3ddc0a9340 con 0x7f3de8108f00 2024-05-17T09:19:25.686 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-17T09:19:25.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3dd00b2880 msgr2=0x7f3dd00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:25.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3dd00b2880 0x7f3dd00b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8000c00 tx=0x7f3dd8005fb0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 msgr2=0x7f3de8075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:25.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 0x7f3de8075ff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d980 tx=0x7f3ddc00dc90 comp rx=0 tx=0).stop 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 shutdown_connections 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3dd00b2880 0x7f3dd00b4d90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3de8109850 0x7f3de8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3de81045d0 0x7f3de8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 --2- 172.21.15.19:0/3512713001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de8108f00 0x7f3de8075ff0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 >> 172.21.15.19:0/3512713001 conn(0x7f3de80ffe60 msgr2=0x7f3de81072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 shutdown_connections 2024-05-17T09:19:25.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:25.688+0000 7f3decbe4700 1 -- 172.21.15.19:0/3512713001 wait complete. 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T09:19:26.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T09:19:26.456 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T09:19:26.456 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T09:19:26.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:26.456 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-05-17T09:19:26.475 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T09:19:26.475 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T09:19:26.475 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-05-17T09:19:26.476 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-17T09:19:26.477 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41951sec preferred_lft 41951sec 2024-05-17T09:19:26.477 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-17T09:19:26.477 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-17T09:19:26.477 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T09:19:26.477 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T09:19:26.477 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-05-17T09:19:26.477 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T09:19:26.477 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:19:26.477 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:19:26.477 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-05-17T09:19:26.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-05-17T09:19:26.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-05-17T09:19:26.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T09:19:26.478 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:19:26.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:19:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:19:26.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:26 smithi019 ceph-mon[31620]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:26.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:26 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3512713001' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T09:19:26.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:26.629 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T09:19:26.655 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-05-17T09:19:26.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:26.656 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-05-17T09:19:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:26 smithi069 ceph-mon[35081]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:26 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3512713001' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T09:19:26.714 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-05-17T09:19:26.714 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T09:19:26.714 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-17T09:19:26.714 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-17T09:19:26.714 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T09:19:26.715 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:19:26.715 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:19:26.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-17T09:19:26.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-17T09:19:26.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-17T09:19:26.715 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:19:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:26 smithi079 ceph-mon[35502]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T09:19:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:26 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3512713001' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T09:19:26.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:19:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:19:26.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:26.845 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-17T09:19:26.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:26.862 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-05-17T09:19:26.918 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:19:26.918 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:19:26.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:26.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-17T09:19:26.919 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:19:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:19:27.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:19:27.041 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-17T09:19:27.041 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:19:27.041 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:19:27.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-17T09:19:27.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T09:19:27.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-17T09:19:27.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-17T09:19:27.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-17T09:19:27.042 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:19:27.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:19:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:19:27.161 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T09:19:27.161 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-17T09:19:27.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-05-17T09:19:27.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T09:19:27.161 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-17T09:19:27.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T09:19:27.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:19:27.241 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-17T09:19:27.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T09:19:27.241 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:27.257 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T09:19:27.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T09:19:27.257 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:27.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-17T09:19:27.380 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T09:19:27.437 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-17T09:19:27.437 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-17T09:19:27.438 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T09:19:27.439 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e38ed570e27c2067f159537cd6dbdfa48d8341d814f3621d8480c1538e1a4584/merged 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/915aa4a49a9b427eb59253962f3294adfba243c780d4d98ee3c676f165537864/merged 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/771ae42c6c974cb5cabd1b84160e64ee5b728e6f89a90e9084f37af07bf202ca/merged 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/15c08818cc91279bb50691009af2904dd353b8e48fc373a71f6533e7854cb130/merged 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/c475fe19d3080f437062d2798ecff9d5f633dcbca27b63186d2b5ec30bd65b63/merged 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/ae1a36aa63ed826bf37af7568322fabd17e78e97ec6f90ea7927fc98b6237e01/merged 2024-05-17T09:19:27.440 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5b2ae6a0ee44dfc255eae88a62aad5da201f3f85045af9d76bd8f7878ffbfdc0/merged 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/91b165d8ae06b68d3ad58ce9b6cc0c52b9497643b8677875ed636e90c46f5d2a/merged 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/41c98f10fe54919ef49d91ad3a9d0b7ff2951ef2f452b33252b3e1d458a3ae34/merged 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/0a1d060f75c509c6772e39741b59a06bd5e2926810bec6a60fc1fdc83b11e572/merged 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:27.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:27.441 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-17T09:19:27.494 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T09:19:27.494 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df --client_fs=cephfs) 2024-05-17T09:19:27.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-17T09:19:27.562 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T09:19:27.620 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-17T09:19:27.620 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-17T09:19:27.620 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-17T09:19:27.621 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-17T09:19:27.622 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e38ed570e27c2067f159537cd6dbdfa48d8341d814f3621d8480c1538e1a4584/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/915aa4a49a9b427eb59253962f3294adfba243c780d4d98ee3c676f165537864/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/771ae42c6c974cb5cabd1b84160e64ee5b728e6f89a90e9084f37af07bf202ca/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/15c08818cc91279bb50691009af2904dd353b8e48fc373a71f6533e7854cb130/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/c475fe19d3080f437062d2798ecff9d5f633dcbca27b63186d2b5ec30bd65b63/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/ae1a36aa63ed826bf37af7568322fabd17e78e97ec6f90ea7927fc98b6237e01/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5b2ae6a0ee44dfc255eae88a62aad5da201f3f85045af9d76bd8f7878ffbfdc0/merged 2024-05-17T09:19:27.623 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/91b165d8ae06b68d3ad58ce9b6cc0c52b9497643b8677875ed636e90c46f5d2a/merged 2024-05-17T09:19:27.624 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/41c98f10fe54919ef49d91ad3a9d0b7ff2951ef2f452b33252b3e1d458a3ae34/merged 2024-05-17T09:19:27.624 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/0a1d060f75c509c6772e39741b59a06bd5e2926810bec6a60fc1fdc83b11e572/merged 2024-05-17T09:19:27.624 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-17T09:19:27.624 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:27.624 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:27.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:27.624 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-17T09:19:27.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:2024-05-17T09:19:27.631+0000 7fc298583480 -1 init, newargv = 0x7fc27c004720 newargc=15 2024-05-17T09:19:27.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:ceph-fuse[88770]: starting ceph client 2024-05-17T09:19:27.648 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:ceph-fuse[88770]: starting fuse 2024-05-17T09:19:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:28 smithi069 ceph-mon[35081]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:28 smithi019 ceph-mon[31620]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:28.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-17T09:19:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:28 smithi079 ceph-mon[35502]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:19:28.782 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-17T09:19:28.842 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T09:19:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e38ed570e27c2067f159537cd6dbdfa48d8341d814f3621d8480c1538e1a4584/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/915aa4a49a9b427eb59253962f3294adfba243c780d4d98ee3c676f165537864/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/771ae42c6c974cb5cabd1b84160e64ee5b728e6f89a90e9084f37af07bf202ca/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/15c08818cc91279bb50691009af2904dd353b8e48fc373a71f6533e7854cb130/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/c475fe19d3080f437062d2798ecff9d5f633dcbca27b63186d2b5ec30bd65b63/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/ae1a36aa63ed826bf37af7568322fabd17e78e97ec6f90ea7927fc98b6237e01/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/5b2ae6a0ee44dfc255eae88a62aad5da201f3f85045af9d76bd8f7878ffbfdc0/merged 2024-05-17T09:19:28.844 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/91b165d8ae06b68d3ad58ce9b6cc0c52b9497643b8677875ed636e90c46f5d2a/merged 2024-05-17T09:19:28.845 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/41c98f10fe54919ef49d91ad3a9d0b7ff2951ef2f452b33252b3e1d458a3ae34/merged 2024-05-17T09:19:28.845 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/0a1d060f75c509c6772e39741b59a06bd5e2926810bec6a60fc1fdc83b11e572/merged 2024-05-17T09:19:28.845 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-17T09:19:28.845 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:28.845 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:19:28.845 INFO:teuthology.orchestra.run.smithi019.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:28.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:28.846 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-17T09:19:28.906 INFO:teuthology.orchestra.run.smithi019.stdout:107 2024-05-17T09:19:28.906 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-17T09:19:28.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo stdin-killer -- python3 -c ' 2024-05-17T09:19:28.906 DEBUG:teuthology.orchestra.run.smithi019:> import glob 2024-05-17T09:19:28.906 DEBUG:teuthology.orchestra.run.smithi019:> import re 2024-05-17T09:19:28.906 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> import subprocess 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> def _find_admin_socket(client_name): 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> files = glob.glob(asok_path) 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> # Given a non-glob path, it better be there 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> if "*" not in asok_path: 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> assert(len(files) == 1) 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> return files[0] 2024-05-17T09:19:28.907 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> for f in files: 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> contents = proc_f.read() 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> if mountpoint in contents: 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> return f 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> print(_find_admin_socket("client.0")) 2024-05-17T09:19:28.908 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-05-17T09:19:29.038 INFO:teuthology.orchestra.run.smithi019.stdout:/var/run/ceph/ceph-client.0.88770.asok 2024-05-17T09:19:29.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T09:19:29.051 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.88770.asok 2024-05-17T09:19:29.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:29.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.88770.asok status 2024-05-17T09:19:29.201 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:19:29.201 INFO:teuthology.orchestra.run.smithi019.stdout: "metadata": { 2024-05-17T09:19:29.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_sha1": "3790c0feaaca36c33bb1b31470733325d7d91e6a", 2024-05-17T09:19:29.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3778-g3790c0fe (3790c0feaaca36c33bb1b31470733325d7d91e6a) squid (dev)", 2024-05-17T09:19:29.201 INFO:teuthology.orchestra.run.smithi019.stdout: "entity_id": "0", 2024-05-17T09:19:29.201 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": "smithi019", 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "pid": "88770", 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "root": "/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df" 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_count": 0, 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_pinned_count": 0, 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 15228, 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "inst": { 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": { 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "client", 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 15228 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:19:29.202 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 849183059 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 849183059 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "inst_str": "client.15228 192.168.0.1:0/849183059", 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "addr_str": "192.168.0.1:0/849183059", 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "inode_count": 5, 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_epoch": 26, 2024-05-17T09:19:29.203 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch": 89, 2024-05-17T09:19:29.204 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch_barrier": 0, 2024-05-17T09:19:29.204 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklisted": false, 2024-05-17T09:19:29.204 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs" 2024-05-17T09:19:29.204 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:19:29.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:29.210 DEBUG:teuthology.orchestra.run.smithi019:> stat --file-system '--printf=%T 2024-05-17T09:19:29.210 DEBUG:teuthology.orchestra.run.smithi019:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:29.226 INFO:teuthology.orchestra.run.smithi019.stdout:fuseblk 2024-05-17T09:19:29.226 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:29.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:19:29.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T09:19:29.296 INFO:teuthology.run_tasks:Running task exec... 2024-05-17T09:19:29.306 INFO:teuthology.task.exec:Executing custom commands... 2024-05-17T09:19:29.306 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:19:29.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-17T09:19:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/2032059335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4103d60 msgr2=0x7f5cd4104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/2032059335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4103d60 0x7f5cd4104170 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009900 tx=0x7f5cc402a410 comp rx=0 tx=0).stop 2024-05-17T09:19:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/2032059335 shutdown_connections 2024-05-17T09:19:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/2032059335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cd4108a60 0x7f5cd410ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/2032059335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4103d60 0x7f5cd4104170 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/2032059335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4102b20 0x7f5cd4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/2032059335 >> 172.21.15.19:0/2032059335 conn(0x7f5cd40fdd10 msgr2=0x7f5cd41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/2032059335 shutdown_connections 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/2032059335 wait complete. 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 Processor -- start 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.453+0000 7f5cdb6c8640 1 -- start start 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cdb6c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 0x7f5cd419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cdb6c8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4103d60 0x7f5cd419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cdb6c8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cd4108a60 0x7f5cd41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cdb6c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410e4d0 con 0x7f5cd4102b20 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cdb6c8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410e350 con 0x7f5cd4103d60 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cdb6c8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410e650 con 0x7f5cd4108a60 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 0x7f5cd419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:29.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 0x7f5cd419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42692/0 (socket says 172.21.15.19:42692) 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 -- 172.21.15.19:0/446719830 learned_addr learned my addr 172.21.15.19:0/446719830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9c65640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cd4108a60 0x7f5cd41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd8c63640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4103d60 0x7f5cd419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 -- 172.21.15.19:0/446719830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cd4108a60 msgr2=0x7f5cd41a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cd4108a60 0x7f5cd41a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 -- 172.21.15.19:0/446719830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4103d60 msgr2=0x7f5cd419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4103d60 0x7f5cd419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 -- 172.21.15.19:0/446719830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd41a1c80 con 0x7f5cd4102b20 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd8c63640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4103d60 0x7f5cd419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.454+0000 7f5cd9464640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 0x7f5cd419a1f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.455+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0017070 con 0x7f5cd4102b20 2024-05-17T09:19:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.455+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41a1f70 con 0x7f5cd4102b20 2024-05-17T09:19:29.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.455+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cd0009c30 con 0x7f5cd4102b20 2024-05-17T09:19:29.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.455+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003c90 con 0x7f5cd4102b20 2024-05-17T09:19:29.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.455+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a23d0 con 0x7f5cd4102b20 2024-05-17T09:19:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.456+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd41a2630 con 0x7f5cd4102b20 2024-05-17T09:19:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.457+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5cd0003e30 con 0x7f5cd4102b20 2024-05-17T09:19:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.458+0000 7f5cca7fc640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc00b28a0 0x7f5cc00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.458+0000 7f5cd8c63640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc00b28a0 0x7f5cc00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.458+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5cd00e2af0 con 0x7f5cd4102b20 2024-05-17T09:19:29.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.458+0000 7f5cd8c63640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc00b28a0 0x7f5cc00b4db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4006010 tx=0x7f5cc4031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:29.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.462+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cd0020030 con 0x7f5cd4102b20 2024-05-17T09:19:29.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:29.694+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f5cd41a2800 con 0x7f5cd4102b20 2024-05-17T09:19:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.378+0000 7f5cca7fc640 1 -- 172.21.15.19:0/446719830 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f5cd00aa400 con 0x7f5cd4102b20 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc00b28a0 msgr2=0x7f5cc00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc00b28a0 0x7f5cc00b4db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4006010 tx=0x7f5cc4031420 comp rx=0 tx=0).stop 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 msgr2=0x7f5cd419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 0x7f5cd419a1f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd000dde0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 shutdown_connections 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cd4108a60 0x7f5cd41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cd4103d60 0x7f5cd419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc00b28a0 0x7f5cc00b4db0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 --2- 172.21.15.19:0/446719830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd4102b20 0x7f5cd419a1f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 >> 172.21.15.19:0/446719830 conn(0x7f5cd40fdd10 msgr2=0x7f5cd410a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:30.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.381+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 shutdown_connections 2024-05-17T09:19:30.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.382+0000 7f5cdb6c8640 1 -- 172.21.15.19:0/446719830 wait complete. 2024-05-17T09:19:30.396 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-17T09:19:30.407 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-17T09:19:30.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 -- 172.21.15.19:0/3689942591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c108a70 msgr2=0x7f977c10ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 --2- 172.21.15.19:0/3689942591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c108a70 0x7f977c10ae90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9774009ca0 tx=0x7f977402a8c0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 -- 172.21.15.19:0/3689942591 shutdown_connections 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 --2- 172.21.15.19:0/3689942591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c108a70 0x7f977c10ae90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 --2- 172.21.15.19:0/3689942591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f977c103d70 0x7f977c104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 --2- 172.21.15.19:0/3689942591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f977c102b30 0x7f977c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.543+0000 7f9783484640 1 -- 172.21.15.19:0/3689942591 >> 172.21.15.19:0/3689942591 conn(0x7f977c0fdd20 msgr2=0x7f977c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 -- 172.21.15.19:0/3689942591 shutdown_connections 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 -- 172.21.15.19:0/3689942591 wait complete. 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 Processor -- start 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 -- start start 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f977c102b30 0x7f977c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 0x7f977c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f977c108a70 0x7f977c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.544+0000 7f9783484640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10f670 con 0x7f977c103d70 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9783484640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10f4f0 con 0x7f977c102b30 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9783484640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10f7f0 con 0x7f977c108a70 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 0x7f977c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 0x7f977c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42706/0 (socket says 172.21.15.19:42706) 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 -- 172.21.15.19:0/186648007 learned_addr learned my addr 172.21.15.19:0/186648007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9781220640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f977c102b30 0x7f977c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9781a21640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f977c108a70 0x7f977c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 -- 172.21.15.19:0/186648007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f977c108a70 msgr2=0x7f977c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f977c108a70 0x7f977c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 -- 172.21.15.19:0/186648007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f977c102b30 msgr2=0x7f977c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f977c102b30 0x7f977c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 -- 172.21.15.19:0/186648007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1a1ae0 con 0x7f977c103d70 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9780a1f640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 0x7f977c19a590 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f977000c920 tx=0x7f977000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770007c70 con 0x7f977c103d70 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9770007e10 con 0x7f977c103d70 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1a1dd0 con 0x7f977c103d70 2024-05-17T09:19:30.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770018710 con 0x7f977c103d70 2024-05-17T09:19:30.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.545+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1a2290 con 0x7f977c103d70 2024-05-17T09:19:30.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.547+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c107170 con 0x7f977c103d70 2024-05-17T09:19:30.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.548+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f977000f450 con 0x7f977c103d70 2024-05-17T09:19:30.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.549+0000 7f976e7fc640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97680b2920 0x7f97680b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.549+0000 7f9781220640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97680b2920 0x7f97680b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.549+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f97700e2800 con 0x7f977c103d70 2024-05-17T09:19:30.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.549+0000 7f9781220640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97680b2920 0x7f97680b4e30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97780058c0 tx=0x7f9778005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:30.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.554+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9770014030 con 0x7f977c103d70 2024-05-17T09:19:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:30 smithi069 ceph-mon[35081]: pgmap v430: 129 pgs: 129 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-17T09:19:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:30 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/446719830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T09:19:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:30 smithi019 ceph-mon[31620]: pgmap v430: 129 pgs: 129 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-17T09:19:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:30 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/446719830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T09:19:30.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.749+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f977c0638f0 con 0x7f977c103d70 2024-05-17T09:19:30.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.751+0000 7f976e7fc640 1 -- 172.21.15.19:0/186648007 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f97700aa110 con 0x7f977c103d70 2024-05-17T09:19:30.751 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:30.752 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:19:30.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97680b2920 msgr2=0x7f97680b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97680b2920 0x7f97680b4e30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97780058c0 tx=0x7f9778005850 comp rx=0 tx=0).stop 2024-05-17T09:19:30.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 msgr2=0x7f977c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 0x7f977c19a590 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f977000c920 tx=0x7f977000cde0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 shutdown_connections 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f977c108a70 0x7f977c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f97680b2920 0x7f97680b4e30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f977c103d70 0x7f977c19a590 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 --2- 172.21.15.19:0/186648007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f977c102b30 0x7f977c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 >> 172.21.15.19:0/186648007 conn(0x7f977c0fdd20 msgr2=0x7f977c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 shutdown_connections 2024-05-17T09:19:30.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.753+0000 7f9783484640 1 -- 172.21.15.19:0/186648007 wait complete. 2024-05-17T09:19:30.765 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T09:10:10.865915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-17T09:15:01.931223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-17T09:15:05.954895+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-17T09:15:18.012606+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-17T09:19:30.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T09:19:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:30 smithi079 ceph-mon[35502]: pgmap v430: 129 pgs: 129 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-17T09:19:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:30 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/446719830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T09:19:30.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/1835755835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0108a80 msgr2=0x7f9ee010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/1835755835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0108a80 0x7f9ee010aea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009ca0 tx=0x7f9ed802a8c0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/1835755835 shutdown_connections 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/1835755835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0108a80 0x7f9ee010aea0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/1835755835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ee0103d80 0x7f9ee0104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/1835755835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee0102b40 0x7f9ee0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/1835755835 >> 172.21.15.19:0/1835755835 conn(0x7f9ee00fdd10 msgr2=0x7f9ee01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/1835755835 shutdown_connections 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/1835755835 wait complete. 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 Processor -- start 2024-05-17T09:19:30.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.883+0000 7f9ee6d5f640 1 -- start start 2024-05-17T09:19:30.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee6d5f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee0102b40 0x7f9ee00ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee6d5f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 0x7f9ee00fe240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee6d5f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ee0108a80 0x7f9ee00fe780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee6d5f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee010e570 con 0x7f9ee0103d80 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee6d5f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee010e3f0 con 0x7f9ee0108a80 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee6d5f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee010e6f0 con 0x7f9ee0102b40 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 0x7f9ee00fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 0x7f9ee00fe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42722/0 (socket says 172.21.15.19:42722) 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 -- 172.21.15.19:0/3480070350 learned_addr learned my addr 172.21.15.19:0/3480070350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee4afb640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee0102b40 0x7f9ee00ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 -- 172.21.15.19:0/3480070350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee0102b40 msgr2=0x7f9ee00ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee0102b40 0x7f9ee00ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 -- 172.21.15.19:0/3480070350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ee0108a80 msgr2=0x7f9ee00fe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9ee52fc640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ee0108a80 0x7f9ee00fe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ee0108a80 0x7f9ee00fe780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 -- 172.21.15.19:0/3480070350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00feeb0 con 0x7f9ee0103d80 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9edffff640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 0x7f9ee00fe240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400d950 tx=0x7f9ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.884+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4017070 con 0x7f9ee0103d80 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.885+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ed4009c30 con 0x7f9ee0103d80 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.885+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400f620 con 0x7f9ee0103d80 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.885+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee01a4270 con 0x7f9ee0103d80 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.885+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee01a4760 con 0x7f9ee0103d80 2024-05-17T09:19:30.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.886+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee01a49c0 con 0x7f9ee0103d80 2024-05-17T09:19:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.887+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9ed4010460 con 0x7f9ee0103d80 2024-05-17T09:19:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.888+0000 7f9eddffb640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec80b2870 0x7f9ec80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.888+0000 7f9ee4afb640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec80b2870 0x7f9ec80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.888+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9ed40e21d0 con 0x7f9ee0103d80 2024-05-17T09:19:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.888+0000 7f9ee4afb640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec80b2870 0x7f9ec80b4d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0103740 tx=0x7f9ed0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:30.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:30.893+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ed40a9ae0 con 0x7f9ee0103d80 2024-05-17T09:19:31.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.085+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ee01a4b90 con 0x7f9ee0103d80 2024-05-17T09:19:31.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.085+0000 7f9eddffb640 1 -- 172.21.15.19:0/3480070350 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f9ed4009de0 con 0x7f9ee0103d80 2024-05-17T09:19:31.085 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-17T09:19:31.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec80b2870 msgr2=0x7f9ec80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec80b2870 0x7f9ec80b4d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0103740 tx=0x7f9ed0005f90 comp rx=0 tx=0).stop 2024-05-17T09:19:31.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 msgr2=0x7f9ee00fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 0x7f9ee00fe240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400d950 tx=0x7f9ed400de10 comp rx=0 tx=0).stop 2024-05-17T09:19:31.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 shutdown_connections 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ee0108a80 0x7f9ee00fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec80b2870 0x7f9ec80b4d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee0103d80 0x7f9ee00fe240 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 --2- 172.21.15.19:0/3480070350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ee0102b40 0x7f9ee00ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.087+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 >> 172.21.15.19:0/3480070350 conn(0x7f9ee00fdd10 msgr2=0x7f9ee00757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.088+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 shutdown_connections 2024-05-17T09:19:31.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.088+0000 7f9ee6d5f640 1 -- 172.21.15.19:0/3480070350 wait complete. 2024-05-17T09:19:31.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-17T09:19:31.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 -- 172.21.15.19:0/3158525950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c50101fd0 msgr2=0x7f7c501023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 --2- 172.21.15.19:0/3158525950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c50101fd0 0x7f7c501023e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009900 tx=0x7f7c4c02a410 comp rx=0 tx=0).stop 2024-05-17T09:19:31.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 -- 172.21.15.19:0/3158525950 shutdown_connections 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 --2- 172.21.15.19:0/3158525950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c501098d0 0x7f7c5010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 --2- 172.21.15.19:0/3158525950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c50106d60 0x7f7c50109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 --2- 172.21.15.19:0/3158525950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c50101fd0 0x7f7c501023e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 -- 172.21.15.19:0/3158525950 >> 172.21.15.19:0/3158525950 conn(0x7f7c500fd4a0 msgr2=0x7f7c500ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 -- 172.21.15.19:0/3158525950 shutdown_connections 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 -- 172.21.15.19:0/3158525950 wait complete. 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 Processor -- start 2024-05-17T09:19:31.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.217+0000 7f7c57776640 1 -- start start 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c57776640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c50101fd0 0x7f7c5019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c57776640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c50106d60 0x7f7c5019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c57776640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 0x7f7c501a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c57776640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5010f5c0 con 0x7f7c501098d0 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c57776640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5010f440 con 0x7f7c50106d60 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c57776640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5010f740 con 0x7f7c50101fd0 2024-05-17T09:19:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 0x7f7c501a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 0x7f7c501a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42752/0 (socket says 172.21.15.19:42752) 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 -- 172.21.15.19:0/3435786506 learned_addr learned my addr 172.21.15.19:0/3435786506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c54d11640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c50106d60 0x7f7c5019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55512640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c50101fd0 0x7f7c5019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 -- 172.21.15.19:0/3435786506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c50101fd0 msgr2=0x7f7c5019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c50101fd0 0x7f7c5019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 -- 172.21.15.19:0/3435786506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c50106d60 msgr2=0x7f7c5019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c50106d60 0x7f7c5019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 -- 172.21.15.19:0/3435786506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c501a1da0 con 0x7f7c501098d0 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.218+0000 7f7c55d13640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 0x7f7c501a1730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009fd0 tx=0x7f7c4800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:31.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.219+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48017070 con 0x7f7c501098d0 2024-05-17T09:19:31.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.219+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c48003e00 con 0x7f7c501098d0 2024-05-17T09:19:31.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.219+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c501a2090 con 0x7f7c501098d0 2024-05-17T09:19:31.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.219+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48010460 con 0x7f7c501098d0 2024-05-17T09:19:31.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.219+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501a25e0 con 0x7f7c501098d0 2024-05-17T09:19:31.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.220+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c500fefc0 con 0x7f7c501098d0 2024-05-17T09:19:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.221+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7c48003970 con 0x7f7c501098d0 2024-05-17T09:19:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.222+0000 7f7c467fc640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c3c0b2870 0x7f7c3c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.222+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7c480e25c0 con 0x7f7c501098d0 2024-05-17T09:19:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.222+0000 7f7c55512640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c3c0b2870 0x7f7c3c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.223+0000 7f7c55512640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c3c0b2870 0x7f7c3c0b4d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c005760 tx=0x7f7c4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.226+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c480a9960 con 0x7f7c501098d0 2024-05-17T09:19:31.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.418+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c500638c0 con 0x7f7c501098d0 2024-05-17T09:19:31.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.418+0000 7f7c467fc640 1 -- 172.21.15.19:0/3435786506 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f7c480adc60 con 0x7f7c501098d0 2024-05-17T09:19:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-17T09:19:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.420+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c3c0b2870 msgr2=0x7f7c3c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.420+0000 7f7c57776640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c3c0b2870 0x7f7c3c0b4d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c005760 tx=0x7f7c4c000bc0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.420+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 msgr2=0x7f7c501a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.420+0000 7f7c57776640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 0x7f7c501a1730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009fd0 tx=0x7f7c4800b260 comp rx=0 tx=0).stop 2024-05-17T09:19:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 shutdown_connections 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c3c0b2870 0x7f7c3c0b4d80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c501098d0 0x7f7c501a1730 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c50106d60 0x7f7c5019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 --2- 172.21.15.19:0/3435786506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c50101fd0 0x7f7c5019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 >> 172.21.15.19:0/3435786506 conn(0x7f7c500fd4a0 msgr2=0x7f7c501078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 shutdown_connections 2024-05-17T09:19:31.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.421+0000 7f7c57776640 1 -- 172.21.15.19:0/3435786506 wait complete. 2024-05-17T09:19:31.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 -- 172.21.15.19:0/4164767682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50102b20 msgr2=0x7f8f50103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4164767682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50102b20 0x7f8f50103820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009900 tx=0x7f8f4c02a410 comp rx=0 tx=0).stop 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 -- 172.21.15.19:0/4164767682 shutdown_connections 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4164767682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50108a60 0x7f8f5010ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4164767682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 0x7f8f50104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4164767682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50102b20 0x7f8f50103820 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 -- 172.21.15.19:0/4164767682 >> 172.21.15.19:0/4164767682 conn(0x7f8f500fdd50 msgr2=0x7f8f501001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 -- 172.21.15.19:0/4164767682 shutdown_connections 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.549+0000 7f8f5792e640 1 -- 172.21.15.19:0/4164767682 wait complete. 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 Processor -- start 2024-05-17T09:19:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 -- start start 2024-05-17T09:19:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50102b20 0x7f8f5019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 0x7f8f5019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 0x7f8f501a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5010f600 con 0x7f8f50108a60 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5010f480 con 0x7f8f50102b20 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f5792e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5010f780 con 0x7f8f50103d60 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 0x7f8f501a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 0x7f8f501a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42778/0 (socket says 172.21.15.19:42778) 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 -- 172.21.15.19:0/4205322135 learned_addr learned my addr 172.21.15.19:0/4205322135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f54ec9640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 0x7f8f5019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f556ca640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50102b20 0x7f8f5019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 -- 172.21.15.19:0/4205322135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 msgr2=0x7f8f5019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 0x7f8f5019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 -- 172.21.15.19:0/4205322135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50102b20 msgr2=0x7f8f5019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50102b20 0x7f8f5019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 -- 172.21.15.19:0/4205322135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f501a1dc0 con 0x7f8f50108a60 2024-05-17T09:19:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f54ec9640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 0x7f8f5019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f556ca640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50102b20 0x7f8f5019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.550+0000 7f8f55ecb640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 0x7f8f501a1750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800eaa0 tx=0x7f8f4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.551+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800cdb0 con 0x7f8f50108a60 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.551+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f501a20b0 con 0x7f8f50108a60 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.551+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f48005020 con 0x7f8f50108a60 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.551+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f480116d0 con 0x7f8f50108a60 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.551+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f501a2600 con 0x7f8f50108a60 2024-05-17T09:19:31.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.552+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f50107870 con 0x7f8f50108a60 2024-05-17T09:19:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.553+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f480051d0 con 0x7f8f50108a60 2024-05-17T09:19:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.554+0000 7f8f467fc640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f3c0b27f0 0x7f8f3c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.554+0000 7f8f556ca640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f3c0b27f0 0x7f8f3c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.554+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8f480e28c0 con 0x7f8f50108a60 2024-05-17T09:19:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.555+0000 7f8f556ca640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f3c0b27f0 0x7f8f3c0b4d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0057c0 tx=0x7f8f4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.559+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f48016030 con 0x7f8f50108a60 2024-05-17T09:19:31.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/446719830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T09:19:31.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:31 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:19:31.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/186648007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:31.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3480070350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:19:31.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/446719830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T09:19:31.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:31 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:19:31.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/186648007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:31.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3480070350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:19:31.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.749+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f500638c0 con 0x7f8f50108a60 2024-05-17T09:19:31.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.749+0000 7f8f467fc640 1 -- 172.21.15.19:0/4205322135 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f8f480aa1d0 con 0x7f8f50108a60 2024-05-17T09:19:31.750 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f3c0b27f0 msgr2=0x7f8f3c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f3c0b27f0 0x7f8f3c0b4d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0057c0 tx=0x7f8f4c000bc0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 msgr2=0x7f8f501a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 0x7f8f501a1750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800eaa0 tx=0x7f8f4800ef60 comp rx=0 tx=0).stop 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 shutdown_connections 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f3c0b27f0 0x7f8f3c0b4d00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f50108a60 0x7f8f501a1750 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f50103d60 0x7f8f5019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 --2- 172.21.15.19:0/4205322135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f50102b20 0x7f8f5019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.751+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 >> 172.21.15.19:0/4205322135 conn(0x7f8f500fdd50 msgr2=0x7f8f50104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:31.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.752+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 shutdown_connections 2024-05-17T09:19:31.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.752+0000 7f8f5792e640 1 -- 172.21.15.19:0/4205322135 wait complete. 2024-05-17T09:19:31.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-17T09:19:31.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/446719830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T09:19:31.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:31 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:19:31.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/186648007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:31.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3480070350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/901234473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4108a30 msgr2=0x7f6cc410ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/901234473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4108a30 0x7f6cc410ae50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009900 tx=0x7f6cbc02a410 comp rx=0 tx=0).stop 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/901234473 shutdown_connections 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/901234473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4108a30 0x7f6cc410ae50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/901234473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc4103d30 0x7f6cc4104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/901234473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4102af0 0x7f6cc41037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/901234473 >> 172.21.15.19:0/901234473 conn(0x7f6cc40fdce0 msgr2=0x7f6cc4100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/901234473 shutdown_connections 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.881+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/901234473 wait complete. 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 Processor -- start 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 -- start start 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 0x7f6cc419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4103d30 0x7f6cc419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc4108a30 0x7f6cc41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e4d0 con 0x7f6cc4102af0 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e350 con 0x7f6cc4108a30 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6ccb9dc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e650 con 0x7f6cc4103d30 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 0x7f6cc419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 0x7f6cc419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42790/0 (socket says 172.21.15.19:42790) 2024-05-17T09:19:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 -- 172.21.15.19:0/1741188421 learned_addr learned my addr 172.21.15.19:0/1741188421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9f79640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc4108a30 0x7f6cc41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc8f77640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4103d30 0x7f6cc419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 -- 172.21.15.19:0/1741188421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4103d30 msgr2=0x7f6cc419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4103d30 0x7f6cc419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 -- 172.21.15.19:0/1741188421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc4108a30 msgr2=0x7f6cc41a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc4108a30 0x7f6cc41a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc9778640 1 -- 172.21.15.19:0/1741188421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc41a1c80 con 0x7f6cc4102af0 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.882+0000 7f6cc8f77640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4103d30 0x7f6cc419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.883+0000 7f6cc9778640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 0x7f6cc419a1f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000ead0 tx=0x7f6cc000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.883+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000cde0 con 0x7f6cc4102af0 2024-05-17T09:19:31.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.883+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cc0003e00 con 0x7f6cc4102af0 2024-05-17T09:19:31.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.883+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc41a1f70 con 0x7f6cc4102af0 2024-05-17T09:19:31.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.883+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00116d0 con 0x7f6cc4102af0 2024-05-17T09:19:31.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.883+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc41a2380 con 0x7f6cc4102af0 2024-05-17T09:19:31.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.884+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc41a2640 con 0x7f6cc4102af0 2024-05-17T09:19:31.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.886+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6cc0003970 con 0x7f6cc4102af0 2024-05-17T09:19:31.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.887+0000 7f6cba7fc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb00b27f0 0x7f6cb00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:31.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.887+0000 7f6cc8f77640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb00b27f0 0x7f6cb00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:31.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.887+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6cc00e2d40 con 0x7f6cc4102af0 2024-05-17T09:19:31.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.888+0000 7f6cc8f77640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb00b27f0 0x7f6cb00b4d00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cc419b290 tx=0x7f6cb4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:31.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:31.891+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6cc0015030 con 0x7f6cc4102af0 2024-05-17T09:19:32.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.088+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6cc41a2810 con 0x7f6cc4102af0 2024-05-17T09:19:32.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.088+0000 7f6cba7fc640 1 -- 172.21.15.19:0/1741188421 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f6cc00aa060 con 0x7f6cc4102af0 2024-05-17T09:19:32.088 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb00b27f0 msgr2=0x7f6cb00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb00b27f0 0x7f6cb00b4d00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cc419b290 tx=0x7f6cb4005db0 comp rx=0 tx=0).stop 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 msgr2=0x7f6cc419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 0x7f6cc419a1f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000ead0 tx=0x7f6cc000ef90 comp rx=0 tx=0).stop 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 shutdown_connections 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc4108a30 0x7f6cc41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cc4103d30 0x7f6cc419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb00b27f0 0x7f6cb00b4d00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 --2- 172.21.15.19:0/1741188421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cc4102af0 0x7f6cc419a1f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 >> 172.21.15.19:0/1741188421 conn(0x7f6cc40fdce0 msgr2=0x7f6cc406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 shutdown_connections 2024-05-17T09:19:32.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:32.090+0000 7f6ccb9dc640 1 -- 172.21.15.19:0/1741188421 wait complete. 2024-05-17T09:19:32.103 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-17T09:19:32.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:19:32.359 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:32.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:32 smithi019 ceph-mon[31620]: pgmap v431: 129 pgs: 129 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-17T09:19:32.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:32 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3435786506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:32 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4205322135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:32 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1741188421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:32 smithi069 ceph-mon[35081]: pgmap v431: 129 pgs: 129 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-17T09:19:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:32 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3435786506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:32 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4205322135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:32 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1741188421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:32 smithi079 ceph-mon[35502]: pgmap v431: 129 pgs: 129 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-17T09:19:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:32 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3435786506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:32 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4205322135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T09:19:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:32 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1741188421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 -- 172.21.15.19:0/1402250686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4107e00 msgr2=0x7fa3f4108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1402250686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4107e00 0x7fa3f4108210 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80099c0 tx=0x7fa3e8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 -- 172.21.15.19:0/1402250686 shutdown_connections 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1402250686 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f4108750 0x7fa3f41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1402250686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3f41034d0 0x7fa3f41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1402250686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4107e00 0x7fa3f4108210 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 -- 172.21.15.19:0/1402250686 >> 172.21.15.19:0/1402250686 conn(0x7fa3f40fef20 msgr2=0x7fa3f4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 -- 172.21.15.19:0/1402250686 shutdown_connections 2024-05-17T09:19:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.267+0000 7fa3fa058700 1 -- 172.21.15.19:0/1402250686 wait complete. 2024-05-17T09:19:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 Processor -- start 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 -- start start 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3f41034d0 0x7fa3f419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f4107e00 0x7fa3f419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 0x7fa3f41a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113210 con 0x7fa3f4108750 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113090 con 0x7fa3f41034d0 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3fa058700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113390 con 0x7fa3f4107e00 2024-05-17T09:19:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 0x7fa3f41a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 0x7fa3f41a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35204/0 (socket says 172.21.15.19:35204) 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.268+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1749687769 learned_addr learned my addr 172.21.15.19:0/1749687769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f2ffd700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f4107e00 0x7fa3f419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f37fe700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3f41034d0 0x7fa3f419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1749687769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f4107e00 msgr2=0x7fa3f419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f3fff700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f4107e00 0x7fa3f419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1749687769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3f41034d0 msgr2=0x7fa3f419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f3fff700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3f41034d0 0x7fa3f419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1749687769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41a1e90 con 0x7fa3f4108750 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f3fff700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 0x7fa3f41a17f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00d980 tx=0x7fa3ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec017070 con 0x7fa3f4108750 2024-05-17T09:19:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f41a2180 con 0x7fa3f4108750 2024-05-17T09:19:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3ec005020 con 0x7fa3f4108750 2024-05-17T09:19:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.269+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00f980 con 0x7fa3f4108750 2024-05-17T09:19:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.270+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a9aa0 con 0x7fa3f4108750 2024-05-17T09:19:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.271+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f4104980 con 0x7fa3f4108750 2024-05-17T09:19:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.272+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa3ec004a30 con 0x7fa3f4108750 2024-05-17T09:19:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.273+0000 7fa3f0ff9700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.273+0000 7fa3f37fe700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.273+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa3ec0e22c0 con 0x7fa3f4108750 2024-05-17T09:19:34.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.274+0000 7fa3f37fe700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009550 tx=0x7fa3e40091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:34.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.278+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3ec020020 con 0x7fa3f4108750 2024-05-17T09:19:34.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.546+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f402e5e0 con 0x7fa3f4108750 2024-05-17T09:19:34.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:34 smithi019 ceph-mon[31620]: pgmap v432: 129 pgs: 129 active+clean; 646 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:19:34.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.547+0000 7fa3f0ff9700 1 -- 172.21.15.19:0/1749687769 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fa3ec0a9bd0 con 0x7fa3f4108750 2024-05-17T09:19:34.548 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:34.548 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3dc0b2900 msgr2=0x7fa3dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009550 tx=0x7fa3e40091f0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 msgr2=0x7fa3f41a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 0x7fa3f41a17f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00d980 tx=0x7fa3ec00dc90 comp rx=0 tx=0).stop 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 shutdown_connections 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f4108750 0x7fa3f41a17f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3f41034d0 0x7fa3f419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 --2- 172.21.15.19:0/1749687769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3f4107e00 0x7fa3f419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.549+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 >> 172.21.15.19:0/1749687769 conn(0x7fa3f40fef20 msgr2=0x7fa3f41097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.550+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 shutdown_connections 2024-05-17T09:19:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:34.550+0000 7fa3fa058700 1 -- 172.21.15.19:0/1749687769 wait complete. 2024-05-17T09:19:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:19:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:34 smithi069 ceph-mon[35081]: pgmap v432: 129 pgs: 129 active+clean; 646 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:19:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:34 smithi079 ceph-mon[35502]: pgmap v432: 129 pgs: 129 active+clean; 646 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:19:35.193 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-17T09:19:35.193 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-17T09:19:35.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 -- 172.21.15.19:0/367082470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc103c50 msgr2=0x7fabfc104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 --2- 172.21.15.19:0/367082470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc103c50 0x7fabfc104950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fabec009900 tx=0x7fabec02a410 comp rx=0 tx=0).stop 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 -- 172.21.15.19:0/367082470 shutdown_connections 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 --2- 172.21.15.19:0/367082470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc109b90 0x7fabfc10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 --2- 172.21.15.19:0/367082470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabfc104e90 0x7fabfc1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 --2- 172.21.15.19:0/367082470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc103c50 0x7fabfc104950 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 -- 172.21.15.19:0/367082470 >> 172.21.15.19:0/367082470 conn(0x7fabfc0fee60 msgr2=0x7fabfc1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 -- 172.21.15.19:0/367082470 shutdown_connections 2024-05-17T09:19:35.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.311+0000 7fac01027640 1 -- 172.21.15.19:0/367082470 wait complete. 2024-05-17T09:19:35.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 Processor -- start 2024-05-17T09:19:35.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 -- start start 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabfc103c50 0x7fabfc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc104e90 0x7fabfc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 0x7fabfc1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10f600 con 0x7fabfc109b90 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10f480 con 0x7fabfc103c50 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.312+0000 7fac01027640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10f780 con 0x7fabfc104e90 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 0x7fabfc1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 0x7fabfc1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35212/0 (socket says 172.21.15.19:35212) 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 -- 172.21.15.19:0/2398015785 learned_addr learned my addr 172.21.15.19:0/2398015785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfad9d640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabfc103c50 0x7fabfc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabf259c640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc104e90 0x7fabfc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 -- 172.21.15.19:0/2398015785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc104e90 msgr2=0x7fabfc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc104e90 0x7fabfc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 -- 172.21.15.19:0/2398015785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabfc103c50 msgr2=0x7fabfc19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabfc103c50 0x7fabfc19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 -- 172.21.15.19:0/2398015785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc1a1d10 con 0x7fabfc109b90 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabf259c640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc104e90 0x7fabfc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabfb59e640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 0x7fabfc1a16a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fabf400eb10 tx=0x7fabf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4004b20 con 0x7fabfc109b90 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf4004cc0 con 0x7fabfc109b90 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc1a1fa0 con 0x7fabfc109b90 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4010690 con 0x7fabfc109b90 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.313+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a2460 con 0x7fabfc109b90 2024-05-17T09:19:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.314+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc1089a0 con 0x7fabfc109b90 2024-05-17T09:19:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.316+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fabf4005000 con 0x7fabfc109b90 2024-05-17T09:19:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.317+0000 7fabf8d99640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabe80b28a0 0x7fabe80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.317+0000 7fabfad9d640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabe80b28a0 0x7fabe80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.317+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fabf40e2dd0 con 0x7fabfc109b90 2024-05-17T09:19:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.317+0000 7fabfad9d640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabe80b28a0 0x7fabe80b4db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fabec005760 tx=0x7fabec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.320+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabf4054df0 con 0x7fabfc109b90 2024-05-17T09:19:35.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:35 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1749687769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.511+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabfc063100 con 0x7fabe80b28a0 2024-05-17T09:19:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.520+0000 7fabf8d99640 1 -- 172.21.15.19:0/2398015785 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340732 (secure 0 0 0) 0x7fabfc063100 con 0x7fabe80b28a0 2024-05-17T09:19:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabe80b28a0 msgr2=0x7fabe80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabe80b28a0 0x7fabe80b4db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fabec005760 tx=0x7fabec000bc0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 msgr2=0x7fabfc1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 0x7fabfc1a16a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fabf400eb10 tx=0x7fabf400c2d0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 shutdown_connections 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fabe80b28a0 0x7fabe80b4db0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabfc109b90 0x7fabfc1a16a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fabfc104e90 0x7fabfc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 --2- 172.21.15.19:0/2398015785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabfc103c50 0x7fabfc19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 >> 172.21.15.19:0/2398015785 conn(0x7fabfc0fee60 msgr2=0x7fabfc105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 shutdown_connections 2024-05-17T09:19:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.524+0000 7fac01027640 1 -- 172.21.15.19:0/2398015785 wait complete. 2024-05-17T09:19:35.536 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-05-17T09:19:34.948495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1276,"num_read_kb":746,"num_write":313,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348648,"kb_used_data":11976,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593176,"statfs":{"total":1151940427776,"available":1151583412224,"internally_reserved":0,"allocated":12263424,"data_stored":8144780,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":48,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003881"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906823+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906823+0000","last_peered":"2024-05-17T09:17:46.906823+0000","last_clean":"2024-05-17T09:17:46.906823+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906823+0000","last_undegraded":"2024-05-17T09:17:46.906823+0000","last_fullsized":"2024-05-17T09:17:46.906823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901351+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.901351+0000","last_peered":"2024-05-17T09:17:46.901351+0000","last_clean":"2024-05-17T09:17:46.901351+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.901351+0000","last_undegraded":"2024-05-17T09:17:46.901351+0000","last_fullsized":"2024-05-17T09:17:46.901351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905015+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.905015+0000","last_peered":"2024-05-17T09:17:46.905015+0000","last_clean":"2024-05-17T09:17:46.905015+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.905015+0000","last_undegraded":"2024-05-17T09:17:46.905015+0000","last_fullsized":"2024-05-17T09:17:46.905015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886153+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.886153+0000","last_peered":"2024-05-17T09:17:46.886153+0000","last_clean":"2024-05-17T09:17:46.886153+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.886153+0000","last_undegraded":"2024-05-17T09:17:46.886153+0000","last_fullsized":"2024-05-17T09:17:46.886153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897920+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.897920+0000","last_peered":"2024-05-17T09:17:46.897920+0000","last_clean":"2024-05-17T09:17:46.897920+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.897920+0000","last_undegraded":"2024-05-17T09:17:46.897920+0000","last_fullsized":"2024-05-17T09:17:46.897920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893468+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893468+0000","last_peered":"2024-05-17T09:17:46.893468+0000","last_clean":"2024-05-17T09:17:46.893468+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893468+0000","last_undegraded":"2024-05-17T09:17:46.893468+0000","last_fullsized":"2024-05-17T09:17:46.893468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908147+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908147+0000","last_peered":"2024-05-17T09:17:46.908147+0000","last_clean":"2024-05-17T09:17:46.908147+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908147+0000","last_undegraded":"2024-05-17T09:17:46.908147+0000","last_fullsized":"2024-05-17T09:17:46.908147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889231+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889231+0000","last_peered":"2024-05-17T09:17:46.889231+0000","last_clean":"2024-05-17T09:17:46.889231+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889231+0000","last_undegraded":"2024-05-17T09:17:46.889231+0000","last_fullsized":"2024-05-17T09:17:46.889231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934972+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:17:11.934972+0000","last_peered":"2024-05-17T09:17:11.934972+0000","last_clean":"2024-05-17T09:17:11.934972+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:17:11.934972+0000","last_undegraded":"2024-05-17T09:17:11.934972+0000","last_fullsized":"2024-05-17T09:17:11.934972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898655+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898655+0000","last_peered":"2024-05-17T09:17:46.898655+0000","last_clean":"2024-05-17T09:17:46.898655+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898655+0000","last_undegraded":"2024-05-17T09:17:46.898655+0000","last_fullsized":"2024-05-17T09:17:46.898655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:05.632115+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:17:05.632115+0000","last_peered":"2024-05-17T09:17:05.632115+0000","last_clean":"2024-05-17T09:17:05.632115+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:17:05.632115+0000","last_undegraded":"2024-05-17T09:17:05.632115+0000","last_fullsized":"2024-05-17T09:17:05.632115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905191+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905191+0000","last_peered":"2024-05-17T09:17:46.905191+0000","last_clean":"2024-05-17T09:17:46.905191+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905191+0000","last_undegraded":"2024-05-17T09:17:46.905191+0000","last_fullsized":"2024-05-17T09:17:46.905191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:30.536421+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:19:30.536421+0000","last_peered":"2024-05-17T09:19:30.536421+0000","last_clean":"2024-05-17T09:19:30.536421+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:19:30.536421+0000","last_undegraded":"2024-05-17T09:19:30.536421+0000","last_fullsized":"2024-05-17T09:19:30.536421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112225+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:17:24.112225+0000","last_peered":"2024-05-17T09:17:24.112225+0000","last_clean":"2024-05-17T09:17:24.112225+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:17:24.112225+0000","last_undegraded":"2024-05-17T09:17:24.112225+0000","last_fullsized":"2024-05-17T09:17:24.112225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.430695+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:17:23.430695+0000","last_peered":"2024-05-17T09:17:23.430695+0000","last_clean":"2024-05-17T09:17:23.430695+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:17:23.430695+0000","last_undegraded":"2024-05-17T09:17:23.430695+0000","last_fullsized":"2024-05-17T09:17:23.430695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:32.538829+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:19:32.538829+0000","last_peered":"2024-05-17T09:19:32.538829+0000","last_clean":"2024-05-17T09:19:32.538829+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:19:32.538829+0000","last_undegraded":"2024-05-17T09:19:32.538829+0000","last_fullsized":"2024-05-17T09:19:32.538829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":486,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:33.541041+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:19:33.541041+0000","last_peered":"2024-05-17T09:19:33.541041+0000","last_clean":"2024-05-17T09:19:33.541041+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:19:33.541041+0000","last_undegraded":"2024-05-17T09:19:33.541041+0000","last_fullsized":"2024-05-17T09:19:33.541041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":269,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903452+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903452+0000","last_peered":"2024-05-17T09:17:46.903452+0000","last_clean":"2024-05-17T09:17:46.903452+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903452+0000","last_undegraded":"2024-05-17T09:17:46.903452+0000","last_fullsized":"2024-05-17T09:17:46.903452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423686+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423686+0000","last_peered":"2024-05-17T09:16:48.423686+0000","last_clean":"2024-05-17T09:16:48.423686+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423686+0000","last_undegraded":"2024-05-17T09:16:48.423686+0000","last_fullsized":"2024-05-17T09:16:48.423686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904979+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.904979+0000","last_peered":"2024-05-17T09:17:46.904979+0000","last_clean":"2024-05-17T09:17:46.904979+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.904979+0000","last_undegraded":"2024-05-17T09:17:46.904979+0000","last_fullsized":"2024-05-17T09:17:46.904979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":247,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:29.533641+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:19:29.533641+0000","last_peered":"2024-05-17T09:19:29.533641+0000","last_clean":"2024-05-17T09:19:29.533641+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:19:29.533641+0000","last_undegraded":"2024-05-17T09:19:29.533641+0000","last_fullsized":"2024-05-17T09:19:29.533641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900338+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.900338+0000","last_peered":"2024-05-17T09:17:46.900338+0000","last_clean":"2024-05-17T09:17:46.900338+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.900338+0000","last_undegraded":"2024-05-17T09:17:46.900338+0000","last_fullsized":"2024-05-17T09:17:46.900338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829668+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829668+0000","last_peered":"2024-05-17T09:16:48.829668+0000","last_clean":"2024-05-17T09:16:48.829668+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829668+0000","last_undegraded":"2024-05-17T09:16:48.829668+0000","last_fullsized":"2024-05-17T09:16:48.829668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423264+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423264+0000","last_peered":"2024-05-17T09:16:48.423264+0000","last_clean":"2024-05-17T09:16:48.423264+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423264+0000","last_undegraded":"2024-05-17T09:16:48.423264+0000","last_fullsized":"2024-05-17T09:16:48.423264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897951+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.897951+0000","last_peered":"2024-05-17T09:17:46.897951+0000","last_clean":"2024-05-17T09:17:46.897951+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.897951+0000","last_undegraded":"2024-05-17T09:17:46.897951+0000","last_fullsized":"2024-05-17T09:17:46.897951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.939280+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:17:28.939280+0000","last_peered":"2024-05-17T09:17:28.939280+0000","last_clean":"2024-05-17T09:17:28.939280+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:17:28.939280+0000","last_undegraded":"2024-05-17T09:17:28.939280+0000","last_fullsized":"2024-05-17T09:17:28.939280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934940+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:17:11.934940+0000","last_peered":"2024-05-17T09:17:11.934940+0000","last_clean":"2024-05-17T09:17:11.934940+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:17:11.934940+0000","last_undegraded":"2024-05-17T09:17:11.934940+0000","last_fullsized":"2024-05-17T09:17:11.934940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904550+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904550+0000","last_peered":"2024-05-17T09:17:46.904550+0000","last_clean":"2024-05-17T09:17:46.904550+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904550+0000","last_undegraded":"2024-05-17T09:17:46.904550+0000","last_fullsized":"2024-05-17T09:17:46.904550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:27.771474+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:17:27.771474+0000","last_peered":"2024-05-17T09:17:27.771474+0000","last_clean":"2024-05-17T09:17:27.771474+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:17:27.771474+0000","last_undegraded":"2024-05-17T09:17:27.771474+0000","last_fullsized":"2024-05-17T09:17:27.771474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.384811+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:17:08.384811+0000","last_peered":"2024-05-17T09:17:08.384811+0000","last_clean":"2024-05-17T09:17:08.384811+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:17:08.384811+0000","last_undegraded":"2024-05-17T09:17:08.384811+0000","last_fullsized":"2024-05-17T09:17:08.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900921+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.900921+0000","last_peered":"2024-05-17T09:17:46.900921+0000","last_clean":"2024-05-17T09:17:46.900921+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.900921+0000","last_undegraded":"2024-05-17T09:17:46.900921+0000","last_fullsized":"2024-05-17T09:17:46.900921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320360+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320360+0000","last_peered":"2024-05-17T09:16:48.320360+0000","last_clean":"2024-05-17T09:16:48.320360+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320360+0000","last_undegraded":"2024-05-17T09:16:48.320360+0000","last_fullsized":"2024-05-17T09:16:48.320360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901492+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901492+0000","last_peered":"2024-05-17T09:17:46.901492+0000","last_clean":"2024-05-17T09:17:46.901492+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901492+0000","last_undegraded":"2024-05-17T09:17:46.901492+0000","last_fullsized":"2024-05-17T09:17:46.901492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:31.536246+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:19:31.536246+0000","last_peered":"2024-05-17T09:19:31.536246+0000","last_clean":"2024-05-17T09:19:31.536246+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:19:31.536246+0000","last_undegraded":"2024-05-17T09:19:31.536246+0000","last_fullsized":"2024-05-17T09:19:31.536246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387326+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:17:28.387326+0000","last_peered":"2024-05-17T09:17:28.387326+0000","last_clean":"2024-05-17T09:17:28.387326+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:17:28.387326+0000","last_undegraded":"2024-05-17T09:17:28.387326+0000","last_fullsized":"2024-05-17T09:17:28.387326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904968+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.904968+0000","last_peered":"2024-05-17T09:17:46.904968+0000","last_clean":"2024-05-17T09:17:46.904968+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.904968+0000","last_undegraded":"2024-05-17T09:17:46.904968+0000","last_fullsized":"2024-05-17T09:17:46.904968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.411802+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:17:11.411802+0000","last_peered":"2024-05-17T09:17:11.411802+0000","last_clean":"2024-05-17T09:17:11.411802+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:17:11.411802+0000","last_undegraded":"2024-05-17T09:17:11.411802+0000","last_fullsized":"2024-05-17T09:17:11.411802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595216+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595216+0000","last_peered":"2024-05-17T09:16:48.595216+0000","last_clean":"2024-05-17T09:16:48.595216+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595216+0000","last_undegraded":"2024-05-17T09:16:48.595216+0000","last_fullsized":"2024-05-17T09:16:48.595216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423844+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423844+0000","last_peered":"2024-05-17T09:16:48.423844+0000","last_clean":"2024-05-17T09:16:48.423844+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423844+0000","last_undegraded":"2024-05-17T09:16:48.423844+0000","last_fullsized":"2024-05-17T09:16:48.423844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903562+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903562+0000","last_peered":"2024-05-17T09:17:46.903562+0000","last_clean":"2024-05-17T09:17:46.903562+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903562+0000","last_undegraded":"2024-05-17T09:17:46.903562+0000","last_fullsized":"2024-05-17T09:17:46.903562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.384322+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:17:03.384322+0000","last_peered":"2024-05-17T09:17:03.384322+0000","last_clean":"2024-05-17T09:17:03.384322+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:17:03.384322+0000","last_undegraded":"2024-05-17T09:17:03.384322+0000","last_fullsized":"2024-05-17T09:17:03.384322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112162+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:17:24.112162+0000","last_peered":"2024-05-17T09:17:24.112162+0000","last_clean":"2024-05-17T09:17:24.112162+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:17:24.112162+0000","last_undegraded":"2024-05-17T09:17:24.112162+0000","last_fullsized":"2024-05-17T09:17:24.112162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.936560+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:17:08.936560+0000","last_peered":"2024-05-17T09:17:08.936560+0000","last_clean":"2024-05-17T09:17:08.936560+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:17:08.936560+0000","last_undegraded":"2024-05-17T09:17:08.936560+0000","last_fullsized":"2024-05-17T09:17:08.936560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903852+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903852+0000","last_peered":"2024-05-17T09:17:46.903852+0000","last_clean":"2024-05-17T09:17:46.903852+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903852+0000","last_undegraded":"2024-05-17T09:17:46.903852+0000","last_fullsized":"2024-05-17T09:17:46.903852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:20.633799+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:17:20.633799+0000","last_peered":"2024-05-17T09:17:20.633799+0000","last_clean":"2024-05-17T09:17:20.633799+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:17:20.633799+0000","last_undegraded":"2024-05-17T09:17:20.633799+0000","last_fullsized":"2024-05-17T09:17:20.633799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.948913+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:19:26.948913+0000","last_peered":"2024-05-17T09:19:26.948913+0000","last_clean":"2024-05-17T09:19:26.948913+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:19:26.948913+0000","last_undegraded":"2024-05-17T09:19:26.948913+0000","last_fullsized":"2024-05-17T09:19:26.948913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889973+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889973+0000","last_peered":"2024-05-17T09:17:46.889973+0000","last_clean":"2024-05-17T09:17:46.889973+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889973+0000","last_undegraded":"2024-05-17T09:17:46.889973+0000","last_fullsized":"2024-05-17T09:17:46.889973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.193083+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:17:26.193083+0000","last_peered":"2024-05-17T09:17:26.193083+0000","last_clean":"2024-05-17T09:17:26.193083+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:17:26.193083+0000","last_undegraded":"2024-05-17T09:17:26.193083+0000","last_fullsized":"2024-05-17T09:17:26.193083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.412427+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:17:16.412427+0000","last_peered":"2024-05-17T09:17:16.412427+0000","last_clean":"2024-05-17T09:17:16.412427+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:17:16.412427+0000","last_undegraded":"2024-05-17T09:17:16.412427+0000","last_fullsized":"2024-05-17T09:17:16.412427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898859+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898859+0000","last_peered":"2024-05-17T09:17:46.898859+0000","last_clean":"2024-05-17T09:17:46.898859+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898859+0000","last_undegraded":"2024-05-17T09:17:46.898859+0000","last_fullsized":"2024-05-17T09:17:46.898859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.908768+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:17:16.908768+0000","last_peered":"2024-05-17T09:17:16.908768+0000","last_clean":"2024-05-17T09:17:16.908768+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:17:16.908768+0000","last_undegraded":"2024-05-17T09:17:16.908768+0000","last_fullsized":"2024-05-17T09:17:16.908768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898286+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.898286+0000","last_peered":"2024-05-17T09:17:46.898286+0000","last_clean":"2024-05-17T09:17:46.898286+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.898286+0000","last_undegraded":"2024-05-17T09:17:46.898286+0000","last_fullsized":"2024-05-17T09:17:46.898286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:15.633296+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:17:15.633296+0000","last_peered":"2024-05-17T09:17:15.633296+0000","last_clean":"2024-05-17T09:17:15.633296+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:17:15.633296+0000","last_undegraded":"2024-05-17T09:17:15.633296+0000","last_fullsized":"2024-05-17T09:17:15.633296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901878+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.901878+0000","last_peered":"2024-05-17T09:17:46.901878+0000","last_clean":"2024-05-17T09:17:46.901878+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.901878+0000","last_undegraded":"2024-05-17T09:17:46.901878+0000","last_fullsized":"2024-05-17T09:17:46.901878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.206872+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:19:26.206872+0000","last_peered":"2024-05-17T09:19:26.206872+0000","last_clean":"2024-05-17T09:19:26.206872+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:19:26.206872+0000","last_undegraded":"2024-05-17T09:19:26.206872+0000","last_fullsized":"2024-05-17T09:19:26.206872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900941+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.900941+0000","last_peered":"2024-05-17T09:17:46.900941+0000","last_clean":"2024-05-17T09:17:46.900941+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.900941+0000","last_undegraded":"2024-05-17T09:17:46.900941+0000","last_fullsized":"2024-05-17T09:17:46.900941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:18.937946+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:17:18.937946+0000","last_peered":"2024-05-17T09:17:18.937946+0000","last_clean":"2024-05-17T09:17:18.937946+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:17:18.937946+0000","last_undegraded":"2024-05-17T09:17:18.937946+0000","last_fullsized":"2024-05-17T09:17:18.937946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908869+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908869+0000","last_peered":"2024-05-17T09:17:46.908869+0000","last_clean":"2024-05-17T09:17:46.908869+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908869+0000","last_undegraded":"2024-05-17T09:17:46.908869+0000","last_fullsized":"2024-05-17T09:17:46.908869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859520+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859520+0000","last_peered":"2024-05-17T09:16:48.859520+0000","last_clean":"2024-05-17T09:16:48.859520+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859520+0000","last_undegraded":"2024-05-17T09:16:48.859520+0000","last_fullsized":"2024-05-17T09:16:48.859520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319435+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319435+0000","last_peered":"2024-05-17T09:16:48.319435+0000","last_clean":"2024-05-17T09:16:48.319435+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319435+0000","last_undegraded":"2024-05-17T09:16:48.319435+0000","last_fullsized":"2024-05-17T09:16:48.319435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908319+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908319+0000","last_peered":"2024-05-17T09:17:46.908319+0000","last_clean":"2024-05-17T09:17:46.908319+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908319+0000","last_undegraded":"2024-05-17T09:17:46.908319+0000","last_fullsized":"2024-05-17T09:17:46.908319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423786+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423786+0000","last_peered":"2024-05-17T09:16:48.423786+0000","last_clean":"2024-05-17T09:16:48.423786+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423786+0000","last_undegraded":"2024-05-17T09:16:48.423786+0000","last_fullsized":"2024-05-17T09:16:48.423786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901697+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901697+0000","last_peered":"2024-05-17T09:17:46.901697+0000","last_clean":"2024-05-17T09:17:46.901697+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901697+0000","last_undegraded":"2024-05-17T09:17:46.901697+0000","last_fullsized":"2024-05-17T09:17:46.901697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423060+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.423060+0000","last_peered":"2024-05-17T09:16:48.423060+0000","last_clean":"2024-05-17T09:16:48.423060+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.423060+0000","last_undegraded":"2024-05-17T09:16:48.423060+0000","last_fullsized":"2024-05-17T09:16:48.423060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320546+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320546+0000","last_peered":"2024-05-17T09:16:48.320546+0000","last_clean":"2024-05-17T09:16:48.320546+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320546+0000","last_undegraded":"2024-05-17T09:16:48.320546+0000","last_fullsized":"2024-05-17T09:16:48.320546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897876+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897876+0000","last_peered":"2024-05-17T09:17:46.897876+0000","last_clean":"2024-05-17T09:17:46.897876+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897876+0000","last_undegraded":"2024-05-17T09:17:46.897876+0000","last_fullsized":"2024-05-17T09:17:46.897876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899078+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.899078+0000","last_peered":"2024-05-17T09:17:46.899078+0000","last_clean":"2024-05-17T09:17:46.899078+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.899078+0000","last_undegraded":"2024-05-17T09:17:46.899078+0000","last_fullsized":"2024-05-17T09:17:46.899078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888934+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888934+0000","last_peered":"2024-05-17T09:17:46.888934+0000","last_clean":"2024-05-17T09:17:46.888934+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888934+0000","last_undegraded":"2024-05-17T09:17:46.888934+0000","last_fullsized":"2024-05-17T09:17:46.888934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896944+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896944+0000","last_peered":"2024-05-17T09:17:46.896944+0000","last_clean":"2024-05-17T09:17:46.896944+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896944+0000","last_undegraded":"2024-05-17T09:17:46.896944+0000","last_fullsized":"2024-05-17T09:17:46.896944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904847+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904847+0000","last_peered":"2024-05-17T09:17:46.904847+0000","last_clean":"2024-05-17T09:17:46.904847+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904847+0000","last_undegraded":"2024-05-17T09:17:46.904847+0000","last_fullsized":"2024-05-17T09:17:46.904847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904438+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904438+0000","last_peered":"2024-05-17T09:17:46.904438+0000","last_clean":"2024-05-17T09:17:46.904438+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904438+0000","last_undegraded":"2024-05-17T09:17:46.904438+0000","last_fullsized":"2024-05-17T09:17:46.904438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899186+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899186+0000","last_peered":"2024-05-17T09:17:46.899186+0000","last_clean":"2024-05-17T09:17:46.899186+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899186+0000","last_undegraded":"2024-05-17T09:17:46.899186+0000","last_fullsized":"2024-05-17T09:17:46.899186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891995+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891995+0000","last_peered":"2024-05-17T09:17:46.891995+0000","last_clean":"2024-05-17T09:17:46.891995+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891995+0000","last_undegraded":"2024-05-17T09:17:46.891995+0000","last_fullsized":"2024-05-17T09:17:46.891995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886503+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.886503+0000","last_peered":"2024-05-17T09:17:46.886503+0000","last_clean":"2024-05-17T09:17:46.886503+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.886503+0000","last_undegraded":"2024-05-17T09:17:46.886503+0000","last_fullsized":"2024-05-17T09:17:46.886503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899965+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899965+0000","last_peered":"2024-05-17T09:17:46.899965+0000","last_clean":"2024-05-17T09:17:46.899965+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899965+0000","last_undegraded":"2024-05-17T09:17:46.899965+0000","last_fullsized":"2024-05-17T09:17:46.899965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894550+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894550+0000","last_peered":"2024-05-17T09:17:46.894550+0000","last_clean":"2024-05-17T09:17:46.894550+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894550+0000","last_undegraded":"2024-05-17T09:17:46.894550+0000","last_fullsized":"2024-05-17T09:17:46.894550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908241+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908241+0000","last_peered":"2024-05-17T09:17:46.908241+0000","last_clean":"2024-05-17T09:17:46.908241+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908241+0000","last_undegraded":"2024-05-17T09:17:46.908241+0000","last_fullsized":"2024-05-17T09:17:46.908241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904498+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904498+0000","last_peered":"2024-05-17T09:17:46.904498+0000","last_clean":"2024-05-17T09:17:46.904498+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904498+0000","last_undegraded":"2024-05-17T09:17:46.904498+0000","last_fullsized":"2024-05-17T09:17:46.904498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899413+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899413+0000","last_peered":"2024-05-17T09:17:46.899413+0000","last_clean":"2024-05-17T09:17:46.899413+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899413+0000","last_undegraded":"2024-05-17T09:17:46.899413+0000","last_fullsized":"2024-05-17T09:17:46.899413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1200,"num_read_kb":682,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":215928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1176674,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266773,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1112817,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626528,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":575293,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":577634,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":516180,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822757,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1103284,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:19:35.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T09:19:35.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:35 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1749687769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.690+0000 7f22e9696640 1 -- 172.21.15.19:0/773658896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4103950 msgr2=0x7f22e4103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 --2- 172.21.15.19:0/773658896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4103950 0x7f22e4103d60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009900 tx=0x7f22d802a410 comp rx=0 tx=0).stop 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 -- 172.21.15.19:0/773658896 shutdown_connections 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 --2- 172.21.15.19:0/773658896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4109b90 0x7f22e410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 --2- 172.21.15.19:0/773658896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4103950 0x7f22e4103d60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 --2- 172.21.15.19:0/773658896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4105290 0x7f22e4103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 -- 172.21.15.19:0/773658896 >> 172.21.15.19:0/773658896 conn(0x7f22e40fee60 msgr2=0x7f22e41012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 -- 172.21.15.19:0/773658896 shutdown_connections 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 -- 172.21.15.19:0/773658896 wait complete. 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 Processor -- start 2024-05-17T09:19:35.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.691+0000 7f22e9696640 1 -- start start 2024-05-17T09:19:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e9696640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4103950 0x7f22e4077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e9696640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 0x7f22e4076120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e9696640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4109b90 0x7f22e4076660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e9696640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22e410f790 con 0x7f22e4105290 2024-05-17T09:19:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e9696640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22e410f610 con 0x7f22e4103950 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e9696640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22e410f910 con 0x7f22e4109b90 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 0x7f22e4076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 0x7f22e4076120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35222/0 (socket says 172.21.15.19:35222) 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 -- 172.21.15.19:0/3442646276 learned_addr learned my addr 172.21.15.19:0/3442646276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e37fe640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4109b90 0x7f22e4076660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e2ffd640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4103950 0x7f22e4077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 -- 172.21.15.19:0/3442646276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4109b90 msgr2=0x7f22e4076660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4109b90 0x7f22e4076660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 -- 172.21.15.19:0/3442646276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4103950 msgr2=0x7f22e4077a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4103950 0x7f22e4077a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 -- 172.21.15.19:0/3442646276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e4076d90 con 0x7f22e4105290 2024-05-17T09:19:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e2ffd640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4103950 0x7f22e4077a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.692+0000 7f22e27fc640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 0x7f22e4076120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005760 tx=0x7f22d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.693+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d803f070 con 0x7f22e4105290 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.693+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22d8031ba0 con 0x7f22e4105290 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.693+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e41a4390 con 0x7f22e4105290 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.693+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8035030 con 0x7f22e4105290 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.693+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e41a4790 con 0x7f22e4105290 2024-05-17T09:19:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.694+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e410ff50 con 0x7f22e4105290 2024-05-17T09:19:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.695+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f22d8042070 con 0x7f22e4105290 2024-05-17T09:19:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.696+0000 7f22cbfff640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22cc0b28a0 0x7f22cc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.696+0000 7f22e2ffd640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22cc0b28a0 0x7f22cc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.696+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f22d8102f80 con 0x7f22e4105290 2024-05-17T09:19:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.696+0000 7f22e2ffd640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22cc0b28a0 0x7f22cc0b4db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22e4104850 tx=0x7f22d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.701+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22d80ca890 con 0x7f22e4105290 2024-05-17T09:19:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:35 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1749687769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:35.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.888+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22e402df80 con 0x7f22cc0b28a0 2024-05-17T09:19:35.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.897+0000 7f22cbfff640 1 -- 172.21.15.19:0/3442646276 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340732 (secure 0 0 0) 0x7f22e402df80 con 0x7f22cc0b28a0 2024-05-17T09:19:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:35.904 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:19:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22cc0b28a0 msgr2=0x7f22cc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22cc0b28a0 0x7f22cc0b4db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22e4104850 tx=0x7f22d4005f90 comp rx=0 tx=0).stop 2024-05-17T09:19:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 msgr2=0x7f22e4076120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 0x7f22e4076120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005760 tx=0x7f22d8003c10 comp rx=0 tx=0).stop 2024-05-17T09:19:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 shutdown_connections 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4109b90 0x7f22e4076660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22cc0b28a0 0x7f22cc0b4db0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22e4105290 0x7f22e4076120 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 --2- 172.21.15.19:0/3442646276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22e4103950 0x7f22e4077a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 >> 172.21.15.19:0/3442646276 conn(0x7f22e40fee60 msgr2=0x7f22e4105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 shutdown_connections 2024-05-17T09:19:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:35.901+0000 7f22e9696640 1 -- 172.21.15.19:0/3442646276 wait complete. 2024-05-17T09:19:35.913 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-05-17T09:19:34.948495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1276,"num_read_kb":746,"num_write":313,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348648,"kb_used_data":11976,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593176,"statfs":{"total":1151940427776,"available":1151583412224,"internally_reserved":0,"allocated":12263424,"data_stored":8144780,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":48,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003881"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906823+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906823+0000","last_peered":"2024-05-17T09:17:46.906823+0000","last_clean":"2024-05-17T09:17:46.906823+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906823+0000","last_undegraded":"2024-05-17T09:17:46.906823+0000","last_fullsized":"2024-05-17T09:17:46.906823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901351+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.901351+0000","last_peered":"2024-05-17T09:17:46.901351+0000","last_clean":"2024-05-17T09:17:46.901351+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.901351+0000","last_undegraded":"2024-05-17T09:17:46.901351+0000","last_fullsized":"2024-05-17T09:17:46.901351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905015+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.905015+0000","last_peered":"2024-05-17T09:17:46.905015+0000","last_clean":"2024-05-17T09:17:46.905015+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.905015+0000","last_undegraded":"2024-05-17T09:17:46.905015+0000","last_fullsized":"2024-05-17T09:17:46.905015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886153+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.886153+0000","last_peered":"2024-05-17T09:17:46.886153+0000","last_clean":"2024-05-17T09:17:46.886153+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.886153+0000","last_undegraded":"2024-05-17T09:17:46.886153+0000","last_fullsized":"2024-05-17T09:17:46.886153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897920+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.897920+0000","last_peered":"2024-05-17T09:17:46.897920+0000","last_clean":"2024-05-17T09:17:46.897920+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.897920+0000","last_undegraded":"2024-05-17T09:17:46.897920+0000","last_fullsized":"2024-05-17T09:17:46.897920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893468+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893468+0000","last_peered":"2024-05-17T09:17:46.893468+0000","last_clean":"2024-05-17T09:17:46.893468+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893468+0000","last_undegraded":"2024-05-17T09:17:46.893468+0000","last_fullsized":"2024-05-17T09:17:46.893468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908147+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908147+0000","last_peered":"2024-05-17T09:17:46.908147+0000","last_clean":"2024-05-17T09:17:46.908147+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908147+0000","last_undegraded":"2024-05-17T09:17:46.908147+0000","last_fullsized":"2024-05-17T09:17:46.908147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889231+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889231+0000","last_peered":"2024-05-17T09:17:46.889231+0000","last_clean":"2024-05-17T09:17:46.889231+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889231+0000","last_undegraded":"2024-05-17T09:17:46.889231+0000","last_fullsized":"2024-05-17T09:17:46.889231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934972+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:17:11.934972+0000","last_peered":"2024-05-17T09:17:11.934972+0000","last_clean":"2024-05-17T09:17:11.934972+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:17:11.934972+0000","last_undegraded":"2024-05-17T09:17:11.934972+0000","last_fullsized":"2024-05-17T09:17:11.934972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898655+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898655+0000","last_peered":"2024-05-17T09:17:46.898655+0000","last_clean":"2024-05-17T09:17:46.898655+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898655+0000","last_undegraded":"2024-05-17T09:17:46.898655+0000","last_fullsized":"2024-05-17T09:17:46.898655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:05.632115+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:17:05.632115+0000","last_peered":"2024-05-17T09:17:05.632115+0000","last_clean":"2024-05-17T09:17:05.632115+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:17:05.632115+0000","last_undegraded":"2024-05-17T09:17:05.632115+0000","last_fullsized":"2024-05-17T09:17:05.632115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905191+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905191+0000","last_peered":"2024-05-17T09:17:46.905191+0000","last_clean":"2024-05-17T09:17:46.905191+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905191+0000","last_undegraded":"2024-05-17T09:17:46.905191+0000","last_fullsized":"2024-05-17T09:17:46.905191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:30.536421+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:19:30.536421+0000","last_peered":"2024-05-17T09:19:30.536421+0000","last_clean":"2024-05-17T09:19:30.536421+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:19:30.536421+0000","last_undegraded":"2024-05-17T09:19:30.536421+0000","last_fullsized":"2024-05-17T09:19:30.536421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112225+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:17:24.112225+0000","last_peered":"2024-05-17T09:17:24.112225+0000","last_clean":"2024-05-17T09:17:24.112225+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:17:24.112225+0000","last_undegraded":"2024-05-17T09:17:24.112225+0000","last_fullsized":"2024-05-17T09:17:24.112225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.430695+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:17:23.430695+0000","last_peered":"2024-05-17T09:17:23.430695+0000","last_clean":"2024-05-17T09:17:23.430695+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:17:23.430695+0000","last_undegraded":"2024-05-17T09:17:23.430695+0000","last_fullsized":"2024-05-17T09:17:23.430695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:32.538829+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:19:32.538829+0000","last_peered":"2024-05-17T09:19:32.538829+0000","last_clean":"2024-05-17T09:19:32.538829+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:19:32.538829+0000","last_undegraded":"2024-05-17T09:19:32.538829+0000","last_fullsized":"2024-05-17T09:19:32.538829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":486,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:33.541041+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:19:33.541041+0000","last_peered":"2024-05-17T09:19:33.541041+0000","last_clean":"2024-05-17T09:19:33.541041+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:19:33.541041+0000","last_undegraded":"2024-05-17T09:19:33.541041+0000","last_fullsized":"2024-05-17T09:19:33.541041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":269,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903452+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903452+0000","last_peered":"2024-05-17T09:17:46.903452+0000","last_clean":"2024-05-17T09:17:46.903452+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903452+0000","last_undegraded":"2024-05-17T09:17:46.903452+0000","last_fullsized":"2024-05-17T09:17:46.903452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423686+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423686+0000","last_peered":"2024-05-17T09:16:48.423686+0000","last_clean":"2024-05-17T09:16:48.423686+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423686+0000","last_undegraded":"2024-05-17T09:16:48.423686+0000","last_fullsized":"2024-05-17T09:16:48.423686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904979+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.904979+0000","last_peered":"2024-05-17T09:17:46.904979+0000","last_clean":"2024-05-17T09:17:46.904979+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.904979+0000","last_undegraded":"2024-05-17T09:17:46.904979+0000","last_fullsized":"2024-05-17T09:17:46.904979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":247,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:29.533641+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:19:29.533641+0000","last_peered":"2024-05-17T09:19:29.533641+0000","last_clean":"2024-05-17T09:19:29.533641+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:19:29.533641+0000","last_undegraded":"2024-05-17T09:19:29.533641+0000","last_fullsized":"2024-05-17T09:19:29.533641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900338+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.900338+0000","last_peered":"2024-05-17T09:17:46.900338+0000","last_clean":"2024-05-17T09:17:46.900338+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.900338+0000","last_undegraded":"2024-05-17T09:17:46.900338+0000","last_fullsized":"2024-05-17T09:17:46.900338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829668+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829668+0000","last_peered":"2024-05-17T09:16:48.829668+0000","last_clean":"2024-05-17T09:16:48.829668+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829668+0000","last_undegraded":"2024-05-17T09:16:48.829668+0000","last_fullsized":"2024-05-17T09:16:48.829668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423264+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423264+0000","last_peered":"2024-05-17T09:16:48.423264+0000","last_clean":"2024-05-17T09:16:48.423264+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423264+0000","last_undegraded":"2024-05-17T09:16:48.423264+0000","last_fullsized":"2024-05-17T09:16:48.423264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897951+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.897951+0000","last_peered":"2024-05-17T09:17:46.897951+0000","last_clean":"2024-05-17T09:17:46.897951+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.897951+0000","last_undegraded":"2024-05-17T09:17:46.897951+0000","last_fullsized":"2024-05-17T09:17:46.897951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.939280+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:17:28.939280+0000","last_peered":"2024-05-17T09:17:28.939280+0000","last_clean":"2024-05-17T09:17:28.939280+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:17:28.939280+0000","last_undegraded":"2024-05-17T09:17:28.939280+0000","last_fullsized":"2024-05-17T09:17:28.939280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934940+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:17:11.934940+0000","last_peered":"2024-05-17T09:17:11.934940+0000","last_clean":"2024-05-17T09:17:11.934940+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:17:11.934940+0000","last_undegraded":"2024-05-17T09:17:11.934940+0000","last_fullsized":"2024-05-17T09:17:11.934940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904550+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904550+0000","last_peered":"2024-05-17T09:17:46.904550+0000","last_clean":"2024-05-17T09:17:46.904550+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904550+0000","last_undegraded":"2024-05-17T09:17:46.904550+0000","last_fullsized":"2024-05-17T09:17:46.904550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:27.771474+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:17:27.771474+0000","last_peered":"2024-05-17T09:17:27.771474+0000","last_clean":"2024-05-17T09:17:27.771474+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:17:27.771474+0000","last_undegraded":"2024-05-17T09:17:27.771474+0000","last_fullsized":"2024-05-17T09:17:27.771474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.384811+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:17:08.384811+0000","last_peered":"2024-05-17T09:17:08.384811+0000","last_clean":"2024-05-17T09:17:08.384811+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:17:08.384811+0000","last_undegraded":"2024-05-17T09:17:08.384811+0000","last_fullsized":"2024-05-17T09:17:08.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900921+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.900921+0000","last_peered":"2024-05-17T09:17:46.900921+0000","last_clean":"2024-05-17T09:17:46.900921+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.900921+0000","last_undegraded":"2024-05-17T09:17:46.900921+0000","last_fullsized":"2024-05-17T09:17:46.900921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320360+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320360+0000","last_peered":"2024-05-17T09:16:48.320360+0000","last_clean":"2024-05-17T09:16:48.320360+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320360+0000","last_undegraded":"2024-05-17T09:16:48.320360+0000","last_fullsized":"2024-05-17T09:16:48.320360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901492+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901492+0000","last_peered":"2024-05-17T09:17:46.901492+0000","last_clean":"2024-05-17T09:17:46.901492+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901492+0000","last_undegraded":"2024-05-17T09:17:46.901492+0000","last_fullsized":"2024-05-17T09:17:46.901492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:31.536246+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:19:31.536246+0000","last_peered":"2024-05-17T09:19:31.536246+0000","last_clean":"2024-05-17T09:19:31.536246+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:19:31.536246+0000","last_undegraded":"2024-05-17T09:19:31.536246+0000","last_fullsized":"2024-05-17T09:19:31.536246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387326+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:17:28.387326+0000","last_peered":"2024-05-17T09:17:28.387326+0000","last_clean":"2024-05-17T09:17:28.387326+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:17:28.387326+0000","last_undegraded":"2024-05-17T09:17:28.387326+0000","last_fullsized":"2024-05-17T09:17:28.387326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904968+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.904968+0000","last_peered":"2024-05-17T09:17:46.904968+0000","last_clean":"2024-05-17T09:17:46.904968+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.904968+0000","last_undegraded":"2024-05-17T09:17:46.904968+0000","last_fullsized":"2024-05-17T09:17:46.904968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.411802+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:17:11.411802+0000","last_peered":"2024-05-17T09:17:11.411802+0000","last_clean":"2024-05-17T09:17:11.411802+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:17:11.411802+0000","last_undegraded":"2024-05-17T09:17:11.411802+0000","last_fullsized":"2024-05-17T09:17:11.411802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595216+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595216+0000","last_peered":"2024-05-17T09:16:48.595216+0000","last_clean":"2024-05-17T09:16:48.595216+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595216+0000","last_undegraded":"2024-05-17T09:16:48.595216+0000","last_fullsized":"2024-05-17T09:16:48.595216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423844+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423844+0000","last_peered":"2024-05-17T09:16:48.423844+0000","last_clean":"2024-05-17T09:16:48.423844+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423844+0000","last_undegraded":"2024-05-17T09:16:48.423844+0000","last_fullsized":"2024-05-17T09:16:48.423844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903562+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903562+0000","last_peered":"2024-05-17T09:17:46.903562+0000","last_clean":"2024-05-17T09:17:46.903562+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903562+0000","last_undegraded":"2024-05-17T09:17:46.903562+0000","last_fullsized":"2024-05-17T09:17:46.903562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.384322+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:17:03.384322+0000","last_peered":"2024-05-17T09:17:03.384322+0000","last_clean":"2024-05-17T09:17:03.384322+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:17:03.384322+0000","last_undegraded":"2024-05-17T09:17:03.384322+0000","last_fullsized":"2024-05-17T09:17:03.384322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112162+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:17:24.112162+0000","last_peered":"2024-05-17T09:17:24.112162+0000","last_clean":"2024-05-17T09:17:24.112162+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:17:24.112162+0000","last_undegraded":"2024-05-17T09:17:24.112162+0000","last_fullsized":"2024-05-17T09:17:24.112162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.936560+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:17:08.936560+0000","last_peered":"2024-05-17T09:17:08.936560+0000","last_clean":"2024-05-17T09:17:08.936560+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:17:08.936560+0000","last_undegraded":"2024-05-17T09:17:08.936560+0000","last_fullsized":"2024-05-17T09:17:08.936560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903852+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903852+0000","last_peered":"2024-05-17T09:17:46.903852+0000","last_clean":"2024-05-17T09:17:46.903852+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903852+0000","last_undegraded":"2024-05-17T09:17:46.903852+0000","last_fullsized":"2024-05-17T09:17:46.903852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:20.633799+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:17:20.633799+0000","last_peered":"2024-05-17T09:17:20.633799+0000","last_clean":"2024-05-17T09:17:20.633799+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:17:20.633799+0000","last_undegraded":"2024-05-17T09:17:20.633799+0000","last_fullsized":"2024-05-17T09:17:20.633799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.948913+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:19:26.948913+0000","last_peered":"2024-05-17T09:19:26.948913+0000","last_clean":"2024-05-17T09:19:26.948913+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:19:26.948913+0000","last_undegraded":"2024-05-17T09:19:26.948913+0000","last_fullsized":"2024-05-17T09:19:26.948913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889973+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889973+0000","last_peered":"2024-05-17T09:17:46.889973+0000","last_clean":"2024-05-17T09:17:46.889973+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889973+0000","last_undegraded":"2024-05-17T09:17:46.889973+0000","last_fullsized":"2024-05-17T09:17:46.889973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.193083+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:17:26.193083+0000","last_peered":"2024-05-17T09:17:26.193083+0000","last_clean":"2024-05-17T09:17:26.193083+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:17:26.193083+0000","last_undegraded":"2024-05-17T09:17:26.193083+0000","last_fullsized":"2024-05-17T09:17:26.193083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.412427+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:17:16.412427+0000","last_peered":"2024-05-17T09:17:16.412427+0000","last_clean":"2024-05-17T09:17:16.412427+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:17:16.412427+0000","last_undegraded":"2024-05-17T09:17:16.412427+0000","last_fullsized":"2024-05-17T09:17:16.412427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898859+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898859+0000","last_peered":"2024-05-17T09:17:46.898859+0000","last_clean":"2024-05-17T09:17:46.898859+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898859+0000","last_undegraded":"2024-05-17T09:17:46.898859+0000","last_fullsized":"2024-05-17T09:17:46.898859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.908768+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:17:16.908768+0000","last_peered":"2024-05-17T09:17:16.908768+0000","last_clean":"2024-05-17T09:17:16.908768+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:17:16.908768+0000","last_undegraded":"2024-05-17T09:17:16.908768+0000","last_fullsized":"2024-05-17T09:17:16.908768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898286+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.898286+0000","last_peered":"2024-05-17T09:17:46.898286+0000","last_clean":"2024-05-17T09:17:46.898286+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.898286+0000","last_undegraded":"2024-05-17T09:17:46.898286+0000","last_fullsized":"2024-05-17T09:17:46.898286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:15.633296+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:17:15.633296+0000","last_peered":"2024-05-17T09:17:15.633296+0000","last_clean":"2024-05-17T09:17:15.633296+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:17:15.633296+0000","last_undegraded":"2024-05-17T09:17:15.633296+0000","last_fullsized":"2024-05-17T09:17:15.633296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901878+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.901878+0000","last_peered":"2024-05-17T09:17:46.901878+0000","last_clean":"2024-05-17T09:17:46.901878+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.901878+0000","last_undegraded":"2024-05-17T09:17:46.901878+0000","last_fullsized":"2024-05-17T09:17:46.901878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.206872+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:19:26.206872+0000","last_peered":"2024-05-17T09:19:26.206872+0000","last_clean":"2024-05-17T09:19:26.206872+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:19:26.206872+0000","last_undegraded":"2024-05-17T09:19:26.206872+0000","last_fullsized":"2024-05-17T09:19:26.206872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900941+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.900941+0000","last_peered":"2024-05-17T09:17:46.900941+0000","last_clean":"2024-05-17T09:17:46.900941+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.900941+0000","last_undegraded":"2024-05-17T09:17:46.900941+0000","last_fullsized":"2024-05-17T09:17:46.900941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:18.937946+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:17:18.937946+0000","last_peered":"2024-05-17T09:17:18.937946+0000","last_clean":"2024-05-17T09:17:18.937946+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:17:18.937946+0000","last_undegraded":"2024-05-17T09:17:18.937946+0000","last_fullsized":"2024-05-17T09:17:18.937946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908869+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908869+0000","last_peered":"2024-05-17T09:17:46.908869+0000","last_clean":"2024-05-17T09:17:46.908869+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908869+0000","last_undegraded":"2024-05-17T09:17:46.908869+0000","last_fullsized":"2024-05-17T09:17:46.908869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859520+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859520+0000","last_peered":"2024-05-17T09:16:48.859520+0000","last_clean":"2024-05-17T09:16:48.859520+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859520+0000","last_undegraded":"2024-05-17T09:16:48.859520+0000","last_fullsized":"2024-05-17T09:16:48.859520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319435+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319435+0000","last_peered":"2024-05-17T09:16:48.319435+0000","last_clean":"2024-05-17T09:16:48.319435+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319435+0000","last_undegraded":"2024-05-17T09:16:48.319435+0000","last_fullsized":"2024-05-17T09:16:48.319435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908319+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908319+0000","last_peered":"2024-05-17T09:17:46.908319+0000","last_clean":"2024-05-17T09:17:46.908319+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908319+0000","last_undegraded":"2024-05-17T09:17:46.908319+0000","last_fullsized":"2024-05-17T09:17:46.908319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423786+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423786+0000","last_peered":"2024-05-17T09:16:48.423786+0000","last_clean":"2024-05-17T09:16:48.423786+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423786+0000","last_undegraded":"2024-05-17T09:16:48.423786+0000","last_fullsized":"2024-05-17T09:16:48.423786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901697+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901697+0000","last_peered":"2024-05-17T09:17:46.901697+0000","last_clean":"2024-05-17T09:17:46.901697+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901697+0000","last_undegraded":"2024-05-17T09:17:46.901697+0000","last_fullsized":"2024-05-17T09:17:46.901697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423060+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.423060+0000","last_peered":"2024-05-17T09:16:48.423060+0000","last_clean":"2024-05-17T09:16:48.423060+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.423060+0000","last_undegraded":"2024-05-17T09:16:48.423060+0000","last_fullsized":"2024-05-17T09:16:48.423060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320546+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320546+0000","last_peered":"2024-05-17T09:16:48.320546+0000","last_clean":"2024-05-17T09:16:48.320546+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320546+0000","last_undegraded":"2024-05-17T09:16:48.320546+0000","last_fullsized":"2024-05-17T09:16:48.320546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897876+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897876+0000","last_peered":"2024-05-17T09:17:46.897876+0000","last_clean":"2024-05-17T09:17:46.897876+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897876+0000","last_undegraded":"2024-05-17T09:17:46.897876+0000","last_fullsized":"2024-05-17T09:17:46.897876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899078+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.899078+0000","last_peered":"2024-05-17T09:17:46.899078+0000","last_clean":"2024-05-17T09:17:46.899078+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.899078+0000","last_undegraded":"2024-05-17T09:17:46.899078+0000","last_fullsized":"2024-05-17T09:17:46.899078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888934+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888934+0000","last_peered":"2024-05-17T09:17:46.888934+0000","last_clean":"2024-05-17T09:17:46.888934+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888934+0000","last_undegraded":"2024-05-17T09:17:46.888934+0000","last_fullsized":"2024-05-17T09:17:46.888934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896944+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896944+0000","last_peered":"2024-05-17T09:17:46.896944+0000","last_clean":"2024-05-17T09:17:46.896944+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896944+0000","last_undegraded":"2024-05-17T09:17:46.896944+0000","last_fullsized":"2024-05-17T09:17:46.896944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904847+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904847+0000","last_peered":"2024-05-17T09:17:46.904847+0000","last_clean":"2024-05-17T09:17:46.904847+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904847+0000","last_undegraded":"2024-05-17T09:17:46.904847+0000","last_fullsized":"2024-05-17T09:17:46.904847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904438+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904438+0000","last_peered":"2024-05-17T09:17:46.904438+0000","last_clean":"2024-05-17T09:17:46.904438+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904438+0000","last_undegraded":"2024-05-17T09:17:46.904438+0000","last_fullsized":"2024-05-17T09:17:46.904438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899186+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899186+0000","last_peered":"2024-05-17T09:17:46.899186+0000","last_clean":"2024-05-17T09:17:46.899186+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899186+0000","last_undegraded":"2024-05-17T09:17:46.899186+0000","last_fullsized":"2024-05-17T09:17:46.899186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891995+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891995+0000","last_peered":"2024-05-17T09:17:46.891995+0000","last_clean":"2024-05-17T09:17:46.891995+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891995+0000","last_undegraded":"2024-05-17T09:17:46.891995+0000","last_fullsized":"2024-05-17T09:17:46.891995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886503+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.886503+0000","last_peered":"2024-05-17T09:17:46.886503+0000","last_clean":"2024-05-17T09:17:46.886503+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.886503+0000","last_undegraded":"2024-05-17T09:17:46.886503+0000","last_fullsized":"2024-05-17T09:17:46.886503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899965+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899965+0000","last_peered":"2024-05-17T09:17:46.899965+0000","last_clean":"2024-05-17T09:17:46.899965+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899965+0000","last_undegraded":"2024-05-17T09:17:46.899965+0000","last_fullsized":"2024-05-17T09:17:46.899965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894550+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894550+0000","last_peered":"2024-05-17T09:17:46.894550+0000","last_clean":"2024-05-17T09:17:46.894550+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894550+0000","last_undegraded":"2024-05-17T09:17:46.894550+0000","last_fullsized":"2024-05-17T09:17:46.894550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908241+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908241+0000","last_peered":"2024-05-17T09:17:46.908241+0000","last_clean":"2024-05-17T09:17:46.908241+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908241+0000","last_undegraded":"2024-05-17T09:17:46.908241+0000","last_fullsized":"2024-05-17T09:17:46.908241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904498+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904498+0000","last_peered":"2024-05-17T09:17:46.904498+0000","last_clean":"2024-05-17T09:17:46.904498+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904498+0000","last_undegraded":"2024-05-17T09:17:46.904498+0000","last_fullsized":"2024-05-17T09:17:46.904498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899413+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899413+0000","last_peered":"2024-05-17T09:17:46.899413+0000","last_clean":"2024-05-17T09:17:46.899413+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899413+0000","last_undegraded":"2024-05-17T09:17:46.899413+0000","last_fullsized":"2024-05-17T09:17:46.899413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1200,"num_read_kb":682,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":215928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1176674,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266773,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1112817,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626528,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":575293,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":577634,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":516180,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822757,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1103284,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:19:35.918 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-17T09:19:35.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T09:19:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 -- 172.21.15.19:0/2582971289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc105290 msgr2=0x7f00fc103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 --2- 172.21.15.19:0/2582971289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc105290 0x7f00fc103410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f802a410 comp rx=0 tx=0).stop 2024-05-17T09:19:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 -- 172.21.15.19:0/2582971289 shutdown_connections 2024-05-17T09:19:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 --2- 172.21.15.19:0/2582971289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00fc109b90 0x7f00fc10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 --2- 172.21.15.19:0/2582971289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00fc103950 0x7f00fc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 --2- 172.21.15.19:0/2582971289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc105290 0x7f00fc103410 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.069+0000 7f01033b2640 1 -- 172.21.15.19:0/2582971289 >> 172.21.15.19:0/2582971289 conn(0x7f00fc0fee60 msgr2=0x7f00fc1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.070+0000 7f01033b2640 1 -- 172.21.15.19:0/2582971289 shutdown_connections 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.070+0000 7f01033b2640 1 -- 172.21.15.19:0/2582971289 wait complete. 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.070+0000 7f01033b2640 1 Processor -- start 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.070+0000 7f01033b2640 1 -- start start 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 0x7f00fc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00fc105290 0x7f00fc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00fc109b90 0x7f00fc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc10f710 con 0x7f00fc103950 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc10f590 con 0x7f00fc105290 2024-05-17T09:19:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc10f890 con 0x7f00fc109b90 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 0x7f00fc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 0x7f00fc19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35246/0 (socket says 172.21.15.19:35246) 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 -- 172.21.15.19:0/1671921934 learned_addr learned my addr 172.21.15.19:0/1671921934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010094d640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00fc105290 0x7f00fc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010194f640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00fc109b90 0x7f00fc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 -- 172.21.15.19:0/1671921934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00fc109b90 msgr2=0x7f00fc1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00fc109b90 0x7f00fc1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 -- 172.21.15.19:0/1671921934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00fc105290 msgr2=0x7f00fc19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00fc105290 0x7f00fc19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 -- 172.21.15.19:0/1671921934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc1a1e50 con 0x7f00fc103950 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f010114e640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 0x7f00fc19a310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f00f8005760 tx=0x7f00f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f803f030 con 0x7f00fc103950 2024-05-17T09:19:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00f8003a80 con 0x7f00fc103950 2024-05-17T09:19:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc1a20e0 con 0x7f00fc103950 2024-05-17T09:19:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.071+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8035070 con 0x7f00fc103950 2024-05-17T09:19:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.072+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc1a25d0 con 0x7f00fc103950 2024-05-17T09:19:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.073+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc06a330 con 0x7f00fc103950 2024-05-17T09:19:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.074+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f00f8042030 con 0x7f00fc103950 2024-05-17T09:19:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.075+0000 7f00f27fc640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f00e80b28a0 0x7f00e80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.075+0000 7f010094d640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f00e80b28a0 0x7f00e80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.075+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f00f8103140 con 0x7f00fc103950 2024-05-17T09:19:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.076+0000 7f010094d640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f00e80b28a0 0x7f00e80b4db0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00fc19b3b0 tx=0x7f00ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.079+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00f80caa50 con 0x7f00fc103950 2024-05-17T09:19:36.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.266+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00fc063100 con 0x7f00e80b28a0 2024-05-17T09:19:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.276+0000 7f00f27fc640 1 -- 172.21.15.19:0/1671921934 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340732 (secure 0 0 0) 0x7f00fc063100 con 0x7f00e80b28a0 2024-05-17T09:19:36.280 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:19:36.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f00e80b28a0 msgr2=0x7f00e80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:36.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f00e80b28a0 0x7f00e80b4db0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00fc19b3b0 tx=0x7f00ec00b390 comp rx=0 tx=0).stop 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 msgr2=0x7f00fc19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 0x7f00fc19a310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f00f8005760 tx=0x7f00f8004270 comp rx=0 tx=0).stop 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 shutdown_connections 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00fc109b90 0x7f00fc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00fc105290 0x7f00fc19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f00e80b28a0 0x7f00e80b4db0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 --2- 172.21.15.19:0/1671921934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00fc103950 0x7f00fc19a310 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.279+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 >> 172.21.15.19:0/1671921934 conn(0x7f00fc0fee60 msgr2=0x7f00fc105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.280+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 shutdown_connections 2024-05-17T09:19:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:36.280+0000 7f01033b2640 1 -- 172.21.15.19:0/1671921934 wait complete. 2024-05-17T09:19:36.291 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-05-17T09:19:34.948495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1276,"num_read_kb":746,"num_write":313,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348648,"kb_used_data":11976,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593176,"statfs":{"total":1151940427776,"available":1151583412224,"internally_reserved":0,"allocated":12263424,"data_stored":8144780,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":48,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003881"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906823+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906823+0000","last_peered":"2024-05-17T09:17:46.906823+0000","last_clean":"2024-05-17T09:17:46.906823+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906823+0000","last_undegraded":"2024-05-17T09:17:46.906823+0000","last_fullsized":"2024-05-17T09:17:46.906823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901351+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.901351+0000","last_peered":"2024-05-17T09:17:46.901351+0000","last_clean":"2024-05-17T09:17:46.901351+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.901351+0000","last_undegraded":"2024-05-17T09:17:46.901351+0000","last_fullsized":"2024-05-17T09:17:46.901351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905015+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.905015+0000","last_peered":"2024-05-17T09:17:46.905015+0000","last_clean":"2024-05-17T09:17:46.905015+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.905015+0000","last_undegraded":"2024-05-17T09:17:46.905015+0000","last_fullsized":"2024-05-17T09:17:46.905015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886153+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.886153+0000","last_peered":"2024-05-17T09:17:46.886153+0000","last_clean":"2024-05-17T09:17:46.886153+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.886153+0000","last_undegraded":"2024-05-17T09:17:46.886153+0000","last_fullsized":"2024-05-17T09:17:46.886153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897920+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.897920+0000","last_peered":"2024-05-17T09:17:46.897920+0000","last_clean":"2024-05-17T09:17:46.897920+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.897920+0000","last_undegraded":"2024-05-17T09:17:46.897920+0000","last_fullsized":"2024-05-17T09:17:46.897920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893468+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893468+0000","last_peered":"2024-05-17T09:17:46.893468+0000","last_clean":"2024-05-17T09:17:46.893468+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893468+0000","last_undegraded":"2024-05-17T09:17:46.893468+0000","last_fullsized":"2024-05-17T09:17:46.893468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908147+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908147+0000","last_peered":"2024-05-17T09:17:46.908147+0000","last_clean":"2024-05-17T09:17:46.908147+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908147+0000","last_undegraded":"2024-05-17T09:17:46.908147+0000","last_fullsized":"2024-05-17T09:17:46.908147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889231+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889231+0000","last_peered":"2024-05-17T09:17:46.889231+0000","last_clean":"2024-05-17T09:17:46.889231+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889231+0000","last_undegraded":"2024-05-17T09:17:46.889231+0000","last_fullsized":"2024-05-17T09:17:46.889231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934972+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:17:11.934972+0000","last_peered":"2024-05-17T09:17:11.934972+0000","last_clean":"2024-05-17T09:17:11.934972+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:17:11.934972+0000","last_undegraded":"2024-05-17T09:17:11.934972+0000","last_fullsized":"2024-05-17T09:17:11.934972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898655+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898655+0000","last_peered":"2024-05-17T09:17:46.898655+0000","last_clean":"2024-05-17T09:17:46.898655+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898655+0000","last_undegraded":"2024-05-17T09:17:46.898655+0000","last_fullsized":"2024-05-17T09:17:46.898655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:05.632115+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:17:05.632115+0000","last_peered":"2024-05-17T09:17:05.632115+0000","last_clean":"2024-05-17T09:17:05.632115+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:17:05.632115+0000","last_undegraded":"2024-05-17T09:17:05.632115+0000","last_fullsized":"2024-05-17T09:17:05.632115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905191+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905191+0000","last_peered":"2024-05-17T09:17:46.905191+0000","last_clean":"2024-05-17T09:17:46.905191+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905191+0000","last_undegraded":"2024-05-17T09:17:46.905191+0000","last_fullsized":"2024-05-17T09:17:46.905191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:30.536421+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:19:30.536421+0000","last_peered":"2024-05-17T09:19:30.536421+0000","last_clean":"2024-05-17T09:19:30.536421+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:19:30.536421+0000","last_undegraded":"2024-05-17T09:19:30.536421+0000","last_fullsized":"2024-05-17T09:19:30.536421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112225+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:17:24.112225+0000","last_peered":"2024-05-17T09:17:24.112225+0000","last_clean":"2024-05-17T09:17:24.112225+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:17:24.112225+0000","last_undegraded":"2024-05-17T09:17:24.112225+0000","last_fullsized":"2024-05-17T09:17:24.112225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.430695+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:17:23.430695+0000","last_peered":"2024-05-17T09:17:23.430695+0000","last_clean":"2024-05-17T09:17:23.430695+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:17:23.430695+0000","last_undegraded":"2024-05-17T09:17:23.430695+0000","last_fullsized":"2024-05-17T09:17:23.430695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:32.538829+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:19:32.538829+0000","last_peered":"2024-05-17T09:19:32.538829+0000","last_clean":"2024-05-17T09:19:32.538829+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:19:32.538829+0000","last_undegraded":"2024-05-17T09:19:32.538829+0000","last_fullsized":"2024-05-17T09:19:32.538829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":486,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:33.541041+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:19:33.541041+0000","last_peered":"2024-05-17T09:19:33.541041+0000","last_clean":"2024-05-17T09:19:33.541041+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:19:33.541041+0000","last_undegraded":"2024-05-17T09:19:33.541041+0000","last_fullsized":"2024-05-17T09:19:33.541041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":269,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903452+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903452+0000","last_peered":"2024-05-17T09:17:46.903452+0000","last_clean":"2024-05-17T09:17:46.903452+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903452+0000","last_undegraded":"2024-05-17T09:17:46.903452+0000","last_fullsized":"2024-05-17T09:17:46.903452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423686+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:16:48.423686+0000","last_peered":"2024-05-17T09:16:48.423686+0000","last_clean":"2024-05-17T09:16:48.423686+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:16:48.423686+0000","last_undegraded":"2024-05-17T09:16:48.423686+0000","last_fullsized":"2024-05-17T09:16:48.423686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904979+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.904979+0000","last_peered":"2024-05-17T09:17:46.904979+0000","last_clean":"2024-05-17T09:17:46.904979+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.904979+0000","last_undegraded":"2024-05-17T09:17:46.904979+0000","last_fullsized":"2024-05-17T09:17:46.904979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":247,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:29.533641+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:19:29.533641+0000","last_peered":"2024-05-17T09:19:29.533641+0000","last_clean":"2024-05-17T09:19:29.533641+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:19:29.533641+0000","last_undegraded":"2024-05-17T09:19:29.533641+0000","last_fullsized":"2024-05-17T09:19:29.533641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900338+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.900338+0000","last_peered":"2024-05-17T09:17:46.900338+0000","last_clean":"2024-05-17T09:17:46.900338+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.900338+0000","last_undegraded":"2024-05-17T09:17:46.900338+0000","last_fullsized":"2024-05-17T09:17:46.900338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829668+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:16:48.829668+0000","last_peered":"2024-05-17T09:16:48.829668+0000","last_clean":"2024-05-17T09:16:48.829668+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:16:48.829668+0000","last_undegraded":"2024-05-17T09:16:48.829668+0000","last_fullsized":"2024-05-17T09:16:48.829668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423264+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:16:48.423264+0000","last_peered":"2024-05-17T09:16:48.423264+0000","last_clean":"2024-05-17T09:16:48.423264+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:16:48.423264+0000","last_undegraded":"2024-05-17T09:16:48.423264+0000","last_fullsized":"2024-05-17T09:16:48.423264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897951+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.897951+0000","last_peered":"2024-05-17T09:17:46.897951+0000","last_clean":"2024-05-17T09:17:46.897951+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.897951+0000","last_undegraded":"2024-05-17T09:17:46.897951+0000","last_fullsized":"2024-05-17T09:17:46.897951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.939280+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:17:28.939280+0000","last_peered":"2024-05-17T09:17:28.939280+0000","last_clean":"2024-05-17T09:17:28.939280+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:17:28.939280+0000","last_undegraded":"2024-05-17T09:17:28.939280+0000","last_fullsized":"2024-05-17T09:17:28.939280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934940+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:17:11.934940+0000","last_peered":"2024-05-17T09:17:11.934940+0000","last_clean":"2024-05-17T09:17:11.934940+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:17:11.934940+0000","last_undegraded":"2024-05-17T09:17:11.934940+0000","last_fullsized":"2024-05-17T09:17:11.934940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904550+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904550+0000","last_peered":"2024-05-17T09:17:46.904550+0000","last_clean":"2024-05-17T09:17:46.904550+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904550+0000","last_undegraded":"2024-05-17T09:17:46.904550+0000","last_fullsized":"2024-05-17T09:17:46.904550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:27.771474+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:17:27.771474+0000","last_peered":"2024-05-17T09:17:27.771474+0000","last_clean":"2024-05-17T09:17:27.771474+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:17:27.771474+0000","last_undegraded":"2024-05-17T09:17:27.771474+0000","last_fullsized":"2024-05-17T09:17:27.771474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.384811+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:17:08.384811+0000","last_peered":"2024-05-17T09:17:08.384811+0000","last_clean":"2024-05-17T09:17:08.384811+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:17:08.384811+0000","last_undegraded":"2024-05-17T09:17:08.384811+0000","last_fullsized":"2024-05-17T09:17:08.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900921+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.900921+0000","last_peered":"2024-05-17T09:17:46.900921+0000","last_clean":"2024-05-17T09:17:46.900921+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.900921+0000","last_undegraded":"2024-05-17T09:17:46.900921+0000","last_fullsized":"2024-05-17T09:17:46.900921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320360+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:16:48.320360+0000","last_peered":"2024-05-17T09:16:48.320360+0000","last_clean":"2024-05-17T09:16:48.320360+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:16:48.320360+0000","last_undegraded":"2024-05-17T09:16:48.320360+0000","last_fullsized":"2024-05-17T09:16:48.320360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901492+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901492+0000","last_peered":"2024-05-17T09:17:46.901492+0000","last_clean":"2024-05-17T09:17:46.901492+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901492+0000","last_undegraded":"2024-05-17T09:17:46.901492+0000","last_fullsized":"2024-05-17T09:17:46.901492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:31.536246+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:19:31.536246+0000","last_peered":"2024-05-17T09:19:31.536246+0000","last_clean":"2024-05-17T09:19:31.536246+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:19:31.536246+0000","last_undegraded":"2024-05-17T09:19:31.536246+0000","last_fullsized":"2024-05-17T09:19:31.536246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387326+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:17:28.387326+0000","last_peered":"2024-05-17T09:17:28.387326+0000","last_clean":"2024-05-17T09:17:28.387326+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:17:28.387326+0000","last_undegraded":"2024-05-17T09:17:28.387326+0000","last_fullsized":"2024-05-17T09:17:28.387326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904968+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.904968+0000","last_peered":"2024-05-17T09:17:46.904968+0000","last_clean":"2024-05-17T09:17:46.904968+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.904968+0000","last_undegraded":"2024-05-17T09:17:46.904968+0000","last_fullsized":"2024-05-17T09:17:46.904968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.411802+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:17:11.411802+0000","last_peered":"2024-05-17T09:17:11.411802+0000","last_clean":"2024-05-17T09:17:11.411802+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:17:11.411802+0000","last_undegraded":"2024-05-17T09:17:11.411802+0000","last_fullsized":"2024-05-17T09:17:11.411802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595216+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595216+0000","last_peered":"2024-05-17T09:16:48.595216+0000","last_clean":"2024-05-17T09:16:48.595216+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595216+0000","last_undegraded":"2024-05-17T09:16:48.595216+0000","last_fullsized":"2024-05-17T09:16:48.595216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423844+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:16:48.423844+0000","last_peered":"2024-05-17T09:16:48.423844+0000","last_clean":"2024-05-17T09:16:48.423844+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:16:48.423844+0000","last_undegraded":"2024-05-17T09:16:48.423844+0000","last_fullsized":"2024-05-17T09:16:48.423844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903562+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903562+0000","last_peered":"2024-05-17T09:17:46.903562+0000","last_clean":"2024-05-17T09:17:46.903562+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903562+0000","last_undegraded":"2024-05-17T09:17:46.903562+0000","last_fullsized":"2024-05-17T09:17:46.903562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.384322+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:17:03.384322+0000","last_peered":"2024-05-17T09:17:03.384322+0000","last_clean":"2024-05-17T09:17:03.384322+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:17:03.384322+0000","last_undegraded":"2024-05-17T09:17:03.384322+0000","last_fullsized":"2024-05-17T09:17:03.384322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112162+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:17:24.112162+0000","last_peered":"2024-05-17T09:17:24.112162+0000","last_clean":"2024-05-17T09:17:24.112162+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:17:24.112162+0000","last_undegraded":"2024-05-17T09:17:24.112162+0000","last_fullsized":"2024-05-17T09:17:24.112162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.936560+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:17:08.936560+0000","last_peered":"2024-05-17T09:17:08.936560+0000","last_clean":"2024-05-17T09:17:08.936560+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:17:08.936560+0000","last_undegraded":"2024-05-17T09:17:08.936560+0000","last_fullsized":"2024-05-17T09:17:08.936560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903852+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903852+0000","last_peered":"2024-05-17T09:17:46.903852+0000","last_clean":"2024-05-17T09:17:46.903852+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903852+0000","last_undegraded":"2024-05-17T09:17:46.903852+0000","last_fullsized":"2024-05-17T09:17:46.903852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:20.633799+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:17:20.633799+0000","last_peered":"2024-05-17T09:17:20.633799+0000","last_clean":"2024-05-17T09:17:20.633799+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:17:20.633799+0000","last_undegraded":"2024-05-17T09:17:20.633799+0000","last_fullsized":"2024-05-17T09:17:20.633799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.948913+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:19:26.948913+0000","last_peered":"2024-05-17T09:19:26.948913+0000","last_clean":"2024-05-17T09:19:26.948913+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:19:26.948913+0000","last_undegraded":"2024-05-17T09:19:26.948913+0000","last_fullsized":"2024-05-17T09:19:26.948913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889973+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889973+0000","last_peered":"2024-05-17T09:17:46.889973+0000","last_clean":"2024-05-17T09:17:46.889973+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889973+0000","last_undegraded":"2024-05-17T09:17:46.889973+0000","last_fullsized":"2024-05-17T09:17:46.889973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.193083+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:17:26.193083+0000","last_peered":"2024-05-17T09:17:26.193083+0000","last_clean":"2024-05-17T09:17:26.193083+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:17:26.193083+0000","last_undegraded":"2024-05-17T09:17:26.193083+0000","last_fullsized":"2024-05-17T09:17:26.193083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.412427+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:17:16.412427+0000","last_peered":"2024-05-17T09:17:16.412427+0000","last_clean":"2024-05-17T09:17:16.412427+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:17:16.412427+0000","last_undegraded":"2024-05-17T09:17:16.412427+0000","last_fullsized":"2024-05-17T09:17:16.412427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898859+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898859+0000","last_peered":"2024-05-17T09:17:46.898859+0000","last_clean":"2024-05-17T09:17:46.898859+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898859+0000","last_undegraded":"2024-05-17T09:17:46.898859+0000","last_fullsized":"2024-05-17T09:17:46.898859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.908768+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:17:16.908768+0000","last_peered":"2024-05-17T09:17:16.908768+0000","last_clean":"2024-05-17T09:17:16.908768+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:17:16.908768+0000","last_undegraded":"2024-05-17T09:17:16.908768+0000","last_fullsized":"2024-05-17T09:17:16.908768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898286+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.898286+0000","last_peered":"2024-05-17T09:17:46.898286+0000","last_clean":"2024-05-17T09:17:46.898286+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.898286+0000","last_undegraded":"2024-05-17T09:17:46.898286+0000","last_fullsized":"2024-05-17T09:17:46.898286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:15.633296+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:17:15.633296+0000","last_peered":"2024-05-17T09:17:15.633296+0000","last_clean":"2024-05-17T09:17:15.633296+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:17:15.633296+0000","last_undegraded":"2024-05-17T09:17:15.633296+0000","last_fullsized":"2024-05-17T09:17:15.633296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901878+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.901878+0000","last_peered":"2024-05-17T09:17:46.901878+0000","last_clean":"2024-05-17T09:17:46.901878+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.901878+0000","last_undegraded":"2024-05-17T09:17:46.901878+0000","last_fullsized":"2024-05-17T09:17:46.901878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.206872+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:19:26.206872+0000","last_peered":"2024-05-17T09:19:26.206872+0000","last_clean":"2024-05-17T09:19:26.206872+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:19:26.206872+0000","last_undegraded":"2024-05-17T09:19:26.206872+0000","last_fullsized":"2024-05-17T09:19:26.206872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900941+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.900941+0000","last_peered":"2024-05-17T09:17:46.900941+0000","last_clean":"2024-05-17T09:17:46.900941+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.900941+0000","last_undegraded":"2024-05-17T09:17:46.900941+0000","last_fullsized":"2024-05-17T09:17:46.900941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:18.937946+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:17:18.937946+0000","last_peered":"2024-05-17T09:17:18.937946+0000","last_clean":"2024-05-17T09:17:18.937946+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:17:18.937946+0000","last_undegraded":"2024-05-17T09:17:18.937946+0000","last_fullsized":"2024-05-17T09:17:18.937946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908869+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908869+0000","last_peered":"2024-05-17T09:17:46.908869+0000","last_clean":"2024-05-17T09:17:46.908869+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908869+0000","last_undegraded":"2024-05-17T09:17:46.908869+0000","last_fullsized":"2024-05-17T09:17:46.908869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859520+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:16:48.859520+0000","last_peered":"2024-05-17T09:16:48.859520+0000","last_clean":"2024-05-17T09:16:48.859520+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:16:48.859520+0000","last_undegraded":"2024-05-17T09:16:48.859520+0000","last_fullsized":"2024-05-17T09:16:48.859520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319435+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:16:48.319435+0000","last_peered":"2024-05-17T09:16:48.319435+0000","last_clean":"2024-05-17T09:16:48.319435+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:16:48.319435+0000","last_undegraded":"2024-05-17T09:16:48.319435+0000","last_fullsized":"2024-05-17T09:16:48.319435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908319+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908319+0000","last_peered":"2024-05-17T09:17:46.908319+0000","last_clean":"2024-05-17T09:17:46.908319+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908319+0000","last_undegraded":"2024-05-17T09:17:46.908319+0000","last_fullsized":"2024-05-17T09:17:46.908319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423786+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:16:48.423786+0000","last_peered":"2024-05-17T09:16:48.423786+0000","last_clean":"2024-05-17T09:16:48.423786+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:16:48.423786+0000","last_undegraded":"2024-05-17T09:16:48.423786+0000","last_fullsized":"2024-05-17T09:16:48.423786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901697+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901697+0000","last_peered":"2024-05-17T09:17:46.901697+0000","last_clean":"2024-05-17T09:17:46.901697+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901697+0000","last_undegraded":"2024-05-17T09:17:46.901697+0000","last_fullsized":"2024-05-17T09:17:46.901697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423060+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:16:48.423060+0000","last_peered":"2024-05-17T09:16:48.423060+0000","last_clean":"2024-05-17T09:16:48.423060+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:16:48.423060+0000","last_undegraded":"2024-05-17T09:16:48.423060+0000","last_fullsized":"2024-05-17T09:16:48.423060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320546+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:16:48.320546+0000","last_peered":"2024-05-17T09:16:48.320546+0000","last_clean":"2024-05-17T09:16:48.320546+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:16:48.320546+0000","last_undegraded":"2024-05-17T09:16:48.320546+0000","last_fullsized":"2024-05-17T09:16:48.320546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897876+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897876+0000","last_peered":"2024-05-17T09:17:46.897876+0000","last_clean":"2024-05-17T09:17:46.897876+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897876+0000","last_undegraded":"2024-05-17T09:17:46.897876+0000","last_fullsized":"2024-05-17T09:17:46.897876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899078+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.899078+0000","last_peered":"2024-05-17T09:17:46.899078+0000","last_clean":"2024-05-17T09:17:46.899078+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.899078+0000","last_undegraded":"2024-05-17T09:17:46.899078+0000","last_fullsized":"2024-05-17T09:17:46.899078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888934+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888934+0000","last_peered":"2024-05-17T09:17:46.888934+0000","last_clean":"2024-05-17T09:17:46.888934+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888934+0000","last_undegraded":"2024-05-17T09:17:46.888934+0000","last_fullsized":"2024-05-17T09:17:46.888934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896944+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896944+0000","last_peered":"2024-05-17T09:17:46.896944+0000","last_clean":"2024-05-17T09:17:46.896944+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896944+0000","last_undegraded":"2024-05-17T09:17:46.896944+0000","last_fullsized":"2024-05-17T09:17:46.896944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904847+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904847+0000","last_peered":"2024-05-17T09:17:46.904847+0000","last_clean":"2024-05-17T09:17:46.904847+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904847+0000","last_undegraded":"2024-05-17T09:17:46.904847+0000","last_fullsized":"2024-05-17T09:17:46.904847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904438+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904438+0000","last_peered":"2024-05-17T09:17:46.904438+0000","last_clean":"2024-05-17T09:17:46.904438+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904438+0000","last_undegraded":"2024-05-17T09:17:46.904438+0000","last_fullsized":"2024-05-17T09:17:46.904438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899186+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899186+0000","last_peered":"2024-05-17T09:17:46.899186+0000","last_clean":"2024-05-17T09:17:46.899186+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899186+0000","last_undegraded":"2024-05-17T09:17:46.899186+0000","last_fullsized":"2024-05-17T09:17:46.899186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891995+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891995+0000","last_peered":"2024-05-17T09:17:46.891995+0000","last_clean":"2024-05-17T09:17:46.891995+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891995+0000","last_undegraded":"2024-05-17T09:17:46.891995+0000","last_fullsized":"2024-05-17T09:17:46.891995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886503+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.886503+0000","last_peered":"2024-05-17T09:17:46.886503+0000","last_clean":"2024-05-17T09:17:46.886503+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.886503+0000","last_undegraded":"2024-05-17T09:17:46.886503+0000","last_fullsized":"2024-05-17T09:17:46.886503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899965+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899965+0000","last_peered":"2024-05-17T09:17:46.899965+0000","last_clean":"2024-05-17T09:17:46.899965+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899965+0000","last_undegraded":"2024-05-17T09:17:46.899965+0000","last_fullsized":"2024-05-17T09:17:46.899965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894550+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894550+0000","last_peered":"2024-05-17T09:17:46.894550+0000","last_clean":"2024-05-17T09:17:46.894550+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894550+0000","last_undegraded":"2024-05-17T09:17:46.894550+0000","last_fullsized":"2024-05-17T09:17:46.894550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908241+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908241+0000","last_peered":"2024-05-17T09:17:46.908241+0000","last_clean":"2024-05-17T09:17:46.908241+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908241+0000","last_undegraded":"2024-05-17T09:17:46.908241+0000","last_fullsized":"2024-05-17T09:17:46.908241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904498+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904498+0000","last_peered":"2024-05-17T09:17:46.904498+0000","last_clean":"2024-05-17T09:17:46.904498+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904498+0000","last_undegraded":"2024-05-17T09:17:46.904498+0000","last_fullsized":"2024-05-17T09:17:46.904498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899413+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899413+0000","last_peered":"2024-05-17T09:17:46.899413+0000","last_clean":"2024-05-17T09:17:46.899413+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899413+0000","last_undegraded":"2024-05-17T09:17:46.899413+0000","last_fullsized":"2024-05-17T09:17:46.899413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1200,"num_read_kb":682,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":215928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1176674,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266773,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1112817,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626528,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":575293,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":577634,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":516180,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822757,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1103284,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:19:36.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:19:36.584 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:36.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:36 smithi019 ceph-mon[31620]: pgmap v433: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:36 smithi069 ceph-mon[35081]: pgmap v433: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:36 smithi079 ceph-mon[35502]: pgmap v433: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:37.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:37 smithi069 ceph-mon[35081]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:37 smithi069 ceph-mon[35081]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:37 smithi069 ceph-mon[35081]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:37 smithi019 ceph-mon[31620]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:37 smithi019 ceph-mon[31620]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:37 smithi019 ceph-mon[31620]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:37 smithi079 ceph-mon[35502]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:37 smithi079 ceph-mon[35502]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:37 smithi079 ceph-mon[35502]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:38.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 -- 172.21.15.19:0/2581476883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48109860 msgr2=0x7efe481111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:38.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 --2- 172.21.15.19:0/2581476883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48109860 0x7efe481111c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efe400099c0 tx=0x7efe40009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 -- 172.21.15.19:0/2581476883 shutdown_connections 2024-05-17T09:19:38.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 --2- 172.21.15.19:0/2581476883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe481045e0 0x7efe481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 --2- 172.21.15.19:0/2581476883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48109860 0x7efe481111c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 --2- 172.21.15.19:0/2581476883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe48108f10 0x7efe48109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 -- 172.21.15.19:0/2581476883 >> 172.21.15.19:0/2581476883 conn(0x7efe48100030 msgr2=0x7efe48102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:38.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 -- 172.21.15.19:0/2581476883 shutdown_connections 2024-05-17T09:19:38.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.366+0000 7efe4ef57700 1 -- 172.21.15.19:0/2581476883 wait complete. 2024-05-17T09:19:38.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 Processor -- start 2024-05-17T09:19:38.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 -- start start 2024-05-17T09:19:38.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 0x7efe48075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:38.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe48108f10 0x7efe48075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:38.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe48109860 0x7efe48076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:38.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe48114340 con 0x7efe481045e0 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe481141c0 con 0x7efe48109860 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ef57700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe481144c0 con 0x7efe48108f10 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ccf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 0x7efe48075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ccf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 0x7efe48075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35274/0 (socket says 172.21.15.19:35274) 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4ccf3700 1 -- 172.21.15.19:0/1398933411 learned_addr learned my addr 172.21.15.19:0/1398933411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe4d4f4700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe48109860 0x7efe48076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.367+0000 7efe47fff700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe48108f10 0x7efe48075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ccf3700 1 -- 172.21.15.19:0/1398933411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe48108f10 msgr2=0x7efe48075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ccf3700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe48108f10 0x7efe48075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ccf3700 1 -- 172.21.15.19:0/1398933411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe48109860 msgr2=0x7efe48076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ccf3700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe48109860 0x7efe48076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ccf3700 1 -- 172.21.15.19:0/1398933411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe481a9060 con 0x7efe481045e0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ccf3700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 0x7efe48075ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c920 tx=0x7efe3000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30007b00 con 0x7efe481045e0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe481a9350 con 0x7efe481045e0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe30004cc0 con 0x7efe481045e0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30005440 con 0x7efe481045e0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.368+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe481a9890 con 0x7efe481045e0 2024-05-17T09:19:38.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.369+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe480512b0 con 0x7efe481045e0 2024-05-17T09:19:38.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.371+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efe30004750 con 0x7efe481045e0 2024-05-17T09:19:38.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.371+0000 7efe45ffb700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe340b2880 0x7efe340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:38.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.372+0000 7efe47fff700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe340b2880 0x7efe340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:38.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.372+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efe300e2150 con 0x7efe481045e0 2024-05-17T09:19:38.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.372+0000 7efe47fff700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe340b2880 0x7efe340b4d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe48107870 tx=0x7efe38009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:38.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.376+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe30014030 con 0x7efe481045e0 2024-05-17T09:19:38.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.645+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe4802e5e0 con 0x7efe481045e0 2024-05-17T09:19:38.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:38 smithi019 ceph-mon[31620]: pgmap v434: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:38.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.646+0000 7efe45ffb700 1 -- 172.21.15.19:0/1398933411 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7efe300a9ac0 con 0x7efe481045e0 2024-05-17T09:19:38.647 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:38.647 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:19:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe340b2880 msgr2=0x7efe340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe340b2880 0x7efe340b4d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe48107870 tx=0x7efe38009040 comp rx=0 tx=0).stop 2024-05-17T09:19:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 msgr2=0x7efe48075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 0x7efe48075ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c920 tx=0x7efe3000cc30 comp rx=0 tx=0).stop 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 shutdown_connections 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe48109860 0x7efe48076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe481045e0 0x7efe48075ab0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe340b2880 0x7efe340b4d90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 --2- 172.21.15.19:0/1398933411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe48108f10 0x7efe48075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 >> 172.21.15.19:0/1398933411 conn(0x7efe48100030 msgr2=0x7efe481086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 shutdown_connections 2024-05-17T09:19:38.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:38.649+0000 7efe4ef57700 1 -- 172.21.15.19:0/1398933411 wait complete. 2024-05-17T09:19:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:19:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:38 smithi069 ceph-mon[35081]: pgmap v434: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:38 smithi079 ceph-mon[35502]: pgmap v434: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:39.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:19:39.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:39 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1398933411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:39.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:39 smithi069 ceph-mon[35081]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:39.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:39 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1398933411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:39.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:39 smithi019 ceph-mon[31620]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:39 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1398933411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:39 smithi079 ceph-mon[35502]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 -- 172.21.15.19:0/2750220241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f241034b0 msgr2=0x7f1f241038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 --2- 172.21.15.19:0/2750220241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f241034b0 0x7f1f241038c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1f140099c0 tx=0x7f1f14009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 -- 172.21.15.19:0/2750220241 shutdown_connections 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 --2- 172.21.15.19:0/2750220241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24108730 0x7f1f24110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 --2- 172.21.15.19:0/2750220241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f241034b0 0x7f1f241038c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 --2- 172.21.15.19:0/2750220241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f24107de0 0x7f1f241081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 -- 172.21.15.19:0/2750220241 >> 172.21.15.19:0/2750220241 conn(0x7f1f240fedb0 msgr2=0x7f1f241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 -- 172.21.15.19:0/2750220241 shutdown_connections 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 -- 172.21.15.19:0/2750220241 wait complete. 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.384+0000 7f1f297fc700 1 Processor -- start 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 -- start start 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f241034b0 0x7f1f2419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24107de0 0x7f1f2419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 0x7f1f2419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24113210 con 0x7f1f24108730 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24113090 con 0x7f1f24107de0 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f297fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24113390 con 0x7f1f241034b0 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 0x7f1f2419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 0x7f1f2419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35300/0 (socket says 172.21.15.19:35300) 2024-05-17T09:19:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 -- 172.21.15.19:0/3714357345 learned_addr learned my addr 172.21.15.19:0/3714357345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f241034b0 0x7f1f2419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f227fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24107de0 0x7f1f2419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 -- 172.21.15.19:0/3714357345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f241034b0 msgr2=0x7f1f2419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f241034b0 0x7f1f2419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 -- 172.21.15.19:0/3714357345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24107de0 msgr2=0x7f1f2419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24107de0 0x7f1f2419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f237fe700 1 -- 172.21.15.19:0/3714357345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f241a6e50 con 0x7f1f24108730 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.385+0000 7f1f227fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24107de0 0x7f1f2419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.386+0000 7f1f237fe700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 0x7f1f2419c630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00ada0 tx=0x7f1f1c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.386+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017020 con 0x7f1f24108730 2024-05-17T09:19:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.386+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f241a7140 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.386+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f1c00ce10 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.386+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c005440 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.386+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f241a7680 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.387+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f240512b0 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.389+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f1c004ad0 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.389+0000 7f1f0bfff700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f0c0b2880 0x7f1f0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.389+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f0c0b2880 0x7f1f0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.389+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1f1c0e2390 con 0x7f1f24108730 2024-05-17T09:19:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.390+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f0c0b2880 0x7f1f0c0b4d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f140058e0 tx=0x7f1f14005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.393+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f1c021020 con 0x7f1f24108730 2024-05-17T09:19:41.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.608+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f2402e700 con 0x7f1f24108730 2024-05-17T09:19:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.609+0000 7f1f0bfff700 1 -- 172.21.15.19:0/3714357345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f1f1c0a9d30 con 0x7f1f24108730 2024-05-17T09:19:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:41.610 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:19:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f0c0b2880 msgr2=0x7f1f0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f0c0b2880 0x7f1f0c0b4d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f140058e0 tx=0x7f1f14005940 comp rx=0 tx=0).stop 2024-05-17T09:19:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 msgr2=0x7f1f2419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 0x7f1f2419c630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00ada0 tx=0x7f1f1c00c330 comp rx=0 tx=0).stop 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 shutdown_connections 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f0c0b2880 0x7f1f0c0b4d90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f241034b0 0x7f1f2419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f24108730 0x7f1f2419c630 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 --2- 172.21.15.19:0/3714357345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f24107de0 0x7f1f2419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 >> 172.21.15.19:0/3714357345 conn(0x7f1f240fedb0 msgr2=0x7f1f24101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 shutdown_connections 2024-05-17T09:19:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:41.612+0000 7f1f297fc700 1 -- 172.21.15.19:0/3714357345 wait complete. 2024-05-17T09:19:42.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:42 smithi079 ceph-mon[35502]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-17T09:19:42.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:42 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3714357345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:42.330 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:42 smithi019 ceph-mon[31620]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-17T09:19:42.330 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:42 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3714357345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:42.362 DEBUG:tasks.fwd_scrub:Yielding 2024-05-17T09:19:42.362 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-17T09:19:42.372 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:19:42.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-17T09:19:42.373 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-17T09:19:42.373 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:19:42.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:19:42.376 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-17T09:19:42.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T09:19:42.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:19:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:42 smithi069 ceph-mon[35081]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-17T09:19:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:42 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3714357345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:19:42.642 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:42.661 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:44.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:44 smithi079 ceph-mon[35502]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-17T09:19:44.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:44 smithi069 ceph-mon[35081]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-17T09:19:44.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:44 smithi019 ceph-mon[31620]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/443498561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 msgr2=0x7fb8e010a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/443498561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e010a7e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009990 tx=0x7fb8d0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/443498561 shutdown_connections 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/443498561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e0103120 0x7fb8e0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/443498561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e010a7e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/443498561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8e01027d0 0x7fb8e0102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/443498561 >> 172.21.15.19:0/443498561 conn(0x7fb8e00fe6e0 msgr2=0x7fb8e0100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/443498561 shutdown_connections 2024-05-17T09:19:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.558+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/443498561 wait complete. 2024-05-17T09:19:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.559+0000 7fb8e5cd1700 1 Processor -- start 2024-05-17T09:19:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.559+0000 7fb8e5cd1700 1 -- start start 2024-05-17T09:19:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8e5cd1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e01027d0 0x7fb8e019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8e5cd1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8e0103120 0x7fb8e019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8e5cd1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8e5cd1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0114320 con 0x7fb8e010a3d0 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8e5cd1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01141a0 con 0x7fb8e0103120 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8e5cd1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01144a0 con 0x7fb8e01027d0 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37564/0 (socket says 172.21.15.19:37564) 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 -- 172.21.15.19:0/4173932947 learned_addr learned my addr 172.21.15.19:0/4173932947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e01027d0 0x7fb8e019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8deffd700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8e0103120 0x7fb8e019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 -- 172.21.15.19:0/4173932947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e01027d0 msgr2=0x7fb8e019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e01027d0 0x7fb8e019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 -- 172.21.15.19:0/4173932947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8e0103120 msgr2=0x7fb8e019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8e0103120 0x7fb8e019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 -- 172.21.15.19:0/4173932947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e019fc00 con 0x7fb8e010a3d0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dffff700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e019f560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8007ee0 tx=0x7fb8d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.560+0000 7fb8dcff9700 1 -- 172.21.15.19:0/4173932947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8004bb0 con 0x7fb8e010a3d0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.561+0000 7fb8dcff9700 1 -- 172.21.15.19:0/4173932947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d8004d50 con 0x7fb8e010a3d0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.561+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e019fef0 con 0x7fb8e010a3d0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.561+0000 7fb8dcff9700 1 -- 172.21.15.19:0/4173932947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80058b0 con 0x7fb8e010a3d0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.561+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a7810 con 0x7fb8e010a3d0 2024-05-17T09:19:45.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.563+0000 7fb8dcff9700 1 -- 172.21.15.19:0/4173932947 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb8d801a020 con 0x7fb8e010a3d0 2024-05-17T09:19:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.564+0000 7fb8dcff9700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8c80b2880 0x7fb8c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.564+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8c80b2880 0x7fb8c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.564+0000 7fb8dcff9700 1 -- 172.21.15.19:0/4173932947 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb8d80e1c20 con 0x7fb8e010a3d0 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8c80b2880 0x7fb8c80b4d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000b5c0 tx=0x7fb8d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 Processor -- start 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 -- start start 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d0f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 0x55e964d0a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e964c37df0 con 0x55e964d09d50 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e964c27990 con 0x55e964b8aec0 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e964c2b9d0 con 0x55e964c764b0 2024-05-17T09:19:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 0x55e964d0a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 0x55e964d0a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37578/0 (socket says 172.21.15.19:37578) 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8dffff700 1 -- 172.21.15.19:0/2494132866 learned_addr learned my addr 172.21.15.19:0/2494132866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8deffd700 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.565+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d0f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8dffff700 1 -- 172.21.15.19:0/2494132866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 msgr2=0x55e964d0f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8dffff700 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d0f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8dffff700 1 -- 172.21.15.19:0/2494132866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 msgr2=0x55e964d0ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8dffff700 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8dffff700 1 -- 172.21.15.19:0/2494132866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e964bd1030 con 0x55e964d09d50 2024-05-17T09:19:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8dffff700 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 0x55e964d0a220 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80a8b90 tx=0x7fb8d80a8d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d30d72e8e3cb730 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8c5ffb700 1 -- 172.21.15.19:0/2494132866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80a9590 con 0x55e964d09d50 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8c5ffb700 1 -- 172.21.15.19:0/2494132866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d80a8e90 con 0x55e964d09d50 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.566+0000 7fb8c5ffb700 1 -- 172.21.15.19:0/2494132866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80a90f0 con 0x55e964d09d50 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/2494132866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 msgr2=0x55e964d0a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 0x55e964d0a220 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80a8b90 tx=0x7fb8d80a8d60 comp rx=0 tx=0).stop 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/2494132866 shutdown_connections 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d0f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964d09d50 0x55e964d0a220 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/2494132866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/2494132866 >> 172.21.15.19:0/2494132866 conn(0x55e964beb240 msgr2=0x55e964d04850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/2494132866 shutdown_connections 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/2494132866 wait complete. 2024-05-17T09:19:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.567+0000 7fb8f46e6b80 1 -- start start 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 Processor -- start 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 0x55e964d13680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d13bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d14100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e964b2e020 con 0x55e964b8aec0 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e964b8fe20 con 0x55e964d09d50 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e964d16620 con 0x55e964c764b0 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 0x55e964d13680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8f46e6b80 10 client.0 _ll_register_callbacks cb 0x55e964bc29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 0x55e964d13680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37594/0 (socket says 172.21.15.19:37594) 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8df7fe700 1 -- 172.21.15.19:0/1926886471 learned_addr learned my addr 172.21.15.19:0/1926886471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8deffd700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d13bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.568+0000 7fb8dffff700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8df7fe700 1 -- 172.21.15.19:0/1926886471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 msgr2=0x55e964d13bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d13bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8df7fe700 1 -- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 msgr2=0x55e964d14100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d14100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8df7fe700 1 -- 172.21.15.19:0/1926886471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e964b77950 con 0x55e964b8aec0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8dffff700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d14100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 0x55e964d13680 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0015000 tx=0x7fb8d001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa8968b99c7a311 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8beffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8beffd700 1 -- 172.21.15.19:0/1926886471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000be90 con 0x55e964b8aec0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8beffd700 1 -- 172.21.15.19:0/1926886471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb8d000f460 con 0x55e964b8aec0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d80a8e90 con 0x55e964b8aec0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8beffd700 1 -- 172.21.15.19:0/1926886471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8d003d050 con 0x55e964b8aec0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8f46e6b80 10 client.15312 fetch_fsmap learned FSMap version 27 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e964be17e0 con 0x55e964b8aec0 2024-05-17T09:19:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.569+0000 7fb8beffd700 1 -- 172.21.15.19:0/1926886471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d80a8e90 con 0x55e964b8aec0 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8beffd700 1 -- 172.21.15.19:0/1926886471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8d0022070 con 0x55e964b8aec0 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8f46e6b80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8f46e6b80 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8f46e6b80 20 client.15312 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d25a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8f46e6b80 4 client.15312 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e964cd5f50 con 0x55e964d23570 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8deffd700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d25a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8deffd700 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d25a40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f3dd8010d7d039 server_cookie=2ddd52458314cff in_seq=0 out_seq=0 2024-05-17T09:19:45.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.570+0000 7fb8beffd700 10 client.15312 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:45.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8beffd700 1 -- 172.21.15.19:0/1926886471 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e964cd5f50 con 0x55e964d23570 2024-05-17T09:19:45.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8beffd700 10 client.15312 handle_command_reply: tid=0 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 client.15312 shutdown 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 msgr2=0x55e964d13680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 0x55e964d13680 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0015000 tx=0x7fb8d001a050 comp rx=0 tx=0).stop 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 shutdown_connections 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d25a40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8beffd700 0 client.15312 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d14100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964b8aec0 0x55e964d13680 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.571+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/1926886471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964c764b0 0x55e964d13bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.572+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 >> 172.21.15.19:0/1926886471 conn(0x55e964beb240 msgr2=0x55e964d04810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.572+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 shutdown_connections 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.572+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/1926886471 wait complete. 2024-05-17T09:19:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.572+0000 7fb8f46e6b80 20 client.15312 trim_cache size 0 max 16384 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.592+0000 7fb8f46e6b80 1 Processor -- start 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.592+0000 7fb8f46e6b80 1 -- start start 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964d09d50 0x55e964d12550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01b2c20 con 0x55e964c764b0 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01b04b0 con 0x55e964b8aec0 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01ae300 con 0x55e964d09d50 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d0dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37604/0 (socket says 172.21.15.19:37604) 2024-05-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 -- 172.21.15.19:0/948214986 learned_addr learned my addr 172.21.15.19:0/948214986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8dffff700 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964d09d50 0x55e964d12550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 -- 172.21.15.19:0/948214986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964d09d50 msgr2=0x55e964d12550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964d09d50 0x55e964d12550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 -- 172.21.15.19:0/948214986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 msgr2=0x55e964d0ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 -- 172.21.15.19:0/948214986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e964d04d80 con 0x55e964c764b0 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.593+0000 7fb8deffd700 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d0dcd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01002d0 tx=0x7fb8d400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacd9aa69594a405 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8c57fa700 1 -- 172.21.15.19:0/948214986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4003b80 con 0x55e964c764b0 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8c57fa700 1 -- 172.21.15.19:0/948214986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d4004cc0 con 0x55e964c764b0 2024-05-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8c57fa700 1 -- 172.21.15.19:0/948214986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4005590 con 0x55e964c764b0 2024-05-17T09:19:45.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/948214986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 msgr2=0x55e964d0dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d0dcd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01002d0 tx=0x7fb8d400b410 comp rx=0 tx=0).stop 2024-05-17T09:19:45.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/948214986 shutdown_connections 2024-05-17T09:19:45.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964d09d50 0x55e964d12550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964b8aec0 0x55e964d0ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/948214986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d0dcd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/948214986 >> 172.21.15.19:0/948214986 conn(0x55e964beb240 msgr2=0x55e964d11e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.594+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/948214986 shutdown_connections 2024-05-17T09:19:45.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/948214986 wait complete. 2024-05-17T09:19:45.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 -- start start 2024-05-17T09:19:45.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 Processor -- start 2024-05-17T09:19:45.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964b8aec0 0x55e964d18840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d18d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01ae970 con 0x55e964c764b0 2024-05-17T09:19:45.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01afde0 con 0x55e964d09d50 2024-05-17T09:19:45.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01af040 con 0x55e964b8aec0 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.595+0000 7fb8f46e6b80 10 client.0 _ll_register_callbacks cb 0x55e964bc29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d18d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37620/0 (socket says 172.21.15.19:37620) 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 -- 172.21.15.19:0/3067067105 learned_addr learned my addr 172.21.15.19:0/3067067105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964b8aec0 0x55e964d18840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8dffff700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 -- 172.21.15.19:0/3067067105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964b8aec0 msgr2=0x55e964d18840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964b8aec0 0x55e964d18840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 -- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 msgr2=0x55e964d192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 -- 172.21.15.19:0/3067067105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e964d8e0e0 con 0x55e964c764b0 2024-05-17T09:19:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8dffff700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8deffd700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d18d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005130 tx=0x7fb8d4005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded21c05ed16a0c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8beffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8beffd700 1 -- 172.21.15.19:0/3067067105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4004b20 con 0x55e964c764b0 2024-05-17T09:19:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8beffd700 1 -- 172.21.15.19:0/3067067105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb8d4003d90 con 0x55e964c764b0 2024-05-17T09:19:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d4004cc0 con 0x55e964c764b0 2024-05-17T09:19:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.596+0000 7fb8beffd700 1 -- 172.21.15.19:0/3067067105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8d403a050 con 0x55e964c764b0 2024-05-17T09:19:45.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 10 client.15318 fetch_fsmap learned FSMap version 27 2024-05-17T09:19:45.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e964d8f300 con 0x55e964c764b0 2024-05-17T09:19:45.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8beffd700 1 -- 172.21.15.19:0/3067067105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d4004cc0 con 0x55e964c764b0 2024-05-17T09:19:45.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8beffd700 1 -- 172.21.15.19:0/3067067105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8d402f020 con 0x55e964c764b0 2024-05-17T09:19:45.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:19:45.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:19:45.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 20 client.15318 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:45.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d91940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:45.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 4 client.15318 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:19:45.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e964c39d40 con 0x55e964d23570 2024-05-17T09:19:45.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.597+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d91940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:45.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.598+0000 7fb8beffd700 10 client.15318 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.598+0000 7fb8df7fe700 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d91940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c891f782ad38af server_cookie=c8802ecb0d02c5d5 in_seq=0 out_seq=0 2024-05-17T09:19:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8beffd700 1 -- 172.21.15.19:0/3067067105 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e964c39d40 con 0x55e964d23570 2024-05-17T09:19:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8beffd700 10 client.15318 handle_command_reply: tid=0 2024-05-17T09:19:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 client.15318 shutdown 2024-05-17T09:19:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 msgr2=0x55e964d18d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d18d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4005130 tx=0x7fb8d4005160 comp rx=0 tx=0).stop 2024-05-17T09:19:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 shutdown_connections 2024-05-17T09:19:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e964b8aec0 0x55e964d18840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e964d23570 0x55e964d91940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8beffd700 0 client.15318 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e964d09d50 0x55e964d192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 --2- 172.21.15.19:0/3067067105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e964c764b0 0x55e964d18d80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 >> 172.21.15.19:0/3067067105 conn(0x55e964beb240 msgr2=0x55e964d0da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 shutdown_connections 2024-05-17T09:19:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 1 -- 172.21.15.19:0/3067067105 wait complete. 2024-05-17T09:19:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.599+0000 7fb8f46e6b80 20 client.15318 trim_cache size 0 max 16384 2024-05-17T09:19:45.613 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:19:45.613 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:19:45.613 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "a531b57f-a265-466e-9985-1375efb7f296", 2024-05-17T09:19:45.613 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:19:45.614 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:19:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8c80b2880 msgr2=0x7fb8c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8c80b2880 0x7fb8c80b4d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000b5c0 tx=0x7fb8d0005960 comp rx=0 tx=0).stop 2024-05-17T09:19:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 msgr2=0x7fb8e019f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e019f560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8007ee0 tx=0x7fb8d800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 shutdown_connections 2024-05-17T09:19:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8c80b2880 0x7fb8c80b4d90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e01027d0 0x7fb8e019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8e010a3d0 0x7fb8e019f560 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 --2- 172.21.15.19:0/4173932947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8e0103120 0x7fb8e019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 >> 172.21.15.19:0/4173932947 conn(0x7fb8e00fe6e0 msgr2=0x7fb8e0100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 shutdown_connections 2024-05-17T09:19:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:45.600+0000 7fb8e5cd1700 1 -- 172.21.15.19:0/4173932947 wait complete. 2024-05-17T09:19:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:46 smithi079 ceph-mon[35502]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T09:19:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:46 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:19:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:46 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:19:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:46 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:19:46.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:46 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:19:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:46 smithi069 ceph-mon[35081]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T09:19:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:46 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:19:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:46 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:19:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:46 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:19:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:46 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:19:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:46 smithi019 ceph-mon[31620]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T09:19:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:46 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:19:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:46 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:19:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:46 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:19:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:46 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:19:46.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:19:46.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.744+0000 7f0d1de01700 1 -- 172.21.15.19:0/1127281918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18107de0 msgr2=0x7f0d181081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.744+0000 7f0d1de01700 1 --2- 172.21.15.19:0/1127281918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18107de0 0x7f0d181081f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0099c0 tx=0x7f0d0c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 -- 172.21.15.19:0/1127281918 shutdown_connections 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 --2- 172.21.15.19:0/1127281918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d181034b0 0x7f0d181038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 --2- 172.21.15.19:0/1127281918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d18108730 0x7f0d18110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 --2- 172.21.15.19:0/1127281918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18107de0 0x7f0d181081f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 -- 172.21.15.19:0/1127281918 >> 172.21.15.19:0/1127281918 conn(0x7f0d180fedb0 msgr2=0x7f0d181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 -- 172.21.15.19:0/1127281918 shutdown_connections 2024-05-17T09:19:46.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.745+0000 7f0d1de01700 1 -- 172.21.15.19:0/1127281918 wait complete. 2024-05-17T09:19:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.746+0000 7f0d1de01700 1 Processor -- start 2024-05-17T09:19:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.746+0000 7f0d1de01700 1 -- start start 2024-05-17T09:19:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.746+0000 7f0d1de01700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 0x7f0d1819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.746+0000 7f0d1de01700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d18107de0 0x7f0d1819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.746+0000 7f0d1de01700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d18108730 0x7f0d181a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.746+0000 7f0d1de01700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18114320 con 0x7f0d181034b0 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d1de01700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181141a0 con 0x7f0d18107de0 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d1de01700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181144a0 con 0x7f0d18108730 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d18108730 0x7f0d181a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 0x7f0d1819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 0x7f0d1819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37644/0 (socket says 172.21.15.19:37644) 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d18108730 0x7f0d181a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:44356/0 (socket says 172.21.15.19:44356) 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 -- 172.21.15.19:0/593887712 learned_addr learned my addr 172.21.15.19:0/593887712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d16ffd700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d18107de0 0x7f0d1819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 -- 172.21.15.19:0/593887712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d18108730 msgr2=0x7f0d181a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d18108730 0x7f0d181a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 -- 172.21.15.19:0/593887712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d18107de0 msgr2=0x7f0d1819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d18107de0 0x7f0d1819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 -- 172.21.15.19:0/593887712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d181a1ea0 con 0x7f0d181034b0 2024-05-17T09:19:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.747+0000 7f0d177fe700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 0x7f0d1819cfd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800d950 tx=0x7f0d0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.748+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08016070 con 0x7f0d181034b0 2024-05-17T09:19:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.748+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d181a2190 con 0x7f0d181034b0 2024-05-17T09:19:46.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.748+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d181a9ab0 con 0x7f0d181034b0 2024-05-17T09:19:46.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.748+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d0800fd20 con 0x7f0d181034b0 2024-05-17T09:19:46.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.749+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800e3b0 con 0x7f0d181034b0 2024-05-17T09:19:46.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.750+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d04005330 con 0x7f0d181034b0 2024-05-17T09:19:46.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.751+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0d08019070 con 0x7f0d181034b0 2024-05-17T09:19:46.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.752+0000 7f0d14ff9700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d000b2900 0x7f0d000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:46.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.752+0000 7f0d16ffd700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d000b2900 0x7f0d000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:46.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.752+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0d080e2300 con 0x7f0d181034b0 2024-05-17T09:19:46.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.753+0000 7f0d16ffd700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d000b2900 0x7f0d000b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009990 tx=0x7f0d0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:46.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.757+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d080a9c70 con 0x7f0d181034b0 2024-05-17T09:19:46.790 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.980+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d04000bc0 con 0x7f0d000b2900 2024-05-17T09:19:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.990+0000 7f0d14ff9700 1 -- 172.21.15.19:0/593887712 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340727 (secure 0 0 0) 0x7f0d0c07b310 con 0x7f0d000b2900 2024-05-17T09:19:46.995 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d000b2900 msgr2=0x7f0d000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d000b2900 0x7f0d000b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009990 tx=0x7f0d0c005fb0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 msgr2=0x7f0d1819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 0x7f0d1819cfd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800d950 tx=0x7f0d0800dc60 comp rx=0 tx=0).stop 2024-05-17T09:19:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 shutdown_connections 2024-05-17T09:19:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0d000b2900 0x7f0d000b4e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d18108730 0x7f0d181a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d181034b0 0x7f0d1819cfd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 --2- 172.21.15.19:0/593887712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d18107de0 0x7f0d1819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.993+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 >> 172.21.15.19:0/593887712 conn(0x7f0d180fedb0 msgr2=0x7f0d18109730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.994+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 shutdown_connections 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:46.994+0000 7f0d1de01700 1 -- 172.21.15.19:0/593887712 wait complete. 2024-05-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:19:47.798 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-05-17T09:19:46.952641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1347,"num_read_kb":777,"num_write":313,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348656,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593168,"statfs":{"total":1151940427776,"available":1151583404032,"internally_reserved":0,"allocated":12271616,"data_stored":8153408,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":31,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004146"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906823+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906823+0000","last_peered":"2024-05-17T09:17:46.906823+0000","last_clean":"2024-05-17T09:17:46.906823+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906823+0000","last_undegraded":"2024-05-17T09:17:46.906823+0000","last_fullsized":"2024-05-17T09:17:46.906823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901351+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.901351+0000","last_peered":"2024-05-17T09:17:46.901351+0000","last_clean":"2024-05-17T09:17:46.901351+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.901351+0000","last_undegraded":"2024-05-17T09:17:46.901351+0000","last_fullsized":"2024-05-17T09:17:46.901351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905015+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.905015+0000","last_peered":"2024-05-17T09:17:46.905015+0000","last_clean":"2024-05-17T09:17:46.905015+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.905015+0000","last_undegraded":"2024-05-17T09:17:46.905015+0000","last_fullsized":"2024-05-17T09:17:46.905015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886153+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.886153+0000","last_peered":"2024-05-17T09:17:46.886153+0000","last_clean":"2024-05-17T09:17:46.886153+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.886153+0000","last_undegraded":"2024-05-17T09:17:46.886153+0000","last_fullsized":"2024-05-17T09:17:46.886153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897920+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.897920+0000","last_peered":"2024-05-17T09:17:46.897920+0000","last_clean":"2024-05-17T09:17:46.897920+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.897920+0000","last_undegraded":"2024-05-17T09:17:46.897920+0000","last_fullsized":"2024-05-17T09:17:46.897920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893468+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893468+0000","last_peered":"2024-05-17T09:17:46.893468+0000","last_clean":"2024-05-17T09:17:46.893468+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893468+0000","last_undegraded":"2024-05-17T09:17:46.893468+0000","last_fullsized":"2024-05-17T09:17:46.893468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908147+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908147+0000","last_peered":"2024-05-17T09:17:46.908147+0000","last_clean":"2024-05-17T09:17:46.908147+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908147+0000","last_undegraded":"2024-05-17T09:17:46.908147+0000","last_fullsized":"2024-05-17T09:17:46.908147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889416+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889416+0000","last_peered":"2024-05-17T09:17:46.889416+0000","last_clean":"2024-05-17T09:17:46.889416+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889416+0000","last_undegraded":"2024-05-17T09:17:46.889416+0000","last_fullsized":"2024-05-17T09:17:46.889416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:16.948046+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:19:16.948046+0000","last_peered":"2024-05-17T09:19:16.948046+0000","last_clean":"2024-05-17T09:19:16.948046+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:19:16.948046+0000","last_undegraded":"2024-05-17T09:19:16.948046+0000","last_fullsized":"2024-05-17T09:19:16.948046+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898655+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898655+0000","last_peered":"2024-05-17T09:17:46.898655+0000","last_clean":"2024-05-17T09:17:46.898655+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898655+0000","last_undegraded":"2024-05-17T09:17:46.898655+0000","last_fullsized":"2024-05-17T09:17:46.898655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:05.632115+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:17:05.632115+0000","last_peered":"2024-05-17T09:17:05.632115+0000","last_clean":"2024-05-17T09:17:05.632115+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:17:05.632115+0000","last_undegraded":"2024-05-17T09:17:05.632115+0000","last_fullsized":"2024-05-17T09:17:05.632115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905191+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905191+0000","last_peered":"2024-05-17T09:17:46.905191+0000","last_clean":"2024-05-17T09:17:46.905191+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905191+0000","last_undegraded":"2024-05-17T09:17:46.905191+0000","last_fullsized":"2024-05-17T09:17:46.905191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":265,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:45.560667+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:19:45.560667+0000","last_peered":"2024-05-17T09:19:45.560667+0000","last_clean":"2024-05-17T09:19:45.560667+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:19:45.560667+0000","last_undegraded":"2024-05-17T09:19:45.560667+0000","last_fullsized":"2024-05-17T09:19:45.560667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112225+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:17:24.112225+0000","last_peered":"2024-05-17T09:17:24.112225+0000","last_clean":"2024-05-17T09:17:24.112225+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:17:24.112225+0000","last_undegraded":"2024-05-17T09:17:24.112225+0000","last_fullsized":"2024-05-17T09:17:24.112225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.430695+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:17:23.430695+0000","last_peered":"2024-05-17T09:17:23.430695+0000","last_clean":"2024-05-17T09:17:23.430695+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:17:23.430695+0000","last_undegraded":"2024-05-17T09:17:23.430695+0000","last_fullsized":"2024-05-17T09:17:23.430695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":255,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:42.554856+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:19:42.554856+0000","last_peered":"2024-05-17T09:19:42.554856+0000","last_clean":"2024-05-17T09:19:42.554856+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:19:42.554856+0000","last_undegraded":"2024-05-17T09:19:42.554856+0000","last_fullsized":"2024-05-17T09:19:42.554856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":507,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:43.557180+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:19:43.557180+0000","last_peered":"2024-05-17T09:19:43.557180+0000","last_clean":"2024-05-17T09:19:43.557180+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:19:43.557180+0000","last_undegraded":"2024-05-17T09:19:43.557180+0000","last_fullsized":"2024-05-17T09:19:43.557180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":279,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903452+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903452+0000","last_peered":"2024-05-17T09:17:46.903452+0000","last_clean":"2024-05-17T09:17:46.903452+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903452+0000","last_undegraded":"2024-05-17T09:17:46.903452+0000","last_fullsized":"2024-05-17T09:17:46.903452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.769113+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:17:26.769113+0000","last_peered":"2024-05-17T09:17:26.769113+0000","last_clean":"2024-05-17T09:17:26.769113+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:17:26.769113+0000","last_undegraded":"2024-05-17T09:17:26.769113+0000","last_fullsized":"2024-05-17T09:17:26.769113+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904979+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.904979+0000","last_peered":"2024-05-17T09:17:46.904979+0000","last_clean":"2024-05-17T09:17:46.904979+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.904979+0000","last_undegraded":"2024-05-17T09:17:46.904979+0000","last_fullsized":"2024-05-17T09:17:46.904979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":263,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:45.560310+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:19:45.560310+0000","last_peered":"2024-05-17T09:19:45.560310+0000","last_clean":"2024-05-17T09:19:45.560310+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:19:45.560310+0000","last_undegraded":"2024-05-17T09:19:45.560310+0000","last_fullsized":"2024-05-17T09:19:45.560310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900338+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.900338+0000","last_peered":"2024-05-17T09:17:46.900338+0000","last_clean":"2024-05-17T09:17:46.900338+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.900338+0000","last_undegraded":"2024-05-17T09:17:46.900338+0000","last_fullsized":"2024-05-17T09:17:46.900338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.938637+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:17:23.938637+0000","last_peered":"2024-05-17T09:17:23.938637+0000","last_clean":"2024-05-17T09:17:23.938637+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:17:23.938637+0000","last_undegraded":"2024-05-17T09:17:23.938637+0000","last_fullsized":"2024-05-17T09:17:23.938637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769733+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:17:31.769733+0000","last_peered":"2024-05-17T09:17:31.769733+0000","last_clean":"2024-05-17T09:17:31.769733+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:17:31.769733+0000","last_undegraded":"2024-05-17T09:17:31.769733+0000","last_fullsized":"2024-05-17T09:17:31.769733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898756+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.898756+0000","last_peered":"2024-05-17T09:17:46.898756+0000","last_clean":"2024-05-17T09:17:46.898756+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.898756+0000","last_undegraded":"2024-05-17T09:17:46.898756+0000","last_fullsized":"2024-05-17T09:17:46.898756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.939280+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:17:28.939280+0000","last_peered":"2024-05-17T09:17:28.939280+0000","last_clean":"2024-05-17T09:17:28.939280+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:17:28.939280+0000","last_undegraded":"2024-05-17T09:17:28.939280+0000","last_fullsized":"2024-05-17T09:17:28.939280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934940+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:17:11.934940+0000","last_peered":"2024-05-17T09:17:11.934940+0000","last_clean":"2024-05-17T09:17:11.934940+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:17:11.934940+0000","last_undegraded":"2024-05-17T09:17:11.934940+0000","last_fullsized":"2024-05-17T09:17:11.934940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904550+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904550+0000","last_peered":"2024-05-17T09:17:46.904550+0000","last_clean":"2024-05-17T09:17:46.904550+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904550+0000","last_undegraded":"2024-05-17T09:17:46.904550+0000","last_fullsized":"2024-05-17T09:17:46.904550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:27.771474+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:17:27.771474+0000","last_peered":"2024-05-17T09:17:27.771474+0000","last_clean":"2024-05-17T09:17:27.771474+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:17:27.771474+0000","last_undegraded":"2024-05-17T09:17:27.771474+0000","last_fullsized":"2024-05-17T09:17:27.771474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.384811+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:17:08.384811+0000","last_peered":"2024-05-17T09:17:08.384811+0000","last_clean":"2024-05-17T09:17:08.384811+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:17:08.384811+0000","last_undegraded":"2024-05-17T09:17:08.384811+0000","last_fullsized":"2024-05-17T09:17:08.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900921+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.900921+0000","last_peered":"2024-05-17T09:17:46.900921+0000","last_clean":"2024-05-17T09:17:46.900921+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.900921+0000","last_undegraded":"2024-05-17T09:17:46.900921+0000","last_fullsized":"2024-05-17T09:17:46.900921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:32.771965+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:17:32.771965+0000","last_peered":"2024-05-17T09:17:32.771965+0000","last_clean":"2024-05-17T09:17:32.771965+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:17:32.771965+0000","last_undegraded":"2024-05-17T09:17:32.771965+0000","last_fullsized":"2024-05-17T09:17:32.771965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901492+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901492+0000","last_peered":"2024-05-17T09:17:46.901492+0000","last_clean":"2024-05-17T09:17:46.901492+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901492+0000","last_undegraded":"2024-05-17T09:17:46.901492+0000","last_fullsized":"2024-05-17T09:17:46.901492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":255,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:41.552449+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:19:41.552449+0000","last_peered":"2024-05-17T09:19:41.552449+0000","last_clean":"2024-05-17T09:19:41.552449+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:19:41.552449+0000","last_undegraded":"2024-05-17T09:19:41.552449+0000","last_fullsized":"2024-05-17T09:19:41.552449+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387326+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:17:28.387326+0000","last_peered":"2024-05-17T09:17:28.387326+0000","last_clean":"2024-05-17T09:17:28.387326+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:17:28.387326+0000","last_undegraded":"2024-05-17T09:17:28.387326+0000","last_fullsized":"2024-05-17T09:17:28.387326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904968+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.904968+0000","last_peered":"2024-05-17T09:17:46.904968+0000","last_clean":"2024-05-17T09:17:46.904968+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.904968+0000","last_undegraded":"2024-05-17T09:17:46.904968+0000","last_fullsized":"2024-05-17T09:17:46.904968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.411802+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:17:11.411802+0000","last_peered":"2024-05-17T09:17:11.411802+0000","last_clean":"2024-05-17T09:17:11.411802+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:17:11.411802+0000","last_undegraded":"2024-05-17T09:17:11.411802+0000","last_fullsized":"2024-05-17T09:17:11.411802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595216+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595216+0000","last_peered":"2024-05-17T09:16:48.595216+0000","last_clean":"2024-05-17T09:16:48.595216+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595216+0000","last_undegraded":"2024-05-17T09:16:48.595216+0000","last_fullsized":"2024-05-17T09:16:48.595216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769780+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:17:31.769780+0000","last_peered":"2024-05-17T09:17:31.769780+0000","last_clean":"2024-05-17T09:17:31.769780+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:17:31.769780+0000","last_undegraded":"2024-05-17T09:17:31.769780+0000","last_fullsized":"2024-05-17T09:17:31.769780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903562+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903562+0000","last_peered":"2024-05-17T09:17:46.903562+0000","last_clean":"2024-05-17T09:17:46.903562+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903562+0000","last_undegraded":"2024-05-17T09:17:46.903562+0000","last_fullsized":"2024-05-17T09:17:46.903562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.384322+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:17:03.384322+0000","last_peered":"2024-05-17T09:17:03.384322+0000","last_clean":"2024-05-17T09:17:03.384322+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:17:03.384322+0000","last_undegraded":"2024-05-17T09:17:03.384322+0000","last_fullsized":"2024-05-17T09:17:03.384322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112162+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:17:24.112162+0000","last_peered":"2024-05-17T09:17:24.112162+0000","last_clean":"2024-05-17T09:17:24.112162+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:17:24.112162+0000","last_undegraded":"2024-05-17T09:17:24.112162+0000","last_fullsized":"2024-05-17T09:17:24.112162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.936560+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:17:08.936560+0000","last_peered":"2024-05-17T09:17:08.936560+0000","last_clean":"2024-05-17T09:17:08.936560+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:17:08.936560+0000","last_undegraded":"2024-05-17T09:17:08.936560+0000","last_fullsized":"2024-05-17T09:17:08.936560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903852+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903852+0000","last_peered":"2024-05-17T09:17:46.903852+0000","last_clean":"2024-05-17T09:17:46.903852+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903852+0000","last_undegraded":"2024-05-17T09:17:46.903852+0000","last_fullsized":"2024-05-17T09:17:46.903852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:20.633799+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:17:20.633799+0000","last_peered":"2024-05-17T09:17:20.633799+0000","last_clean":"2024-05-17T09:17:20.633799+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:17:20.633799+0000","last_undegraded":"2024-05-17T09:17:20.633799+0000","last_fullsized":"2024-05-17T09:17:20.633799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.948913+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:19:26.948913+0000","last_peered":"2024-05-17T09:19:26.948913+0000","last_clean":"2024-05-17T09:19:26.948913+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:19:26.948913+0000","last_undegraded":"2024-05-17T09:19:26.948913+0000","last_fullsized":"2024-05-17T09:19:26.948913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889973+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889973+0000","last_peered":"2024-05-17T09:17:46.889973+0000","last_clean":"2024-05-17T09:17:46.889973+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889973+0000","last_undegraded":"2024-05-17T09:17:46.889973+0000","last_fullsized":"2024-05-17T09:17:46.889973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.193083+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:17:26.193083+0000","last_peered":"2024-05-17T09:17:26.193083+0000","last_clean":"2024-05-17T09:17:26.193083+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:17:26.193083+0000","last_undegraded":"2024-05-17T09:17:26.193083+0000","last_fullsized":"2024-05-17T09:17:26.193083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.412427+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:17:16.412427+0000","last_peered":"2024-05-17T09:17:16.412427+0000","last_clean":"2024-05-17T09:17:16.412427+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:17:16.412427+0000","last_undegraded":"2024-05-17T09:17:16.412427+0000","last_fullsized":"2024-05-17T09:17:16.412427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898859+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898859+0000","last_peered":"2024-05-17T09:17:46.898859+0000","last_clean":"2024-05-17T09:17:46.898859+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898859+0000","last_undegraded":"2024-05-17T09:17:46.898859+0000","last_fullsized":"2024-05-17T09:17:46.898859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.908768+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:17:16.908768+0000","last_peered":"2024-05-17T09:17:16.908768+0000","last_clean":"2024-05-17T09:17:16.908768+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:17:16.908768+0000","last_undegraded":"2024-05-17T09:17:16.908768+0000","last_fullsized":"2024-05-17T09:17:16.908768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898286+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.898286+0000","last_peered":"2024-05-17T09:17:46.898286+0000","last_clean":"2024-05-17T09:17:46.898286+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.898286+0000","last_undegraded":"2024-05-17T09:17:46.898286+0000","last_fullsized":"2024-05-17T09:17:46.898286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:15.633296+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:17:15.633296+0000","last_peered":"2024-05-17T09:17:15.633296+0000","last_clean":"2024-05-17T09:17:15.633296+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:17:15.633296+0000","last_undegraded":"2024-05-17T09:17:15.633296+0000","last_fullsized":"2024-05-17T09:17:15.633296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901878+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.901878+0000","last_peered":"2024-05-17T09:17:46.901878+0000","last_clean":"2024-05-17T09:17:46.901878+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.901878+0000","last_undegraded":"2024-05-17T09:17:46.901878+0000","last_fullsized":"2024-05-17T09:17:46.901878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.206872+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:19:26.206872+0000","last_peered":"2024-05-17T09:19:26.206872+0000","last_clean":"2024-05-17T09:19:26.206872+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:19:26.206872+0000","last_undegraded":"2024-05-17T09:19:26.206872+0000","last_fullsized":"2024-05-17T09:19:26.206872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901144+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.901144+0000","last_peered":"2024-05-17T09:17:46.901144+0000","last_clean":"2024-05-17T09:17:46.901144+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.901144+0000","last_undegraded":"2024-05-17T09:17:46.901144+0000","last_fullsized":"2024-05-17T09:17:46.901144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:18.937946+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:17:18.937946+0000","last_peered":"2024-05-17T09:17:18.937946+0000","last_clean":"2024-05-17T09:17:18.937946+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:17:18.937946+0000","last_undegraded":"2024-05-17T09:17:18.937946+0000","last_fullsized":"2024-05-17T09:17:18.937946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908869+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908869+0000","last_peered":"2024-05-17T09:17:46.908869+0000","last_clean":"2024-05-17T09:17:46.908869+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908869+0000","last_undegraded":"2024-05-17T09:17:46.908869+0000","last_fullsized":"2024-05-17T09:17:46.908869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.910346+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:17:31.910346+0000","last_peered":"2024-05-17T09:17:31.910346+0000","last_clean":"2024-05-17T09:17:31.910346+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:17:31.910346+0000","last_undegraded":"2024-05-17T09:17:31.910346+0000","last_fullsized":"2024-05-17T09:17:31.910346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:21.192445+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:17:21.192445+0000","last_peered":"2024-05-17T09:17:21.192445+0000","last_clean":"2024-05-17T09:17:21.192445+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:17:21.192445+0000","last_undegraded":"2024-05-17T09:17:21.192445+0000","last_fullsized":"2024-05-17T09:17:21.192445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908474+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908474+0000","last_peered":"2024-05-17T09:17:46.908474+0000","last_clean":"2024-05-17T09:17:46.908474+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908474+0000","last_undegraded":"2024-05-17T09:17:46.908474+0000","last_fullsized":"2024-05-17T09:17:46.908474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:21.768494+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:17:21.768494+0000","last_peered":"2024-05-17T09:17:21.768494+0000","last_clean":"2024-05-17T09:17:21.768494+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:17:21.768494+0000","last_undegraded":"2024-05-17T09:17:21.768494+0000","last_fullsized":"2024-05-17T09:17:21.768494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901818+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901818+0000","last_peered":"2024-05-17T09:17:46.901818+0000","last_clean":"2024-05-17T09:17:46.901818+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901818+0000","last_undegraded":"2024-05-17T09:17:46.901818+0000","last_fullsized":"2024-05-17T09:17:46.901818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769697+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:17:31.769697+0000","last_peered":"2024-05-17T09:17:31.769697+0000","last_clean":"2024-05-17T09:17:31.769697+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:17:31.769697+0000","last_undegraded":"2024-05-17T09:17:31.769697+0000","last_fullsized":"2024-05-17T09:17:31.769697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:22.770777+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:17:22.770777+0000","last_peered":"2024-05-17T09:17:22.770777+0000","last_clean":"2024-05-17T09:17:22.770777+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:17:22.770777+0000","last_undegraded":"2024-05-17T09:17:22.770777+0000","last_fullsized":"2024-05-17T09:17:22.770777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897999+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897999+0000","last_peered":"2024-05-17T09:17:46.897999+0000","last_clean":"2024-05-17T09:17:46.897999+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897999+0000","last_undegraded":"2024-05-17T09:17:46.897999+0000","last_fullsized":"2024-05-17T09:17:46.897999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899078+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.899078+0000","last_peered":"2024-05-17T09:17:46.899078+0000","last_clean":"2024-05-17T09:17:46.899078+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.899078+0000","last_undegraded":"2024-05-17T09:17:46.899078+0000","last_fullsized":"2024-05-17T09:17:46.899078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888934+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888934+0000","last_peered":"2024-05-17T09:17:46.888934+0000","last_clean":"2024-05-17T09:17:46.888934+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888934+0000","last_undegraded":"2024-05-17T09:17:46.888934+0000","last_fullsized":"2024-05-17T09:17:46.888934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896944+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896944+0000","last_peered":"2024-05-17T09:17:46.896944+0000","last_clean":"2024-05-17T09:17:46.896944+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896944+0000","last_undegraded":"2024-05-17T09:17:46.896944+0000","last_fullsized":"2024-05-17T09:17:46.896944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904847+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904847+0000","last_peered":"2024-05-17T09:17:46.904847+0000","last_clean":"2024-05-17T09:17:46.904847+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904847+0000","last_undegraded":"2024-05-17T09:17:46.904847+0000","last_fullsized":"2024-05-17T09:17:46.904847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904438+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904438+0000","last_peered":"2024-05-17T09:17:46.904438+0000","last_clean":"2024-05-17T09:17:46.904438+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904438+0000","last_undegraded":"2024-05-17T09:17:46.904438+0000","last_fullsized":"2024-05-17T09:17:46.904438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899186+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899186+0000","last_peered":"2024-05-17T09:17:46.899186+0000","last_clean":"2024-05-17T09:17:46.899186+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899186+0000","last_undegraded":"2024-05-17T09:17:46.899186+0000","last_fullsized":"2024-05-17T09:17:46.899186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891995+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891995+0000","last_peered":"2024-05-17T09:17:46.891995+0000","last_clean":"2024-05-17T09:17:46.891995+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891995+0000","last_undegraded":"2024-05-17T09:17:46.891995+0000","last_fullsized":"2024-05-17T09:17:46.891995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886503+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.886503+0000","last_peered":"2024-05-17T09:17:46.886503+0000","last_clean":"2024-05-17T09:17:46.886503+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.886503+0000","last_undegraded":"2024-05-17T09:17:46.886503+0000","last_fullsized":"2024-05-17T09:17:46.886503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899965+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899965+0000","last_peered":"2024-05-17T09:17:46.899965+0000","last_clean":"2024-05-17T09:17:46.899965+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899965+0000","last_undegraded":"2024-05-17T09:17:46.899965+0000","last_fullsized":"2024-05-17T09:17:46.899965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894683+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894683+0000","last_peered":"2024-05-17T09:17:46.894683+0000","last_clean":"2024-05-17T09:17:46.894683+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894683+0000","last_undegraded":"2024-05-17T09:17:46.894683+0000","last_fullsized":"2024-05-17T09:17:46.894683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908531+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908531+0000","last_peered":"2024-05-17T09:17:46.908531+0000","last_clean":"2024-05-17T09:17:46.908531+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908531+0000","last_undegraded":"2024-05-17T09:17:46.908531+0000","last_fullsized":"2024-05-17T09:17:46.908531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904591+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904591+0000","last_peered":"2024-05-17T09:17:46.904591+0000","last_clean":"2024-05-17T09:17:46.904591+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904591+0000","last_undegraded":"2024-05-17T09:17:46.904591+0000","last_fullsized":"2024-05-17T09:17:46.904591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899578+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899578+0000","last_peered":"2024-05-17T09:17:46.899578+0000","last_clean":"2024-05-17T09:17:46.899578+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899578+0000","last_undegraded":"2024-05-17T09:17:46.899578+0000","last_fullsized":"2024-05-17T09:17:46.899578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1271,"num_read_kb":713,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":224556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710792,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907020,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1176674,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266775,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463004,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1112817,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626530,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":579607,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":581948,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182513,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986284,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":516180,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822760,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1103284,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:19:47.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:19:48.021 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:48 smithi019 ceph-mon[31620]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T09:19:48.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:48 smithi019 ceph-mon[31620]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:48.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:48 smithi079 ceph-mon[35502]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T09:19:48.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:48 smithi079 ceph-mon[35502]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:48 smithi069 ceph-mon[35081]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T09:19:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:48 smithi069 ceph-mon[35081]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:48.915 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 -- 172.21.15.19:0/1028705748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c107e00 msgr2=0x7f4a9c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 --2- 172.21.15.19:0/1028705748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c107e00 0x7f4a9c108210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4a900099c0 tx=0x7f4a90009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 -- 172.21.15.19:0/1028705748 shutdown_connections 2024-05-17T09:19:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 --2- 172.21.15.19:0/1028705748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a9c108750 0x7f4a9c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 --2- 172.21.15.19:0/1028705748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c1034d0 0x7f4a9c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 --2- 172.21.15.19:0/1028705748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c107e00 0x7f4a9c108210 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 -- 172.21.15.19:0/1028705748 >> 172.21.15.19:0/1028705748 conn(0x7f4a9c0fef20 msgr2=0x7f4a9c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 -- 172.21.15.19:0/1028705748 shutdown_connections 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.510+0000 7f4aa226a700 1 -- 172.21.15.19:0/1028705748 wait complete. 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 Processor -- start 2024-05-17T09:19:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 -- start start 2024-05-17T09:19:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a9c1034d0 0x7f4a9c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c107e00 0x7f4a9c19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 0x7f4a9c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c113210 con 0x7f4a9c108750 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c113090 con 0x7f4a9c107e00 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa226a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c113390 con 0x7f4a9c1034d0 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa0807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 0x7f4a9c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4a9b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c107e00 0x7f4a9c19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a9c1034d0 0x7f4a9c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa0807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 0x7f4a9c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37656/0 (socket says 172.21.15.19:37656) 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.511+0000 7f4aa0807700 1 -- 172.21.15.19:0/823336755 learned_addr learned my addr 172.21.15.19:0/823336755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa0807700 1 -- 172.21.15.19:0/823336755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a9c1034d0 msgr2=0x7f4a9c19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa0807700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a9c1034d0 0x7f4a9c19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa0807700 1 -- 172.21.15.19:0/823336755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c107e00 msgr2=0x7f4a9c19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa0807700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c107e00 0x7f4a9c19c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa0807700 1 -- 172.21.15.19:0/823336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c1a6f20 con 0x7f4a9c108750 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4a9b7fe700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c107e00 0x7f4a9c19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa0807700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 0x7f4a9c19c670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007ee0 tx=0x7f4a9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4a997fa700 1 -- 172.21.15.19:0/823336755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004bb0 con 0x7f4a9c108750 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c1a7210 con 0x7f4a9c108750 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4a997fa700 1 -- 172.21.15.19:0/823336755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a94004d50 con 0x7f4a9c108750 2024-05-17T09:19:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4a997fa700 1 -- 172.21.15.19:0/823336755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a940058b0 con 0x7f4a9c108750 2024-05-17T09:19:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.512+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a7720 con 0x7f4a9c108750 2024-05-17T09:19:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.515+0000 7f4a997fa700 1 -- 172.21.15.19:0/823336755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4a9401a020 con 0x7f4a9c108750 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.516+0000 7f4a997fa700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a840b2880 0x7f4a840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.516+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a840b2880 0x7f4a840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.516+0000 7f4a997fa700 1 -- 172.21.15.19:0/823336755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4a940e1c50 con 0x7f4a9c108750 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 Processor -- start 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 -- start start 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a840b2880 0x7f4a840b4d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c005950 tx=0x7f4a8c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 0x559665ae3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55966595fe40 0x559665ae3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665ae77c0 0x559665ae7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559665a0cd70 con 0x559665a4b430 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5596659fc910 con 0x55966595fe40 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559665a00950 con 0x559665ae77c0 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 0x559665ae3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 0x559665ae3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37670/0 (socket says 172.21.15.19:37670) 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4081278716 learned_addr learned my addr 172.21.15.19:0/4081278716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.517+0000 7f4a9b7fe700 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55966595fe40 0x559665ae3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4aa0807700 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665ae77c0 0x559665ae7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4081278716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665ae77c0 msgr2=0x559665ae7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665ae77c0 0x559665ae7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4081278716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55966595fe40 msgr2=0x559665ae3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55966595fe40 0x559665ae3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4081278716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596659a5fb0 con 0x559665a4b430 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4aa0807700 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665ae77c0 0x559665ae7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 0x559665ae3380 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c010fa0 tx=0x7f4a8c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7990f7325268110a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a827fc700 1 -- 172.21.15.19:0/4081278716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c00be60 con 0x559665a4b430 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a827fc700 1 -- 172.21.15.19:0/4081278716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8c011db0 con 0x559665a4b430 2024-05-17T09:19:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.518+0000 7f4a827fc700 1 -- 172.21.15.19:0/4081278716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c01c6b0 con 0x559665a4b430 2024-05-17T09:19:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4081278716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 msgr2=0x559665ae3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 0x559665ae3380 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c010fa0 tx=0x7f4a8c013000 comp rx=0 tx=0).stop 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4081278716 shutdown_connections 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665ae77c0 0x559665ae7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665a4b430 0x559665ae3380 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4081278716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55966595fe40 0x559665ae3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4081278716 >> 172.21.15.19:0/4081278716 conn(0x5596659c01c0 msgr2=0x559665ad97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4081278716 shutdown_connections 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4081278716 wait complete. 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.519+0000 7f4ab0c7fb80 1 -- start start 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 Processor -- start 2024-05-17T09:19:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665aeca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665aecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665aed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559665902fa0 con 0x559665ae77c0 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559665964da0 con 0x559665a4b430 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559665ae8570 con 0x55966595fe40 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4ab0c7fb80 10 client.0 _ll_register_callbacks cb 0x559665997950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4aa0807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665aed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4aa0807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665aed4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37674/0 (socket says 172.21.15.19:37674) 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4aa0807700 1 -- 172.21.15.19:0/73236604 learned_addr learned my addr 172.21.15.19:0/73236604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665aeca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.520+0000 7f4a9b7fe700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665aecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4aa0807700 1 -- 172.21.15.19:0/73236604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 msgr2=0x559665aeca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4aa0807700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665aeca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4aa0807700 1 -- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 msgr2=0x559665aecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4aa0807700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665aecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4aa0807700 1 -- 172.21.15.19:0/73236604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966594c8d0 con 0x559665ae77c0 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a9b7fe700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665aecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4aa0807700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665aed4d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a940df2b0 tx=0x7f4a940a8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524400ab14010d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:49.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/73236604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a940a9560 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/73236604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4a940b1a40 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8c011db0 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/73236604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a940a8e60 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4ab0c7fb80 10 client.15336 fetch_fsmap learned FSMap version 27 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596659b6760 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/73236604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8c011db0 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/73236604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a940a47d0 con 0x559665ae77c0 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.521+0000 7f4ab0c7fb80 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4ab0c7fb80 10 client.15336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4ab0c7fb80 20 client.15336 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665afc6a0 0x559665afeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4ab0c7fb80 4 client.15336 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559665aaaed0 con 0x559665afc6a0 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665afc6a0 0x559665afeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4a7b7fe700 10 client.15336 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.522+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665afc6a0 0x559665afeac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da07c0f2fe02b06 server_cookie=224b476f6f51198f in_seq=0 out_seq=0 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.523+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/73236604 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559665aaaed0 con 0x559665afc6a0 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.523+0000 7f4a7b7fe700 10 client.15336 handle_command_reply: tid=0 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.523+0000 7f4ab0c7fb80 1 client.15336 shutdown 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 msgr2=0x559665aed4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665aed4d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a940df2b0 tx=0x7f4a940a8640 comp rx=0 tx=0).stop 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 shutdown_connections 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665aeca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665afc6a0 0x559665afeac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665aed4d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/73236604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665aecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4a7b7fe700 0 client.15336 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 >> 172.21.15.19:0/73236604 conn(0x5596659c01c0 msgr2=0x559665ad9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 shutdown_connections 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/73236604 wait complete. 2024-05-17T09:19:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.524+0000 7f4ab0c7fb80 20 client.15336 trim_cache size 0 max 16384 2024-05-17T09:19:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.544+0000 7f4ab0c7fb80 1 Processor -- start 2024-05-17T09:19:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.544+0000 7f4ab0c7fb80 1 -- start start 2024-05-17T09:19:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 0x559665ade6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665a4b430 0x559665af8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665ae77c0 0x559665ad3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1b09e0 con 0x55966595fe40 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1ae270 con 0x559665ae77c0 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1ac0c0 con 0x559665a4b430 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 0x559665ade6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 0x559665ade6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37688/0 (socket says 172.21.15.19:37688) 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4212164966 learned_addr learned my addr 172.21.15.19:0/4212164966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9b7fe700 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665a4b430 0x559665af8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4212164966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665a4b430 msgr2=0x559665af8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665a4b430 0x559665af8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4aa0807700 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665ae77c0 0x559665ad3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4212164966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665ae77c0 msgr2=0x559665ad3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665ae77c0 0x559665ad3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 -- 172.21.15.19:0/4212164966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559665ae8070 con 0x55966595fe40 2024-05-17T09:19:49.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4aa0807700 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665ae77c0 0x559665ad3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:49.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.545+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 0x559665ade6f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c01eed0 tx=0x7f4a8c011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78047ea57e1d0fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:49.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4a81ffb700 1 -- 172.21.15.19:0/4212164966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c005220 con 0x55966595fe40 2024-05-17T09:19:49.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4a81ffb700 1 -- 172.21.15.19:0/4212164966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8c0053c0 con 0x55966595fe40 2024-05-17T09:19:49.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4a81ffb700 1 -- 172.21.15.19:0/4212164966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c01e640 con 0x55966595fe40 2024-05-17T09:19:49.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4212164966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 msgr2=0x559665ade6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 0x559665ade6f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c01eed0 tx=0x7f4a8c011c60 comp rx=0 tx=0).stop 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4212164966 shutdown_connections 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665ae77c0 0x559665ad3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55966595fe40 0x559665ade6f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/4212164966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559665a4b430 0x559665af8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.546+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4212164966 >> 172.21.15.19:0/4212164966 conn(0x5596659c01c0 msgr2=0x559665ad3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4212164966 shutdown_connections 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/4212164966 wait complete. 2024-05-17T09:19:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 -- start start 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 Processor -- start 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665af51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665ae95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665ae9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1ac730 con 0x559665ae77c0 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1adba0 con 0x559665a4b430 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.547+0000 7f4ab0c7fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1ace00 con 0x55966595fe40 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4ab0c7fb80 10 client.0 _ll_register_callbacks cb 0x559665997950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665ae9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665ae9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37692/0 (socket says 172.21.15.19:37692) 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 -- 172.21.15.19:0/2854166283 learned_addr learned my addr 172.21.15.19:0/2854166283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665af51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4a9b7fe700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665ae95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 -- 172.21.15.19:0/2854166283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 msgr2=0x559665af51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665af51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 -- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 msgr2=0x559665ae95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665ae95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 -- 172.21.15.19:0/2854166283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559665aea170 con 0x559665ae77c0 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4aa0807700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665ae9b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009510 tx=0x7f4a94007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc521152eec108e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4a7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/2854166283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a940a9560 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/2854166283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4a940a5450 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.548+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8c0053c0 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/2854166283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a9408a050 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 10 client.15342 fetch_fsmap learned FSMap version 27 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559665b66ea0 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/2854166283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8c0053c0 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/2854166283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a9400dcb0 con 0x559665ae77c0 2024-05-17T09:19:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 20 client.15342 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665b675b0 0x559665b699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 4 client.15342 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559665a0ecc0 con 0x559665b675b0 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.549+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665b675b0 0x559665b699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4a7b7fe700 10 client.15342 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4a9bfff700 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665b675b0 0x559665b699d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9956c4dbb9d4262 server_cookie=50c87970f0194720 in_seq=0 out_seq=0 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4a7b7fe700 1 -- 172.21.15.19:0/2854166283 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559665a0ecc0 con 0x559665b675b0 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4a7b7fe700 10 client.15342 handle_command_reply: tid=0 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 client.15342 shutdown 2024-05-17T09:19:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 msgr2=0x559665ae9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665ae9b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009510 tx=0x7f4a94007f90 comp rx=0 tx=0).stop 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 shutdown_connections 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55966595fe40 0x559665af51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559665b675b0 0x559665b699d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559665ae77c0 0x559665ae9b30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 --2- 172.21.15.19:0/2854166283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559665a4b430 0x559665ae95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 >> 172.21.15.19:0/2854166283 conn(0x5596659c01c0 msgr2=0x559665ade450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.550+0000 7f4a7b7fe700 0 client.15342 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 shutdown_connections 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4ab0c7fb80 1 -- 172.21.15.19:0/2854166283 wait complete. 2024-05-17T09:19:49.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4ab0c7fb80 20 client.15342 trim_cache size 0 max 16384 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: "a531b57f-a265-466e-9985-1375efb7f296": { 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "a531b57f-a265-466e-9985-1375efb7f296", 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-17T09:19:49.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:19:49.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:19:49.560 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:19:49.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a840b2880 msgr2=0x7f4a840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a840b2880 0x7f4a840b4d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c005950 tx=0x7f4a8c00b410 comp rx=0 tx=0).stop 2024-05-17T09:19:49.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 msgr2=0x7f4a9c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 0x7f4a9c19c670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007ee0 tx=0x7f4a9400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 shutdown_connections 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4a840b2880 0x7f4a840b4d90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a9c1034d0 0x7f4a9c19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108750 0x7f4a9c19c670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 --2- 172.21.15.19:0/823336755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a9c107e00 0x7f4a9c19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 >> 172.21.15.19:0/823336755 conn(0x7f4a9c0fef20 msgr2=0x7f4a9c1096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.551+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 shutdown_connections 2024-05-17T09:19:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:49.552+0000 7f4aa226a700 1 -- 172.21.15.19:0/823336755 wait complete. 2024-05-17T09:19:50.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:50 smithi019 ceph-mon[31620]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-17T09:19:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:50 smithi079 ceph-mon[35502]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-17T09:19:50.309 INFO:tasks.cephfs.filesystem:scrub status for tag:a531b57f-a265-466e-9985-1375efb7f296 - {'path': '/', 'tag': 'a531b57f-a265-466e-9985-1375efb7f296', 'options': 'recursive,force'} 2024-05-17T09:19:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:50 smithi069 ceph-mon[35081]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 -- 172.21.15.19:0/808780946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4105e00 msgr2=0x7f0fb410e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/808780946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4105e00 0x7f0fb410e350 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009990 tx=0x7f0fa8009ca0 comp rx=0 tx=0).stop 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 -- 172.21.15.19:0/808780946 shutdown_connections 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/808780946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fb410e8e0 0x7f0fb4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/808780946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb41054b0 0x7f0fb41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/808780946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4105e00 0x7f0fb410e350 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 -- 172.21.15.19:0/808780946 >> 172.21.15.19:0/808780946 conn(0x7f0fb40fede0 msgr2=0x7f0fb4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 -- 172.21.15.19:0/808780946 shutdown_connections 2024-05-17T09:19:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 -- 172.21.15.19:0/808780946 wait complete. 2024-05-17T09:19:51.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.858+0000 7f0fbbd64700 1 Processor -- start 2024-05-17T09:19:51.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 -- start start 2024-05-17T09:19:51.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 0x7f0fb41031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:51.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fb4105e00 0x7f0fb41036e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb410e8e0 0x7f0fb4103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41142f0 con 0x7f0fb41054b0 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4114170 con 0x7f0fb410e8e0 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fbbd64700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4114470 con 0x7f0fb4105e00 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 0x7f0fb41031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 0x7f0fb41031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37698/0 (socket says 172.21.15.19:37698) 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 -- 172.21.15.19:0/1667141232 learned_addr learned my addr 172.21.15.19:0/1667141232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb92ff700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fb4105e00 0x7f0fb41036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fba301700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb410e8e0 0x7f0fb4103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 -- 172.21.15.19:0/1667141232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fb4105e00 msgr2=0x7f0fb41036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fb4105e00 0x7f0fb41036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 -- 172.21.15.19:0/1667141232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb410e8e0 msgr2=0x7f0fb4103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb410e8e0 0x7f0fb4103c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fb9b00700 1 -- 172.21.15.19:0/1667141232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb41a9050 con 0x7f0fb41054b0 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.859+0000 7f0fba301700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb410e8e0 0x7f0fb4103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.860+0000 7f0fb9b00700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 0x7f0fb41031a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4007f10 tx=0x7f0fa400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.860+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa400dbd0 con 0x7f0fb41054b0 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.860+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb41a9340 con 0x7f0fb41054b0 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.860+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fa400dd70 con 0x7f0fb41054b0 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.860+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4004890 con 0x7f0fb41054b0 2024-05-17T09:19:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.860+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb41a98a0 con 0x7f0fb41054b0 2024-05-17T09:19:51.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.861+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb40512b0 con 0x7f0fb41054b0 2024-05-17T09:19:51.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.862+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0fa401a020 con 0x7f0fb41054b0 2024-05-17T09:19:51.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.863+0000 7f0fb2ffd700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0f9c0b2880 0x7f0f9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:51.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.863+0000 7f0fb92ff700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0f9c0b2880 0x7f0f9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:51.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.863+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0fa40e22d0 con 0x7f0fb41054b0 2024-05-17T09:19:51.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.864+0000 7f0fb92ff700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0f9c0b2880 0x7f0f9c0b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80098e0 tx=0x7f0fa8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:51.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:51.867+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fa4010070 con 0x7f0fb41054b0 2024-05-17T09:19:52.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.082+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fb4063bf0 con 0x7f0f9c0b2880 2024-05-17T09:19:52.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.091+0000 7f0fb2ffd700 1 -- 172.21.15.19:0/1667141232 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340727 (secure 0 0 0) 0x7f0fa807b310 con 0x7f0f9c0b2880 2024-05-17T09:19:52.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:52 smithi019 ceph-mon[31620]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-17T09:19:52.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0f9c0b2880 msgr2=0x7f0f9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0f9c0b2880 0x7f0f9c0b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80098e0 tx=0x7f0fa8005960 comp rx=0 tx=0).stop 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 msgr2=0x7f0fb41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 0x7f0fb41031a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4007f10 tx=0x7f0fa400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 shutdown_connections 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fb410e8e0 0x7f0fb4103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41054b0 0x7f0fb41031a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0f9c0b2880 0x7f0f9c0b4d90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 --2- 172.21.15.19:0/1667141232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fb4105e00 0x7f0fb41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 >> 172.21.15.19:0/1667141232 conn(0x7f0fb40fede0 msgr2=0x7f0fb40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 shutdown_connections 2024-05-17T09:19:52.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:52.095+0000 7f0fbbd64700 1 -- 172.21.15.19:0/1667141232 wait complete. 2024-05-17T09:19:52.101 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:19:52.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:52 smithi079 ceph-mon[35502]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-17T09:19:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:52 smithi069 ceph-mon[35081]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-17T09:19:52.795 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-05-17T09:19:50.953893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1368,"num_read_kb":793,"num_write":313,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348656,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593168,"statfs":{"total":1151940427776,"available":1151583404032,"internally_reserved":0,"allocated":12271616,"data_stored":8153408,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004135"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906823+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906823+0000","last_peered":"2024-05-17T09:17:46.906823+0000","last_clean":"2024-05-17T09:17:46.906823+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906823+0000","last_undegraded":"2024-05-17T09:17:46.906823+0000","last_fullsized":"2024-05-17T09:17:46.906823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901351+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.901351+0000","last_peered":"2024-05-17T09:17:46.901351+0000","last_clean":"2024-05-17T09:17:46.901351+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.901351+0000","last_undegraded":"2024-05-17T09:17:46.901351+0000","last_fullsized":"2024-05-17T09:17:46.901351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905015+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.905015+0000","last_peered":"2024-05-17T09:17:46.905015+0000","last_clean":"2024-05-17T09:17:46.905015+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.905015+0000","last_undegraded":"2024-05-17T09:17:46.905015+0000","last_fullsized":"2024-05-17T09:17:46.905015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886153+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.886153+0000","last_peered":"2024-05-17T09:17:46.886153+0000","last_clean":"2024-05-17T09:17:46.886153+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.886153+0000","last_undegraded":"2024-05-17T09:17:46.886153+0000","last_fullsized":"2024-05-17T09:17:46.886153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898029+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.898029+0000","last_peered":"2024-05-17T09:17:46.898029+0000","last_clean":"2024-05-17T09:17:46.898029+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.898029+0000","last_undegraded":"2024-05-17T09:17:46.898029+0000","last_fullsized":"2024-05-17T09:17:46.898029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893666+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893666+0000","last_peered":"2024-05-17T09:17:46.893666+0000","last_clean":"2024-05-17T09:17:46.893666+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893666+0000","last_undegraded":"2024-05-17T09:17:46.893666+0000","last_fullsized":"2024-05-17T09:17:46.893666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908085+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908085+0000","last_peered":"2024-05-17T09:17:46.908085+0000","last_clean":"2024-05-17T09:17:46.908085+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908085+0000","last_undegraded":"2024-05-17T09:17:46.908085+0000","last_fullsized":"2024-05-17T09:17:46.908085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908289+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908289+0000","last_peered":"2024-05-17T09:17:46.908289+0000","last_clean":"2024-05-17T09:17:46.908289+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908289+0000","last_undegraded":"2024-05-17T09:17:46.908289+0000","last_fullsized":"2024-05-17T09:17:46.908289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.892004+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.892004+0000","last_peered":"2024-05-17T09:17:46.892004+0000","last_clean":"2024-05-17T09:17:46.892004+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.892004+0000","last_undegraded":"2024-05-17T09:17:46.892004+0000","last_fullsized":"2024-05-17T09:17:46.892004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889416+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889416+0000","last_peered":"2024-05-17T09:17:46.889416+0000","last_clean":"2024-05-17T09:17:46.889416+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889416+0000","last_undegraded":"2024-05-17T09:17:46.889416+0000","last_fullsized":"2024-05-17T09:17:46.889416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:16.948046+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:19:16.948046+0000","last_peered":"2024-05-17T09:19:16.948046+0000","last_clean":"2024-05-17T09:19:16.948046+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:19:16.948046+0000","last_undegraded":"2024-05-17T09:19:16.948046+0000","last_fullsized":"2024-05-17T09:19:16.948046+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898655+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898655+0000","last_peered":"2024-05-17T09:17:46.898655+0000","last_clean":"2024-05-17T09:17:46.898655+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898655+0000","last_undegraded":"2024-05-17T09:17:46.898655+0000","last_fullsized":"2024-05-17T09:17:46.898655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:05.632115+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:17:05.632115+0000","last_peered":"2024-05-17T09:17:05.632115+0000","last_clean":"2024-05-17T09:17:05.632115+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:17:05.632115+0000","last_undegraded":"2024-05-17T09:17:05.632115+0000","last_fullsized":"2024-05-17T09:17:05.632115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905191+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905191+0000","last_peered":"2024-05-17T09:17:46.905191+0000","last_clean":"2024-05-17T09:17:46.905191+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905191+0000","last_undegraded":"2024-05-17T09:17:46.905191+0000","last_fullsized":"2024-05-17T09:17:46.905191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":265,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:45.560667+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:19:45.560667+0000","last_peered":"2024-05-17T09:19:45.560667+0000","last_clean":"2024-05-17T09:19:45.560667+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:19:45.560667+0000","last_undegraded":"2024-05-17T09:19:45.560667+0000","last_fullsized":"2024-05-17T09:19:45.560667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112225+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:17:24.112225+0000","last_peered":"2024-05-17T09:17:24.112225+0000","last_clean":"2024-05-17T09:17:24.112225+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:17:24.112225+0000","last_undegraded":"2024-05-17T09:17:24.112225+0000","last_fullsized":"2024-05-17T09:17:24.112225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.430695+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:17:23.430695+0000","last_peered":"2024-05-17T09:17:23.430695+0000","last_clean":"2024-05-17T09:17:23.430695+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:17:23.430695+0000","last_undegraded":"2024-05-17T09:17:23.430695+0000","last_fullsized":"2024-05-17T09:17:23.430695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:47.563502+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:19:47.563502+0000","last_peered":"2024-05-17T09:19:47.563502+0000","last_clean":"2024-05-17T09:19:47.563502+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:19:47.563502+0000","last_undegraded":"2024-05-17T09:19:47.563502+0000","last_fullsized":"2024-05-17T09:19:47.563502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595792+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:16:48.595792+0000","last_peered":"2024-05-17T09:16:48.595792+0000","last_clean":"2024-05-17T09:16:48.595792+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:16:48.595792+0000","last_undegraded":"2024-05-17T09:16:48.595792+0000","last_fullsized":"2024-05-17T09:16:48.595792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908774+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908774+0000","last_peered":"2024-05-17T09:17:46.908774+0000","last_clean":"2024-05-17T09:17:46.908774+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908774+0000","last_undegraded":"2024-05-17T09:17:46.908774+0000","last_fullsized":"2024-05-17T09:17:46.908774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":518,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:48.565225+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:19:48.565225+0000","last_peered":"2024-05-17T09:19:48.565225+0000","last_clean":"2024-05-17T09:19:48.565225+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:19:48.565225+0000","last_undegraded":"2024-05-17T09:19:48.565225+0000","last_fullsized":"2024-05-17T09:19:48.565225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":433,"num_read_kb":285,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903556+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903556+0000","last_peered":"2024-05-17T09:17:46.903556+0000","last_clean":"2024-05-17T09:17:46.903556+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903556+0000","last_undegraded":"2024-05-17T09:17:46.903556+0000","last_fullsized":"2024-05-17T09:17:46.903556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.769113+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:17:26.769113+0000","last_peered":"2024-05-17T09:17:26.769113+0000","last_clean":"2024-05-17T09:17:26.769113+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:17:26.769113+0000","last_undegraded":"2024-05-17T09:17:26.769113+0000","last_fullsized":"2024-05-17T09:17:26.769113+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905106+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.905106+0000","last_peered":"2024-05-17T09:17:46.905106+0000","last_clean":"2024-05-17T09:17:46.905106+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.905106+0000","last_undegraded":"2024-05-17T09:17:46.905106+0000","last_fullsized":"2024-05-17T09:17:46.905106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":263,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:45.560310+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:19:45.560310+0000","last_peered":"2024-05-17T09:19:45.560310+0000","last_clean":"2024-05-17T09:19:45.560310+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:19:45.560310+0000","last_undegraded":"2024-05-17T09:19:45.560310+0000","last_fullsized":"2024-05-17T09:19:45.560310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901798+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.901798+0000","last_peered":"2024-05-17T09:17:46.901798+0000","last_clean":"2024-05-17T09:17:46.901798+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.901798+0000","last_undegraded":"2024-05-17T09:17:46.901798+0000","last_fullsized":"2024-05-17T09:17:46.901798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.938637+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:17:23.938637+0000","last_peered":"2024-05-17T09:17:23.938637+0000","last_clean":"2024-05-17T09:17:23.938637+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:17:23.938637+0000","last_undegraded":"2024-05-17T09:17:23.938637+0000","last_fullsized":"2024-05-17T09:17:23.938637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908343+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908343+0000","last_peered":"2024-05-17T09:17:46.908343+0000","last_clean":"2024-05-17T09:17:46.908343+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908343+0000","last_undegraded":"2024-05-17T09:17:46.908343+0000","last_fullsized":"2024-05-17T09:17:46.908343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769733+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:17:31.769733+0000","last_peered":"2024-05-17T09:17:31.769733+0000","last_clean":"2024-05-17T09:17:31.769733+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:17:31.769733+0000","last_undegraded":"2024-05-17T09:17:31.769733+0000","last_fullsized":"2024-05-17T09:17:31.769733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898756+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.898756+0000","last_peered":"2024-05-17T09:17:46.898756+0000","last_clean":"2024-05-17T09:17:46.898756+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.898756+0000","last_undegraded":"2024-05-17T09:17:46.898756+0000","last_fullsized":"2024-05-17T09:17:46.898756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.939280+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:17:28.939280+0000","last_peered":"2024-05-17T09:17:28.939280+0000","last_clean":"2024-05-17T09:17:28.939280+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:17:28.939280+0000","last_undegraded":"2024-05-17T09:17:28.939280+0000","last_fullsized":"2024-05-17T09:17:28.939280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934940+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:17:11.934940+0000","last_peered":"2024-05-17T09:17:11.934940+0000","last_clean":"2024-05-17T09:17:11.934940+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:17:11.934940+0000","last_undegraded":"2024-05-17T09:17:11.934940+0000","last_fullsized":"2024-05-17T09:17:11.934940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904550+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904550+0000","last_peered":"2024-05-17T09:17:46.904550+0000","last_clean":"2024-05-17T09:17:46.904550+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904550+0000","last_undegraded":"2024-05-17T09:17:46.904550+0000","last_fullsized":"2024-05-17T09:17:46.904550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:27.771474+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:17:27.771474+0000","last_peered":"2024-05-17T09:17:27.771474+0000","last_clean":"2024-05-17T09:17:27.771474+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:17:27.771474+0000","last_undegraded":"2024-05-17T09:17:27.771474+0000","last_fullsized":"2024-05-17T09:17:27.771474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.384811+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:17:08.384811+0000","last_peered":"2024-05-17T09:17:08.384811+0000","last_clean":"2024-05-17T09:17:08.384811+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:17:08.384811+0000","last_undegraded":"2024-05-17T09:17:08.384811+0000","last_fullsized":"2024-05-17T09:17:08.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900921+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.900921+0000","last_peered":"2024-05-17T09:17:46.900921+0000","last_clean":"2024-05-17T09:17:46.900921+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.900921+0000","last_undegraded":"2024-05-17T09:17:46.900921+0000","last_fullsized":"2024-05-17T09:17:46.900921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:32.771965+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:17:32.771965+0000","last_peered":"2024-05-17T09:17:32.771965+0000","last_clean":"2024-05-17T09:17:32.771965+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:17:32.771965+0000","last_undegraded":"2024-05-17T09:17:32.771965+0000","last_fullsized":"2024-05-17T09:17:32.771965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901492+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901492+0000","last_peered":"2024-05-17T09:17:46.901492+0000","last_clean":"2024-05-17T09:17:46.901492+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901492+0000","last_undegraded":"2024-05-17T09:17:46.901492+0000","last_fullsized":"2024-05-17T09:17:46.901492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:46.561064+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:19:46.561064+0000","last_peered":"2024-05-17T09:19:46.561064+0000","last_clean":"2024-05-17T09:19:46.561064+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:19:46.561064+0000","last_undegraded":"2024-05-17T09:19:46.561064+0000","last_fullsized":"2024-05-17T09:19:46.561064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":209,"num_read_kb":209,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387326+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:17:28.387326+0000","last_peered":"2024-05-17T09:17:28.387326+0000","last_clean":"2024-05-17T09:17:28.387326+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:17:28.387326+0000","last_undegraded":"2024-05-17T09:17:28.387326+0000","last_fullsized":"2024-05-17T09:17:28.387326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904968+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.904968+0000","last_peered":"2024-05-17T09:17:46.904968+0000","last_clean":"2024-05-17T09:17:46.904968+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.904968+0000","last_undegraded":"2024-05-17T09:17:46.904968+0000","last_fullsized":"2024-05-17T09:17:46.904968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.411802+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:17:11.411802+0000","last_peered":"2024-05-17T09:17:11.411802+0000","last_clean":"2024-05-17T09:17:11.411802+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:17:11.411802+0000","last_undegraded":"2024-05-17T09:17:11.411802+0000","last_fullsized":"2024-05-17T09:17:11.411802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595216+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595216+0000","last_peered":"2024-05-17T09:16:48.595216+0000","last_clean":"2024-05-17T09:16:48.595216+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595216+0000","last_undegraded":"2024-05-17T09:16:48.595216+0000","last_fullsized":"2024-05-17T09:16:48.595216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769780+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:17:31.769780+0000","last_peered":"2024-05-17T09:17:31.769780+0000","last_clean":"2024-05-17T09:17:31.769780+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:17:31.769780+0000","last_undegraded":"2024-05-17T09:17:31.769780+0000","last_fullsized":"2024-05-17T09:17:31.769780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903562+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903562+0000","last_peered":"2024-05-17T09:17:46.903562+0000","last_clean":"2024-05-17T09:17:46.903562+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903562+0000","last_undegraded":"2024-05-17T09:17:46.903562+0000","last_fullsized":"2024-05-17T09:17:46.903562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.384322+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:17:03.384322+0000","last_peered":"2024-05-17T09:17:03.384322+0000","last_clean":"2024-05-17T09:17:03.384322+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:17:03.384322+0000","last_undegraded":"2024-05-17T09:17:03.384322+0000","last_fullsized":"2024-05-17T09:17:03.384322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112162+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:17:24.112162+0000","last_peered":"2024-05-17T09:17:24.112162+0000","last_clean":"2024-05-17T09:17:24.112162+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:17:24.112162+0000","last_undegraded":"2024-05-17T09:17:24.112162+0000","last_fullsized":"2024-05-17T09:17:24.112162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.936560+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:17:08.936560+0000","last_peered":"2024-05-17T09:17:08.936560+0000","last_clean":"2024-05-17T09:17:08.936560+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:17:08.936560+0000","last_undegraded":"2024-05-17T09:17:08.936560+0000","last_fullsized":"2024-05-17T09:17:08.936560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903852+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903852+0000","last_peered":"2024-05-17T09:17:46.903852+0000","last_clean":"2024-05-17T09:17:46.903852+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903852+0000","last_undegraded":"2024-05-17T09:17:46.903852+0000","last_fullsized":"2024-05-17T09:17:46.903852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:20.633799+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:17:20.633799+0000","last_peered":"2024-05-17T09:17:20.633799+0000","last_clean":"2024-05-17T09:17:20.633799+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:17:20.633799+0000","last_undegraded":"2024-05-17T09:17:20.633799+0000","last_fullsized":"2024-05-17T09:17:20.633799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.948913+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:19:26.948913+0000","last_peered":"2024-05-17T09:19:26.948913+0000","last_clean":"2024-05-17T09:19:26.948913+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:19:26.948913+0000","last_undegraded":"2024-05-17T09:19:26.948913+0000","last_fullsized":"2024-05-17T09:19:26.948913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889973+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889973+0000","last_peered":"2024-05-17T09:17:46.889973+0000","last_clean":"2024-05-17T09:17:46.889973+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889973+0000","last_undegraded":"2024-05-17T09:17:46.889973+0000","last_fullsized":"2024-05-17T09:17:46.889973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.193083+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:17:26.193083+0000","last_peered":"2024-05-17T09:17:26.193083+0000","last_clean":"2024-05-17T09:17:26.193083+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:17:26.193083+0000","last_undegraded":"2024-05-17T09:17:26.193083+0000","last_fullsized":"2024-05-17T09:17:26.193083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.412427+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:17:16.412427+0000","last_peered":"2024-05-17T09:17:16.412427+0000","last_clean":"2024-05-17T09:17:16.412427+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:17:16.412427+0000","last_undegraded":"2024-05-17T09:17:16.412427+0000","last_fullsized":"2024-05-17T09:17:16.412427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898859+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898859+0000","last_peered":"2024-05-17T09:17:46.898859+0000","last_clean":"2024-05-17T09:17:46.898859+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898859+0000","last_undegraded":"2024-05-17T09:17:46.898859+0000","last_fullsized":"2024-05-17T09:17:46.898859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731377+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:16:48.731377+0000","last_peered":"2024-05-17T09:16:48.731377+0000","last_clean":"2024-05-17T09:16:48.731377+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:16:48.731377+0000","last_undegraded":"2024-05-17T09:16:48.731377+0000","last_fullsized":"2024-05-17T09:16:48.731377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.908768+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:17:16.908768+0000","last_peered":"2024-05-17T09:17:16.908768+0000","last_clean":"2024-05-17T09:17:16.908768+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:17:16.908768+0000","last_undegraded":"2024-05-17T09:17:16.908768+0000","last_fullsized":"2024-05-17T09:17:16.908768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898286+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.898286+0000","last_peered":"2024-05-17T09:17:46.898286+0000","last_clean":"2024-05-17T09:17:46.898286+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.898286+0000","last_undegraded":"2024-05-17T09:17:46.898286+0000","last_fullsized":"2024-05-17T09:17:46.898286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317648+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:16:48.317648+0000","last_peered":"2024-05-17T09:16:48.317648+0000","last_clean":"2024-05-17T09:16:48.317648+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:16:48.317648+0000","last_undegraded":"2024-05-17T09:16:48.317648+0000","last_fullsized":"2024-05-17T09:16:48.317648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:15.633296+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:17:15.633296+0000","last_peered":"2024-05-17T09:17:15.633296+0000","last_clean":"2024-05-17T09:17:15.633296+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:17:15.633296+0000","last_undegraded":"2024-05-17T09:17:15.633296+0000","last_fullsized":"2024-05-17T09:17:15.633296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901878+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.901878+0000","last_peered":"2024-05-17T09:17:46.901878+0000","last_clean":"2024-05-17T09:17:46.901878+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.901878+0000","last_undegraded":"2024-05-17T09:17:46.901878+0000","last_fullsized":"2024-05-17T09:17:46.901878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731157+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:16:48.731157+0000","last_peered":"2024-05-17T09:16:48.731157+0000","last_clean":"2024-05-17T09:16:48.731157+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:16:48.731157+0000","last_undegraded":"2024-05-17T09:16:48.731157+0000","last_fullsized":"2024-05-17T09:16:48.731157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.206872+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:19:26.206872+0000","last_peered":"2024-05-17T09:19:26.206872+0000","last_clean":"2024-05-17T09:19:26.206872+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:19:26.206872+0000","last_undegraded":"2024-05-17T09:19:26.206872+0000","last_fullsized":"2024-05-17T09:19:26.206872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901144+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.901144+0000","last_peered":"2024-05-17T09:17:46.901144+0000","last_clean":"2024-05-17T09:17:46.901144+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.901144+0000","last_undegraded":"2024-05-17T09:17:46.901144+0000","last_fullsized":"2024-05-17T09:17:46.901144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731424+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:16:48.731424+0000","last_peered":"2024-05-17T09:16:48.731424+0000","last_clean":"2024-05-17T09:16:48.731424+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:16:48.731424+0000","last_undegraded":"2024-05-17T09:16:48.731424+0000","last_fullsized":"2024-05-17T09:16:48.731424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:18.937946+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:17:18.937946+0000","last_peered":"2024-05-17T09:17:18.937946+0000","last_clean":"2024-05-17T09:17:18.937946+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:17:18.937946+0000","last_undegraded":"2024-05-17T09:17:18.937946+0000","last_fullsized":"2024-05-17T09:17:18.937946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908869+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908869+0000","last_peered":"2024-05-17T09:17:46.908869+0000","last_clean":"2024-05-17T09:17:46.908869+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908869+0000","last_undegraded":"2024-05-17T09:17:46.908869+0000","last_fullsized":"2024-05-17T09:17:46.908869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.910346+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:17:31.910346+0000","last_peered":"2024-05-17T09:17:31.910346+0000","last_clean":"2024-05-17T09:17:31.910346+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:17:31.910346+0000","last_undegraded":"2024-05-17T09:17:31.910346+0000","last_fullsized":"2024-05-17T09:17:31.910346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:21.192445+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:17:21.192445+0000","last_peered":"2024-05-17T09:17:21.192445+0000","last_clean":"2024-05-17T09:17:21.192445+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:17:21.192445+0000","last_undegraded":"2024-05-17T09:17:21.192445+0000","last_fullsized":"2024-05-17T09:17:21.192445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908474+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908474+0000","last_peered":"2024-05-17T09:17:46.908474+0000","last_clean":"2024-05-17T09:17:46.908474+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908474+0000","last_undegraded":"2024-05-17T09:17:46.908474+0000","last_fullsized":"2024-05-17T09:17:46.908474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.859893+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:16:48.859893+0000","last_peered":"2024-05-17T09:16:48.859893+0000","last_clean":"2024-05-17T09:16:48.859893+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:16:48.859893+0000","last_undegraded":"2024-05-17T09:16:48.859893+0000","last_fullsized":"2024-05-17T09:16:48.859893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:21.768494+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:17:21.768494+0000","last_peered":"2024-05-17T09:17:21.768494+0000","last_clean":"2024-05-17T09:17:21.768494+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:17:21.768494+0000","last_undegraded":"2024-05-17T09:17:21.768494+0000","last_fullsized":"2024-05-17T09:17:21.768494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901818+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901818+0000","last_peered":"2024-05-17T09:17:46.901818+0000","last_clean":"2024-05-17T09:17:46.901818+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901818+0000","last_undegraded":"2024-05-17T09:17:46.901818+0000","last_fullsized":"2024-05-17T09:17:46.901818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769697+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:17:31.769697+0000","last_peered":"2024-05-17T09:17:31.769697+0000","last_clean":"2024-05-17T09:17:31.769697+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:17:31.769697+0000","last_undegraded":"2024-05-17T09:17:31.769697+0000","last_fullsized":"2024-05-17T09:17:31.769697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:22.770777+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:17:22.770777+0000","last_peered":"2024-05-17T09:17:22.770777+0000","last_clean":"2024-05-17T09:17:22.770777+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:17:22.770777+0000","last_undegraded":"2024-05-17T09:17:22.770777+0000","last_fullsized":"2024-05-17T09:17:22.770777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897999+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897999+0000","last_peered":"2024-05-17T09:17:46.897999+0000","last_clean":"2024-05-17T09:17:46.897999+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897999+0000","last_undegraded":"2024-05-17T09:17:46.897999+0000","last_fullsized":"2024-05-17T09:17:46.897999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899078+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.899078+0000","last_peered":"2024-05-17T09:17:46.899078+0000","last_clean":"2024-05-17T09:17:46.899078+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.899078+0000","last_undegraded":"2024-05-17T09:17:46.899078+0000","last_fullsized":"2024-05-17T09:17:46.899078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888934+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888934+0000","last_peered":"2024-05-17T09:17:46.888934+0000","last_clean":"2024-05-17T09:17:46.888934+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888934+0000","last_undegraded":"2024-05-17T09:17:46.888934+0000","last_fullsized":"2024-05-17T09:17:46.888934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896944+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896944+0000","last_peered":"2024-05-17T09:17:46.896944+0000","last_clean":"2024-05-17T09:17:46.896944+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896944+0000","last_undegraded":"2024-05-17T09:17:46.896944+0000","last_fullsized":"2024-05-17T09:17:46.896944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904847+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904847+0000","last_peered":"2024-05-17T09:17:46.904847+0000","last_clean":"2024-05-17T09:17:46.904847+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904847+0000","last_undegraded":"2024-05-17T09:17:46.904847+0000","last_fullsized":"2024-05-17T09:17:46.904847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904438+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904438+0000","last_peered":"2024-05-17T09:17:46.904438+0000","last_clean":"2024-05-17T09:17:46.904438+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904438+0000","last_undegraded":"2024-05-17T09:17:46.904438+0000","last_fullsized":"2024-05-17T09:17:46.904438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899186+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899186+0000","last_peered":"2024-05-17T09:17:46.899186+0000","last_clean":"2024-05-17T09:17:46.899186+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899186+0000","last_undegraded":"2024-05-17T09:17:46.899186+0000","last_fullsized":"2024-05-17T09:17:46.899186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891995+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891995+0000","last_peered":"2024-05-17T09:17:46.891995+0000","last_clean":"2024-05-17T09:17:46.891995+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891995+0000","last_undegraded":"2024-05-17T09:17:46.891995+0000","last_fullsized":"2024-05-17T09:17:46.891995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886503+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.886503+0000","last_peered":"2024-05-17T09:17:46.886503+0000","last_clean":"2024-05-17T09:17:46.886503+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.886503+0000","last_undegraded":"2024-05-17T09:17:46.886503+0000","last_fullsized":"2024-05-17T09:17:46.886503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899965+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899965+0000","last_peered":"2024-05-17T09:17:46.899965+0000","last_clean":"2024-05-17T09:17:46.899965+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899965+0000","last_undegraded":"2024-05-17T09:17:46.899965+0000","last_fullsized":"2024-05-17T09:17:46.899965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894683+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894683+0000","last_peered":"2024-05-17T09:17:46.894683+0000","last_clean":"2024-05-17T09:17:46.894683+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894683+0000","last_undegraded":"2024-05-17T09:17:46.894683+0000","last_fullsized":"2024-05-17T09:17:46.894683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908531+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908531+0000","last_peered":"2024-05-17T09:17:46.908531+0000","last_clean":"2024-05-17T09:17:46.908531+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908531+0000","last_undegraded":"2024-05-17T09:17:46.908531+0000","last_fullsized":"2024-05-17T09:17:46.908531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904591+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904591+0000","last_peered":"2024-05-17T09:17:46.904591+0000","last_clean":"2024-05-17T09:17:46.904591+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904591+0000","last_undegraded":"2024-05-17T09:17:46.904591+0000","last_fullsized":"2024-05-17T09:17:46.904591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899578+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899578+0000","last_peered":"2024-05-17T09:17:46.899578+0000","last_clean":"2024-05-17T09:17:46.899578+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899578+0000","last_undegraded":"2024-05-17T09:17:46.899578+0000","last_fullsized":"2024-05-17T09:17:46.899578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1292,"num_read_kb":729,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":224556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907021,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1176674,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266776,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463005,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1112817,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626531,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":579607,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":581948,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182514,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986284,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":516180,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822760,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1103284,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:19:52.800 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T09:19:52.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T09:19:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:53.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:53 smithi019 ceph-mon[31620]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:53.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:53 smithi069 ceph-mon[35081]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:53.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:53 smithi079 ceph-mon[35502]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:54.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:54 smithi019 ceph-mon[31620]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-17T09:19:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:54 smithi069 ceph-mon[35081]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-17T09:19:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:54 smithi079 ceph-mon[35502]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 -- 172.21.15.19:0/651191924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4109870 msgr2=0x7f25f41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 --2- 172.21.15.19:0/651191924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4109870 0x7f25f41111d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0099c0 tx=0x7f25ec009cd0 comp rx=0 tx=0).stop 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 -- 172.21.15.19:0/651191924 shutdown_connections 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 --2- 172.21.15.19:0/651191924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4109870 0x7f25f41111d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 --2- 172.21.15.19:0/651191924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25f4104640 0x7f25f4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 --2- 172.21.15.19:0/651191924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4108f20 0x7f25f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 -- 172.21.15.19:0/651191924 >> 172.21.15.19:0/651191924 conn(0x7f25f4079ef0 msgr2=0x7f25f407a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 -- 172.21.15.19:0/651191924 shutdown_connections 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.829+0000 7f25faae5700 1 -- 172.21.15.19:0/651191924 wait complete. 2024-05-17T09:19:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 Processor -- start 2024-05-17T09:19:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 -- start start 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25f4104640 0x7f25f419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 0x7f25f419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4109870 0x7f25f41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25f4114350 con 0x7f25f4108f20 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41141d0 con 0x7f25f4104640 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25faae5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41144d0 con 0x7f25f4109870 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 0x7f25f419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 0x7f25f419d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46614/0 (socket says 172.21.15.19:46614) 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 -- 172.21.15.19:0/2744500475 learned_addr learned my addr 172.21.15.19:0/2744500475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f9082700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4109870 0x7f25f41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f8881700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25f4104640 0x7f25f419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 -- 172.21.15.19:0/2744500475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4109870 msgr2=0x7f25f41a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4109870 0x7f25f41a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 -- 172.21.15.19:0/2744500475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25f4104640 msgr2=0x7f25f419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25f4104640 0x7f25f419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.830+0000 7f25f3fff700 1 -- 172.21.15.19:0/2744500475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f41a1d80 con 0x7f25f4108f20 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.831+0000 7f25f3fff700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 0x7f25f419d500 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f25dc007df0 tx=0x7f25dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.831+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc016020 con 0x7f25f4108f20 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.831+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25dc00ac90 con 0x7f25f4108f20 2024-05-17T09:19:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.831+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f41a2070 con 0x7f25f4108f20 2024-05-17T09:19:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.831+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc0115e0 con 0x7f25f4108f20 2024-05-17T09:19:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.831+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f41a25b0 con 0x7f25f4108f20 2024-05-17T09:19:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.832+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f40512b0 con 0x7f25f4108f20 2024-05-17T09:19:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.833+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f25dc019020 con 0x7f25f4108f20 2024-05-17T09:19:54.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.835+0000 7f25f1ffb700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25e00b2880 0x7f25e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:54.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.835+0000 7f25f8881700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25e00b2880 0x7f25e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:54.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.835+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f25dc0e1ef0 con 0x7f25f4108f20 2024-05-17T09:19:54.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.835+0000 7f25f8881700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25e00b2880 0x7f25e00b4d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25e4005950 tx=0x7f25e400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:54.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:54.840+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25dc020070 con 0x7f25f4108f20 2024-05-17T09:19:55.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.052+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 --> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25f410fca0 con 0x7f25e00b2880 2024-05-17T09:19:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.061+0000 7f25f1ffb700 1 -- 172.21.15.19:0/2744500475 <== mgr.14152 v2:172.21.15.19:6800/3017687651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340727 (secure 0 0 0) 0x7f25f410fca0 con 0x7f25e00b2880 2024-05-17T09:19:55.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25e00b2880 msgr2=0x7f25e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25e00b2880 0x7f25e00b4d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25e4005950 tx=0x7f25e400a300 comp rx=0 tx=0).stop 2024-05-17T09:19:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 msgr2=0x7f25f419d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 0x7f25f419d500 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f25dc007df0 tx=0x7f25dc00a340 comp rx=0 tx=0).stop 2024-05-17T09:19:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 shutdown_connections 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25e00b2880 0x7f25e00b4d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25f4109870 0x7f25f41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25f4104640 0x7f25f419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 --2- 172.21.15.19:0/2744500475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25f4108f20 0x7f25f419d500 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 >> 172.21.15.19:0/2744500475 conn(0x7f25f4079ef0 msgr2=0x7f25f410f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 shutdown_connections 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:55.065+0000 7f25faae5700 1 -- 172.21.15.19:0/2744500475 wait complete. 2024-05-17T09:19:55.073 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-17T09:19:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 -- 172.21.15.19:0/2605692983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 msgr2=0x7f753410aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 --2- 172.21.15.19:0/2605692983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f753410aea0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f752c009900 tx=0x7f752c02a410 comp rx=0 tx=0).stop 2024-05-17T09:19:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 -- 172.21.15.19:0/2605692983 shutdown_connections 2024-05-17T09:19:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 --2- 172.21.15.19:0/2605692983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f753410aea0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 --2- 172.21.15.19:0/2605692983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7534103d80 0x7f7534104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 --2- 172.21.15.19:0/2605692983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7534102b40 0x7f7534103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 -- 172.21.15.19:0/2605692983 >> 172.21.15.19:0/2605692983 conn(0x7f75340fdd30 msgr2=0x7f75341001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 -- 172.21.15.19:0/2605692983 shutdown_connections 2024-05-17T09:19:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.529+0000 7f7538c8b640 1 -- 172.21.15.19:0/2605692983 wait complete. 2024-05-17T09:19:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 Processor -- start 2024-05-17T09:19:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 -- start start 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7534102b40 0x7f753419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7534103d80 0x7f753419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f75341a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f753410f610 con 0x7f7534108a80 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f753410f490 con 0x7f7534102b40 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7538c8b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f753410f790 con 0x7f7534103d80 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.530+0000 7f7532d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f75341a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f75341a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46634/0 (socket says 172.21.15.19:46634) 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 -- 172.21.15.19:0/3113511127 learned_addr learned my addr 172.21.15.19:0/3113511127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7531d9b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7534103d80 0x7f753419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f753259c640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7534102b40 0x7f753419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 -- 172.21.15.19:0/3113511127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7534103d80 msgr2=0x7f753419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7534103d80 0x7f753419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 -- 172.21.15.19:0/3113511127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7534102b40 msgr2=0x7f753419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7534102b40 0x7f753419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 -- 172.21.15.19:0/3113511127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75341a1be0 con 0x7f7534108a80 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7532d9d640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f75341a1620 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f752c005ab0 tx=0x7f752c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f75237fe640 1 -- 172.21.15.19:0/3113511127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c03f030 con 0x7f7534108a80 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f75237fe640 1 -- 172.21.15.19:0/3113511127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f752c003c90 con 0x7f7534108a80 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75341a1e70 con 0x7f7534108a80 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f75237fe640 1 -- 172.21.15.19:0/3113511127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c035070 con 0x7f7534108a80 2024-05-17T09:19:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.531+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75341a2280 con 0x7f7534108a80 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.534+0000 7f753fc13740 1 Processor -- start 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.534+0000 7f75237fe640 1 -- 172.21.15.19:0/3113511127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f752c042030 con 0x7f7534108a80 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.534+0000 7f753fc13740 1 -- start start 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753fc13740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753fc13740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d931920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753fc13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d938820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753fc13740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e81d8a3c30 con 0x55e81d5672f0 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753fc13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e81d8937d0 con 0x55e81d56f4d0 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753fc13740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e81d897810 con 0x55e81d5901b0 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d931920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d931920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46640/0 (socket says 172.21.15.19:46640) 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 -- 172.21.15.19:0/555106664 learned_addr learned my addr 172.21.15.19:0/555106664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f7531d9b640 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f7531d9b640 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d938820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 -- 172.21.15.19:0/555106664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 msgr2=0x55e81d9313a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9313a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 -- 172.21.15.19:0/555106664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 msgr2=0x55e81d938820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d938820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 -- 172.21.15.19:0/555106664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81d5a8210 con 0x55e81d5672f0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f753259c640 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d931920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f752400bcd0 tx=0x7f752400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8f080a88780d3b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f75237fe640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f751c0b28a0 0x7f751c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.535+0000 7f7520ff9640 1 -- 172.21.15.19:0/555106664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524015070 con 0x55e81d5672f0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f7531d9b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f751c0b28a0 0x7f751c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f7520ff9640 1 -- 172.21.15.19:0/555106664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75240054b0 con 0x55e81d5672f0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f7520ff9640 1 -- 172.21.15.19:0/555106664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524011440 con 0x55e81d5672f0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f75237fe640 1 -- 172.21.15.19:0/3113511127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f752c102f80 con 0x7f7534108a80 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f7531d9b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f751c0b28a0 0x7f751c0b4db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55e81d932570 tx=0x7f75280075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f753fc13740 1 -- 172.21.15.19:0/555106664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 msgr2=0x55e81d931920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.536+0000 7f753fc13740 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d931920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f752400bcd0 tx=0x7f752400f730 comp rx=0 tx=0).stop 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- 172.21.15.19:0/555106664 shutdown_connections 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d938820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d931920 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 --2- 172.21.15.19:0/555106664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- 172.21.15.19:0/555106664 >> 172.21.15.19:0/555106664 conn(0x55e81d584300 msgr2=0x55e81d92cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- 172.21.15.19:0/555106664 shutdown_connections 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- 172.21.15.19:0/555106664 wait complete. 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- start start 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 Processor -- start 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5672f0 0x55e81d93acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d93b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5901b0 0x55e81d942160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e81d579790 con 0x55e81d56f4d0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e81d84f890 con 0x55e81d5901b0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e81d939270 con 0x55e81d5672f0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.537+0000 7f753fc13740 10 client.0 _ll_register_callbacks cb 0x55e81d852d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d93b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d93b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46656/0 (socket says 172.21.15.19:46656) 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 -- 172.21.15.19:0/366571428 learned_addr learned my addr 172.21.15.19:0/366571428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7532d9d640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5901b0 0x55e81d942160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f753259c640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5672f0 0x55e81d93acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 -- 172.21.15.19:0/366571428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5672f0 msgr2=0x55e81d93acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5672f0 0x55e81d93acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 -- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5901b0 msgr2=0x55e81d942160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5901b0 0x55e81d942160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 -- 172.21.15.19:0/366571428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81d674620 con 0x55e81d56f4d0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7531d9b640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d93b200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7528014e60 tx=0x7f7528004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc5bdc4774528e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7519ffb640 1 -- 172.21.15.19:0/366571428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752800fbc0 con 0x55e81d56f4d0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7519ffb640 1 -- 172.21.15.19:0/366571428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f752800fd60 con 0x55e81d56f4d0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75240054b0 con 0x55e81d56f4d0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f7519ffb640 1 -- 172.21.15.19:0/366571428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f752802a470 con 0x55e81d56f4d0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f753fc13740 10 client.15366 fetch_fsmap learned FSMap version 27 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.538+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e81d945160 con 0x55e81d56f4d0 2024-05-17T09:19:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f7519ffb640 1 -- 172.21.15.19:0/366571428 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75240054b0 con 0x55e81d56f4d0 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f7519ffb640 1 -- 172.21.15.19:0/366571428 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f752801d030 con 0x55e81d56f4d0 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753fc13740 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753fc13740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753fc13740 20 client.15366 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753fc13740 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d54ed80 0x55e81d9489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753fc13740 4 client.15366 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e81d8a5b80 con 0x55e81d54ed80 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.539+0000 7f753259c640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d54ed80 0x55e81d9489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.540+0000 7f7519ffb640 10 client.15366 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.540+0000 7f753259c640 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d54ed80 0x55e81d9489c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fd537e24f74e29 server_cookie=71902ab19986e88a in_seq=0 out_seq=0 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.540+0000 7f7519ffb640 1 -- 172.21.15.19:0/366571428 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e81d8a5b80 con 0x55e81d54ed80 2024-05-17T09:19:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.540+0000 7f7519ffb640 10 client.15366 handle_command_reply: tid=0 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.540+0000 7f753fc13740 1 client.15366 shutdown 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 msgr2=0x55e81d93b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d93b200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7528014e60 tx=0x7f7528004a40 comp rx=0 tx=0).stop 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 shutdown_connections 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d54ed80 0x55e81d9489c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5901b0 0x55e81d942160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d93b200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f7519ffb640 0 client.15366 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 --2- 172.21.15.19:0/366571428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5672f0 0x55e81d93acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 >> 172.21.15.19:0/366571428 conn(0x55e81d584300 msgr2=0x55e81d92fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 shutdown_connections 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 1 -- 172.21.15.19:0/366571428 wait complete. 2024-05-17T09:19:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.541+0000 7f753fc13740 20 client.15366 trim_cache size 0 max 16384 2024-05-17T09:19:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.556+0000 7f753fc13740 1 Processor -- start 2024-05-17T09:19:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.556+0000 7f753fc13740 1 -- start start 2024-05-17T09:19:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753fc13740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d94e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753fc13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d942080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753fc13740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d94e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753fc13740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75341af1d0 con 0x55e81d5672f0 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753fc13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75341acaa0 con 0x55e81d56f4d0 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753fc13740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f75341aa8f0 con 0x55e81d5901b0 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d94e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d94e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46658/0 (socket says 172.21.15.19:46658) 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 -- 172.21.15.19:0/2493197440 learned_addr learned my addr 172.21.15.19:0/2493197440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f7531d9b640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d942080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 -- 172.21.15.19:0/2493197440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 msgr2=0x55e81d94e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f7532d9d640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d94e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d94e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 -- 172.21.15.19:0/2493197440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 msgr2=0x55e81d942080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d942080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 -- 172.21.15.19:0/2493197440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81d9325a0 con 0x55e81d5672f0 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f7532d9d640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d94e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f7531d9b640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d942080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f753259c640 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d94e180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55e81d932780 tx=0x7f752400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd75072ae6b3c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.557+0000 7f751bfff640 1 -- 172.21.15.19:0/2493197440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524015070 con 0x55e81d5672f0 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f751bfff640 1 -- 172.21.15.19:0/2493197440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f752402b070 con 0x55e81d5672f0 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f751bfff640 1 -- 172.21.15.19:0/2493197440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240118a0 con 0x55e81d5672f0 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 -- 172.21.15.19:0/2493197440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 msgr2=0x55e81d94e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d94e180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55e81d932780 tx=0x7f752400b5c0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 -- 172.21.15.19:0/2493197440 shutdown_connections 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d94e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d56f4d0 0x55e81d942080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 --2- 172.21.15.19:0/2493197440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d5672f0 0x55e81d94e180 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 -- 172.21.15.19:0/2493197440 >> 172.21.15.19:0/2493197440 conn(0x55e81d584300 msgr2=0x55e81d9347c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 -- 172.21.15.19:0/2493197440 shutdown_connections 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.558+0000 7f753fc13740 1 -- 172.21.15.19:0/2493197440 wait complete. 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 -- start start 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 Processor -- start 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5672f0 0x55e81d941480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d9419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75341aaf60 con 0x55e81d56f4d0 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75341ac3d0 con 0x55e81d5672f0 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f75341ab630 con 0x55e81d5901b0 2024-05-17T09:19:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753fc13740 10 client.0 _ll_register_callbacks cb 0x7f7528014b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f7531d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d9419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f7531d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d9419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46668/0 (socket says 172.21.15.19:46668) 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f7531d9b640 1 -- 172.21.15.19:0/3511937810 learned_addr learned my addr 172.21.15.19:0/3511937810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f753259c640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5672f0 0x55e81d941480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.559+0000 7f7532d9d640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7531d9b640 1 -- 172.21.15.19:0/3511937810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 msgr2=0x55e81d9af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7531d9b640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7531d9b640 1 -- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5672f0 msgr2=0x55e81d941480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7531d9b640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5672f0 0x55e81d941480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7531d9b640 1 -- 172.21.15.19:0/3511937810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81d9af930 con 0x55e81d56f4d0 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7532d9d640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7531d9b640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d9419c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f75280079a0 tx=0x7f7528004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcde6392e5678ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7519ffb640 1 -- 172.21.15.19:0/3511937810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528021db0 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7519ffb640 1 -- 172.21.15.19:0/3511937810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f752800fa20 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f752402b070 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7519ffb640 1 -- 172.21.15.19:0/3511937810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7528045050 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f753fc13740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e81d9b0b60 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.560+0000 7f7519ffb640 1 -- 172.21.15.19:0/3511937810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f752402b070 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f7519ffb640 1 -- 172.21.15.19:0/3511937810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f752801d040 con 0x55e81d56f4d0 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753fc13740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753fc13740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:19:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753fc13740 20 client.15372 populate_metadata read hostname 'smithi019' 2024-05-17T09:19:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753fc13740 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d5241c0 0x55e81d9b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753fc13740 4 client.15372 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:19:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e81d89f360 con 0x55e81d5241c0 2024-05-17T09:19:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753259c640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d5241c0 0x55e81d9b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f753259c640 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d5241c0 0x55e81d9b35d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bbfe1d8366bea server_cookie=7c60380f3081eace in_seq=0 out_seq=0 2024-05-17T09:19:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.561+0000 7f7519ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.574+0000 7f7519ffb640 1 -- 172.21.15.19:0/3511937810 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e81d89f360 con 0x55e81d5241c0 2024-05-17T09:19:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.574+0000 7f7519ffb640 10 client.15372 handle_command_reply: tid=0 2024-05-17T09:19:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.574+0000 7f753fc13740 1 client.15372 shutdown 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.574+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 msgr2=0x55e81d9419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d9419c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f75280079a0 tx=0x7f7528004ce0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 shutdown_connections 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e81d5241c0 0x55e81d9b35d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e81d5901b0 0x55e81d9af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81d56f4d0 0x55e81d9419c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7519ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 --2- 172.21.15.19:0/3511937810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e81d5672f0 0x55e81d941480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 >> 172.21.15.19:0/3511937810 conn(0x55e81d584300 msgr2=0x55e81d9351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 shutdown_connections 2024-05-17T09:19:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 1 -- 172.21.15.19:0/3511937810 wait complete. 2024-05-17T09:19:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f753fc13740 20 client.15372 trim_cache size 0 max 16384 2024-05-17T09:19:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:19:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:19:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:19:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f751c0b28a0 msgr2=0x7f751c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f751c0b28a0 0x7f751c0b4db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55e81d932570 tx=0x7f75280075b0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 msgr2=0x7f75341a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f75341a1620 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f752c005ab0 tx=0x7f752c0042f0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 shutdown_connections 2024-05-17T09:19:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f751c0b28a0 0x7f751c0b4db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7534108a80 0x7f75341a1620 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7534103d80 0x7f753419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 --2- 172.21.15.19:0/3113511127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7534102b40 0x7f753419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.575+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 >> 172.21.15.19:0/3113511127 conn(0x7f75340fdd30 msgr2=0x7f7534104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.576+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 shutdown_connections 2024-05-17T09:19:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:19:55.576+0000 7f7538c8b640 1 -- 172.21.15.19:0/3113511127 wait complete. 2024-05-17T09:19:55.903 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-05-17T09:19:54.954987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1398,"num_read_kb":808,"num_write":313,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348656,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593168,"statfs":{"total":1151940427776,"available":1151583404032,"internally_reserved":0,"allocated":12271616,"data_stored":8153408,"data_compressed":495939,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":42,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004048"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.906823+0000","last_change":"2024-05-17T09:15:07.962824+0000","last_active":"2024-05-17T09:17:46.906823+0000","last_peered":"2024-05-17T09:17:46.906823+0000","last_clean":"2024-05-17T09:17:46.906823+0000","last_became_active":"2024-05-17T09:15:07.962381+0000","last_became_peered":"2024-05-17T09:15:07.962381+0000","last_unstale":"2024-05-17T09:17:46.906823+0000","last_undegraded":"2024-05-17T09:17:46.906823+0000","last_fullsized":"2024-05-17T09:17:46.906823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:27:33.396779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901351+0000","last_change":"2024-05-17T09:15:07.957043+0000","last_active":"2024-05-17T09:17:46.901351+0000","last_peered":"2024-05-17T09:17:46.901351+0000","last_clean":"2024-05-17T09:17:46.901351+0000","last_became_active":"2024-05-17T09:15:07.956700+0000","last_became_peered":"2024-05-17T09:15:07.956700+0000","last_unstale":"2024-05-17T09:17:46.901351+0000","last_undegraded":"2024-05-17T09:17:46.901351+0000","last_fullsized":"2024-05-17T09:17:46.901351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:29:43.835294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905015+0000","last_change":"2024-05-17T09:15:07.963609+0000","last_active":"2024-05-17T09:17:46.905015+0000","last_peered":"2024-05-17T09:17:46.905015+0000","last_clean":"2024-05-17T09:17:46.905015+0000","last_became_active":"2024-05-17T09:15:07.963211+0000","last_became_peered":"2024-05-17T09:15:07.963211+0000","last_unstale":"2024-05-17T09:17:46.905015+0000","last_undegraded":"2024-05-17T09:17:46.905015+0000","last_fullsized":"2024-05-17T09:17:46.905015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:32:09.935707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886153+0000","last_change":"2024-05-17T09:15:07.963816+0000","last_active":"2024-05-17T09:17:46.886153+0000","last_peered":"2024-05-17T09:17:46.886153+0000","last_clean":"2024-05-17T09:17:46.886153+0000","last_became_active":"2024-05-17T09:15:07.962667+0000","last_became_peered":"2024-05-17T09:15:07.962667+0000","last_unstale":"2024-05-17T09:17:46.886153+0000","last_undegraded":"2024-05-17T09:17:46.886153+0000","last_fullsized":"2024-05-17T09:17:46.886153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:18:08.319935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888169+0000","last_change":"2024-05-17T09:15:07.963368+0000","last_active":"2024-05-17T09:17:46.888169+0000","last_peered":"2024-05-17T09:17:46.888169+0000","last_clean":"2024-05-17T09:17:46.888169+0000","last_became_active":"2024-05-17T09:15:07.962969+0000","last_became_peered":"2024-05-17T09:15:07.962969+0000","last_unstale":"2024-05-17T09:17:46.888169+0000","last_undegraded":"2024-05-17T09:17:46.888169+0000","last_fullsized":"2024-05-17T09:17:46.888169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:47.064195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902246+0000","last_change":"2024-05-17T09:15:07.960434+0000","last_active":"2024-05-17T09:17:46.902246+0000","last_peered":"2024-05-17T09:17:46.902246+0000","last_clean":"2024-05-17T09:17:46.902246+0000","last_became_active":"2024-05-17T09:15:07.959997+0000","last_became_peered":"2024-05-17T09:15:07.959997+0000","last_unstale":"2024-05-17T09:17:46.902246+0000","last_undegraded":"2024-05-17T09:17:46.902246+0000","last_fullsized":"2024-05-17T09:17:46.902246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:25:55.627254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900418+0000","last_change":"2024-05-17T09:15:07.971731+0000","last_active":"2024-05-17T09:17:46.900418+0000","last_peered":"2024-05-17T09:17:46.900418+0000","last_clean":"2024-05-17T09:17:46.900418+0000","last_became_active":"2024-05-17T09:15:07.971374+0000","last_became_peered":"2024-05-17T09:15:07.971374+0000","last_unstale":"2024-05-17T09:17:46.900418+0000","last_undegraded":"2024-05-17T09:17:46.900418+0000","last_fullsized":"2024-05-17T09:17:46.900418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:48:56.047017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898447+0000","last_change":"2024-05-17T09:15:07.956913+0000","last_active":"2024-05-17T09:17:46.898447+0000","last_peered":"2024-05-17T09:17:46.898447+0000","last_clean":"2024-05-17T09:17:46.898447+0000","last_became_active":"2024-05-17T09:15:07.956585+0000","last_became_peered":"2024-05-17T09:15:07.956585+0000","last_unstale":"2024-05-17T09:17:46.898447+0000","last_undegraded":"2024-05-17T09:17:46.898447+0000","last_fullsized":"2024-05-17T09:17:46.898447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:48:19.100951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.907234+0000","last_change":"2024-05-17T09:15:07.957569+0000","last_active":"2024-05-17T09:17:46.907234+0000","last_peered":"2024-05-17T09:17:46.907234+0000","last_clean":"2024-05-17T09:17:46.907234+0000","last_became_active":"2024-05-17T09:15:07.957195+0000","last_became_peered":"2024-05-17T09:15:07.957195+0000","last_unstale":"2024-05-17T09:17:46.907234+0000","last_undegraded":"2024-05-17T09:17:46.907234+0000","last_fullsized":"2024-05-17T09:17:46.907234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:59:14.765133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902126+0000","last_change":"2024-05-17T09:15:07.960270+0000","last_active":"2024-05-17T09:17:46.902126+0000","last_peered":"2024-05-17T09:17:46.902126+0000","last_clean":"2024-05-17T09:17:46.902126+0000","last_became_active":"2024-05-17T09:15:07.959880+0000","last_became_peered":"2024-05-17T09:15:07.959880+0000","last_unstale":"2024-05-17T09:17:46.902126+0000","last_undegraded":"2024-05-17T09:17:46.902126+0000","last_fullsized":"2024-05-17T09:17:46.902126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:40:32.046562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.890489+0000","last_change":"2024-05-17T09:15:07.963834+0000","last_active":"2024-05-17T09:17:46.890489+0000","last_peered":"2024-05-17T09:17:46.890489+0000","last_clean":"2024-05-17T09:17:46.890489+0000","last_became_active":"2024-05-17T09:15:07.963430+0000","last_became_peered":"2024-05-17T09:15:07.963430+0000","last_unstale":"2024-05-17T09:17:46.890489+0000","last_undegraded":"2024-05-17T09:17:46.890489+0000","last_fullsized":"2024-05-17T09:17:46.890489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:58:26.102071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893936+0000","last_change":"2024-05-17T09:15:07.972209+0000","last_active":"2024-05-17T09:17:46.893936+0000","last_peered":"2024-05-17T09:17:46.893936+0000","last_clean":"2024-05-17T09:17:46.893936+0000","last_became_active":"2024-05-17T09:15:07.971850+0000","last_became_peered":"2024-05-17T09:15:07.971850+0000","last_unstale":"2024-05-17T09:17:46.893936+0000","last_undegraded":"2024-05-17T09:17:46.893936+0000","last_fullsized":"2024-05-17T09:17:46.893936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:33:30.936014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902609+0000","last_change":"2024-05-17T09:15:07.958120+0000","last_active":"2024-05-17T09:17:46.902609+0000","last_peered":"2024-05-17T09:17:46.902609+0000","last_clean":"2024-05-17T09:17:46.902609+0000","last_became_active":"2024-05-17T09:15:07.957686+0000","last_became_peered":"2024-05-17T09:15:07.957686+0000","last_unstale":"2024-05-17T09:17:46.902609+0000","last_undegraded":"2024-05-17T09:17:46.902609+0000","last_fullsized":"2024-05-17T09:17:46.902609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:27.092779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898029+0000","last_change":"2024-05-17T09:15:07.956996+0000","last_active":"2024-05-17T09:17:46.898029+0000","last_peered":"2024-05-17T09:17:46.898029+0000","last_clean":"2024-05-17T09:17:46.898029+0000","last_became_active":"2024-05-17T09:15:07.956631+0000","last_became_peered":"2024-05-17T09:15:07.956631+0000","last_unstale":"2024-05-17T09:17:46.898029+0000","last_undegraded":"2024-05-17T09:17:46.898029+0000","last_fullsized":"2024-05-17T09:17:46.898029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:48:57.029869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.893666+0000","last_change":"2024-05-17T09:15:48.159911+0000","last_active":"2024-05-17T09:17:46.893666+0000","last_peered":"2024-05-17T09:17:46.893666+0000","last_clean":"2024-05-17T09:17:46.893666+0000","last_became_active":"2024-05-17T09:15:48.159544+0000","last_became_peered":"2024-05-17T09:15:48.159544+0000","last_unstale":"2024-05-17T09:17:46.893666+0000","last_undegraded":"2024-05-17T09:17:46.893666+0000","last_fullsized":"2024-05-17T09:17:46.893666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:08:05.746622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908222+0000","last_change":"2024-05-17T09:15:07.964957+0000","last_active":"2024-05-17T09:17:46.908222+0000","last_peered":"2024-05-17T09:17:46.908222+0000","last_clean":"2024-05-17T09:17:46.908222+0000","last_became_active":"2024-05-17T09:15:07.964569+0000","last_became_peered":"2024-05-17T09:15:07.964569+0000","last_unstale":"2024-05-17T09:17:46.908222+0000","last_undegraded":"2024-05-17T09:17:46.908222+0000","last_fullsized":"2024-05-17T09:17:46.908222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:33.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908289+0000","last_change":"2024-05-17T09:15:07.957760+0000","last_active":"2024-05-17T09:17:46.908289+0000","last_peered":"2024-05-17T09:17:46.908289+0000","last_clean":"2024-05-17T09:17:46.908289+0000","last_became_active":"2024-05-17T09:15:07.957264+0000","last_became_peered":"2024-05-17T09:15:07.957264+0000","last_unstale":"2024-05-17T09:17:46.908289+0000","last_undegraded":"2024-05-17T09:17:46.908289+0000","last_fullsized":"2024-05-17T09:17:46.908289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:27:48.038049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.902727+0000","last_change":"2024-05-17T09:15:07.954982+0000","last_active":"2024-05-17T09:17:46.902727+0000","last_peered":"2024-05-17T09:17:46.902727+0000","last_clean":"2024-05-17T09:17:46.902727+0000","last_became_active":"2024-05-17T09:15:07.954654+0000","last_became_peered":"2024-05-17T09:15:07.954654+0000","last_unstale":"2024-05-17T09:17:46.902727+0000","last_undegraded":"2024-05-17T09:17:46.902727+0000","last_fullsized":"2024-05-17T09:17:46.902727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:09:34.761472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889416+0000","last_change":"2024-05-17T09:15:07.960858+0000","last_active":"2024-05-17T09:17:46.889416+0000","last_peered":"2024-05-17T09:17:46.889416+0000","last_clean":"2024-05-17T09:17:46.889416+0000","last_became_active":"2024-05-17T09:15:07.960201+0000","last_became_peered":"2024-05-17T09:15:07.960201+0000","last_unstale":"2024-05-17T09:17:46.889416+0000","last_undegraded":"2024-05-17T09:17:46.889416+0000","last_fullsized":"2024-05-17T09:17:46.889416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:27:00.698554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.318718+0000","last_change":"2024-05-17T09:15:20.041144+0000","last_active":"2024-05-17T09:16:48.318718+0000","last_peered":"2024-05-17T09:16:48.318718+0000","last_clean":"2024-05-17T09:16:48.318718+0000","last_became_active":"2024-05-17T09:15:20.040664+0000","last_became_peered":"2024-05-17T09:15:20.040664+0000","last_unstale":"2024-05-17T09:16:48.318718+0000","last_undegraded":"2024-05-17T09:16:48.318718+0000","last_fullsized":"2024-05-17T09:16:48.318718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:55:47.251341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:16.948046+0000","last_change":"2024-05-17T09:15:03.932098+0000","last_active":"2024-05-17T09:19:16.948046+0000","last_peered":"2024-05-17T09:19:16.948046+0000","last_clean":"2024-05-17T09:19:16.948046+0000","last_became_active":"2024-05-17T09:15:03.931779+0000","last_became_peered":"2024-05-17T09:15:03.931779+0000","last_unstale":"2024-05-17T09:19:16.948046+0000","last_undegraded":"2024-05-17T09:19:16.948046+0000","last_fullsized":"2024-05-17T09:19:16.948046+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:18:07.565837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898655+0000","last_change":"2024-05-17T09:15:07.960431+0000","last_active":"2024-05-17T09:17:46.898655+0000","last_peered":"2024-05-17T09:17:46.898655+0000","last_clean":"2024-05-17T09:17:46.898655+0000","last_became_active":"2024-05-17T09:15:07.958344+0000","last_became_peered":"2024-05-17T09:15:07.958344+0000","last_unstale":"2024-05-17T09:17:46.898655+0000","last_undegraded":"2024-05-17T09:17:46.898655+0000","last_fullsized":"2024-05-17T09:17:46.898655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:22:56.846589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730956+0000","last_change":"2024-05-17T09:15:20.038136+0000","last_active":"2024-05-17T09:16:48.730956+0000","last_peered":"2024-05-17T09:16:48.730956+0000","last_clean":"2024-05-17T09:16:48.730956+0000","last_became_active":"2024-05-17T09:15:20.037679+0000","last_became_peered":"2024-05-17T09:15:20.037679+0000","last_unstale":"2024-05-17T09:16:48.730956+0000","last_undegraded":"2024-05-17T09:16:48.730956+0000","last_fullsized":"2024-05-17T09:16:48.730956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:41:44.981419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:05.632115+0000","last_change":"2024-05-17T09:15:04.127691+0000","last_active":"2024-05-17T09:17:05.632115+0000","last_peered":"2024-05-17T09:17:05.632115+0000","last_clean":"2024-05-17T09:17:05.632115+0000","last_became_active":"2024-05-17T09:15:04.127288+0000","last_became_peered":"2024-05-17T09:15:04.127288+0000","last_unstale":"2024-05-17T09:17:05.632115+0000","last_undegraded":"2024-05-17T09:17:05.632115+0000","last_fullsized":"2024-05-17T09:17:05.632115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:35:41.842344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905191+0000","last_change":"2024-05-17T09:15:07.963564+0000","last_active":"2024-05-17T09:17:46.905191+0000","last_peered":"2024-05-17T09:17:46.905191+0000","last_clean":"2024-05-17T09:17:46.905191+0000","last_became_active":"2024-05-17T09:15:07.963180+0000","last_became_peered":"2024-05-17T09:15:07.963180+0000","last_unstale":"2024-05-17T09:17:46.905191+0000","last_undegraded":"2024-05-17T09:17:46.905191+0000","last_fullsized":"2024-05-17T09:17:46.905191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:53:19.577642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731016+0000","last_change":"2024-05-17T09:15:20.036425+0000","last_active":"2024-05-17T09:16:48.731016+0000","last_peered":"2024-05-17T09:16:48.731016+0000","last_clean":"2024-05-17T09:16:48.731016+0000","last_became_active":"2024-05-17T09:15:20.035613+0000","last_became_peered":"2024-05-17T09:15:20.035613+0000","last_unstale":"2024-05-17T09:16:48.731016+0000","last_undegraded":"2024-05-17T09:16:48.731016+0000","last_fullsized":"2024-05-17T09:16:48.731016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:42:41.461407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830356+0000","last_change":"2024-05-17T09:15:48.162756+0000","last_active":"2024-05-17T09:16:48.830356+0000","last_peered":"2024-05-17T09:16:48.830356+0000","last_clean":"2024-05-17T09:16:48.830356+0000","last_became_active":"2024-05-17T09:15:48.155037+0000","last_became_peered":"2024-05-17T09:15:48.155037+0000","last_unstale":"2024-05-17T09:16:48.830356+0000","last_undegraded":"2024-05-17T09:16:48.830356+0000","last_fullsized":"2024-05-17T09:16:48.830356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:33.270469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904312+0000","last_change":"2024-05-17T09:15:07.967823+0000","last_active":"2024-05-17T09:17:46.904312+0000","last_peered":"2024-05-17T09:17:46.904312+0000","last_clean":"2024-05-17T09:17:46.904312+0000","last_became_active":"2024-05-17T09:15:07.967432+0000","last_became_peered":"2024-05-17T09:15:07.967432+0000","last_unstale":"2024-05-17T09:17:46.904312+0000","last_undegraded":"2024-05-17T09:17:46.904312+0000","last_fullsized":"2024-05-17T09:17:46.904312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:27:36.331893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.730917+0000","last_change":"2024-05-17T09:15:20.036980+0000","last_active":"2024-05-17T09:16:48.730917+0000","last_peered":"2024-05-17T09:16:48.730917+0000","last_clean":"2024-05-17T09:16:48.730917+0000","last_became_active":"2024-05-17T09:15:20.033351+0000","last_became_peered":"2024-05-17T09:15:20.033351+0000","last_unstale":"2024-05-17T09:16:48.730917+0000","last_undegraded":"2024-05-17T09:16:48.730917+0000","last_fullsized":"2024-05-17T09:16:48.730917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:52:54.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.829835+0000","last_change":"2024-05-17T09:15:03.933658+0000","last_active":"2024-05-17T09:16:48.829835+0000","last_peered":"2024-05-17T09:16:48.829835+0000","last_clean":"2024-05-17T09:16:48.829835+0000","last_became_active":"2024-05-17T09:15:03.933214+0000","last_became_peered":"2024-05-17T09:15:03.933214+0000","last_unstale":"2024-05-17T09:16:48.829835+0000","last_undegraded":"2024-05-17T09:16:48.829835+0000","last_fullsized":"2024-05-17T09:16:48.829835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:58:43.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908538+0000","last_change":"2024-05-17T09:15:07.958617+0000","last_active":"2024-05-17T09:17:46.908538+0000","last_peered":"2024-05-17T09:17:46.908538+0000","last_clean":"2024-05-17T09:17:46.908538+0000","last_became_active":"2024-05-17T09:15:07.958284+0000","last_became_peered":"2024-05-17T09:15:07.958284+0000","last_unstale":"2024-05-17T09:17:46.908538+0000","last_undegraded":"2024-05-17T09:17:46.908538+0000","last_fullsized":"2024-05-17T09:17:46.908538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:59:52.214903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319326+0000","last_change":"2024-05-17T09:15:20.035189+0000","last_active":"2024-05-17T09:16:48.319326+0000","last_peered":"2024-05-17T09:16:48.319326+0000","last_clean":"2024-05-17T09:16:48.319326+0000","last_became_active":"2024-05-17T09:15:20.034758+0000","last_became_peered":"2024-05-17T09:15:20.034758+0000","last_unstale":"2024-05-17T09:16:48.319326+0000","last_undegraded":"2024-05-17T09:16:48.319326+0000","last_fullsized":"2024-05-17T09:16:48.319326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:08.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":270,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:50.569030+0000","last_change":"2024-05-17T09:15:03.932973+0000","last_active":"2024-05-17T09:19:50.569030+0000","last_peered":"2024-05-17T09:19:50.569030+0000","last_clean":"2024-05-17T09:19:50.569030+0000","last_became_active":"2024-05-17T09:15:03.932497+0000","last_became_peered":"2024-05-17T09:15:03.932497+0000","last_unstale":"2024-05-17T09:19:50.569030+0000","last_undegraded":"2024-05-17T09:19:50.569030+0000","last_fullsized":"2024-05-17T09:19:50.569030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:51:11.798442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":218,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.883101+0000","last_change":"2024-05-17T09:15:07.958180+0000","last_active":"2024-05-17T09:17:46.883101+0000","last_peered":"2024-05-17T09:17:46.883101+0000","last_clean":"2024-05-17T09:17:46.883101+0000","last_became_active":"2024-05-17T09:15:07.957831+0000","last_became_peered":"2024-05-17T09:15:07.957831+0000","last_unstale":"2024-05-17T09:17:46.883101+0000","last_undegraded":"2024-05-17T09:17:46.883101+0000","last_fullsized":"2024-05-17T09:17:46.883101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:09:24.635449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.319160+0000","last_change":"2024-05-17T09:15:20.034489+0000","last_active":"2024-05-17T09:16:48.319160+0000","last_peered":"2024-05-17T09:16:48.319160+0000","last_clean":"2024-05-17T09:16:48.319160+0000","last_became_active":"2024-05-17T09:15:20.034022+0000","last_became_peered":"2024-05-17T09:15:20.034022+0000","last_unstale":"2024-05-17T09:16:48.319160+0000","last_undegraded":"2024-05-17T09:16:48.319160+0000","last_fullsized":"2024-05-17T09:16:48.319160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:45:04.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320630+0000","last_change":"2024-05-17T09:15:03.935791+0000","last_active":"2024-05-17T09:16:48.320630+0000","last_peered":"2024-05-17T09:16:48.320630+0000","last_clean":"2024-05-17T09:16:48.320630+0000","last_became_active":"2024-05-17T09:15:03.934550+0000","last_became_peered":"2024-05-17T09:15:03.934550+0000","last_unstale":"2024-05-17T09:16:48.320630+0000","last_undegraded":"2024-05-17T09:16:48.320630+0000","last_fullsized":"2024-05-17T09:16:48.320630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:28:43.671706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900020+0000","last_change":"2024-05-17T09:15:48.527228+0000","last_active":"2024-05-17T09:17:46.900020+0000","last_peered":"2024-05-17T09:17:46.900020+0000","last_clean":"2024-05-17T09:17:46.900020+0000","last_became_active":"2024-05-17T09:15:48.526862+0000","last_became_peered":"2024-05-17T09:15:48.526862+0000","last_unstale":"2024-05-17T09:17:46.900020+0000","last_undegraded":"2024-05-17T09:17:46.900020+0000","last_fullsized":"2024-05-17T09:17:46.900020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:28:09.875800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112225+0000","last_change":"2024-05-17T09:15:48.160379+0000","last_active":"2024-05-17T09:17:24.112225+0000","last_peered":"2024-05-17T09:17:24.112225+0000","last_clean":"2024-05-17T09:17:24.112225+0000","last_became_active":"2024-05-17T09:15:48.159937+0000","last_became_peered":"2024-05-17T09:15:48.159937+0000","last_unstale":"2024-05-17T09:17:24.112225+0000","last_undegraded":"2024-05-17T09:17:24.112225+0000","last_fullsized":"2024-05-17T09:17:24.112225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:19:43.982195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320502+0000","last_change":"2024-05-17T09:15:03.925672+0000","last_active":"2024-05-17T09:16:48.320502+0000","last_peered":"2024-05-17T09:16:48.320502+0000","last_clean":"2024-05-17T09:16:48.320502+0000","last_became_active":"2024-05-17T09:15:03.925271+0000","last_became_peered":"2024-05-17T09:15:03.925271+0000","last_unstale":"2024-05-17T09:16:48.320502+0000","last_undegraded":"2024-05-17T09:16:48.320502+0000","last_fullsized":"2024-05-17T09:16:48.320502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:56:32.347186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894703+0000","last_change":"2024-05-17T09:15:07.953880+0000","last_active":"2024-05-17T09:17:46.894703+0000","last_peered":"2024-05-17T09:17:46.894703+0000","last_clean":"2024-05-17T09:17:46.894703+0000","last_became_active":"2024-05-17T09:15:07.953525+0000","last_became_peered":"2024-05-17T09:15:07.953525+0000","last_unstale":"2024-05-17T09:17:46.894703+0000","last_undegraded":"2024-05-17T09:17:46.894703+0000","last_fullsized":"2024-05-17T09:17:46.894703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:00:25.832891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.430695+0000","last_change":"2024-05-17T09:15:20.041045+0000","last_active":"2024-05-17T09:17:23.430695+0000","last_peered":"2024-05-17T09:17:23.430695+0000","last_clean":"2024-05-17T09:17:23.430695+0000","last_became_active":"2024-05-17T09:15:20.040601+0000","last_became_peered":"2024-05-17T09:15:20.040601+0000","last_unstale":"2024-05-17T09:17:23.430695+0000","last_undegraded":"2024-05-17T09:17:23.430695+0000","last_fullsized":"2024-05-17T09:17:23.430695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:26:13.923554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423172+0000","last_change":"2024-05-17T09:15:03.933285+0000","last_active":"2024-05-17T09:16:48.423172+0000","last_peered":"2024-05-17T09:16:48.423172+0000","last_clean":"2024-05-17T09:16:48.423172+0000","last_became_active":"2024-05-17T09:15:03.932940+0000","last_became_peered":"2024-05-17T09:15:03.932940+0000","last_unstale":"2024-05-17T09:16:48.423172+0000","last_undegraded":"2024-05-17T09:16:48.423172+0000","last_fullsized":"2024-05-17T09:16:48.423172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:19:13.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888442+0000","last_change":"2024-05-17T09:15:07.954003+0000","last_active":"2024-05-17T09:17:46.888442+0000","last_peered":"2024-05-17T09:17:46.888442+0000","last_clean":"2024-05-17T09:17:46.888442+0000","last_became_active":"2024-05-17T09:15:07.953698+0000","last_became_peered":"2024-05-17T09:15:07.953698+0000","last_unstale":"2024-05-17T09:17:46.888442+0000","last_undegraded":"2024-05-17T09:17:46.888442+0000","last_fullsized":"2024-05-17T09:17:46.888442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T21:09:26.249007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.755102+0000","last_change":"2024-05-17T09:15:20.036018+0000","last_active":"2024-05-17T09:16:48.755102+0000","last_peered":"2024-05-17T09:16:48.755102+0000","last_clean":"2024-05-17T09:16:48.755102+0000","last_became_active":"2024-05-17T09:15:20.033210+0000","last_became_peered":"2024-05-17T09:15:20.033210+0000","last_unstale":"2024-05-17T09:16:48.755102+0000","last_undegraded":"2024-05-17T09:16:48.755102+0000","last_fullsized":"2024-05-17T09:16:48.755102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:16:59.239412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":266,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:53.573551+0000","last_change":"2024-05-17T09:15:03.935433+0000","last_active":"2024-05-17T09:19:53.573551+0000","last_peered":"2024-05-17T09:19:53.573551+0000","last_clean":"2024-05-17T09:19:53.573551+0000","last_became_active":"2024-05-17T09:15:03.935069+0000","last_became_peered":"2024-05-17T09:15:03.935069+0000","last_unstale":"2024-05-17T09:19:53.573551+0000","last_undegraded":"2024-05-17T09:19:53.573551+0000","last_fullsized":"2024-05-17T09:19:53.573551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:30:22.611929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903037+0000","last_change":"2024-05-17T09:15:07.961356+0000","last_active":"2024-05-17T09:17:46.903037+0000","last_peered":"2024-05-17T09:17:46.903037+0000","last_clean":"2024-05-17T09:17:46.903037+0000","last_became_active":"2024-05-17T09:15:07.960990+0000","last_became_peered":"2024-05-17T09:15:07.960990+0000","last_unstale":"2024-05-17T09:17:46.903037+0000","last_undegraded":"2024-05-17T09:17:46.903037+0000","last_fullsized":"2024-05-17T09:17:46.903037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:01:55.679220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.317452+0000","last_change":"2024-05-17T09:15:20.043894+0000","last_active":"2024-05-17T09:16:48.317452+0000","last_peered":"2024-05-17T09:16:48.317452+0000","last_clean":"2024-05-17T09:16:48.317452+0000","last_became_active":"2024-05-17T09:15:20.043123+0000","last_became_peered":"2024-05-17T09:15:20.043123+0000","last_unstale":"2024-05-17T09:16:48.317452+0000","last_undegraded":"2024-05-17T09:16:48.317452+0000","last_fullsized":"2024-05-17T09:16:48.317452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:02.604875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387343+0000","last_change":"2024-05-17T09:15:03.933591+0000","last_active":"2024-05-17T09:17:28.387343+0000","last_peered":"2024-05-17T09:17:28.387343+0000","last_clean":"2024-05-17T09:17:28.387343+0000","last_became_active":"2024-05-17T09:15:03.933220+0000","last_became_peered":"2024-05-17T09:15:03.933220+0000","last_unstale":"2024-05-17T09:17:28.387343+0000","last_undegraded":"2024-05-17T09:17:28.387343+0000","last_fullsized":"2024-05-17T09:17:28.387343+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:50:10.614686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886801+0000","last_change":"2024-05-17T09:15:07.961210+0000","last_active":"2024-05-17T09:17:46.886801+0000","last_peered":"2024-05-17T09:17:46.886801+0000","last_clean":"2024-05-17T09:17:46.886801+0000","last_became_active":"2024-05-17T09:15:07.960884+0000","last_became_peered":"2024-05-17T09:15:07.960884+0000","last_unstale":"2024-05-17T09:17:46.886801+0000","last_undegraded":"2024-05-17T09:17:46.886801+0000","last_fullsized":"2024-05-17T09:17:46.886801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:03.192748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.731371+0000","last_change":"2024-05-17T09:15:20.035133+0000","last_active":"2024-05-17T09:16:48.731371+0000","last_peered":"2024-05-17T09:16:48.731371+0000","last_clean":"2024-05-17T09:16:48.731371+0000","last_became_active":"2024-05-17T09:15:20.034698+0000","last_became_peered":"2024-05-17T09:15:20.034698+0000","last_unstale":"2024-05-17T09:16:48.731371+0000","last_undegraded":"2024-05-17T09:16:48.731371+0000","last_fullsized":"2024-05-17T09:16:48.731371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:46:49.292943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423645+0000","last_change":"2024-05-17T09:15:04.128432+0000","last_active":"2024-05-17T09:16:48.423645+0000","last_peered":"2024-05-17T09:16:48.423645+0000","last_clean":"2024-05-17T09:16:48.423645+0000","last_became_active":"2024-05-17T09:15:04.128011+0000","last_became_peered":"2024-05-17T09:15:04.128011+0000","last_unstale":"2024-05-17T09:16:48.423645+0000","last_undegraded":"2024-05-17T09:16:48.423645+0000","last_fullsized":"2024-05-17T09:16:48.423645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:55:40.202444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908920+0000","last_change":"2024-05-17T09:15:07.972011+0000","last_active":"2024-05-17T09:17:46.908920+0000","last_peered":"2024-05-17T09:17:46.908920+0000","last_clean":"2024-05-17T09:17:46.908920+0000","last_became_active":"2024-05-17T09:15:07.971473+0000","last_became_peered":"2024-05-17T09:15:07.971473+0000","last_unstale":"2024-05-17T09:17:46.908920+0000","last_undegraded":"2024-05-17T09:17:46.908920+0000","last_fullsized":"2024-05-17T09:17:46.908920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:00:28.730473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320433+0000","last_change":"2024-05-17T09:15:48.158408+0000","last_active":"2024-05-17T09:16:48.320433+0000","last_peered":"2024-05-17T09:16:48.320433+0000","last_clean":"2024-05-17T09:16:48.320433+0000","last_became_active":"2024-05-17T09:15:48.157915+0000","last_became_peered":"2024-05-17T09:15:48.157915+0000","last_unstale":"2024-05-17T09:16:48.320433+0000","last_undegraded":"2024-05-17T09:16:48.320433+0000","last_fullsized":"2024-05-17T09:16:48.320433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:16.174725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":528,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:53.573535+0000","last_change":"2024-05-17T09:15:03.933686+0000","last_active":"2024-05-17T09:19:53.573535+0000","last_peered":"2024-05-17T09:19:53.573535+0000","last_clean":"2024-05-17T09:19:53.573535+0000","last_became_active":"2024-05-17T09:15:03.932966+0000","last_became_peered":"2024-05-17T09:15:03.932966+0000","last_unstale":"2024-05-17T09:19:53.573535+0000","last_undegraded":"2024-05-17T09:19:53.573535+0000","last_fullsized":"2024-05-17T09:19:53.573535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:03:43.171011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":443,"num_read_kb":290,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898727+0000","last_change":"2024-05-17T09:15:07.964623+0000","last_active":"2024-05-17T09:17:46.898727+0000","last_peered":"2024-05-17T09:17:46.898727+0000","last_clean":"2024-05-17T09:17:46.898727+0000","last_became_active":"2024-05-17T09:15:07.964274+0000","last_became_peered":"2024-05-17T09:15:07.964274+0000","last_unstale":"2024-05-17T09:17:46.898727+0000","last_undegraded":"2024-05-17T09:17:46.898727+0000","last_fullsized":"2024-05-17T09:17:46.898727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:28:52.319981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.423939+0000","last_change":"2024-05-17T09:15:20.035512+0000","last_active":"2024-05-17T09:16:48.423939+0000","last_peered":"2024-05-17T09:16:48.423939+0000","last_clean":"2024-05-17T09:16:48.423939+0000","last_became_active":"2024-05-17T09:15:20.034110+0000","last_became_peered":"2024-05-17T09:15:20.034110+0000","last_unstale":"2024-05-17T09:16:48.423939+0000","last_undegraded":"2024-05-17T09:16:48.423939+0000","last_fullsized":"2024-05-17T09:16:48.423939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:43:01.085430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321842+0000","last_change":"2024-05-17T09:15:48.164916+0000","last_active":"2024-05-17T09:16:48.321842+0000","last_peered":"2024-05-17T09:16:48.321842+0000","last_clean":"2024-05-17T09:16:48.321842+0000","last_became_active":"2024-05-17T09:15:48.155407+0000","last_became_peered":"2024-05-17T09:15:48.155407+0000","last_unstale":"2024-05-17T09:16:48.321842+0000","last_undegraded":"2024-05-17T09:16:48.321842+0000","last_fullsized":"2024-05-17T09:16:48.321842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:11:02.381053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903556+0000","last_change":"2024-05-17T09:16:48.596820+0000","last_active":"2024-05-17T09:17:46.903556+0000","last_peered":"2024-05-17T09:17:46.903556+0000","last_clean":"2024-05-17T09:17:46.903556+0000","last_became_active":"2024-05-17T09:16:48.596386+0000","last_became_peered":"2024-05-17T09:16:48.596386+0000","last_unstale":"2024-05-17T09:17:46.903556+0000","last_undegraded":"2024-05-17T09:17:46.903556+0000","last_fullsized":"2024-05-17T09:17:46.903556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:53:29.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.321910+0000","last_change":"2024-05-17T09:15:20.038505+0000","last_active":"2024-05-17T09:16:48.321910+0000","last_peered":"2024-05-17T09:16:48.321910+0000","last_clean":"2024-05-17T09:16:48.321910+0000","last_became_active":"2024-05-17T09:15:20.037952+0000","last_became_peered":"2024-05-17T09:15:20.037952+0000","last_unstale":"2024-05-17T09:16:48.321910+0000","last_undegraded":"2024-05-17T09:16:48.321910+0000","last_fullsized":"2024-05-17T09:16:48.321910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:28:47.052627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.769113+0000","last_change":"2024-05-17T09:15:03.933313+0000","last_active":"2024-05-17T09:17:26.769113+0000","last_peered":"2024-05-17T09:17:26.769113+0000","last_clean":"2024-05-17T09:17:26.769113+0000","last_became_active":"2024-05-17T09:15:03.932493+0000","last_became_peered":"2024-05-17T09:15:03.932493+0000","last_unstale":"2024-05-17T09:17:26.769113+0000","last_undegraded":"2024-05-17T09:17:26.769113+0000","last_fullsized":"2024-05-17T09:17:26.769113+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:03:43.505291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.905106+0000","last_change":"2024-05-17T09:15:07.961751+0000","last_active":"2024-05-17T09:17:46.905106+0000","last_peered":"2024-05-17T09:17:46.905106+0000","last_clean":"2024-05-17T09:17:46.905106+0000","last_became_active":"2024-05-17T09:15:07.960516+0000","last_became_peered":"2024-05-17T09:15:07.960516+0000","last_unstale":"2024-05-17T09:17:46.905106+0000","last_undegraded":"2024-05-17T09:17:46.905106+0000","last_fullsized":"2024-05-17T09:17:46.905106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:54:37.005035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.830021+0000","last_change":"2024-05-17T09:15:48.158350+0000","last_active":"2024-05-17T09:16:48.830021+0000","last_peered":"2024-05-17T09:16:48.830021+0000","last_clean":"2024-05-17T09:16:48.830021+0000","last_became_active":"2024-05-17T09:15:48.157861+0000","last_became_peered":"2024-05-17T09:15:48.157861+0000","last_unstale":"2024-05-17T09:16:48.830021+0000","last_undegraded":"2024-05-17T09:16:48.830021+0000","last_fullsized":"2024-05-17T09:16:48.830021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:25:29.264407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:49.567065+0000","last_change":"2024-05-17T09:15:03.934358+0000","last_active":"2024-05-17T09:19:49.567065+0000","last_peered":"2024-05-17T09:19:49.567065+0000","last_clean":"2024-05-17T09:19:49.567065+0000","last_became_active":"2024-05-17T09:15:03.933958+0000","last_became_peered":"2024-05-17T09:15:03.933958+0000","last_unstale":"2024-05-17T09:19:49.567065+0000","last_undegraded":"2024-05-17T09:19:49.567065+0000","last_fullsized":"2024-05-17T09:19:49.567065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:50.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":218,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901798+0000","last_change":"2024-05-17T09:15:07.965511+0000","last_active":"2024-05-17T09:17:46.901798+0000","last_peered":"2024-05-17T09:17:46.901798+0000","last_clean":"2024-05-17T09:17:46.901798+0000","last_became_active":"2024-05-17T09:15:07.963942+0000","last_became_peered":"2024-05-17T09:15:07.963942+0000","last_unstale":"2024-05-17T09:17:46.901798+0000","last_undegraded":"2024-05-17T09:17:46.901798+0000","last_fullsized":"2024-05-17T09:17:46.901798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:04:14.432556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.320642+0000","last_change":"2024-05-17T09:15:20.039421+0000","last_active":"2024-05-17T09:16:48.320642+0000","last_peered":"2024-05-17T09:16:48.320642+0000","last_clean":"2024-05-17T09:16:48.320642+0000","last_became_active":"2024-05-17T09:15:20.038947+0000","last_became_peered":"2024-05-17T09:15:20.038947+0000","last_unstale":"2024-05-17T09:16:48.320642+0000","last_undegraded":"2024-05-17T09:16:48.320642+0000","last_fullsized":"2024-05-17T09:16:48.320642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:41:17.104907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:23.938637+0000","last_change":"2024-05-17T09:15:04.127409+0000","last_active":"2024-05-17T09:17:23.938637+0000","last_peered":"2024-05-17T09:17:23.938637+0000","last_clean":"2024-05-17T09:17:23.938637+0000","last_became_active":"2024-05-17T09:15:04.127006+0000","last_became_peered":"2024-05-17T09:15:04.127006+0000","last_unstale":"2024-05-17T09:17:23.938637+0000","last_undegraded":"2024-05-17T09:17:23.938637+0000","last_fullsized":"2024-05-17T09:17:23.938637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:38:05.177150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908460+0000","last_change":"2024-05-17T09:15:07.958905+0000","last_active":"2024-05-17T09:17:46.908460+0000","last_peered":"2024-05-17T09:17:46.908460+0000","last_clean":"2024-05-17T09:17:46.908460+0000","last_became_active":"2024-05-17T09:15:07.958591+0000","last_became_peered":"2024-05-17T09:15:07.958591+0000","last_unstale":"2024-05-17T09:17:46.908460+0000","last_undegraded":"2024-05-17T09:17:46.908460+0000","last_fullsized":"2024-05-17T09:17:46.908460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:20:30.634392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769733+0000","last_change":"2024-05-17T09:15:20.041546+0000","last_active":"2024-05-17T09:17:31.769733+0000","last_peered":"2024-05-17T09:17:31.769733+0000","last_clean":"2024-05-17T09:17:31.769733+0000","last_became_active":"2024-05-17T09:15:20.040665+0000","last_became_peered":"2024-05-17T09:15:20.040665+0000","last_unstale":"2024-05-17T09:17:31.769733+0000","last_undegraded":"2024-05-17T09:17:31.769733+0000","last_fullsized":"2024-05-17T09:17:31.769733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:49:34.042420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.322195+0000","last_change":"2024-05-17T09:15:48.167594+0000","last_active":"2024-05-17T09:16:48.322195+0000","last_peered":"2024-05-17T09:16:48.322195+0000","last_clean":"2024-05-17T09:16:48.322195+0000","last_became_active":"2024-05-17T09:15:48.157609+0000","last_became_peered":"2024-05-17T09:15:48.157609+0000","last_unstale":"2024-05-17T09:16:48.322195+0000","last_undegraded":"2024-05-17T09:16:48.322195+0000","last_fullsized":"2024-05-17T09:16:48.322195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:29:50.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898756+0000","last_change":"2024-05-17T09:15:07.957789+0000","last_active":"2024-05-17T09:17:46.898756+0000","last_peered":"2024-05-17T09:17:46.898756+0000","last_clean":"2024-05-17T09:17:46.898756+0000","last_became_active":"2024-05-17T09:15:07.957411+0000","last_became_peered":"2024-05-17T09:15:07.957411+0000","last_unstale":"2024-05-17T09:17:46.898756+0000","last_undegraded":"2024-05-17T09:17:46.898756+0000","last_fullsized":"2024-05-17T09:17:46.898756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:15:25.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.939280+0000","last_change":"2024-05-17T09:15:20.036763+0000","last_active":"2024-05-17T09:17:28.939280+0000","last_peered":"2024-05-17T09:17:28.939280+0000","last_clean":"2024-05-17T09:17:28.939280+0000","last_became_active":"2024-05-17T09:15:20.036178+0000","last_became_peered":"2024-05-17T09:15:20.036178+0000","last_unstale":"2024-05-17T09:17:28.939280+0000","last_undegraded":"2024-05-17T09:17:28.939280+0000","last_fullsized":"2024-05-17T09:17:28.939280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:51.618566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.934940+0000","last_change":"2024-05-17T09:15:04.127921+0000","last_active":"2024-05-17T09:17:11.934940+0000","last_peered":"2024-05-17T09:17:11.934940+0000","last_clean":"2024-05-17T09:17:11.934940+0000","last_became_active":"2024-05-17T09:15:04.127560+0000","last_became_peered":"2024-05-17T09:15:04.127560+0000","last_unstale":"2024-05-17T09:17:11.934940+0000","last_undegraded":"2024-05-17T09:17:11.934940+0000","last_fullsized":"2024-05-17T09:17:11.934940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:44:25.378580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904550+0000","last_change":"2024-05-17T09:15:07.961172+0000","last_active":"2024-05-17T09:17:46.904550+0000","last_peered":"2024-05-17T09:17:46.904550+0000","last_clean":"2024-05-17T09:17:46.904550+0000","last_became_active":"2024-05-17T09:15:07.960807+0000","last_became_peered":"2024-05-17T09:15:07.960807+0000","last_unstale":"2024-05-17T09:17:46.904550+0000","last_undegraded":"2024-05-17T09:17:46.904550+0000","last_fullsized":"2024-05-17T09:17:46.904550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:07:07.866304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:27.771474+0000","last_change":"2024-05-17T09:15:20.040026+0000","last_active":"2024-05-17T09:17:27.771474+0000","last_peered":"2024-05-17T09:17:27.771474+0000","last_clean":"2024-05-17T09:17:27.771474+0000","last_became_active":"2024-05-17T09:15:20.039600+0000","last_became_peered":"2024-05-17T09:15:20.039600+0000","last_unstale":"2024-05-17T09:17:27.771474+0000","last_undegraded":"2024-05-17T09:17:27.771474+0000","last_fullsized":"2024-05-17T09:17:27.771474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:37:49.177277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.384811+0000","last_change":"2024-05-17T09:15:03.935026+0000","last_active":"2024-05-17T09:17:08.384811+0000","last_peered":"2024-05-17T09:17:08.384811+0000","last_clean":"2024-05-17T09:17:08.384811+0000","last_became_active":"2024-05-17T09:15:03.934620+0000","last_became_peered":"2024-05-17T09:15:03.934620+0000","last_unstale":"2024-05-17T09:17:08.384811+0000","last_undegraded":"2024-05-17T09:17:08.384811+0000","last_fullsized":"2024-05-17T09:17:08.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:23.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.900921+0000","last_change":"2024-05-17T09:15:07.962785+0000","last_active":"2024-05-17T09:17:46.900921+0000","last_peered":"2024-05-17T09:17:46.900921+0000","last_clean":"2024-05-17T09:17:46.900921+0000","last_became_active":"2024-05-17T09:15:07.961468+0000","last_became_peered":"2024-05-17T09:15:07.961468+0000","last_unstale":"2024-05-17T09:17:46.900921+0000","last_undegraded":"2024-05-17T09:17:46.900921+0000","last_fullsized":"2024-05-17T09:17:46.900921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:17:56.823137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:32.771965+0000","last_change":"2024-05-17T09:15:20.045574+0000","last_active":"2024-05-17T09:17:32.771965+0000","last_peered":"2024-05-17T09:17:32.771965+0000","last_clean":"2024-05-17T09:17:32.771965+0000","last_became_active":"2024-05-17T09:15:20.045147+0000","last_became_peered":"2024-05-17T09:15:20.045147+0000","last_unstale":"2024-05-17T09:17:32.771965+0000","last_undegraded":"2024-05-17T09:17:32.771965+0000","last_fullsized":"2024-05-17T09:17:32.771965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:58.136671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901492+0000","last_change":"2024-05-17T09:15:07.963700+0000","last_active":"2024-05-17T09:17:46.901492+0000","last_peered":"2024-05-17T09:17:46.901492+0000","last_clean":"2024-05-17T09:17:46.901492+0000","last_became_active":"2024-05-17T09:15:07.962749+0000","last_became_peered":"2024-05-17T09:15:07.962749+0000","last_unstale":"2024-05-17T09:17:46.901492+0000","last_undegraded":"2024-05-17T09:17:46.901492+0000","last_fullsized":"2024-05-17T09:17:46.901492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:48:42.957616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":265,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:51.569186+0000","last_change":"2024-05-17T09:15:04.127491+0000","last_active":"2024-05-17T09:19:51.569186+0000","last_peered":"2024-05-17T09:19:51.569186+0000","last_clean":"2024-05-17T09:19:51.569186+0000","last_became_active":"2024-05-17T09:15:04.127092+0000","last_became_peered":"2024-05-17T09:15:04.127092+0000","last_unstale":"2024-05-17T09:19:51.569186+0000","last_undegraded":"2024-05-17T09:19:51.569186+0000","last_fullsized":"2024-05-17T09:19:51.569186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:43:46.700140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:28.387326+0000","last_change":"2024-05-17T09:15:48.309568+0000","last_active":"2024-05-17T09:17:28.387326+0000","last_peered":"2024-05-17T09:17:28.387326+0000","last_clean":"2024-05-17T09:17:28.387326+0000","last_became_active":"2024-05-17T09:15:48.309183+0000","last_became_peered":"2024-05-17T09:15:48.309183+0000","last_unstale":"2024-05-17T09:17:28.387326+0000","last_undegraded":"2024-05-17T09:17:28.387326+0000","last_fullsized":"2024-05-17T09:17:28.387326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:38:11.150608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904968+0000","last_change":"2024-05-17T09:15:07.960483+0000","last_active":"2024-05-17T09:17:46.904968+0000","last_peered":"2024-05-17T09:17:46.904968+0000","last_clean":"2024-05-17T09:17:46.904968+0000","last_became_active":"2024-05-17T09:15:07.959960+0000","last_became_peered":"2024-05-17T09:15:07.959960+0000","last_unstale":"2024-05-17T09:17:46.904968+0000","last_undegraded":"2024-05-17T09:17:46.904968+0000","last_fullsized":"2024-05-17T09:17:46.904968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:11:04.559233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:11.411802+0000","last_change":"2024-05-17T09:15:03.932720+0000","last_active":"2024-05-17T09:17:11.411802+0000","last_peered":"2024-05-17T09:17:11.411802+0000","last_clean":"2024-05-17T09:17:11.411802+0000","last_became_active":"2024-05-17T09:15:03.932285+0000","last_became_peered":"2024-05-17T09:15:03.932285+0000","last_unstale":"2024-05-17T09:17:11.411802+0000","last_undegraded":"2024-05-17T09:17:11.411802+0000","last_fullsized":"2024-05-17T09:17:11.411802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:04:54.528072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:16:48.595216+0000","last_change":"2024-05-17T09:13:39.613728+0000","last_active":"2024-05-17T09:16:48.595216+0000","last_peered":"2024-05-17T09:16:48.595216+0000","last_clean":"2024-05-17T09:16:48.595216+0000","last_became_active":"2024-05-17T09:13:39.608383+0000","last_became_peered":"2024-05-17T09:13:39.608383+0000","last_unstale":"2024-05-17T09:16:48.595216+0000","last_undegraded":"2024-05-17T09:16:48.595216+0000","last_fullsized":"2024-05-17T09:16:48.595216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:10:11.153331+0000","last_clean_scrub_stamp":"2024-05-17T09:10:11.153331+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:34.506208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769780+0000","last_change":"2024-05-17T09:15:20.033999+0000","last_active":"2024-05-17T09:17:31.769780+0000","last_peered":"2024-05-17T09:17:31.769780+0000","last_clean":"2024-05-17T09:17:31.769780+0000","last_became_active":"2024-05-17T09:15:20.033285+0000","last_became_peered":"2024-05-17T09:15:20.033285+0000","last_unstale":"2024-05-17T09:17:31.769780+0000","last_undegraded":"2024-05-17T09:17:31.769780+0000","last_fullsized":"2024-05-17T09:17:31.769780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:02.871183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903562+0000","last_change":"2024-05-17T09:16:48.322492+0000","last_active":"2024-05-17T09:17:46.903562+0000","last_peered":"2024-05-17T09:17:46.903562+0000","last_clean":"2024-05-17T09:17:46.903562+0000","last_became_active":"2024-05-17T09:16:48.321995+0000","last_became_peered":"2024-05-17T09:16:48.321995+0000","last_unstale":"2024-05-17T09:17:46.903562+0000","last_undegraded":"2024-05-17T09:17:46.903562+0000","last_fullsized":"2024-05-17T09:17:46.903562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:03:34.388270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:03.384322+0000","last_change":"2024-05-17T09:15:48.403084+0000","last_active":"2024-05-17T09:17:03.384322+0000","last_peered":"2024-05-17T09:17:03.384322+0000","last_clean":"2024-05-17T09:17:03.384322+0000","last_became_active":"2024-05-17T09:15:48.402540+0000","last_became_peered":"2024-05-17T09:15:48.402540+0000","last_unstale":"2024-05-17T09:17:03.384322+0000","last_undegraded":"2024-05-17T09:17:03.384322+0000","last_fullsized":"2024-05-17T09:17:03.384322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:29.760485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:24.112162+0000","last_change":"2024-05-17T09:15:20.033179+0000","last_active":"2024-05-17T09:17:24.112162+0000","last_peered":"2024-05-17T09:17:24.112162+0000","last_clean":"2024-05-17T09:17:24.112162+0000","last_became_active":"2024-05-17T09:15:20.032591+0000","last_became_peered":"2024-05-17T09:15:20.032591+0000","last_unstale":"2024-05-17T09:17:24.112162+0000","last_undegraded":"2024-05-17T09:17:24.112162+0000","last_fullsized":"2024-05-17T09:17:24.112162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:09:51.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:08.936560+0000","last_change":"2024-05-17T09:15:04.128551+0000","last_active":"2024-05-17T09:17:08.936560+0000","last_peered":"2024-05-17T09:17:08.936560+0000","last_clean":"2024-05-17T09:17:08.936560+0000","last_became_active":"2024-05-17T09:15:04.128113+0000","last_became_peered":"2024-05-17T09:15:04.128113+0000","last_unstale":"2024-05-17T09:17:08.936560+0000","last_undegraded":"2024-05-17T09:17:08.936560+0000","last_fullsized":"2024-05-17T09:17:08.936560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:25:51.154374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.903852+0000","last_change":"2024-05-17T09:15:07.957957+0000","last_active":"2024-05-17T09:17:46.903852+0000","last_peered":"2024-05-17T09:17:46.903852+0000","last_clean":"2024-05-17T09:17:46.903852+0000","last_became_active":"2024-05-17T09:15:07.957480+0000","last_became_peered":"2024-05-17T09:15:07.957480+0000","last_unstale":"2024-05-17T09:17:46.903852+0000","last_undegraded":"2024-05-17T09:17:46.903852+0000","last_fullsized":"2024-05-17T09:17:46.903852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:40:25.937094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:20.633799+0000","last_change":"2024-05-17T09:15:20.039689+0000","last_active":"2024-05-17T09:17:20.633799+0000","last_peered":"2024-05-17T09:17:20.633799+0000","last_clean":"2024-05-17T09:17:20.633799+0000","last_became_active":"2024-05-17T09:15:20.039301+0000","last_became_peered":"2024-05-17T09:15:20.039301+0000","last_unstale":"2024-05-17T09:17:20.633799+0000","last_undegraded":"2024-05-17T09:17:20.633799+0000","last_fullsized":"2024-05-17T09:17:20.633799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:49:44.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.948913+0000","last_change":"2024-05-17T09:15:03.929892+0000","last_active":"2024-05-17T09:19:26.948913+0000","last_peered":"2024-05-17T09:19:26.948913+0000","last_clean":"2024-05-17T09:19:26.948913+0000","last_became_active":"2024-05-17T09:15:03.929467+0000","last_became_peered":"2024-05-17T09:15:03.929467+0000","last_unstale":"2024-05-17T09:19:26.948913+0000","last_undegraded":"2024-05-17T09:19:26.948913+0000","last_fullsized":"2024-05-17T09:19:26.948913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.889973+0000","last_change":"2024-05-17T09:15:07.965681+0000","last_active":"2024-05-17T09:17:46.889973+0000","last_peered":"2024-05-17T09:17:46.889973+0000","last_clean":"2024-05-17T09:17:46.889973+0000","last_became_active":"2024-05-17T09:15:07.965237+0000","last_became_peered":"2024-05-17T09:15:07.965237+0000","last_unstale":"2024-05-17T09:17:46.889973+0000","last_undegraded":"2024-05-17T09:17:46.889973+0000","last_fullsized":"2024-05-17T09:17:46.889973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:46:19.797845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:26.193083+0000","last_change":"2024-05-17T09:15:20.041174+0000","last_active":"2024-05-17T09:17:26.193083+0000","last_peered":"2024-05-17T09:17:26.193083+0000","last_clean":"2024-05-17T09:17:26.193083+0000","last_became_active":"2024-05-17T09:15:20.040705+0000","last_became_peered":"2024-05-17T09:15:20.040705+0000","last_unstale":"2024-05-17T09:17:26.193083+0000","last_undegraded":"2024-05-17T09:17:26.193083+0000","last_fullsized":"2024-05-17T09:17:26.193083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:54.808446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.412427+0000","last_change":"2024-05-17T09:15:04.061005+0000","last_active":"2024-05-17T09:17:16.412427+0000","last_peered":"2024-05-17T09:17:16.412427+0000","last_clean":"2024-05-17T09:17:16.412427+0000","last_became_active":"2024-05-17T09:15:04.060650+0000","last_became_peered":"2024-05-17T09:15:04.060650+0000","last_unstale":"2024-05-17T09:17:16.412427+0000","last_undegraded":"2024-05-17T09:17:16.412427+0000","last_fullsized":"2024-05-17T09:17:16.412427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:33:18.035262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898859+0000","last_change":"2024-05-17T09:15:07.960609+0000","last_active":"2024-05-17T09:17:46.898859+0000","last_peered":"2024-05-17T09:17:46.898859+0000","last_clean":"2024-05-17T09:17:46.898859+0000","last_became_active":"2024-05-17T09:15:07.959246+0000","last_became_peered":"2024-05-17T09:15:07.959246+0000","last_unstale":"2024-05-17T09:17:46.898859+0000","last_undegraded":"2024-05-17T09:17:46.898859+0000","last_fullsized":"2024-05-17T09:17:46.898859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:31:54.209231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:36.415183+0000","last_change":"2024-05-17T09:15:20.035325+0000","last_active":"2024-05-17T09:17:36.415183+0000","last_peered":"2024-05-17T09:17:36.415183+0000","last_clean":"2024-05-17T09:17:36.415183+0000","last_became_active":"2024-05-17T09:15:20.034572+0000","last_became_peered":"2024-05-17T09:15:20.034572+0000","last_unstale":"2024-05-17T09:17:36.415183+0000","last_undegraded":"2024-05-17T09:17:36.415183+0000","last_fullsized":"2024-05-17T09:17:36.415183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:50:09.583470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:16.908768+0000","last_change":"2024-05-17T09:15:04.128519+0000","last_active":"2024-05-17T09:17:16.908768+0000","last_peered":"2024-05-17T09:17:16.908768+0000","last_clean":"2024-05-17T09:17:16.908768+0000","last_became_active":"2024-05-17T09:15:04.128210+0000","last_became_peered":"2024-05-17T09:15:04.128210+0000","last_unstale":"2024-05-17T09:17:16.908768+0000","last_undegraded":"2024-05-17T09:17:16.908768+0000","last_fullsized":"2024-05-17T09:17:16.908768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:31:29.166544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.898286+0000","last_change":"2024-05-17T09:15:07.965901+0000","last_active":"2024-05-17T09:17:46.898286+0000","last_peered":"2024-05-17T09:17:46.898286+0000","last_clean":"2024-05-17T09:17:46.898286+0000","last_became_active":"2024-05-17T09:15:07.965526+0000","last_became_peered":"2024-05-17T09:15:07.965526+0000","last_unstale":"2024-05-17T09:17:46.898286+0000","last_undegraded":"2024-05-17T09:17:46.898286+0000","last_fullsized":"2024-05-17T09:17:46.898286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:08:50.322083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:36.937800+0000","last_change":"2024-05-17T09:15:20.034473+0000","last_active":"2024-05-17T09:17:36.937800+0000","last_peered":"2024-05-17T09:17:36.937800+0000","last_clean":"2024-05-17T09:17:36.937800+0000","last_became_active":"2024-05-17T09:15:20.033927+0000","last_became_peered":"2024-05-17T09:15:20.033927+0000","last_unstale":"2024-05-17T09:17:36.937800+0000","last_undegraded":"2024-05-17T09:17:36.937800+0000","last_fullsized":"2024-05-17T09:17:36.937800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:45:00.032396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:15.633296+0000","last_change":"2024-05-17T09:15:03.932393+0000","last_active":"2024-05-17T09:17:15.633296+0000","last_peered":"2024-05-17T09:17:15.633296+0000","last_clean":"2024-05-17T09:17:15.633296+0000","last_became_active":"2024-05-17T09:15:03.931905+0000","last_became_peered":"2024-05-17T09:15:03.931905+0000","last_unstale":"2024-05-17T09:17:15.633296+0000","last_undegraded":"2024-05-17T09:17:15.633296+0000","last_fullsized":"2024-05-17T09:17:15.633296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:28:57.025027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901878+0000","last_change":"2024-05-17T09:15:07.954273+0000","last_active":"2024-05-17T09:17:46.901878+0000","last_peered":"2024-05-17T09:17:46.901878+0000","last_clean":"2024-05-17T09:17:46.901878+0000","last_became_active":"2024-05-17T09:15:07.953917+0000","last_became_peered":"2024-05-17T09:15:07.953917+0000","last_unstale":"2024-05-17T09:17:46.901878+0000","last_undegraded":"2024-05-17T09:17:46.901878+0000","last_fullsized":"2024-05-17T09:17:46.901878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:40:53.557275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:36.415136+0000","last_change":"2024-05-17T09:15:20.034960+0000","last_active":"2024-05-17T09:17:36.415136+0000","last_peered":"2024-05-17T09:17:36.415136+0000","last_clean":"2024-05-17T09:17:36.415136+0000","last_became_active":"2024-05-17T09:15:20.034498+0000","last_became_peered":"2024-05-17T09:15:20.034498+0000","last_unstale":"2024-05-17T09:17:36.415136+0000","last_undegraded":"2024-05-17T09:17:36.415136+0000","last_fullsized":"2024-05-17T09:17:36.415136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T19:51:53.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:19:26.206872+0000","last_change":"2024-05-17T09:15:03.928063+0000","last_active":"2024-05-17T09:19:26.206872+0000","last_peered":"2024-05-17T09:19:26.206872+0000","last_clean":"2024-05-17T09:19:26.206872+0000","last_became_active":"2024-05-17T09:15:03.927644+0000","last_became_peered":"2024-05-17T09:15:03.927644+0000","last_unstale":"2024-05-17T09:19:26.206872+0000","last_undegraded":"2024-05-17T09:19:26.206872+0000","last_fullsized":"2024-05-17T09:19:26.206872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:11:50.011642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901144+0000","last_change":"2024-05-17T09:15:07.952416+0000","last_active":"2024-05-17T09:17:46.901144+0000","last_peered":"2024-05-17T09:17:46.901144+0000","last_clean":"2024-05-17T09:17:46.901144+0000","last_became_active":"2024-05-17T09:15:07.952090+0000","last_became_peered":"2024-05-17T09:15:07.952090+0000","last_unstale":"2024-05-17T09:17:46.901144+0000","last_undegraded":"2024-05-17T09:17:46.901144+0000","last_fullsized":"2024-05-17T09:17:46.901144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:58:34.012440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:36.415172+0000","last_change":"2024-05-17T09:15:20.033761+0000","last_active":"2024-05-17T09:17:36.415172+0000","last_peered":"2024-05-17T09:17:36.415172+0000","last_clean":"2024-05-17T09:17:36.415172+0000","last_became_active":"2024-05-17T09:15:20.033230+0000","last_became_peered":"2024-05-17T09:15:20.033230+0000","last_unstale":"2024-05-17T09:17:36.415172+0000","last_undegraded":"2024-05-17T09:17:36.415172+0000","last_fullsized":"2024-05-17T09:17:36.415172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:09:35.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:18.937946+0000","last_change":"2024-05-17T09:15:03.934327+0000","last_active":"2024-05-17T09:17:18.937946+0000","last_peered":"2024-05-17T09:17:18.937946+0000","last_clean":"2024-05-17T09:17:18.937946+0000","last_became_active":"2024-05-17T09:15:03.933918+0000","last_became_peered":"2024-05-17T09:15:03.933918+0000","last_unstale":"2024-05-17T09:17:18.937946+0000","last_undegraded":"2024-05-17T09:17:18.937946+0000","last_fullsized":"2024-05-17T09:17:18.937946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:49:32.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908869+0000","last_change":"2024-05-17T09:15:07.964740+0000","last_active":"2024-05-17T09:17:46.908869+0000","last_peered":"2024-05-17T09:17:46.908869+0000","last_clean":"2024-05-17T09:17:46.908869+0000","last_became_active":"2024-05-17T09:15:07.963974+0000","last_became_peered":"2024-05-17T09:15:07.963974+0000","last_unstale":"2024-05-17T09:17:46.908869+0000","last_undegraded":"2024-05-17T09:17:46.908869+0000","last_fullsized":"2024-05-17T09:17:46.908869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:59:50.881305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.910346+0000","last_change":"2024-05-17T09:15:20.037166+0000","last_active":"2024-05-17T09:17:31.910346+0000","last_peered":"2024-05-17T09:17:31.910346+0000","last_clean":"2024-05-17T09:17:31.910346+0000","last_became_active":"2024-05-17T09:15:20.034103+0000","last_became_peered":"2024-05-17T09:15:20.034103+0000","last_unstale":"2024-05-17T09:17:31.910346+0000","last_undegraded":"2024-05-17T09:17:31.910346+0000","last_fullsized":"2024-05-17T09:17:31.910346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:41.495698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:21.192445+0000","last_change":"2024-05-17T09:15:04.127677+0000","last_active":"2024-05-17T09:17:21.192445+0000","last_peered":"2024-05-17T09:17:21.192445+0000","last_clean":"2024-05-17T09:17:21.192445+0000","last_became_active":"2024-05-17T09:15:04.127151+0000","last_became_peered":"2024-05-17T09:15:04.127151+0000","last_unstale":"2024-05-17T09:17:21.192445+0000","last_undegraded":"2024-05-17T09:17:21.192445+0000","last_fullsized":"2024-05-17T09:17:21.192445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:34:03.700320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908474+0000","last_change":"2024-05-17T09:15:07.968663+0000","last_active":"2024-05-17T09:17:46.908474+0000","last_peered":"2024-05-17T09:17:46.908474+0000","last_clean":"2024-05-17T09:17:46.908474+0000","last_became_active":"2024-05-17T09:15:07.968271+0000","last_became_peered":"2024-05-17T09:15:07.968271+0000","last_unstale":"2024-05-17T09:17:46.908474+0000","last_undegraded":"2024-05-17T09:17:46.908474+0000","last_fullsized":"2024-05-17T09:17:46.908474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:16:13.433562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:38.432594+0000","last_change":"2024-05-17T09:15:20.046663+0000","last_active":"2024-05-17T09:17:38.432594+0000","last_peered":"2024-05-17T09:17:38.432594+0000","last_clean":"2024-05-17T09:17:38.432594+0000","last_became_active":"2024-05-17T09:15:20.046103+0000","last_became_peered":"2024-05-17T09:15:20.046103+0000","last_unstale":"2024-05-17T09:17:38.432594+0000","last_undegraded":"2024-05-17T09:17:38.432594+0000","last_fullsized":"2024-05-17T09:17:38.432594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:16:29.062344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:21.768494+0000","last_change":"2024-05-17T09:15:03.932919+0000","last_active":"2024-05-17T09:17:21.768494+0000","last_peered":"2024-05-17T09:17:21.768494+0000","last_clean":"2024-05-17T09:17:21.768494+0000","last_became_active":"2024-05-17T09:15:03.932529+0000","last_became_peered":"2024-05-17T09:15:03.932529+0000","last_unstale":"2024-05-17T09:17:21.768494+0000","last_undegraded":"2024-05-17T09:17:21.768494+0000","last_fullsized":"2024-05-17T09:17:21.768494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:42:51.460104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.901818+0000","last_change":"2024-05-17T09:15:07.967216+0000","last_active":"2024-05-17T09:17:46.901818+0000","last_peered":"2024-05-17T09:17:46.901818+0000","last_clean":"2024-05-17T09:17:46.901818+0000","last_became_active":"2024-05-17T09:15:07.966852+0000","last_became_peered":"2024-05-17T09:15:07.966852+0000","last_unstale":"2024-05-17T09:17:46.901818+0000","last_undegraded":"2024-05-17T09:17:46.901818+0000","last_fullsized":"2024-05-17T09:17:46.901818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T20:52:17.887717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:31.769697+0000","last_change":"2024-05-17T09:15:20.032265+0000","last_active":"2024-05-17T09:17:31.769697+0000","last_peered":"2024-05-17T09:17:31.769697+0000","last_clean":"2024-05-17T09:17:31.769697+0000","last_became_active":"2024-05-17T09:15:20.031794+0000","last_became_peered":"2024-05-17T09:15:20.031794+0000","last_unstale":"2024-05-17T09:17:31.769697+0000","last_undegraded":"2024-05-17T09:17:31.769697+0000","last_fullsized":"2024-05-17T09:17:31.769697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:19.004802+0000","last_clean_scrub_stamp":"2024-05-17T09:15:19.004802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T18:06:24.511417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:22.770777+0000","last_change":"2024-05-17T09:15:03.929797+0000","last_active":"2024-05-17T09:17:22.770777+0000","last_peered":"2024-05-17T09:17:22.770777+0000","last_clean":"2024-05-17T09:17:22.770777+0000","last_became_active":"2024-05-17T09:15:03.929413+0000","last_became_peered":"2024-05-17T09:15:03.929413+0000","last_unstale":"2024-05-17T09:17:22.770777+0000","last_undegraded":"2024-05-17T09:17:22.770777+0000","last_fullsized":"2024-05-17T09:17:22.770777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:02.911373+0000","last_clean_scrub_stamp":"2024-05-17T09:15:02.911373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:51:44.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.897999+0000","last_change":"2024-05-17T09:15:07.969020+0000","last_active":"2024-05-17T09:17:46.897999+0000","last_peered":"2024-05-17T09:17:46.897999+0000","last_clean":"2024-05-17T09:17:46.897999+0000","last_became_active":"2024-05-17T09:15:07.968614+0000","last_became_peered":"2024-05-17T09:15:07.968614+0000","last_unstale":"2024-05-17T09:17:46.897999+0000","last_undegraded":"2024-05-17T09:17:46.897999+0000","last_fullsized":"2024-05-17T09:17:46.897999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:40:22.543913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899078+0000","last_change":"2024-05-17T09:15:07.955936+0000","last_active":"2024-05-17T09:17:46.899078+0000","last_peered":"2024-05-17T09:17:46.899078+0000","last_clean":"2024-05-17T09:17:46.899078+0000","last_became_active":"2024-05-17T09:15:07.955597+0000","last_became_peered":"2024-05-17T09:15:07.955597+0000","last_unstale":"2024-05-17T09:17:46.899078+0000","last_undegraded":"2024-05-17T09:17:46.899078+0000","last_fullsized":"2024-05-17T09:17:46.899078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:01:27.628435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.888934+0000","last_change":"2024-05-17T09:15:07.965004+0000","last_active":"2024-05-17T09:17:46.888934+0000","last_peered":"2024-05-17T09:17:46.888934+0000","last_clean":"2024-05-17T09:17:46.888934+0000","last_became_active":"2024-05-17T09:15:07.964508+0000","last_became_peered":"2024-05-17T09:15:07.964508+0000","last_unstale":"2024-05-17T09:17:46.888934+0000","last_undegraded":"2024-05-17T09:17:46.888934+0000","last_fullsized":"2024-05-17T09:17:46.888934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:46:23.918278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.896944+0000","last_change":"2024-05-17T09:15:07.965441+0000","last_active":"2024-05-17T09:17:46.896944+0000","last_peered":"2024-05-17T09:17:46.896944+0000","last_clean":"2024-05-17T09:17:46.896944+0000","last_became_active":"2024-05-17T09:15:07.964885+0000","last_became_peered":"2024-05-17T09:15:07.964885+0000","last_unstale":"2024-05-17T09:17:46.896944+0000","last_undegraded":"2024-05-17T09:17:46.896944+0000","last_fullsized":"2024-05-17T09:17:46.896944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:28.226766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904847+0000","last_change":"2024-05-17T09:15:07.959599+0000","last_active":"2024-05-17T09:17:46.904847+0000","last_peered":"2024-05-17T09:17:46.904847+0000","last_clean":"2024-05-17T09:17:46.904847+0000","last_became_active":"2024-05-17T09:15:07.959089+0000","last_became_peered":"2024-05-17T09:15:07.959089+0000","last_unstale":"2024-05-17T09:17:46.904847+0000","last_undegraded":"2024-05-17T09:17:46.904847+0000","last_fullsized":"2024-05-17T09:17:46.904847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:39:25.111629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904438+0000","last_change":"2024-05-17T09:15:07.962206+0000","last_active":"2024-05-17T09:17:46.904438+0000","last_peered":"2024-05-17T09:17:46.904438+0000","last_clean":"2024-05-17T09:17:46.904438+0000","last_became_active":"2024-05-17T09:15:07.961858+0000","last_became_peered":"2024-05-17T09:15:07.961858+0000","last_unstale":"2024-05-17T09:17:46.904438+0000","last_undegraded":"2024-05-17T09:17:46.904438+0000","last_fullsized":"2024-05-17T09:17:46.904438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:01:28.068276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899186+0000","last_change":"2024-05-17T09:16:48.831533+0000","last_active":"2024-05-17T09:17:46.899186+0000","last_peered":"2024-05-17T09:17:46.899186+0000","last_clean":"2024-05-17T09:17:46.899186+0000","last_became_active":"2024-05-17T09:16:48.831024+0000","last_became_peered":"2024-05-17T09:16:48.831024+0000","last_unstale":"2024-05-17T09:17:46.899186+0000","last_undegraded":"2024-05-17T09:17:46.899186+0000","last_fullsized":"2024-05-17T09:17:46.899186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:23:50.105163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.891995+0000","last_change":"2024-05-17T09:15:07.962144+0000","last_active":"2024-05-17T09:17:46.891995+0000","last_peered":"2024-05-17T09:17:46.891995+0000","last_clean":"2024-05-17T09:17:46.891995+0000","last_became_active":"2024-05-17T09:15:07.961758+0000","last_became_peered":"2024-05-17T09:15:07.961758+0000","last_unstale":"2024-05-17T09:17:46.891995+0000","last_undegraded":"2024-05-17T09:17:46.891995+0000","last_fullsized":"2024-05-17T09:17:46.891995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:20:39.276395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.886503+0000","last_change":"2024-05-17T09:15:07.959962+0000","last_active":"2024-05-17T09:17:46.886503+0000","last_peered":"2024-05-17T09:17:46.886503+0000","last_clean":"2024-05-17T09:17:46.886503+0000","last_became_active":"2024-05-17T09:15:07.959600+0000","last_became_peered":"2024-05-17T09:15:07.959600+0000","last_unstale":"2024-05-17T09:17:46.886503+0000","last_undegraded":"2024-05-17T09:17:46.886503+0000","last_fullsized":"2024-05-17T09:17:46.886503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:57:00.790182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899965+0000","last_change":"2024-05-17T09:15:07.968835+0000","last_active":"2024-05-17T09:17:46.899965+0000","last_peered":"2024-05-17T09:17:46.899965+0000","last_clean":"2024-05-17T09:17:46.899965+0000","last_became_active":"2024-05-17T09:15:07.968524+0000","last_became_peered":"2024-05-17T09:15:07.968524+0000","last_unstale":"2024-05-17T09:17:46.899965+0000","last_undegraded":"2024-05-17T09:17:46.899965+0000","last_fullsized":"2024-05-17T09:17:46.899965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:09:44.153208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.894683+0000","last_change":"2024-05-17T09:15:07.965293+0000","last_active":"2024-05-17T09:17:46.894683+0000","last_peered":"2024-05-17T09:17:46.894683+0000","last_clean":"2024-05-17T09:17:46.894683+0000","last_became_active":"2024-05-17T09:15:07.964809+0000","last_became_peered":"2024-05-17T09:15:07.964809+0000","last_unstale":"2024-05-17T09:17:46.894683+0000","last_undegraded":"2024-05-17T09:17:46.894683+0000","last_fullsized":"2024-05-17T09:17:46.894683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:19.664937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.908531+0000","last_change":"2024-05-17T09:15:07.965524+0000","last_active":"2024-05-17T09:17:46.908531+0000","last_peered":"2024-05-17T09:17:46.908531+0000","last_clean":"2024-05-17T09:17:46.908531+0000","last_became_active":"2024-05-17T09:15:07.965183+0000","last_became_peered":"2024-05-17T09:15:07.965183+0000","last_unstale":"2024-05-17T09:17:46.908531+0000","last_undegraded":"2024-05-17T09:17:46.908531+0000","last_fullsized":"2024-05-17T09:17:46.908531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:47.623342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.904591+0000","last_change":"2024-05-17T09:15:07.966770+0000","last_active":"2024-05-17T09:17:46.904591+0000","last_peered":"2024-05-17T09:17:46.904591+0000","last_clean":"2024-05-17T09:17:46.904591+0000","last_became_active":"2024-05-17T09:15:07.966177+0000","last_became_peered":"2024-05-17T09:15:07.966177+0000","last_unstale":"2024-05-17T09:17:46.904591+0000","last_undegraded":"2024-05-17T09:17:46.904591+0000","last_fullsized":"2024-05-17T09:17:46.904591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:43:37.766383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T09:17:46.899578+0000","last_change":"2024-05-17T09:15:07.958302+0000","last_active":"2024-05-17T09:17:46.899578+0000","last_peered":"2024-05-17T09:17:46.899578+0000","last_clean":"2024-05-17T09:17:46.899578+0000","last_became_active":"2024-05-17T09:15:07.957500+0000","last_became_peered":"2024-05-17T09:15:07.957500+0000","last_unstale":"2024-05-17T09:17:46.899578+0000","last_undegraded":"2024-05-17T09:17:46.899578+0000","last_fullsized":"2024-05-17T09:17:46.899578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T09:15:06.928632+0000","last_clean_scrub_stamp":"2024-05-17T09:15:06.928632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T17:00:28.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246853,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1322,"num_read_kb":744,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":224556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710794,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":515149,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907022,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1176674,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070548,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":519425,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266777,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":515227,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463006,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1112817,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626532,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510505,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":579607,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738498,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":511893,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":581948,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182514,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510699,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986285,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":516180,"data_compressed":32396,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822761,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1103284,"data_compressed":68125,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34104,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35729,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T09:19:55.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:19:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:56.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:56 smithi019 ceph-mon[31620]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-17T09:19:56.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:56 smithi019 ceph-mon[31620]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:56.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:56 smithi069 ceph-mon[35081]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-17T09:19:56.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:56 smithi069 ceph-mon[35081]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:56 smithi079 ceph-mon[35502]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-17T09:19:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:56 smithi079 ceph-mon[35502]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T09:19:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/560783948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb581054e0 msgr2=0x7fdb581058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/560783948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb581054e0 0x7fdb581058f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b0a0 tx=0x7fdb4800b3b0 comp rx=0 tx=0).stop 2024-05-17T09:19:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/560783948 shutdown_connections 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/560783948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb5810e910 0x7fdb58110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/560783948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb581054e0 0x7fdb581058f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/560783948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb58105e30 0x7fdb5810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/560783948 >> 172.21.15.19:0/560783948 conn(0x7fdb58078df0 msgr2=0x7fdb58079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/560783948 shutdown_connections 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.946+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/560783948 wait complete. 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.947+0000 7fdb5ddb4700 1 Processor -- start 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.947+0000 7fdb5ddb4700 1 -- start start 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb5ddb4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb581054e0 0x7fdb581031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb5ddb4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 0x7fdb58103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb5ddb4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb5810e910 0x7fdb58103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb5ddb4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58114310 con 0x7fdb58105e30 2024-05-17T09:19:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb5ddb4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58114190 con 0x7fdb581054e0 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb5ddb4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58114490 con 0x7fdb5810e910 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 0x7fdb58103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 0x7fdb58103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46696/0 (socket says 172.21.15.19:46696) 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 -- 172.21.15.19:0/208764140 learned_addr learned my addr 172.21.15.19:0/208764140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb577fe700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb581054e0 0x7fdb581031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb57fff700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb5810e910 0x7fdb58103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 -- 172.21.15.19:0/208764140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb5810e910 msgr2=0x7fdb58103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb5810e910 0x7fdb58103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 -- 172.21.15.19:0/208764140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb581054e0 msgr2=0x7fdb581031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb581054e0 0x7fdb581031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 -- 172.21.15.19:0/208764140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb581a90b0 con 0x7fdb58105e30 2024-05-17T09:19:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb56ffd700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 0x7fdb58103710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c920 tx=0x7fdb4000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.948+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40004a30 con 0x7fdb58105e30 2024-05-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.949+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb581a9340 con 0x7fdb58105e30 2024-05-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.949+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb400054b0 con 0x7fdb58105e30 2024-05-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.949+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40007b70 con 0x7fdb58105e30 2024-05-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.949+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb581a98d0 con 0x7fdb58105e30 2024-05-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.950+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb580512b0 con 0x7fdb58105e30 2024-05-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.951+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdb40007d10 con 0x7fdb58105e30 2024-05-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.952+0000 7fdb54ff9700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb440b2880 0x7fdb440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.952+0000 7fdb577fe700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb440b2880 0x7fdb440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.952+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdb400e2520 con 0x7fdb58105e30 2024-05-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.953+0000 7fdb577fe700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb440b2880 0x7fdb440b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009e60 tx=0x7fdb4800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:19:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:57.956+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb40014030 con 0x7fdb58105e30 2024-05-17T09:19:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:58 smithi019 ceph-mon[31620]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T09:19:58.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:58 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:19:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.220+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb58104730 con 0x7fdb58105e30 2024-05-17T09:19:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.220+0000 7fdb54ff9700 1 -- 172.21.15.19:0/208764140 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fdb58104730 con 0x7fdb58105e30 2024-05-17T09:19:58.221 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:19:58.221 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb440b2880 msgr2=0x7fdb440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb440b2880 0x7fdb440b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009e60 tx=0x7fdb4800b7b0 comp rx=0 tx=0).stop 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 msgr2=0x7fdb58103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 0x7fdb58103710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c920 tx=0x7fdb4000cc30 comp rx=0 tx=0).stop 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 shutdown_connections 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb440b2880 0x7fdb440b4d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:58.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb5810e910 0x7fdb58103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb581054e0 0x7fdb581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 --2- 172.21.15.19:0/208764140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb58105e30 0x7fdb58103710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:19:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 >> 172.21.15.19:0/208764140 conn(0x7fdb58078df0 msgr2=0x7fdb581012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:19:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 shutdown_connections 2024-05-17T09:19:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:19:58.223+0000 7fdb5ddb4700 1 -- 172.21.15.19:0/208764140 wait complete. 2024-05-17T09:19:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:19:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:58 smithi069 ceph-mon[35081]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T09:19:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:58 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:19:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:58 smithi079 ceph-mon[35502]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T09:19:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:58 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:19:59.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:19:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:19:59.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:19:59 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/208764140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:59.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:19:59 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/208764140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:19:59.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:19:59 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/208764140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:00.331 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:00 smithi069 ceph-mon[35081]: pgmap v445: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-17T09:20:00.331 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:00 smithi069 ceph-mon[35081]: overall HEALTH_OK 2024-05-17T09:20:00.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:00 smithi019 ceph-mon[31620]: pgmap v445: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-17T09:20:00.333 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:00 smithi019 ceph-mon[31620]: overall HEALTH_OK 2024-05-17T09:20:00.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:00 smithi079 ceph-mon[35502]: pgmap v445: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-17T09:20:00.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:00 smithi079 ceph-mon[35502]: overall HEALTH_OK 2024-05-17T09:20:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 -- 172.21.15.19:0/1258673936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 msgr2=0x7fc92c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 --2- 172.21.15.19:0/1258673936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c1038c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0099c0 tx=0x7fc91c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 -- 172.21.15.19:0/1258673936 shutdown_connections 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 --2- 172.21.15.19:0/1258673936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 0x7fc92c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 --2- 172.21.15.19:0/1258673936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c1038c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 --2- 172.21.15.19:0/1258673936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc92c107de0 0x7fc92c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 -- 172.21.15.19:0/1258673936 >> 172.21.15.19:0/1258673936 conn(0x7fc92c0fede0 msgr2=0x7fc92c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 -- 172.21.15.19:0/1258673936 shutdown_connections 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.164+0000 7fc93224d700 1 -- 172.21.15.19:0/1258673936 wait complete. 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.165+0000 7fc93224d700 1 Processor -- start 2024-05-17T09:20:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.165+0000 7fc93224d700 1 -- start start 2024-05-17T09:20:01.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc93224d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c110870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:01.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc93224d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc92c107de0 0x7fc92c110db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:01.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc93224d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 0x7fc92c1112f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:01.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc93224d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c114370 con 0x7fc92c1034b0 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc93224d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c1141f0 con 0x7fc92c107de0 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc93224d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c1144f0 con 0x7fc92c108730 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c110870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46718/0 (socket says 172.21.15.19:46718) 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 -- 172.21.15.19:0/272518775 learned_addr learned my addr 172.21.15.19:0/272518775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc922ffd700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc92c107de0 0x7fc92c110db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92bfff700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 0x7fc92c1112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 -- 172.21.15.19:0/272518775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 msgr2=0x7fc92c1112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 0x7fc92c1112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 -- 172.21.15.19:0/272518775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc92c107de0 msgr2=0x7fc92c110db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc92c107de0 0x7fc92c110db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 -- 172.21.15.19:0/272518775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c1a90c0 con 0x7fc92c1034b0 2024-05-17T09:20:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92bfff700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 0x7fc92c1112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.166+0000 7fc92b7fe700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c110870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006010 tx=0x7fc91c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.167+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c026070 con 0x7fc92c1034b0 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.167+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c1a9350 con 0x7fc92c1034b0 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.167+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc91c00f460 con 0x7fc92c1034b0 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.167+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c009e60 con 0x7fc92c1034b0 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.167+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c1a9800 con 0x7fc92c1034b0 2024-05-17T09:20:01.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.168+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92c0512b0 con 0x7fc92c1034b0 2024-05-17T09:20:01.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.170+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc91c029070 con 0x7fc92c1034b0 2024-05-17T09:20:01.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.170+0000 7fc9297fa700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9180b2880 0x7fc9180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:01.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.171+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc91c0e3350 con 0x7fc92c1034b0 2024-05-17T09:20:01.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.171+0000 7fc922ffd700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9180b2880 0x7fc9180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:01.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.171+0000 7fc922ffd700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9180b2880 0x7fc9180b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc92c1068e0 tx=0x7fc9140073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:01.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.174+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc91c013070 con 0x7fc92c1034b0 2024-05-17T09:20:01.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.433+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc92c02e5e0 con 0x7fc92c1034b0 2024-05-17T09:20:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.434+0000 7fc9297fa700 1 -- 172.21.15.19:0/272518775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fc91c0aadf0 con 0x7fc92c1034b0 2024-05-17T09:20:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:01.435 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:20:01.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9180b2880 msgr2=0x7fc9180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:01.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9180b2880 0x7fc9180b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc92c1068e0 tx=0x7fc9140073d0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 msgr2=0x7fc92c110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:01.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c110870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006010 tx=0x7fc91c0045e0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 shutdown_connections 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9180b2880 0x7fc9180b4d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc92c108730 0x7fc92c1112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc92c1034b0 0x7fc92c110870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 --2- 172.21.15.19:0/272518775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc92c107de0 0x7fc92c110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 >> 172.21.15.19:0/272518775 conn(0x7fc92c0fede0 msgr2=0x7fc92c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 shutdown_connections 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:01.437+0000 7fc93224d700 1 -- 172.21.15.19:0/272518775 wait complete. 2024-05-17T09:20:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:20:02.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:02 smithi019 ceph-mon[31620]: pgmap v446: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-17T09:20:02.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/272518775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:02.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T09:20:02.412 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:02 smithi069 ceph-mon[35081]: pgmap v446: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-17T09:20:02.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/272518775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:02 smithi079 ceph-mon[35502]: pgmap v446: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-17T09:20:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/272518775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.251+0000 7fe44afa0700 1 -- 172.21.15.19:0/2441497003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 msgr2=0x7fe4441081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.251+0000 7fe44afa0700 1 --2- 172.21.15.19:0/2441497003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe4441081f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0099c0 tx=0x7fe42c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 -- 172.21.15.19:0/2441497003 shutdown_connections 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 --2- 172.21.15.19:0/2441497003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4441034b0 0x7fe4441038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 --2- 172.21.15.19:0/2441497003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe444108730 0x7fe444110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 --2- 172.21.15.19:0/2441497003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe4441081f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 -- 172.21.15.19:0/2441497003 >> 172.21.15.19:0/2441497003 conn(0x7fe4440fede0 msgr2=0x7fe444101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 -- 172.21.15.19:0/2441497003 shutdown_connections 2024-05-17T09:20:04.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 -- 172.21.15.19:0/2441497003 wait complete. 2024-05-17T09:20:04.254 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:04 smithi019 ceph-mon[31620]: pgmap v447: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.252+0000 7fe44afa0700 1 Processor -- start 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 -- start start 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4441034b0 0x7fe444110a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe444110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe444108730 0x7fe4441114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe444114360 con 0x7fe444107de0 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441141e0 con 0x7fe444108730 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44afa0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441144e0 con 0x7fe4441034b0 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe444110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe444110fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52446/0 (socket says 172.21.15.19:52446) 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 -- 172.21.15.19:0/597783131 learned_addr learned my addr 172.21.15.19:0/597783131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe448d3c700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4441034b0 0x7fe444110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe44953d700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe444108730 0x7fe4441114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 -- 172.21.15.19:0/597783131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4441034b0 msgr2=0x7fe444110a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4441034b0 0x7fe444110a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 -- 172.21.15.19:0/597783131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe444108730 msgr2=0x7fe4441114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.253+0000 7fe443fff700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe444108730 0x7fe4441114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe443fff700 1 -- 172.21.15.19:0/597783131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4441a9080 con 0x7fe444107de0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe443fff700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe444110fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00b580 tx=0x7fe42c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c027070 con 0x7fe444107de0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe42c022470 con 0x7fe444107de0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c017650 con 0x7fe444107de0 2024-05-17T09:20:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4441a9310 con 0x7fe444107de0 2024-05-17T09:20:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.254+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4441a9710 con 0x7fe444107de0 2024-05-17T09:20:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.256+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440512b0 con 0x7fe444107de0 2024-05-17T09:20:04.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.258+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe42c02a070 con 0x7fe444107de0 2024-05-17T09:20:04.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.259+0000 7fe441ffb700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4300b2900 0x7fe4300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:04.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.259+0000 7fe448d3c700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4300b2900 0x7fe4300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:04.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.259+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe42c0e3430 con 0x7fe444107de0 2024-05-17T09:20:04.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.260+0000 7fe448d3c700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4300b2900 0x7fe4300b4e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe4340060b0 tx=0x7fe43400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:04.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.269+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe42c013070 con 0x7fe444107de0 2024-05-17T09:20:04.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:04 smithi069 ceph-mon[35081]: pgmap v447: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.489+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe44402e5e0 con 0x7fe444107de0 2024-05-17T09:20:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.491+0000 7fe441ffb700 1 -- 172.21.15.19:0/597783131 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7fe42c0aadd0 con 0x7fe444107de0 2024-05-17T09:20:04.492 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:04.492 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:20:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4300b2900 msgr2=0x7fe4300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4300b2900 0x7fe4300b4e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe4340060b0 tx=0x7fe43400c040 comp rx=0 tx=0).stop 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 msgr2=0x7fe444110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe444110fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00b580 tx=0x7fe42c00f690 comp rx=0 tx=0).stop 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 shutdown_connections 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe4300b2900 0x7fe4300b4e10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4441034b0 0x7fe444110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe444108730 0x7fe4441114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 --2- 172.21.15.19:0/597783131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe444107de0 0x7fe444110fb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 >> 172.21.15.19:0/597783131 conn(0x7fe4440fede0 msgr2=0x7fe444101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 shutdown_connections 2024-05-17T09:20:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:04.494+0000 7fe44afa0700 1 -- 172.21.15.19:0/597783131 wait complete. 2024-05-17T09:20:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:04 smithi079 ceph-mon[35502]: pgmap v447: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:05.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:05 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/597783131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:20:05.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-17T09:20:05.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:05 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/597783131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:20:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:05.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:05 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/597783131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:20:06.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:06 smithi019 ceph-mon[31620]: pgmap v448: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:06.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:06 smithi069 ceph-mon[35081]: pgmap v448: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:06 smithi079 ceph-mon[35502]: pgmap v448: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:08.099 INFO:teuthology.orchestra.run.smithi019.stdout:{"run_dir":"/var/run/ceph"} 2024-05-17T09:20:08.100 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-17T09:20:08.100 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1072 2024-05-17T09:20:08.100 DEBUG:tasks.quiescer:Yielding 2024-05-17T09:20:08.101 INFO:teuthology.run_tasks:Running task workunit... 2024-05-17T09:20:08.116 INFO:tasks.workunit:Pulling workunits from ref 3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:20:08.117 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-17T09:20:08.117 INFO:tasks.workunit:timeout=3h 2024-05-17T09:20:08.117 INFO:tasks.workunit:cleanup=True 2024-05-17T09:20:08.117 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-17T09:20:08.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:20:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-17T09:20:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-17T09:20:08.136 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6bh/107d Inode: 1 Links: 2 2024-05-17T09:20:08.136 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T09:20:08.136 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-17T09:20:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-17 09:16:28.466915249 +0000 2024-05-17T09:20:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-17 09:16:28.466915249 +0000 2024-05-17T09:20:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-17 09:19:29.293005555 +0000 2024-05-17T09:20:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2024-05-17T09:20:08.137 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-17T09:20:08.137 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-17T09:20:08.211 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3790c0feaaca36c33bb1b31470733325d7d91e6a 2024-05-17T09:20:08.268 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-17T09:20:08.378 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:08.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:08 smithi069 ceph-mon[35081]: pgmap v449: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:08.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:08 smithi019 ceph-mon[31620]: pgmap v449: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:08 smithi079 ceph-mon[35502]: pgmap v449: 129 pgs: 129 active+clean; 651 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-17T09:20:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.682+0000 7f5716dd5640 1 -- 172.21.15.19:0/3610413765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 msgr2=0x7f571010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 --2- 172.21.15.19:0/3610413765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f571010aea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5708009ca0 tx=0x7f570802a8c0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 -- 172.21.15.19:0/3610413765 shutdown_connections 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 --2- 172.21.15.19:0/3610413765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f571010aea0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 --2- 172.21.15.19:0/3610413765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5710103d80 0x7f5710104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 --2- 172.21.15.19:0/3610413765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5710102b40 0x7f5710103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 -- 172.21.15.19:0/3610413765 >> 172.21.15.19:0/3610413765 conn(0x7f57100fdd50 msgr2=0x7f57101001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 -- 172.21.15.19:0/3610413765 shutdown_connections 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 -- 172.21.15.19:0/3610413765 wait complete. 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 Processor -- start 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.683+0000 7f5716dd5640 1 -- start start 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5716dd5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5710102b40 0x7f57100ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5716dd5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5710103d80 0x7f57100fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5716dd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f57100fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5716dd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f571010e550 con 0x7f5710108a80 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5716dd5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f571010e3d0 con 0x7f5710102b40 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5716dd5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f571010e6d0 con 0x7f5710103d80 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f57100fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f57100fe7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52466/0 (socket says 172.21.15.19:52466) 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 -- 172.21.15.19:0/295894676 learned_addr learned my addr 172.21.15.19:0/295894676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f570ffff640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5710103d80 0x7f57100fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5714b71640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5710102b40 0x7f57100ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 -- 172.21.15.19:0/295894676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5710103d80 msgr2=0x7f57100fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5710103d80 0x7f57100fe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 -- 172.21.15.19:0/295894676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5710102b40 msgr2=0x7f57100ffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5710102b40 0x7f57100ffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 -- 172.21.15.19:0/295894676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57100feef0 con 0x7f5710108a80 2024-05-17T09:20:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.684+0000 7f5715372640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f57100fe7c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57080098d0 tx=0x7f5708003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.685+0000 7f570dffb640 1 -- 172.21.15.19:0/295894676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708003d90 con 0x7f5710108a80 2024-05-17T09:20:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.685+0000 7f570dffb640 1 -- 172.21.15.19:0/295894676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57080044d0 con 0x7f5710108a80 2024-05-17T09:20:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.685+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101a4230 con 0x7f5710108a80 2024-05-17T09:20:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.685+0000 7f570dffb640 1 -- 172.21.15.19:0/295894676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708030d10 con 0x7f5710108a80 2024-05-17T09:20:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.685+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a4690 con 0x7f5710108a80 2024-05-17T09:20:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.687+0000 7f571dd63740 1 Processor -- start 2024-05-17T09:20:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f570dffb640 1 -- 172.21.15.19:0/295894676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5708004040 con 0x7f5710108a80 2024-05-17T09:20:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 -- start start 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af9e1b0 0x55c05b33f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b33f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b346820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c05b2b1c30 con 0x55c05af752f0 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c05b2a17d0 con 0x55c05af7d4d0 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f571dd63740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c05b2a5810 con 0x55c05af9e1b0 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b33f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b33f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52468/0 (socket says 172.21.15.19:52468) 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 -- 172.21.15.19:0/133010394 learned_addr learned my addr 172.21.15.19:0/133010394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f570ffff640 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af9e1b0 0x55c05b33f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f570ffff640 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b346820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 -- 172.21.15.19:0/133010394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af9e1b0 msgr2=0x55c05b33f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f570dffb640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f57000b28a0 0x7f57000b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af9e1b0 0x55c05b33f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 -- 172.21.15.19:0/133010394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 msgr2=0x55c05b346820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b346820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.689+0000 7f5714b71640 1 -- 172.21.15.19:0/133010394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05afb6210 con 0x55c05af752f0 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f570ffff640 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b346820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f5715372640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f57000b28a0 0x7f57000b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f570dffb640 1 -- 172.21.15.19:0/295894676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5708103aa0 con 0x7f5710108a80 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f5714b71640 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b33f920 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f570400bc70 tx=0x7f57040107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9bf083fb27c43d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f56faffd640 1 -- 172.21.15.19:0/133010394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704004220 con 0x55c05af752f0 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f56faffd640 1 -- 172.21.15.19:0/133010394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f570400be20 con 0x55c05af752f0 2024-05-17T09:20:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.690+0000 7f56faffd640 1 -- 172.21.15.19:0/133010394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704004b90 con 0x55c05af752f0 2024-05-17T09:20:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 -- 172.21.15.19:0/133010394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 msgr2=0x55c05b33f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f5715372640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f57000b28a0 0x7f57000b4db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f57080c7f80 tx=0x7f57080907a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b33f920 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f570400bc70 tx=0x7f57040107a0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 -- 172.21.15.19:0/133010394 shutdown_connections 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b346820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b33f920 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 --2- 172.21.15.19:0/133010394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af9e1b0 0x55c05b33f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 -- 172.21.15.19:0/133010394 >> 172.21.15.19:0/133010394 conn(0x55c05af92300 msgr2=0x55c05b33ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 -- 172.21.15.19:0/133010394 shutdown_connections 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 -- 172.21.15.19:0/133010394 wait complete. 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.691+0000 7f571dd63740 1 -- start start 2024-05-17T09:20:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 Processor -- start 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af752f0 0x55c05b348c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af7d4d0 0x55c05b3491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 0x55c05b350120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c05af87790 con 0x55c05af9e1b0 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c05b25d890 con 0x55c05af752f0 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c05b347270 con 0x55c05af7d4d0 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f571dd63740 10 client.0 _ll_register_callbacks cb 0x55c05b260d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f570ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 0x55c05b350120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f570ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 0x55c05b350120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52470/0 (socket says 172.21.15.19:52470) 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f570ffff640 1 -- 172.21.15.19:0/123706007 learned_addr learned my addr 172.21.15.19:0/123706007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.692+0000 7f570ffff640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af752f0 0x55c05b348c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f5714b71640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af7d4d0 0x55c05b3491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f570ffff640 1 -- 172.21.15.19:0/123706007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af7d4d0 msgr2=0x55c05b3491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f570ffff640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af7d4d0 0x55c05b3491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f570ffff640 1 -- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af752f0 msgr2=0x55c05b348c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f570ffff640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af752f0 0x55c05b348c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f570ffff640 1 -- 172.21.15.19:0/123706007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05b082620 con 0x55c05af9e1b0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f5714b71640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af7d4d0 0x55c05b3491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f570ffff640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 0x55c05b350120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012c50 tx=0x7f56fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbfdb3766a583f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f56effff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f56effff640 1 -- 172.21.15.19:0/123706007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc004030 con 0x55c05af9e1b0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f56effff640 1 -- 172.21.15.19:0/123706007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f56fc01d040 con 0x55c05af9e1b0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570400be20 con 0x55c05af9e1b0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f56effff640 1 -- 172.21.15.19:0/123706007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56fc029050 con 0x55c05af9e1b0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f571dd63740 10 client.15402 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.693+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c05b353090 con 0x55c05af9e1b0 2024-05-17T09:20:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f56effff640 1 -- 172.21.15.19:0/123706007 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f570400be20 con 0x55c05af9e1b0 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f56effff640 1 -- 172.21.15.19:0/123706007 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f56fc020070 con 0x55c05af9e1b0 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f571dd63740 10 client.15402 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f571dd63740 10 client.15402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f571dd63740 20 client.15402 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f571dd63740 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af5cd80 0x55c05b3567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f571dd63740 4 client.15402 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c05b2b3b80 con 0x55c05af5cd80 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.694+0000 7f5714b71640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af5cd80 0x55c05b3567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f5714b71640 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af5cd80 0x55c05b3567f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e051a3a29e592cb4 server_cookie=c61442ce200e1537 in_seq=0 out_seq=0 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f56effff640 10 client.15402 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f56effff640 1 -- 172.21.15.19:0/123706007 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c05b2b3b80 con 0x55c05af5cd80 2024-05-17T09:20:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f56effff640 10 client.15402 handle_command_reply: tid=0 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f571dd63740 1 client.15402 shutdown 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 msgr2=0x55c05b350120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.695+0000 7f571dd63740 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 0x55c05b350120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012c50 tx=0x7f56fc0046c0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 shutdown_connections 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af5cd80 0x55c05b3567f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af9e1b0 0x55c05b350120 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af7d4d0 0x55c05b3491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f56effff640 0 client.15402 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 --2- 172.21.15.19:0/123706007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af752f0 0x55c05b348c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 >> 172.21.15.19:0/123706007 conn(0x55c05af92300 msgr2=0x55c05b33dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 shutdown_connections 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 1 -- 172.21.15.19:0/123706007 wait complete. 2024-05-17T09:20:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.696+0000 7f571dd63740 20 client.15402 trim_cache size 0 max 16384 2024-05-17T09:20:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 Processor -- start 2024-05-17T09:20:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 -- start start 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af5cd80 0x55c05b35bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af752f0 0x55c05b35c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 0x55c05b3500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57101b14c0 con 0x55c05af7d4d0 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f57101aed90 con 0x55c05af5cd80 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f571dd63740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f57101acbe0 con 0x55c05af752f0 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f570ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 0x55c05b3500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f570ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 0x55c05b3500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52476/0 (socket says 172.21.15.19:52476) 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f570ffff640 1 -- 172.21.15.19:0/974585744 learned_addr learned my addr 172.21.15.19:0/974585744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.711+0000 7f5714b71640 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af752f0 0x55c05b35c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af5cd80 0x55c05b35bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 -- 172.21.15.19:0/974585744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af752f0 msgr2=0x55c05b35c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af752f0 0x55c05b35c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 -- 172.21.15.19:0/974585744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af5cd80 msgr2=0x55c05b35bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af5cd80 0x55c05b35bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 -- 172.21.15.19:0/974585744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05b356dc0 con 0x55c05af7d4d0 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f570ffff640 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 0x55c05b3500b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012860 tx=0x7f56fc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a043768cd932c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f56fa7fc640 1 -- 172.21.15.19:0/974585744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0120c0 con 0x55c05af7d4d0 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f56fa7fc640 1 -- 172.21.15.19:0/974585744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56fc0035f0 con 0x55c05af7d4d0 2024-05-17T09:20:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.712+0000 7f56fa7fc640 1 -- 172.21.15.19:0/974585744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0214d0 con 0x55c05af7d4d0 2024-05-17T09:20:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 -- 172.21.15.19:0/974585744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 msgr2=0x55c05b3500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 0x55c05b3500b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f56fc012860 tx=0x7f56fc005740 comp rx=0 tx=0).stop 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 -- 172.21.15.19:0/974585744 shutdown_connections 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af7d4d0 0x55c05b3500b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af752f0 0x55c05b35c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 --2- 172.21.15.19:0/974585744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af5cd80 0x55c05b35bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 -- 172.21.15.19:0/974585744 >> 172.21.15.19:0/974585744 conn(0x55c05af92300 msgr2=0x55c05b3427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 -- 172.21.15.19:0/974585744 shutdown_connections 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 -- 172.21.15.19:0/974585744 wait complete. 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 -- start start 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.713+0000 7f571dd63740 1 Processor -- start 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af5cd80 0x55c05b34fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b3501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b3bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ad250 con 0x55c05af752f0 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ae6c0 con 0x55c05af7d4d0 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ad920 con 0x55c05af5cd80 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f571dd63740 10 client.0 _ll_register_callbacks cb 0x7f56fc012890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b3501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b3501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52478/0 (socket says 172.21.15.19:52478) 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 -- 172.21.15.19:0/3844312761 learned_addr learned my addr 172.21.15.19:0/3844312761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f570ffff640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af5cd80 0x55c05b34fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f570ffff640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b3bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 -- 172.21.15.19:0/3844312761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af5cd80 msgr2=0x55c05b34fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af5cd80 0x55c05b34fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 -- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 msgr2=0x55c05b3bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.714+0000 7f5714b71640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b3bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f5714b71640 1 -- 172.21.15.19:0/3844312761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05b3bd710 con 0x55c05af752f0 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f570ffff640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af5cd80 0x55c05b34fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f5714b71640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b3501a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55c05b340780 tx=0x7f5704005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9342b4b933c58cb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f56effff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f56effff640 1 -- 172.21.15.19:0/3844312761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704030070 con 0x55c05af752f0 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f56effff640 1 -- 172.21.15.19:0/3844312761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5704004020 con 0x55c05af752f0 2024-05-17T09:20:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56fc0035f0 con 0x55c05af752f0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f56effff640 1 -- 172.21.15.19:0/3844312761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5704039050 con 0x55c05af752f0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f571dd63740 10 client.15408 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c05b3be9d0 con 0x55c05af752f0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.715+0000 7f56effff640 1 -- 172.21.15.19:0/3844312761 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56fc0035f0 con 0x55c05af752f0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f56effff640 1 -- 172.21.15.19:0/3844312761 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5704033020 con 0x55c05af752f0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f571dd63740 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f571dd63740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f571dd63740 20 client.15408 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f571dd63740 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af9e1b0 0x55c05b3c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f571dd63740 4 client.15408 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c05b2ad360 con 0x55c05af9e1b0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f570ffff640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af9e1b0 0x55c05b3c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f570ffff640 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af9e1b0 0x55c05b3c1190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a274d8c0c0c6121 server_cookie=91f9775dda33fc73 in_seq=0 out_seq=0 2024-05-17T09:20:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.716+0000 7f56effff640 10 client.15408 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.724+0000 7f56effff640 1 -- 172.21.15.19:0/3844312761 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c05b2ad360 con 0x55c05af9e1b0 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.724+0000 7f56effff640 10 client.15408 handle_command_reply: tid=0 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 client.15408 shutdown 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 msgr2=0x55c05b3501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b3501a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55c05b340780 tx=0x7f5704005650 comp rx=0 tx=0).stop 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 shutdown_connections 2024-05-17T09:20:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c05af9e1b0 0x55c05b3c1190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c05af7d4d0 0x55c05b3bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c05af752f0 0x55c05b3501a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f56effff640 0 client.15408 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 --2- 172.21.15.19:0/3844312761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c05af5cd80 0x55c05b34fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 >> 172.21.15.19:0/3844312761 conn(0x55c05af92300 msgr2=0x55c05b3431f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 shutdown_connections 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 1 -- 172.21.15.19:0/3844312761 wait complete. 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.725+0000 7f571dd63740 20 client.15408 trim_cache size 0 max 16384 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:20:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:20:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f57000b28a0 msgr2=0x7f57000b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f57000b28a0 0x7f57000b4db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f57080c7f80 tx=0x7f57080907a0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 msgr2=0x7f57100fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f57100fe7c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57080098d0 tx=0x7f5708003ba0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 shutdown_connections 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f57000b28a0 0x7f57000b4db0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5710108a80 0x7f57100fe7c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5710103d80 0x7f57100fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 --2- 172.21.15.19:0/295894676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5710102b40 0x7f57100ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 >> 172.21.15.19:0/295894676 conn(0x7f57100fdd50 msgr2=0x7f5710075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 shutdown_connections 2024-05-17T09:20:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:08.726+0000 7f5716dd5640 1 -- 172.21.15.19:0/295894676 wait complete. 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.121+0000 7ff02bbf5700 1 -- 172.21.15.19:0/2966830083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 msgr2=0x7ff024110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.121+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/2966830083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024110dc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009990 tx=0x7ff01c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 -- 172.21.15.19:0/2966830083 shutdown_connections 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/2966830083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff024105490 0x7ff0241058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/2966830083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024110dc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/2966830083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff024105de0 0x7ff02410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 -- 172.21.15.19:0/2966830083 >> 172.21.15.19:0/2966830083 conn(0x7ff0240fedb0 msgr2=0x7ff0241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 -- 172.21.15.19:0/2966830083 shutdown_connections 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 -- 172.21.15.19:0/2966830083 wait complete. 2024-05-17T09:20:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.122+0000 7ff02bbf5700 1 Processor -- start 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 -- start start 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff024105490 0x7ff024103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff024105de0 0x7ff0241036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0241143e0 con 0x7ff02410e900 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff024114260 con 0x7ff024105de0 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02bbf5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff024114560 con 0x7ff024105490 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024103c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52504/0 (socket says 172.21.15.19:52504) 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 -- 172.21.15.19:0/129187816 learned_addr learned my addr 172.21.15.19:0/129187816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff029991700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff024105490 0x7ff024103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff029190700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff024105de0 0x7ff0241036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 -- 172.21.15.19:0/129187816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff024105490 msgr2=0x7ff024103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff024105490 0x7ff024103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 -- 172.21.15.19:0/129187816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff024105de0 msgr2=0x7ff0241036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff024105de0 0x7ff0241036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.123+0000 7ff02a192700 1 -- 172.21.15.19:0/129187816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0241a9170 con 0x7ff02410e900 2024-05-17T09:20:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.124+0000 7ff02a192700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024103c00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005b40 tx=0x7ff01c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.124+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c027070 con 0x7ff02410e900 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.124+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff01c00fc00 con 0x7ff02410e900 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.124+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c022c40 con 0x7ff02410e900 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.124+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0241a9400 con 0x7ff02410e900 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.124+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0241a9890 con 0x7ff02410e900 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.125+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0240512b0 con 0x7ff02410e900 2024-05-17T09:20:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.127+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff01c02a070 con 0x7ff02410e900 2024-05-17T09:20:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.127+0000 7ff022ffd700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff00c0b2900 0x7ff00c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.127+0000 7ff029991700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff00c0b2900 0x7ff00c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.128+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff01c0e3500 con 0x7ff02410e900 2024-05-17T09:20:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.128+0000 7ff029991700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff00c0b2900 0x7ff00c0b4e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff0140060b0 tx=0x7ff01400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:10.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.132+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff01c013070 con 0x7ff02410e900 2024-05-17T09:20:10.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:10 smithi019 ceph-mon[31620]: pgmap v450: 129 pgs: 129 active+clean; 656 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-17T09:20:10.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.401+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff02402e7a0 con 0x7ff02410e900 2024-05-17T09:20:10.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.401+0000 7ff022ffd700 1 -- 172.21.15.19:0/129187816 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7ff01c0aaf00 con 0x7ff02410e900 2024-05-17T09:20:10.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:20:10.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff00c0b2900 msgr2=0x7ff00c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff00c0b2900 0x7ff00c0b4e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff0140060b0 tx=0x7ff01400c040 comp rx=0 tx=0).stop 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 msgr2=0x7ff024103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024103c00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005b40 tx=0x7ff01c0057c0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 shutdown_connections 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff00c0b2900 0x7ff00c0b4e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff024105490 0x7ff024103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02410e900 0x7ff024103c00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 --2- 172.21.15.19:0/129187816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff024105de0 0x7ff0241036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 >> 172.21.15.19:0/129187816 conn(0x7ff0240fedb0 msgr2=0x7ff0240ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 shutdown_connections 2024-05-17T09:20:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:10.404+0000 7ff02bbf5700 1 -- 172.21.15.19:0/129187816 wait complete. 2024-05-17T09:20:10.406 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:20:10.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:10 smithi069 ceph-mon[35081]: pgmap v450: 129 pgs: 129 active+clean; 656 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-17T09:20:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:10 smithi079 ceph-mon[35502]: pgmap v450: 129 pgs: 129 active+clean; 656 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-17T09:20:11.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T09:15:09.950009+0000', 'modified': '2024-05-17T09:19:29.696379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24487, 'mds_2': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6835/437548660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 437548660}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 437548660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/1132484996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1132484996}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 1132484996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/720193094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 720193094}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 720193094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.69:6837/1860504493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 1860504493}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 1860504493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.79:6839/2520420584', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2520420584}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2520420584}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.69:6839/814844032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 814844032}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 814844032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24470, 24487, 24497]} 2024-05-17T09:20:11.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T09:20:11.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-17T09:20:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:11.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:11 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/129187816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:11.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:11 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/129187816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:11.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:11 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/129187816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:12.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:12 smithi069 ceph-mon[35081]: pgmap v451: 129 pgs: 129 active+clean; 656 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-17T09:20:12.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:12 smithi019 ceph-mon[31620]: pgmap v451: 129 pgs: 129 active+clean; 656 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-17T09:20:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:12 smithi079 ceph-mon[35502]: pgmap v451: 129 pgs: 129 active+clean; 656 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.091+0000 7f37a2583700 1 -- 172.21.15.19:0/3099737618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 msgr2=0x7f379c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.091+0000 7f37a2583700 1 --2- 172.21.15.19:0/3099737618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c1081f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f37840099c0 tx=0x7f3784009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 -- 172.21.15.19:0/3099737618 shutdown_connections 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 --2- 172.21.15.19:0/3099737618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f379c108730 0x7f379c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 --2- 172.21.15.19:0/3099737618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f379c1034b0 0x7f379c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 --2- 172.21.15.19:0/3099737618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c1081f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 -- 172.21.15.19:0/3099737618 >> 172.21.15.19:0/3099737618 conn(0x7f379c0fedb0 msgr2=0x7f379c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 -- 172.21.15.19:0/3099737618 shutdown_connections 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 -- 172.21.15.19:0/3099737618 wait complete. 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 Processor -- start 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.092+0000 7f37a2583700 1 -- start start 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a2583700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f379c1034b0 0x7f379c0ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a2583700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c0ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a2583700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f379c108730 0x7f379c0ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a2583700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1131f0 con 0x7f379c107de0 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a2583700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f379c113070 con 0x7f379c1034b0 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a2583700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f379c113370 con 0x7f379c108730 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c0ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c0ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52518/0 (socket says 172.21.15.19:52518) 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 -- 172.21.15.19:0/552539573 learned_addr learned my addr 172.21.15.19:0/552539573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f37a0b20700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f379c108730 0x7f379c0ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f379bfff700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f379c1034b0 0x7f379c0ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 -- 172.21.15.19:0/552539573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f379c108730 msgr2=0x7f379c0ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f379c108730 0x7f379c0ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 -- 172.21.15.19:0/552539573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f379c1034b0 msgr2=0x7f379c0ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f379c1034b0 0x7f379c0ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 -- 172.21.15.19:0/552539573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1a9090 con 0x7f379c107de0 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.093+0000 7f3793fff700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c0ff7d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f37840045e0 tx=0x7f3784004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.094+0000 7f3799ffb700 1 -- 172.21.15.19:0/552539573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784026070 con 0x7f379c107de0 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.094+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1a9320 con 0x7f379c107de0 2024-05-17T09:20:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.094+0000 7f3799ffb700 1 -- 172.21.15.19:0/552539573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f378400f460 con 0x7f379c107de0 2024-05-17T09:20:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.094+0000 7f3799ffb700 1 -- 172.21.15.19:0/552539573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784009e60 con 0x7f379c107de0 2024-05-17T09:20:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.094+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1a97d0 con 0x7f379c107de0 2024-05-17T09:20:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.098+0000 7f3799ffb700 1 -- 172.21.15.19:0/552539573 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3784029070 con 0x7f379c107de0 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.098+0000 7f37b0f98b80 1 Processor -- start 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.098+0000 7f37b0f98b80 1 -- start start 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f3799ffb700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f37880b2900 0x7f37880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 0x5594305c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37a0b20700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f37880b2900 0x7f37880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559430443df0 0x5594305c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5594304f0d20 con 0x5594305c2c80 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f3799ffb700 1 -- 172.21.15.19:0/552539573 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f37840e2d30 con 0x7f379c107de0 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5594304e08c0 con 0x559430443df0 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5594304e4900 con 0x55943052f3e0 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 0x5594305c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 0x5594305c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58234/0 (socket says 172.21.15.19:58234) 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 -- 172.21.15.19:0/4268759184 learned_addr learned my addr 172.21.15.19:0/4268759184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f37a0b20700 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f3793fff700 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559430443df0 0x5594305c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 -- 172.21.15.19:0/4268759184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559430443df0 msgr2=0x5594305c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559430443df0 0x5594305c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 -- 172.21.15.19:0/4268759184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 msgr2=0x5594305c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.099+0000 7f379bfff700 1 -- 172.21.15.19:0/4268759184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559430489f60 con 0x55943052f3e0 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.100+0000 7f37a0b20700 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.100+0000 7f37a0b20700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f37880b2900 0x7f37880b4e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f379400f870 tx=0x7f379400fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.100+0000 7f379bfff700 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 0x5594305c88c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f378c007cf0 tx=0x7f378c0103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a48d0455bc23c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.100+0000 7f37927fc700 1 -- 172.21.15.19:0/4268759184 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c010d10 con 0x55943052f3e0 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.100+0000 7f37927fc700 1 -- 172.21.15.19:0/4268759184 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f378c0118d0 con 0x55943052f3e0 2024-05-17T09:20:13.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.100+0000 7f37927fc700 1 -- 172.21.15.19:0/4268759184 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c019630 con 0x55943052f3e0 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 -- 172.21.15.19:0/4268759184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 msgr2=0x5594305c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 0x5594305c88c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f378c007cf0 tx=0x7f378c0103d0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 -- 172.21.15.19:0/4268759184 shutdown_connections 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55943052f3e0 0x5594305c88c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/4268759184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559430443df0 0x5594305c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 -- 172.21.15.19:0/4268759184 >> 172.21.15.19:0/4268759184 conn(0x5594304a4170 msgr2=0x5594305bd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 -- 172.21.15.19:0/4268759184 shutdown_connections 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.101+0000 7f37b0f98b80 1 -- 172.21.15.19:0/4268759184 wait complete. 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 -- start start 2024-05-17T09:20:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 Processor -- start 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 0x5594305cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594305c2c80 0x5594305ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5594303e6f50 con 0x55943052f3e0 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559430448d50 con 0x5594305c2c80 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5594305cf550 con 0x559430443df0 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f3793fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 0x5594305cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37b0f98b80 10 client.0 _ll_register_callbacks cb 0x55943047b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f3793fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 0x5594305cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52524/0 (socket says 172.21.15.19:52524) 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f3793fff700 1 -- 172.21.15.19:0/3241931456 learned_addr learned my addr 172.21.15.19:0/3241931456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.102+0000 7f37a0b20700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594305c2c80 0x5594305ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f379bfff700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37a0b20700 1 -- 172.21.15.19:0/3241931456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 msgr2=0x5594305cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37a0b20700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37a0b20700 1 -- 172.21.15.19:0/3241931456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 msgr2=0x5594305cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37a0b20700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 0x5594305cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37a0b20700 1 -- 172.21.15.19:0/3241931456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559430430880 con 0x5594305c2c80 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f3793fff700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 0x5594305cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37a0b20700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594305c2c80 0x5594305ccfd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3794018640 tx=0x7f3794004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d704d799a5679c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f377b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f377b7fe700 1 -- 172.21.15.19:0/3241931456 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794028030 con 0x5594305c2c80 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f378c0118d0 con 0x5594305c2c80 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f377b7fe700 1 -- 172.21.15.19:0/3241931456 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3794003a00 con 0x5594305c2c80 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.103+0000 7f377b7fe700 1 -- 172.21.15.19:0/3241931456 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3794032050 con 0x5594305c2c80 2024-05-17T09:20:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 10 client.25178 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55943049a710 con 0x5594305c2c80 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f377b7fe700 1 -- 172.21.15.19:0/3241931456 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f378c0118d0 con 0x5594305c2c80 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f377b7fe700 1 -- 172.21.15.19:0/3241931456 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f37940041e0 con 0x5594305c2c80 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 10 client.25178 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 10 client.25178 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 20 client.25178 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5594305dc440 0x5594305dd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 4 client.25178 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55943058ee80 con 0x5594305dc440 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.104+0000 7f379bfff700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5594305dc440 0x5594305dd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.105+0000 7f377b7fe700 10 client.25178 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.105+0000 7f379bfff700 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5594305dc440 0x5594305dd8b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=616b7d962a5779f server_cookie=9719013278d996a1 in_seq=0 out_seq=0 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f377b7fe700 1 -- 172.21.15.19:0/3241931456 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55943058ee80 con 0x5594305dc440 2024-05-17T09:20:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f377b7fe700 10 client.25178 handle_command_reply: tid=0 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 client.25178 shutdown 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594305c2c80 msgr2=0x5594305ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594305c2c80 0x5594305ccfd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3794018640 tx=0x7f3794004b40 comp rx=0 tx=0).stop 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 shutdown_connections 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5594305dc440 0x5594305dd8b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594305c2c80 0x5594305ccfd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/3241931456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55943052f3e0 0x5594305cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f377b7fe700 0 client.25178 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 >> 172.21.15.19:0/3241931456 conn(0x5594304a4170 msgr2=0x5594305bd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 shutdown_connections 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 1 -- 172.21.15.19:0/3241931456 wait complete. 2024-05-17T09:20:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.106+0000 7f37b0f98b80 20 client.25178 trim_cache size 0 max 16384 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.127+0000 7f37b0f98b80 1 Processor -- start 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 -- start start 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5594305dbbf0 con 0x5594305c2c80 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5594305dc2c0 con 0x55943052f3e0 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5594305dc990 con 0x559430443df0 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52534/0 (socket says 172.21.15.19:52534) 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 -- 172.21.15.19:0/2267433355 learned_addr learned my addr 172.21.15.19:0/2267433355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f3793fff700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f379bfff700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 -- 172.21.15.19:0/2267433355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 msgr2=0x5594305b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 -- 172.21.15.19:0/2267433355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 msgr2=0x5594305ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f37a0b20700 1 -- 172.21.15.19:0/2267433355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594305cd560 con 0x5594305c2c80 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f379bfff700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.128+0000 7f3793fff700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.129+0000 7f37a0b20700 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d87b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3794004650 tx=0x7f37940186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf92d8542528667 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.129+0000 7f3791ffb700 1 -- 172.21.15.19:0/2267433355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379403a070 con 0x5594305c2c80 2024-05-17T09:20:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.129+0000 7f3791ffb700 1 -- 172.21.15.19:0/2267433355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794008c80 con 0x5594305c2c80 2024-05-17T09:20:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.129+0000 7f3791ffb700 1 -- 172.21.15.19:0/2267433355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794028040 con 0x5594305c2c80 2024-05-17T09:20:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2267433355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 msgr2=0x5594305d87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d87b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3794004650 tx=0x7f37940186f0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2267433355 shutdown_connections 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d87b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2267433355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2267433355 >> 172.21.15.19:0/2267433355 conn(0x5594304a4170 msgr2=0x5594305cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2267433355 shutdown_connections 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2267433355 wait complete. 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 -- start start 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.130+0000 7f37b0f98b80 1 Processor -- start 2024-05-17T09:20:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5594305dd730 con 0x5594305c2c80 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5594305dd060 con 0x55943052f3e0 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5594305dde00 con 0x559430443df0 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37b0f98b80 10 client.0 _ll_register_callbacks cb 0x55943047b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37a0b20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37a0b20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52548/0 (socket says 172.21.15.19:52548) 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37a0b20700 1 -- 172.21.15.19:0/2645450754 learned_addr learned my addr 172.21.15.19:0/2645450754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f3793fff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 -- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 msgr2=0x5594305ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 -- 172.21.15.19:0/2645450754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 msgr2=0x5594305d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 -- 172.21.15.19:0/2645450754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594305cd340 con 0x559430443df0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f37a0b20700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f379bfff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cdfd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5594305cbdd0 tx=0x7f378c005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74bd249908689f37 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.131+0000 7f377b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f377b7fe700 1 -- 172.21.15.19:0/2645450754 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c01f040 con 0x559430443df0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3794008c80 con 0x559430443df0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f377b7fe700 1 -- 172.21.15.19:0/2645450754 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f378c010720 con 0x559430443df0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f377b7fe700 1 -- 172.21.15.19:0/2645450754 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f378c038050 con 0x559430443df0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 10 client.25198 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559430647cf0 con 0x559430443df0 2024-05-17T09:20:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f377b7fe700 1 -- 172.21.15.19:0/2645450754 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794008c80 con 0x559430443df0 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f377b7fe700 1 -- 172.21.15.19:0/2645450754 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f378c005040 con 0x559430443df0 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 10 client.25198 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 10 client.25198 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 20 client.25198 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x559430649db0 0x55943064b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 4 client.25198 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.132+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "status"}) -- 0x5594304f2c70 con 0x559430649db0 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f3793fff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x559430649db0 0x55943064b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f3793fff700 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x559430649db0 0x55943064b220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c032d71592ce7c1 server_cookie=b0c7e17362280869 in_seq=0 out_seq=0 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f377b7fe700 10 client.25198 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f377b7fe700 1 -- 172.21.15.19:0/2645450754 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5594304f2c70 con 0x559430649db0 2024-05-17T09:20:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f377b7fe700 10 client.25198 handle_command_reply: tid=0 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 client.25198 shutdown 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 msgr2=0x5594305cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cdfd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5594305cbdd0 tx=0x7f378c005970 comp rx=0 tx=0).stop 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 shutdown_connections 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x559430649db0 0x55943064b220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559430443df0 0x5594305cdfd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594305c2c80 0x5594305d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 --2- 172.21.15.19:0/2645450754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55943052f3e0 0x5594305ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f377b7fe700 0 client.25198 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.133+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 >> 172.21.15.19:0/2645450754 conn(0x5594304a4170 msgr2=0x5594305c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 shutdown_connections 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37b0f98b80 1 -- 172.21.15.19:0/2645450754 wait complete. 2024-05-17T09:20:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37b0f98b80 20 client.25198 trim_cache size 0 max 16384 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24470, 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 235.92400311200001, 2024-05-17T09:20:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 333.70228872299998 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f37880b2900 msgr2=0x7f37880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f37880b2900 0x7f37880b4e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f379400f870 tx=0x7f379400fc30 comp rx=0 tx=0).stop 2024-05-17T09:20:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 msgr2=0x7f379c0ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c0ff7d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f37840045e0 tx=0x7f3784004610 comp rx=0 tx=0).stop 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 shutdown_connections 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f37880b2900 0x7f37880b4e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f379c108730 0x7f379c0ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f379c1034b0 0x7f379c0ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 --2- 172.21.15.19:0/552539573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c107de0 0x7f379c0ff7d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 >> 172.21.15.19:0/552539573 conn(0x7f379c0fedb0 msgr2=0x7f379c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.134+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 shutdown_connections 2024-05-17T09:20:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:13.135+0000 7f37a2583700 1 -- 172.21.15.19:0/552539573 wait complete. 2024-05-17T09:20:13.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T09:20:14.150 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:14.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:14 smithi019 ceph-mon[31620]: pgmap v452: 129 pgs: 129 active+clean; 656 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-17T09:20:14.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:20:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:14 smithi069 ceph-mon[35081]: pgmap v452: 129 pgs: 129 active+clean; 656 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-17T09:20:14.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:20:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:14 smithi079 ceph-mon[35502]: pgmap v452: 129 pgs: 129 active+clean; 656 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-17T09:20:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:20:15.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:20:15.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:20:15.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:20:15.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:20:15.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:20:15.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:20:15.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:20:15.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:20:15.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 -- 172.21.15.19:0/3561072005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee881045c0 msgr2=0x7fee881049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 --2- 172.21.15.19:0/3561072005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee881045c0 0x7fee881049d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee780099c0 tx=0x7fee78009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 -- 172.21.15.19:0/3561072005 shutdown_connections 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 --2- 172.21.15.19:0/3561072005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee88109840 0x7fee881111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 --2- 172.21.15.19:0/3561072005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee881045c0 0x7fee881049d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 --2- 172.21.15.19:0/3561072005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee88108ef0 0x7fee88109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 -- 172.21.15.19:0/3561072005 >> 172.21.15.19:0/3561072005 conn(0x7fee880ffe90 msgr2=0x7fee881022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:16.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 -- 172.21.15.19:0/3561072005 shutdown_connections 2024-05-17T09:20:16.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.071+0000 7fee8cab4700 1 -- 172.21.15.19:0/3561072005 wait complete. 2024-05-17T09:20:16.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 Processor -- start 2024-05-17T09:20:16.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 -- start start 2024-05-17T09:20:16.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee881045c0 0x7fee881a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 0x7fee881a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee88109840 0x7fee8819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee88114320 con 0x7fee88108ef0 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee881141a0 con 0x7fee881045c0 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee8cab4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fee881144a0 con 0x7fee88109840 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 0x7fee881a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 0x7fee881a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60240/0 (socket says 172.21.15.19:60240) 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.072+0000 7fee85d9b700 1 -- 172.21.15.19:0/2403599989 learned_addr learned my addr 172.21.15.19:0/2403599989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee8659c700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee881045c0 0x7fee881a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee86d9d700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee88109840 0x7fee8819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee85d9b700 1 -- 172.21.15.19:0/2403599989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee88109840 msgr2=0x7fee8819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee88109840 0x7fee8819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee85d9b700 1 -- 172.21.15.19:0/2403599989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee881045c0 msgr2=0x7fee881a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee881045c0 0x7fee881a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee85d9b700 1 -- 172.21.15.19:0/2403599989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8819b270 con 0x7fee88108ef0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 0x7fee881a0ef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e970 tx=0x7fee7c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee777fe700 1 -- 172.21.15.19:0/2403599989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00cbd0 con 0x7fee88108ef0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee777fe700 1 -- 172.21.15.19:0/2403599989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee7c00cd70 con 0x7fee88108ef0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8819b560 con 0x7fee88108ef0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee777fe700 1 -- 172.21.15.19:0/2403599989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c005570 con 0x7fee88108ef0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.073+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881a7760 con 0x7fee88108ef0 2024-05-17T09:20:16.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.076+0000 7fee777fe700 1 -- 172.21.15.19:0/2403599989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fee7c004750 con 0x7fee88108ef0 2024-05-17T09:20:16.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.077+0000 7fee777fe700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee700b2900 0x7fee700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.077+0000 7fee8659c700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee700b2900 0x7fee700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.077+0000 7fee777fe700 1 -- 172.21.15.19:0/2403599989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fee7c0e27a0 con 0x7fee88108ef0 2024-05-17T09:20:16.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 Processor -- start 2024-05-17T09:20:16.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 -- start start 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee8659c700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee700b2900 0x7fee700b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051dd5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dcf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560051cf8d20 con 0x560051d373e0 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560051ce88c0 con 0x560051dcf090 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560051cec900 con 0x560051c4bdf0 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee8659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee8659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60242/0 (socket says 172.21.15.19:60242) 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee8659c700 1 -- 172.21.15.19:0/1549459697 learned_addr learned my addr 172.21.15.19:0/1549459697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.078+0000 7fee86d9d700 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dcf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee85d9b700 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051dd5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee8659c700 1 -- 172.21.15.19:0/1549459697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 msgr2=0x560051dd5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee8659c700 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051dd5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee8659c700 1 -- 172.21.15.19:0/1549459697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 msgr2=0x560051dcf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee8659c700 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dcf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee8659c700 1 -- 172.21.15.19:0/1549459697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560051c91f60 con 0x560051d373e0 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee8659c700 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd4cd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fee78005b20 tx=0x7fee78009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1069c7badfc8bf8b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee74ff9700 1 -- 172.21.15.19:0/1549459697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78003c20 con 0x560051d373e0 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee74ff9700 1 -- 172.21.15.19:0/1549459697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee78003dc0 con 0x560051d373e0 2024-05-17T09:20:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.079+0000 7fee74ff9700 1 -- 172.21.15.19:0/1549459697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78025a10 con 0x560051d373e0 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/1549459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 msgr2=0x560051dd4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd4cd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fee78005b20 tx=0x7fee78009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/1549459697 shutdown_connections 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dcf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd4cd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/1549459697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051dd5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/1549459697 >> 172.21.15.19:0/1549459697 conn(0x560051cac170 msgr2=0x560051dc5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.080+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/1549459697 shutdown_connections 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.081+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/1549459697 wait complete. 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.081+0000 7fee9b4c9b80 1 -- start start 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.081+0000 7fee9b4c9b80 1 Processor -- start 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 0x560051dd8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051d373e0 0x560051dd8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051dcf090 0x560051dd93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560051beef50 con 0x560051c4bdf0 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560051c50d50 con 0x560051d373e0 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560051ddb960 con 0x560051dcf090 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee9b4c9b80 10 client.0 _ll_register_callbacks cb 0x560051c83900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:16.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 0x560051dd8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 0x560051dd8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60246/0 (socket says 172.21.15.19:60246) 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 -- 172.21.15.19:0/2295000560 learned_addr learned my addr 172.21.15.19:0/2295000560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee86d9d700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051dcf090 0x560051dd93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051d373e0 0x560051dd8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 -- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051dcf090 msgr2=0x560051dd93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051dcf090 0x560051dd93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 -- 172.21.15.19:0/2295000560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051d373e0 msgr2=0x560051dd8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051d373e0 0x560051dd8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 -- 172.21.15.19:0/2295000560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560051c38880 con 0x560051c4bdf0 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee8659c700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 0x560051dd8930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee78003760 tx=0x7fee78005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab6ba2f3a187d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.082+0000 7fee6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee6dffb700 1 -- 172.21.15.19:0/2295000560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7803e060 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee6dffb700 1 -- 172.21.15.19:0/2295000560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fee780256d0 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee78003dc0 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee6dffb700 1 -- 172.21.15.19:0/2295000560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee78035040 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee9b4c9b80 10 client.15444 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560051ca2710 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee6dffb700 1 -- 172.21.15.19:0/2295000560 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee78003dc0 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee6dffb700 1 -- 172.21.15.19:0/2295000560 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fee78038030 con 0x560051c4bdf0 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee9b4c9b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.083+0000 7fee9b4c9b80 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee9b4c9b80 20 client.15444 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051de8310 0x560051de9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee9b4c9b80 4 client.15444 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560051d96e80 con 0x560051de8310 2024-05-17T09:20:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051de8310 0x560051de9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051de8310 0x560051de9780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a054f698d2c2033b server_cookie=52034f6ee69c1dcf in_seq=0 out_seq=0 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.084+0000 7fee6dffb700 10 client.15444 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee6dffb700 1 -- 172.21.15.19:0/2295000560 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560051d96e80 con 0x560051de8310 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee6dffb700 10 client.15444 handle_command_reply: tid=0 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee9b4c9b80 1 client.15444 shutdown 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 msgr2=0x560051dd8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 0x560051dd8930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee78003760 tx=0x7fee78005a00 comp rx=0 tx=0).stop 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 shutdown_connections 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.085+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051de8310 0x560051de9780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051dcf090 0x560051dd93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051c4bdf0 0x560051dd8930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2295000560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051d373e0 0x560051dd8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee6dffb700 0 client.15444 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 >> 172.21.15.19:0/2295000560 conn(0x560051cac170 msgr2=0x560051dc5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 shutdown_connections 2024-05-17T09:20:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2295000560 wait complete. 2024-05-17T09:20:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.086+0000 7fee9b4c9b80 20 client.15444 trim_cache size 0 max 16384 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 Processor -- start 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 -- start start 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051c4bdf0 0x560051dc73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051d373e0 0x560051ddadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 0x560051dd40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560051de7ac0 con 0x560051dcf090 2024-05-17T09:20:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560051de8190 con 0x560051c4bdf0 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560051de8860 con 0x560051d373e0 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 0x560051dd40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 0x560051dd40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60256/0 (socket says 172.21.15.19:60256) 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 -- 172.21.15.19:0/2458969372 learned_addr learned my addr 172.21.15.19:0/2458969372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee85d9b700 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051d373e0 0x560051ddadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee8659c700 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051c4bdf0 0x560051dc73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 -- 172.21.15.19:0/2458969372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051d373e0 msgr2=0x560051ddadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051d373e0 0x560051ddadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 -- 172.21.15.19:0/2458969372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051c4bdf0 msgr2=0x560051dc73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051c4bdf0 0x560051dc73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.108+0000 7fee86d9d700 1 -- 172.21.15.19:0/2458969372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560051dd6900 con 0x560051dcf090 2024-05-17T09:20:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.109+0000 7fee86d9d700 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 0x560051dd40c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee80007850 tx=0x7fee8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36422f660d63a070 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:16.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.109+0000 7fee6ffff700 1 -- 172.21.15.19:0/2458969372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80003c00 con 0x560051dcf090 2024-05-17T09:20:16.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.109+0000 7fee6ffff700 1 -- 172.21.15.19:0/2458969372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee80003da0 con 0x560051dcf090 2024-05-17T09:20:16.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.109+0000 7fee6ffff700 1 -- 172.21.15.19:0/2458969372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80017740 con 0x560051dcf090 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2458969372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 msgr2=0x560051dd40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 0x560051dd40c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee80007850 tx=0x7fee8000b040 comp rx=0 tx=0).stop 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2458969372 shutdown_connections 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051dcf090 0x560051dd40c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051c4bdf0 0x560051dc73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/2458969372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051d373e0 0x560051ddadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2458969372 >> 172.21.15.19:0/2458969372 conn(0x560051cac170 msgr2=0x560051dd39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2458969372 shutdown_connections 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/2458969372 wait complete. 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 -- start start 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.110+0000 7fee9b4c9b80 1 Processor -- start 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051df3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560051de9600 con 0x560051d373e0 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560051de8f30 con 0x560051dcf090 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560051de9cd0 con 0x560051c4bdf0 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee9b4c9b80 10 client.0 _ll_register_callbacks cb 0x560051c83900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60266/0 (socket says 172.21.15.19:60266) 2024-05-17T09:20:16.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 -- 172.21.15.19:0/41455038 learned_addr learned my addr 172.21.15.19:0/41455038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee8659c700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051df3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee86d9d700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 -- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 msgr2=0x560051df3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051df3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 -- 172.21.15.19:0/41455038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 msgr2=0x560051dda580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dda580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.111+0000 7fee85d9b700 1 -- 172.21.15.19:0/41455038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560051ddad00 con 0x560051d373e0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee85d9b700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd9ee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0a6c80 tx=0x7fee7c0dfdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884f2d0d6af345c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee6dffb700 1 -- 172.21.15.19:0/41455038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0c2070 con 0x560051d373e0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee6dffb700 1 -- 172.21.15.19:0/41455038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fee7c0a9e80 con 0x560051d373e0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee80003da0 con 0x560051d373e0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee6dffb700 1 -- 172.21.15.19:0/41455038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee7c0bee60 con 0x560051d373e0 2024-05-17T09:20:16.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 10 client.15450 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560051e53b00 con 0x560051d373e0 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee6dffb700 1 -- 172.21.15.19:0/41455038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee80003da0 con 0x560051d373e0 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee6dffb700 1 -- 172.21.15.19:0/41455038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fee7c089020 con 0x560051d373e0 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 10 client.15450 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 20 client.15450 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051e55f60 0x560051e573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 4 client.15450 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.112+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "status"}) -- 0x560051cfac70 con 0x560051e55f60 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.113+0000 7fee8659c700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051e55f60 0x560051e573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.113+0000 7fee6dffb700 10 client.15450 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.113+0000 7fee8659c700 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051e55f60 0x560051e573d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eceafdb03bc22a07 server_cookie=27cabb6dbe83e244 in_seq=0 out_seq=0 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.113+0000 7fee6dffb700 1 -- 172.21.15.19:0/41455038 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560051cfac70 con 0x560051e55f60 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.113+0000 7fee6dffb700 10 client.15450 handle_command_reply: tid=0 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.113+0000 7fee9b4c9b80 1 client.15450 shutdown 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 msgr2=0x560051dd9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd9ee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0a6c80 tx=0x7fee7c0dfdc0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 shutdown_connections 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560051e55f60 0x560051e573d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560051c4bdf0 0x560051df3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee6dffb700 0 client.15450 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560051dcf090 0x560051dda580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 --2- 172.21.15.19:0/41455038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560051d373e0 0x560051dd9ee0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 >> 172.21.15.19:0/41455038 conn(0x560051cac170 msgr2=0x560051dd25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 shutdown_connections 2024-05-17T09:20:16.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 1 -- 172.21.15.19:0/41455038 wait complete. 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.114+0000 7fee9b4c9b80 20 client.15450 trim_cache size 0 max 16384 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 239.919361839, 2024-05-17T09:20:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 333.92658165099999 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee700b2900 msgr2=0x7fee700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee700b2900 0x7fee700b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee7801a040 comp rx=0 tx=0).stop 2024-05-17T09:20:16.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 msgr2=0x7fee881a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 0x7fee881a0ef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e970 tx=0x7fee7c00ec80 comp rx=0 tx=0).stop 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 shutdown_connections 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fee700b2900 0x7fee700b4e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee88109840 0x7fee8819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee881045c0 0x7fee881a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 --2- 172.21.15.19:0/2403599989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee88108ef0 0x7fee881a0ef0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 >> 172.21.15.19:0/2403599989 conn(0x7fee880ffe90 msgr2=0x7fee881022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 shutdown_connections 2024-05-17T09:20:16.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:16.115+0000 7fee8cab4700 1 -- 172.21.15.19:0/2403599989 wait complete. 2024-05-17T09:20:16.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:16 smithi069 ceph-mon[35081]: pgmap v453: 129 pgs: 129 active+clean; 657 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T09:20:16.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:16 smithi019 ceph-mon[31620]: pgmap v453: 129 pgs: 129 active+clean; 657 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T09:20:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:16 smithi079 ceph-mon[35502]: pgmap v453: 129 pgs: 129 active+clean; 657 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-17T09:20:16.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T09:20:17.150 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:18.336 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:18 smithi019 ceph-mon[31620]: pgmap v454: 129 pgs: 129 active+clean; 657 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-17T09:20:18.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:18 smithi069 ceph-mon[35081]: pgmap v454: 129 pgs: 129 active+clean; 657 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-17T09:20:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:18 smithi079 ceph-mon[35502]: pgmap v454: 129 pgs: 129 active+clean; 657 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-17T09:20:18.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.924+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/602589965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 msgr2=0x7f3c481111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.924+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/602589965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c481111b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3c400099c0 tx=0x7f3c40009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/602589965 shutdown_connections 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/602589965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c481111b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/602589965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481045d0 0x7f3c481049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/602589965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c48108f00 0x7f3c48109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/602589965 >> 172.21.15.19:0/602589965 conn(0x7f3c480ffe60 msgr2=0x7f3c481022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/602589965 shutdown_connections 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/602589965 wait complete. 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 Processor -- start 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.925+0000 7f3c4f6a7700 1 -- start start 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4f6a7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c481045d0 0x7f3c481a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4f6a7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48108f00 0x7f3c481a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4f6a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c4819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4f6a7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48114330 con 0x7f3c48109850 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4f6a7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481141b0 con 0x7f3c48108f00 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4f6a7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481144b0 con 0x7f3c481045d0 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c4819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c4819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60286/0 (socket says 172.21.15.19:60286) 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3192458684 learned_addr learned my addr 172.21.15.19:0/3192458684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4cc42700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48108f00 0x7f3c481a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4d443700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c481045d0 0x7f3c481a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3192458684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c481045d0 msgr2=0x7f3c481a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c481045d0 0x7f3c481a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3192458684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48108f00 msgr2=0x7f3c481a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48108f00 0x7f3c481a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3192458684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4819b1d0 con 0x7f3c48109850 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.926+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c4819aaf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009990 tx=0x7f3c400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.927+0000 7f3c467fc700 1 -- 172.21.15.19:0/3192458684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40004830 con 0x7f3c48109850 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.927+0000 7f3c467fc700 1 -- 172.21.15.19:0/3192458684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c40005020 con 0x7f3c48109850 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.927+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4819b460 con 0x7f3c48109850 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.927+0000 7f3c467fc700 1 -- 172.21.15.19:0/3192458684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000f9d0 con 0x7f3c48109850 2024-05-17T09:20:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.927+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4819b930 con 0x7f3c48109850 2024-05-17T09:20:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.929+0000 7f3c467fc700 1 -- 172.21.15.19:0/3192458684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3c400051d0 con 0x7f3c48109850 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.930+0000 7f3c467fc700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c300b2900 0x7f3c300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.930+0000 7f3c4d443700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c300b2900 0x7f3c300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.930+0000 7f3c467fc700 1 -- 172.21.15.19:0/3192458684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c400e32d0 con 0x7f3c48109850 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 Processor -- start 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 -- start start 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c4d443700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c300b2900 0x7f3c300b4e10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009550 tx=0x7f3c380091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea8ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea8a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56172e9b8c40 con 0x56172ea8a370 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56172e9a87e0 con 0x56172e989380 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56172e9ac820 con 0x56172e90bdf0 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea8a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea8a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60298/0 (socket says 172.21.15.19:60298) 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c4dc44700 1 -- 172.21.15.19:0/2425030484 learned_addr learned my addr 172.21.15.19:0/2425030484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c4d443700 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea8ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.931+0000 7f3c4cc42700 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4dc44700 1 -- 172.21.15.19:0/2425030484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 msgr2=0x56172ea904d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea904d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4dc44700 1 -- 172.21.15.19:0/2425030484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 msgr2=0x56172ea8ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea8ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4dc44700 1 -- 172.21.15.19:0/2425030484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56172e8f8880 con 0x56172ea8a370 2024-05-17T09:20:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4d443700 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea8a780 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3c400a6650 tx=0x7f3c400aa050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d976529a687c5fb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c2f7fe700 1 -- 172.21.15.19:0/2425030484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400ae980 con 0x56172ea8a370 2024-05-17T09:20:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c2f7fe700 1 -- 172.21.15.19:0/2425030484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c400b33a0 con 0x56172ea8a370 2024-05-17T09:20:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.932+0000 7f3c2f7fe700 1 -- 172.21.15.19:0/2425030484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400b3110 con 0x56172ea8a370 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2425030484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 msgr2=0x56172ea8a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea8a780 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3c400a6650 tx=0x7f3c400aa050 comp rx=0 tx=0).stop 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2425030484 shutdown_connections 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea8a780 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2425030484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2425030484 >> 172.21.15.19:0/2425030484 conn(0x56172e94ac30 msgr2=0x56172ea84e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2425030484 shutdown_connections 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.933+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2425030484 wait complete. 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 -- start start 2024-05-17T09:20:18.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 Processor -- start 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea97f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56172e8aef50 con 0x56172ea8a370 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56172e94a770 con 0x56172e989380 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56172ea8afc0 con 0x56172e90bdf0 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c5e0bcb80 10 client.0 _ll_register_callbacks cb 0x56172e9438d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea97f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea97f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60308/0 (socket says 172.21.15.19:60308) 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.934+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3617192949 learned_addr learned my addr 172.21.15.19:0/3617192949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4cc42700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4d443700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3617192949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 msgr2=0x56172ea9db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 msgr2=0x56172ea9e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4dc44700 1 -- 172.21.15.19:0/3617192949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56172e947640 con 0x56172ea8a370 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea97f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3c400a7840 tx=0x7f3c40092280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b826e6e6a369bddc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c2cff9700 1 -- 172.21.15.19:0/3617192949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400c4060 con 0x56172ea8a370 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c400b33a0 con 0x56172ea8a370 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c2cff9700 1 -- 172.21.15.19:0/3617192949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c400aaa40 con 0x56172ea8a370 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c2cff9700 1 -- 172.21.15.19:0/3617192949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c400b2f10 con 0x56172ea8a370 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c5e0bcb80 10 client.15462 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.935+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56172eaa5720 con 0x56172ea8a370 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c2cff9700 1 -- 172.21.15.19:0/3617192949 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c400b33a0 con 0x56172ea8a370 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c2cff9700 1 -- 172.21.15.19:0/3617192949 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3c40090b40 con 0x56172ea8a370 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c5e0bcb80 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c5e0bcb80 10 client.15462 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c5e0bcb80 20 client.15462 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eaa8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c5e0bcb80 4 client.15462 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56172e9bab90 con 0x56172eaa7970 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.936+0000 7f3c4d443700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eaa8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.937+0000 7f3c2cff9700 10 client.15462 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.937+0000 7f3c4d443700 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eaa8de0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed32690ddc584d7d server_cookie=edaac4ef962741a7 in_seq=0 out_seq=0 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.937+0000 7f3c2cff9700 1 -- 172.21.15.19:0/3617192949 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56172e9bab90 con 0x56172eaa7970 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.937+0000 7f3c2cff9700 10 client.15462 handle_command_reply: tid=0 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 client.15462 shutdown 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 msgr2=0x56172ea97f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea97f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3c400a7840 tx=0x7f3c40092280 comp rx=0 tx=0).stop 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 shutdown_connections 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eaa8de0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c2cff9700 0 client.15462 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea97f00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/3617192949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 >> 172.21.15.19:0/3617192949 conn(0x56172e94ac30 msgr2=0x56172ea84e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 shutdown_connections 2024-05-17T09:20:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/3617192949 wait complete. 2024-05-17T09:20:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.938+0000 7f3c5e0bcb80 20 client.15462 trim_cache size 0 max 16384 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 Processor -- start 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 -- start start 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56172eaa7120 con 0x56172ea8a370 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56172eaa77f0 con 0x56172e989380 2024-05-17T09:20:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56172eaa7ec0 con 0x56172e90bdf0 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60316/0 (socket says 172.21.15.19:60316) 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 -- 172.21.15.19:0/246194395 learned_addr learned my addr 172.21.15.19:0/246194395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4d443700 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4cc42700 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 -- 172.21.15.19:0/246194395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 msgr2=0x56172ea9db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 -- 172.21.15.19:0/246194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 msgr2=0x56172ea9e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.960+0000 7f3c4dc44700 1 -- 172.21.15.19:0/246194395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56172ea9d560 con 0x56172ea8a370 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.961+0000 7f3c4d443700 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.961+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea923e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3c400a6650 tx=0x7f3c40009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f207499db10ddb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.961+0000 7f3c2effd700 1 -- 172.21.15.19:0/246194395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400c4050 con 0x56172ea8a370 2024-05-17T09:20:18.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.961+0000 7f3c2effd700 1 -- 172.21.15.19:0/246194395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c400b33a0 con 0x56172ea8a370 2024-05-17T09:20:18.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.961+0000 7f3c2effd700 1 -- 172.21.15.19:0/246194395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400049d0 con 0x56172ea8a370 2024-05-17T09:20:18.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/246194395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 msgr2=0x56172ea923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea923e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3c400a6650 tx=0x7f3c40009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/246194395 shutdown_connections 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e90bdf0 0x56172ea9db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172ea8a370 0x56172ea923e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/246194395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172e989380 0x56172ea9e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/246194395 >> 172.21.15.19:0/246194395 conn(0x56172e94ac30 msgr2=0x56172ea8ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/246194395 shutdown_connections 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/246194395 wait complete. 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 -- start start 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.962+0000 7f3c5e0bcb80 1 Processor -- start 2024-05-17T09:20:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 0x56172ea921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e989380 0x56172ea92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172ea8a370 0x56172eb110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56172eaa8c60 con 0x56172e90bdf0 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56172eaa8590 con 0x56172ea8a370 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56172eaa9330 con 0x56172e989380 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c5e0bcb80 10 client.0 _ll_register_callbacks cb 0x56172e9438d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4d443700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 0x56172ea921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4cc42700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e989380 0x56172ea92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4d443700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 0x56172ea921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60332/0 (socket says 172.21.15.19:60332) 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4cc42700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e989380 0x56172ea92770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:43842/0 (socket says 172.21.15.19:43842) 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4d443700 1 -- 172.21.15.19:0/2926672313 learned_addr learned my addr 172.21.15.19:0/2926672313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4dc44700 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172ea8a370 0x56172eb110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4d443700 1 -- 172.21.15.19:0/2926672313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e989380 msgr2=0x56172ea92770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.963+0000 7f3c4d443700 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e989380 0x56172ea92770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c4d443700 1 -- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172ea8a370 msgr2=0x56172eb110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c4d443700 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172ea8a370 0x56172eb110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c4d443700 1 -- 172.21.15.19:0/2926672313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56172eb116e0 con 0x56172e90bdf0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c4d443700 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 0x56172ea921d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3c38007780 tx=0x7f3c38004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9416d01a7b19214 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c2cff9700 1 -- 172.21.15.19:0/2926672313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801c040 con 0x56172e90bdf0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c400b33a0 con 0x56172e90bdf0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c2cff9700 1 -- 172.21.15.19:0/2926672313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c3802ae00 con 0x56172e90bdf0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c2cff9700 1 -- 172.21.15.19:0/2926672313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c38038050 con 0x56172e90bdf0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c5e0bcb80 10 client.15468 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56172eb12910 con 0x56172e90bdf0 2024-05-17T09:20:18.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.964+0000 7f3c2cff9700 1 -- 172.21.15.19:0/2926672313 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c400b33a0 con 0x56172e90bdf0 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c2cff9700 1 -- 172.21.15.19:0/2926672313 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3c38022040 con 0x56172e90bdf0 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c5e0bcb80 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c5e0bcb80 10 client.15468 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c5e0bcb80 20 client.15468 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eb16290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c5e0bcb80 4 client.15468 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "status"}) -- 0x56172e9b4370 con 0x56172eaa7970 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c4cc42700 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eb16290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c2cff9700 10 client.15468 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.965+0000 7f3c4cc42700 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eb16290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b392b36b7a3aa4f6 server_cookie=10cdae7b9bbdf516 in_seq=0 out_seq=0 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c2cff9700 1 -- 172.21.15.19:0/2926672313 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56172e9b4370 con 0x56172eaa7970 2024-05-17T09:20:18.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c2cff9700 10 client.15468 handle_command_reply: tid=0 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 client.15468 shutdown 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 msgr2=0x56172ea921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 0x56172ea921d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3c38007780 tx=0x7f3c38004b40 comp rx=0 tx=0).stop 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 shutdown_connections 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56172eaa7970 0x56172eb16290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56172ea8a370 0x56172eb110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56172e90bdf0 0x56172ea921d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 --2- 172.21.15.19:0/2926672313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56172e989380 0x56172ea92770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 >> 172.21.15.19:0/2926672313 conn(0x56172e94ac30 msgr2=0x56172ea8ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c2cff9700 0 client.15468 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:18.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 shutdown_connections 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 1 -- 172.21.15.19:0/2926672313 wait complete. 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.966+0000 7f3c5e0bcb80 20 client.15468 trim_cache size 0 max 16384 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9c7a48b0-142c-11ef-bc9a-c7b262605968", 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24497, 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-17T09:20:18.972 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-17T09:20:18.973 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 309.00842820000003, 2024-05-17T09:20:18.973 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-17T09:20:18.973 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-17T09:20:18.973 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 89, 2024-05-17T09:20:18.973 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 326.36014865200002 2024-05-17T09:20:18.973 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c300b2900 msgr2=0x7f3c300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c300b2900 0x7f3c300b4e10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009550 tx=0x7f3c380091b0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 msgr2=0x7f3c4819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c4819aaf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009990 tx=0x7f3c400047b0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 shutdown_connections 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c300b2900 0x7f3c300b4e10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c481045d0 0x7f3c481a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c48109850 0x7f3c4819aaf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 --2- 172.21.15.19:0/3192458684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48108f00 0x7f3c481a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:18.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 >> 172.21.15.19:0/3192458684 conn(0x7f3c480ffe60 msgr2=0x7f3c481070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:18.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 shutdown_connections 2024-05-17T09:20:18.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:18.967+0000 7f3c4f6a7700 1 -- 172.21.15.19:0/3192458684 wait complete. 2024-05-17T09:20:19.782 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-17T09:20:20.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:20:20.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:20 smithi069 ceph-mon[35081]: pgmap v455: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-17T09:20:20.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:20 smithi019 ceph-mon[31620]: pgmap v455: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-17T09:20:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:20 smithi079 ceph-mon[35502]: pgmap v455: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-17T09:20:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 -- 172.21.15.19:0/2441784657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334102370 msgr2=0x7f733410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 --2- 172.21.15.19:0/2441784657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334102370 0x7f733410bdd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f732c009900 tx=0x7f732c02a410 comp rx=0 tx=0).stop 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 -- 172.21.15.19:0/2441784657 shutdown_connections 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 --2- 172.21.15.19:0/2441784657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334102370 0x7f733410bdd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 --2- 172.21.15.19:0/2441784657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334101870 0x7f7334101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 --2- 172.21.15.19:0/2441784657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73341051c0 0x7f73341055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 -- 172.21.15.19:0/2441784657 >> 172.21.15.19:0/2441784657 conn(0x7f73340fd470 msgr2=0x7f73340ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 -- 172.21.15.19:0/2441784657 shutdown_connections 2024-05-17T09:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.841+0000 7f733a649640 1 -- 172.21.15.19:0/2441784657 wait complete. 2024-05-17T09:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 Processor -- start 2024-05-17T09:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 -- start start 2024-05-17T09:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 0x7f733419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334102370 0x7f733419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73341051c0 0x7f73341a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f733410f600 con 0x7f7334101870 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f733410f480 con 0x7f73341051c0 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f733a649640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f733410f780 con 0x7f7334102370 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f7333fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 0x7f733419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f7333fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 0x7f733419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60348/0 (socket says 172.21.15.19:60348) 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f7333fff640 1 -- 172.21.15.19:0/4235187917 learned_addr learned my addr 172.21.15.19:0/4235187917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f73337fe640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334102370 0x7f733419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.842+0000 7f7338be6640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73341051c0 0x7f73341a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f7333fff640 1 -- 172.21.15.19:0/4235187917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334102370 msgr2=0x7f733419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f7333fff640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334102370 0x7f733419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f7333fff640 1 -- 172.21.15.19:0/4235187917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73341051c0 msgr2=0x7f73341a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f7333fff640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73341051c0 0x7f73341a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f7333fff640 1 -- 172.21.15.19:0/4235187917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73341a1c30 con 0x7f7334101870 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f73337fe640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334102370 0x7f733419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f7333fff640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 0x7f733419a1a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f732400c970 tx=0x7f732400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f73317fa640 1 -- 172.21.15.19:0/4235187917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324007c70 con 0x7f7334101870 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f73317fa640 1 -- 172.21.15.19:0/4235187917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7324007e10 con 0x7f7334101870 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73341a1f20 con 0x7f7334101870 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f73317fa640 1 -- 172.21.15.19:0/4235187917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73240186f0 con 0x7f7334101870 2024-05-17T09:20:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.843+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341a2380 con 0x7f7334101870 2024-05-17T09:20:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.846+0000 7f73317fa640 1 -- 172.21.15.19:0/4235187917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f732400fa90 con 0x7f7334101870 2024-05-17T09:20:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.846+0000 7f73415d6740 1 Processor -- start 2024-05-17T09:20:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.846+0000 7f73415d6740 1 -- start start 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73415d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4af67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73415d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4af6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73415d6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4afdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73415d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636b4a64c30 con 0x5636b47511b0 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73415d6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636b4a547d0 con 0x5636b47304d0 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73415d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5636b4a58810 con 0x5636b47282f0 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4af67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4af67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60356/0 (socket says 172.21.15.19:60356) 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 -- 172.21.15.19:0/4114764741 learned_addr learned my addr 172.21.15.19:0/4114764741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7333fff640 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4af6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73337fe640 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4afdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73317fa640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f731c0b28a0 0x7f731c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 -- 172.21.15.19:0/4114764741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 msgr2=0x5636b4af6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4af6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 -- 172.21.15.19:0/4114764741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 msgr2=0x5636b4afdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4afdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 -- 172.21.15.19:0/4114764741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b4769210 con 0x5636b47511b0 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f7338be6640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f731c0b28a0 0x7f731c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.847+0000 7f73317fa640 1 -- 172.21.15.19:0/4235187917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f73240e2600 con 0x7f7334101870 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.848+0000 7f7338be6640 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4af67b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f732c005ab0 tx=0x7f732c031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aeabcc7a675a98c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.848+0000 7f731a7fc640 1 -- 172.21.15.19:0/4114764741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c04d070 con 0x5636b47511b0 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.848+0000 7f7338be6640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f731c0b28a0 0x7f731c0b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f732c005f70 tx=0x7f732c047000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.848+0000 7f731a7fc640 1 -- 172.21.15.19:0/4114764741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f732c047bc0 con 0x5636b47511b0 2024-05-17T09:20:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.848+0000 7f731a7fc640 1 -- 172.21.15.19:0/4114764741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c035040 con 0x5636b47511b0 2024-05-17T09:20:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 -- 172.21.15.19:0/4114764741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 msgr2=0x5636b4af67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4af67b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f732c005ab0 tx=0x7f732c031720 comp rx=0 tx=0).stop 2024-05-17T09:20:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 -- 172.21.15.19:0/4114764741 shutdown_connections 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4afdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4af6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 --2- 172.21.15.19:0/4114764741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4af67b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 -- 172.21.15.19:0/4114764741 >> 172.21.15.19:0/4114764741 conn(0x5636b4745300 msgr2=0x5636b4aedd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 -- 172.21.15.19:0/4114764741 shutdown_connections 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 -- 172.21.15.19:0/4114764741 wait complete. 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.849+0000 7f73415d6740 1 -- start start 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 Processor -- start 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4b000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b00600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4b07560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636b473a790 con 0x5636b47511b0 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636b4a10890 con 0x5636b47304d0 2024-05-17T09:20:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5636b4afe680 con 0x5636b47282f0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73415d6740 10 client.0 _ll_register_callbacks cb 0x5636b4a13d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4b07560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4b07560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60366/0 (socket says 172.21.15.19:60366) 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 -- 172.21.15.19:0/3702115982 learned_addr learned my addr 172.21.15.19:0/3702115982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7333fff640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4b000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73337fe640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b00600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 -- 172.21.15.19:0/3702115982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 msgr2=0x5636b4b000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4b000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 -- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 msgr2=0x5636b4b00600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b00600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f7338be6640 1 -- 172.21.15.19:0/3702115982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b4835620 con 0x5636b47511b0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.850+0000 7f73337fe640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f7338be6640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4b07560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f732c0096c0 tx=0x7f732c0477b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c328039fe60ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73137fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73137fe640 1 -- 172.21.15.19:0/3702115982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c055070 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73137fe640 1 -- 172.21.15.19:0/3702115982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f732c0469b0 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732c047bc0 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73137fe640 1 -- 172.21.15.19:0/3702115982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f732c061050 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73415d6740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5636b4b0a490 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73137fe640 1 -- 172.21.15.19:0/3702115982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f732c047bc0 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73137fe640 1 -- 172.21.15.19:0/3702115982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f732c051ca0 con 0x5636b47511b0 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73415d6740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.851+0000 7f73415d6740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f73415d6740 20 client.15480 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f73415d6740 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b470fd80 0x5636b4b0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f73415d6740 4 client.15480 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b4a66b80 con 0x5636b470fd80 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f7333fff640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b470fd80 0x5636b4b0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f73137fe640 10 client.15480 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.852+0000 7f7333fff640 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b470fd80 0x5636b4b0d7f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c258d2f0a1a162dc server_cookie=36ea96b7eaf074d1 in_seq=0 out_seq=0 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73137fe640 1 -- 172.21.15.19:0/3702115982 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636b4a66b80 con 0x5636b470fd80 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73137fe640 10 client.15480 handle_command_reply: tid=0 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 client.15480 shutdown 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 msgr2=0x5636b4b07560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4b07560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f732c0096c0 tx=0x7f732c0477b0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 shutdown_connections 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b470fd80 0x5636b4b0d7f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47511b0 0x5636b4b07560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73137fe640 0 client.15480 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 --2- 172.21.15.19:0/3702115982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47282f0 0x5636b4b000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 >> 172.21.15.19:0/3702115982 conn(0x5636b4745300 msgr2=0x5636b4af0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 shutdown_connections 2024-05-17T09:20:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.853+0000 7f73415d6740 1 -- 172.21.15.19:0/3702115982 wait complete. 2024-05-17T09:20:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.854+0000 7f73415d6740 20 client.15480 trim_cache size 0 max 16384 2024-05-17T09:20:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 Processor -- start 2024-05-17T09:20:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 -- start start 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b470fd80 0x5636b4afacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b07480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47304d0 0x5636b4b079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73341acf80 con 0x5636b47282f0 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73341aa850 con 0x5636b470fd80 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.869+0000 7f73415d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73341a86a0 con 0x5636b47304d0 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b07480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b07480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60368/0 (socket says 172.21.15.19:60368) 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 -- 172.21.15.19:0/101881126 learned_addr learned my addr 172.21.15.19:0/101881126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f7338be6640 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47304d0 0x5636b4b079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f7333fff640 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b470fd80 0x5636b4afacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 -- 172.21.15.19:0/101881126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47304d0 msgr2=0x5636b4b079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47304d0 0x5636b4b079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 -- 172.21.15.19:0/101881126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b470fd80 msgr2=0x5636b4afacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b470fd80 0x5636b4afacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 -- 172.21.15.19:0/101881126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b4afe7b0 con 0x5636b47282f0 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f7333fff640 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b470fd80 0x5636b4afacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f73337fe640 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b07480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7328009f80 tx=0x7f7328012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b8fbaa7c44ba82 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f7319ffb640 1 -- 172.21.15.19:0/101881126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328020070 con 0x5636b47282f0 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f7319ffb640 1 -- 172.21.15.19:0/101881126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7328010920 con 0x5636b47282f0 2024-05-17T09:20:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.870+0000 7f7319ffb640 1 -- 172.21.15.19:0/101881126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732801ba20 con 0x5636b47282f0 2024-05-17T09:20:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 -- 172.21.15.19:0/101881126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 msgr2=0x5636b4b07480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b07480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7328009f80 tx=0x7f7328012880 comp rx=0 tx=0).stop 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 -- 172.21.15.19:0/101881126 shutdown_connections 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b47304d0 0x5636b4b079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b07480 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 --2- 172.21.15.19:0/101881126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b470fd80 0x5636b4afacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 -- 172.21.15.19:0/101881126 >> 172.21.15.19:0/101881126 conn(0x5636b4745300 msgr2=0x5636b4aec330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 -- 172.21.15.19:0/101881126 shutdown_connections 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.871+0000 7f73415d6740 1 -- 172.21.15.19:0/101881126 wait complete. 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 -- start start 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 Processor -- start 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b470fd80 0x5636b4b05680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b05bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73341a8d10 con 0x5636b47282f0 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73341aa180 con 0x5636b47304d0 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73341a93e0 con 0x5636b470fd80 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.872+0000 7f73415d6740 10 client.0 _ll_register_callbacks cb 0x7f73240a5f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b05bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b05bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60382/0 (socket says 172.21.15.19:60382) 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 -- 172.21.15.19:0/1080594115 learned_addr learned my addr 172.21.15.19:0/1080594115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f7333fff640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b470fd80 0x5636b4b05680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f7338be6640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 -- 172.21.15.19:0/1080594115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b470fd80 msgr2=0x5636b4b05680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b470fd80 0x5636b4b05680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 -- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 msgr2=0x5636b4b06af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b06af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 -- 172.21.15.19:0/1080594115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b4b070b0 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f7338be6640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73337fe640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b05bc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7328000f80 tx=0x7f7328010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d52a750e586d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73137fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73137fe640 1 -- 172.21.15.19:0/1080594115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732802b060 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73137fe640 1 -- 172.21.15.19:0/1080594115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7328010ad0 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.873+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7328010920 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73137fe640 1 -- 172.21.15.19:0/1080594115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7328032050 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5636b4b75e60 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73137fe640 1 -- 172.21.15.19:0/1080594115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7328010920 con 0x5636b47282f0 2024-05-17T09:20:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73137fe640 1 -- 172.21.15.19:0/1080594115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f732800f040 con 0x5636b47282f0 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 20 client.15486 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b47511b0 0x5636b4b787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 4 client.15486 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636b4a60360 con 0x5636b47511b0 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.874+0000 7f7333fff640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b47511b0 0x5636b4b787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.875+0000 7f73137fe640 10 client.15486 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.875+0000 7f7333fff640 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b47511b0 0x5636b4b787d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5fc3bc6ada74e8 server_cookie=f1dbcd6063a5899e in_seq=0 out_seq=0 2024-05-17T09:20:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73137fe640 1 -- 172.21.15.19:0/1080594115 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5636b4a60360 con 0x5636b47511b0 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73137fe640 10 client.15486 handle_command_reply: tid=0 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 client.15486 shutdown 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 msgr2=0x5636b4b05bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b05bc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7328000f80 tx=0x7f7328010740 comp rx=0 tx=0).stop 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 shutdown_connections 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5636b47511b0 0x5636b4b787d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636b47304d0 0x5636b4b06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73137fe640 0 client.15486 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636b47282f0 0x5636b4b05bc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 --2- 172.21.15.19:0/1080594115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636b470fd80 0x5636b4b05680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 >> 172.21.15.19:0/1080594115 conn(0x5636b4745300 msgr2=0x5636b4aecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 shutdown_connections 2024-05-17T09:20:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 1 -- 172.21.15.19:0/1080594115 wait complete. 2024-05-17T09:20:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.883+0000 7f73415d6740 20 client.15486 trim_cache size 0 max 16384 2024-05-17T09:20:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:20:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:20:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f731c0b28a0 msgr2=0x7f731c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f731c0b28a0 0x7f731c0b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f732c005f70 tx=0x7f732c047000 comp rx=0 tx=0).stop 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 msgr2=0x7f733419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 0x7f733419a1a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f732400c970 tx=0x7f732400ce30 comp rx=0 tx=0).stop 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 shutdown_connections 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73341051c0 0x7f73341a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7334102370 0x7f733419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f731c0b28a0 0x7f731c0b4db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 --2- 172.21.15.19:0/4235187917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7334101870 0x7f733419a1a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 >> 172.21.15.19:0/4235187917 conn(0x7f73340fd470 msgr2=0x7f7334107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 shutdown_connections 2024-05-17T09:20:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:21.884+0000 7f733a649640 1 -- 172.21.15.19:0/4235187917 wait complete. 2024-05-17T09:20:22.177 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:22 smithi019 ceph-mon[31620]: pgmap v456: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-17T09:20:22.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:22 smithi069 ceph-mon[35081]: pgmap v456: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 -- 172.21.15.19:0/4257802508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 msgr2=0x7fa74c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 --2- 172.21.15.19:0/4257802508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c109340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa7400099c0 tx=0x7fa740009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 -- 172.21.15.19:0/4257802508 shutdown_connections 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 --2- 172.21.15.19:0/4257802508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa74c104600 0x7fa74c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 --2- 172.21.15.19:0/4257802508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 0x7fa74c1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 --2- 172.21.15.19:0/4257802508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c109340 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 -- 172.21.15.19:0/4257802508 >> 172.21.15.19:0/4257802508 conn(0x7fa74c07af60 msgr2=0x7fa74c07b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 -- 172.21.15.19:0/4257802508 shutdown_connections 2024-05-17T09:20:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.445+0000 7fa752538700 1 -- 172.21.15.19:0/4257802508 wait complete. 2024-05-17T09:20:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.446+0000 7fa752538700 1 Processor -- start 2024-05-17T09:20:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.446+0000 7fa752538700 1 -- start start 2024-05-17T09:20:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.446+0000 7fa752538700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa74c104600 0x7fa74c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa752538700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa752538700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 0x7fa74c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa752538700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c114390 con 0x7fa74c108f30 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa752538700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c114210 con 0x7fa74c109880 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa752538700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c114510 con 0x7fa74c104600 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60400/0 (socket says 172.21.15.19:60400) 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 -- 172.21.15.19:0/1884930694 learned_addr learned my addr 172.21.15.19:0/1884930694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74bfff700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa74c104600 0x7fa74c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa750ad5700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 0x7fa74c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 -- 172.21.15.19:0/1884930694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa74c104600 msgr2=0x7fa74c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa74c104600 0x7fa74c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 -- 172.21.15.19:0/1884930694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 msgr2=0x7fa74c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 0x7fa74c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 -- 172.21.15.19:0/1884930694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c19fba0 con 0x7fa74c108f30 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa750ad5700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 0x7fa74c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c19b320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa7400098e0 tx=0x7fa74000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.447+0000 7fa7497fa700 1 -- 172.21.15.19:0/1884930694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740027070 con 0x7fa74c108f30 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.448+0000 7fa7497fa700 1 -- 172.21.15.19:0/1884930694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa740022470 con 0x7fa74c108f30 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.448+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c19fe30 con 0x7fa74c108f30 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.448+0000 7fa7497fa700 1 -- 172.21.15.19:0/1884930694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740017650 con 0x7fa74c108f30 2024-05-17T09:20:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.448+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c1a0310 con 0x7fa74c108f30 2024-05-17T09:20:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.451+0000 7fa7497fa700 1 -- 172.21.15.19:0/1884930694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa74002a070 con 0x7fa74c108f30 2024-05-17T09:20:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa7497fa700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7340b2880 0x7fa7340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa74bfff700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7340b2880 0x7fa7340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 Processor -- start 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa7497fa700 1 -- 172.21.15.19:0/1884930694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa7400e3510 con 0x7fa74c108f30 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 -- start start 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa74bfff700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7340b2880 0x7fa7340b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00ac50 tx=0x7fa73c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597b91fcc90 con 0x5597b91cd3d0 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5597b91ec830 con 0x5597b92ce3c0 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa760f4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5597b91f0870 con 0x5597b914fe40 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.452+0000 7fa74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92d3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60412/0 (socket says 172.21.15.19:60412) 2024-05-17T09:20:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 -- 172.21.15.19:0/81008593 learned_addr learned my addr 172.21.15.19:0/81008593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92d4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa750ad5700 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 -- 172.21.15.19:0/81008593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 msgr2=0x5597b92d4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92d4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 -- 172.21.15.19:0/81008593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 msgr2=0x5597b92ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 -- 172.21.15.19:0/81008593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b913c8d0 con 0x5597b91cd3d0 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa750ad5700 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa74bfff700 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92d3fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00ff90 tx=0x7fa73c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa377bf2b87b000a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa7327fc700 1 -- 172.21.15.19:0/81008593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c016070 con 0x5597b91cd3d0 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa7327fc700 1 -- 172.21.15.19:0/81008593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa73c014070 con 0x5597b91cd3d0 2024-05-17T09:20:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.453+0000 7fa7327fc700 1 -- 172.21.15.19:0/81008593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c010e40 con 0x5597b91cd3d0 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 -- 172.21.15.19:0/81008593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 msgr2=0x5597b92d3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92d3fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00ff90 tx=0x7fa73c012000 comp rx=0 tx=0).stop 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 -- 172.21.15.19:0/81008593 shutdown_connections 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92d3fe0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 --2- 172.21.15.19:0/81008593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 -- 172.21.15.19:0/81008593 >> 172.21.15.19:0/81008593 conn(0x5597b918ec80 msgr2=0x5597b92c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 -- 172.21.15.19:0/81008593 shutdown_connections 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.454+0000 7fa760f4db80 1 -- 172.21.15.19:0/81008593 wait complete. 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 -- start start 2024-05-17T09:20:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 Processor -- start 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 0x5597b92d7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b91cd3d0 0x5597b92d80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597b90f2fa0 con 0x5597b914fe40 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5597b918e7c0 con 0x5597b92ce3c0 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5597b92cf250 con 0x5597b91cd3d0 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa760f4db80 10 client.0 _ll_register_callbacks cb 0x5597b9187920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 0x5597b92d7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa74bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 0x5597b92d7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60418/0 (socket says 172.21.15.19:60418) 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.455+0000 7fa74bfff700 1 -- 172.21.15.19:0/3131628368 learned_addr learned my addr 172.21.15.19:0/3131628368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b91cd3d0 0x5597b92d80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa750ad5700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74bfff700 1 -- 172.21.15.19:0/3131628368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b91cd3d0 msgr2=0x5597b92d80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74bfff700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b91cd3d0 0x5597b92d80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74bfff700 1 -- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 msgr2=0x5597b92d8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74bfff700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74bfff700 1 -- 172.21.15.19:0/3131628368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b918b690 con 0x5597b914fe40 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa750ad5700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa74bfff700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 0x5597b92d7ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00a4a0 tx=0x7fa73c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0903f217a03788c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa72b7fe700 1 -- 172.21.15.19:0/3131628368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c010e30 con 0x5597b914fe40 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa73c014070 con 0x5597b914fe40 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa72b7fe700 1 -- 172.21.15.19:0/3131628368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa73c003910 con 0x5597b914fe40 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa72b7fe700 1 -- 172.21.15.19:0/3131628368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa73c003b10 con 0x5597b914fe40 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa760f4db80 10 client.15498 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.456+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597b92e5510 con 0x5597b914fe40 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa72b7fe700 1 -- 172.21.15.19:0/3131628368 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa73c014070 con 0x5597b914fe40 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa72b7fe700 1 -- 172.21.15.19:0/3131628368 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa73c0048a0 con 0x5597b914fe40 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa760f4db80 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa760f4db80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa760f4db80 20 client.15498 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b92e7a40 0x5597b92e9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa760f4db80 4 client.15498 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597b91febe0 con 0x5597b92e7a40 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.457+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b92e7a40 0x5597b92e9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.458+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b92e7a40 0x5597b92e9e60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da464ed912d11fc server_cookie=fbfb8c558f8a25d2 in_seq=0 out_seq=0 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.458+0000 7fa72b7fe700 10 client.15498 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.458+0000 7fa72b7fe700 1 -- 172.21.15.19:0/3131628368 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5597b91febe0 con 0x5597b92e7a40 2024-05-17T09:20:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.458+0000 7fa72b7fe700 10 client.15498 handle_command_reply: tid=0 2024-05-17T09:20:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 client.15498 shutdown 2024-05-17T09:20:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 msgr2=0x5597b92d7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 0x5597b92d7ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00a4a0 tx=0x7fa73c0106b0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 shutdown_connections 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b92e7a40 0x5597b92e9e60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b914fe40 0x5597b92d7ba0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3131628368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b91cd3d0 0x5597b92d80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa72b7fe700 0 client.15498 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 >> 172.21.15.19:0/3131628368 conn(0x5597b918ec80 msgr2=0x5597b92c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 shutdown_connections 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 1 -- 172.21.15.19:0/3131628368 wait complete. 2024-05-17T09:20:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.459+0000 7fa760f4db80 20 client.15498 trim_cache size 0 max 16384 2024-05-17T09:20:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 Processor -- start 2024-05-17T09:20:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 -- start start 2024-05-17T09:20:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1b2a60 con 0x5597b91cd3d0 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1b02f0 con 0x5597b92ce3c0 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.483+0000 7fa760f4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1ae140 con 0x5597b914fe40 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92c3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60420/0 (socket says 172.21.15.19:60420) 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 -- 172.21.15.19:0/3047862495 learned_addr learned my addr 172.21.15.19:0/3047862495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74bfff700 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 -- 172.21.15.19:0/3047862495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 msgr2=0x5597b92e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92e3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 -- 172.21.15.19:0/3047862495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 msgr2=0x5597b92d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 -- 172.21.15.19:0/3047862495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b92c54d0 con 0x5597b91cd3d0 2024-05-17T09:20:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92c3990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa7400aa510 tx=0x7fa7400b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0da22733ece07d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:22.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.484+0000 7fa731ffb700 1 -- 172.21.15.19:0/3047862495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7400b35e0 con 0x5597b91cd3d0 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa731ffb700 1 -- 172.21.15.19:0/3047862495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7400a6e00 con 0x5597b91cd3d0 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa731ffb700 1 -- 172.21.15.19:0/3047862495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7400a6c30 con 0x5597b91cd3d0 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa760f4db80 1 -- 172.21.15.19:0/3047862495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 msgr2=0x5597b92c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92c3990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa7400aa510 tx=0x7fa7400b3040 comp rx=0 tx=0).stop 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa760f4db80 1 -- 172.21.15.19:0/3047862495 shutdown_connections 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.485+0000 7fa760f4db80 1 --2- 172.21.15.19:0/3047862495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92c3990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.486+0000 7fa760f4db80 1 -- 172.21.15.19:0/3047862495 >> 172.21.15.19:0/3047862495 conn(0x5597b918ec80 msgr2=0x5597b92d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.486+0000 7fa760f4db80 1 -- 172.21.15.19:0/3047862495 shutdown_connections 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.486+0000 7fa760f4db80 1 -- 172.21.15.19:0/3047862495 wait complete. 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.486+0000 7fa760f4db80 1 -- start start 2024-05-17T09:20:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.486+0000 7fa760f4db80 1 Processor -- start 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1ae7b0 con 0x5597b91cd3d0 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1afc20 con 0x5597b92ce3c0 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1aee80 con 0x5597b914fe40 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa760f4db80 10 client.0 _ll_register_callbacks cb 0x5597b9187920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92eb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60424/0 (socket says 172.21.15.19:60424) 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 -- 172.21.15.19:0/4026552702 learned_addr learned my addr 172.21.15.19:0/4026552702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74bfff700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa750ad5700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 -- 172.21.15.19:0/4026552702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 msgr2=0x5597b92eade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92eade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 -- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 msgr2=0x5597b92eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 -- 172.21.15.19:0/4026552702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b9351420 con 0x5597b91cd3d0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74bfff700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa74b7fe700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92eb320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa7400b3010 tx=0x7fa7400a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0ce0e992d3e1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.487+0000 7fa72b7fe700 1 -- 172.21.15.19:0/4026552702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7400c3020 con 0x5597b91cd3d0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa72b7fe700 1 -- 172.21.15.19:0/4026552702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa7400a6aa0 con 0x5597b91cd3d0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7400a6e00 con 0x5597b91cd3d0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa72b7fe700 1 -- 172.21.15.19:0/4026552702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa740022960 con 0x5597b91cd3d0 2024-05-17T09:20:22.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 10 client.15504 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597b9356aa0 con 0x5597b91cd3d0 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa72b7fe700 1 -- 172.21.15.19:0/4026552702 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7400a6e00 con 0x5597b91cd3d0 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa72b7fe700 1 -- 172.21.15.19:0/4026552702 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa7400b3250 con 0x5597b91cd3d0 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 20 client.15504 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b93570e0 0x5597b9359500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 4 client.15504 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.488+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597b91f83c0 con 0x5597b93570e0 2024-05-17T09:20:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa74bfff700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b93570e0 0x5597b9359500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa74bfff700 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b93570e0 0x5597b9359500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb4706a75b0c87f server_cookie=8dfa5c4c279bef25 in_seq=0 out_seq=0 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa72b7fe700 10 client.15504 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa72b7fe700 1 -- 172.21.15.19:0/4026552702 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa73c029050 con 0x5597b93570e0 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa72b7fe700 10 client.15504 handle_command_reply: tid=0 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 client.15504 shutdown 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 msgr2=0x5597b92eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92eb320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa7400b3010 tx=0x7fa7400a66d0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 shutdown_connections 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597b914fe40 0x5597b92eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5597b93570e0 0x5597b9359500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.489+0000 7fa760f4db80 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5597b92ce3c0 0x5597b92eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa760f4db80 1 --2- 172.21.15.19:0/4026552702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597b91cd3d0 0x5597b92eb320 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa72b7fe700 0 client.15504 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 >> 172.21.15.19:0/4026552702 conn(0x5597b918ec80 msgr2=0x5597b92d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 shutdown_connections 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa760f4db80 1 -- 172.21.15.19:0/4026552702 wait complete. 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa760f4db80 20 client.15504 trim_cache size 0 max 16384 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:20:22.495 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7340b2880 msgr2=0x7fa7340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7340b2880 0x7fa7340b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00ac50 tx=0x7fa73c00a380 comp rx=0 tx=0).stop 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 msgr2=0x7fa74c19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c19b320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa7400098e0 tx=0x7fa74000f690 comp rx=0 tx=0).stop 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 shutdown_connections 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7340b2880 0x7fa7340b4d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa74c104600 0x7fa74c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa74c109880 0x7fa74c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 --2- 172.21.15.19:0/1884930694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa74c108f30 0x7fa74c19b320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.490+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 >> 172.21.15.19:0/1884930694 conn(0x7fa74c07af60 msgr2=0x7fa74c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.491+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 shutdown_connections 2024-05-17T09:20:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:22.491+0000 7fa752538700 1 -- 172.21.15.19:0/1884930694 wait complete. 2024-05-17T09:20:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:22 smithi079 ceph-mon[35502]: pgmap v456: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-17T09:20:23.194 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:20:23.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:20:23.449 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:24.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:24 smithi069 ceph-mon[35081]: pgmap v457: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-17T09:20:24.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:24 smithi019 ceph-mon[31620]: pgmap v457: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-17T09:20:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:24 smithi079 ceph-mon[35502]: pgmap v457: 129 pgs: 129 active+clean; 666 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-17T09:20:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 -- 172.21.15.19:0/956700435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188105470 msgr2=0x7fa188105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/956700435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188105470 0x7fa188105880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa178009990 tx=0x7fa178009ca0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 -- 172.21.15.19:0/956700435 shutdown_connections 2024-05-17T09:20:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/956700435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa18810e8e0 0x7fa188110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/956700435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188105470 0x7fa188105880 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/956700435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa188105dc0 0x7fa18810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 -- 172.21.15.19:0/956700435 >> 172.21.15.19:0/956700435 conn(0x7fa1880fef20 msgr2=0x7fa188101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 -- 172.21.15.19:0/956700435 shutdown_connections 2024-05-17T09:20:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.312+0000 7fa18ca3b700 1 -- 172.21.15.19:0/956700435 wait complete. 2024-05-17T09:20:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.313+0000 7fa18ca3b700 1 Processor -- start 2024-05-17T09:20:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.313+0000 7fa18ca3b700 1 -- start start 2024-05-17T09:20:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18ca3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa188105470 0x7fa18819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18ca3b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa188105dc0 0x7fa18819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18ca3b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 0x7fa18819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18ca3b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881142e0 con 0x7fa18810e8e0 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18ca3b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa188114160 con 0x7fa188105dc0 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18ca3b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa188114460 con 0x7fa188105470 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 0x7fa18819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 0x7fa18819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48990/0 (socket says 172.21.15.19:48990) 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 -- 172.21.15.19:0/4125515385 learned_addr learned my addr 172.21.15.19:0/4125515385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa185d9b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa188105dc0 0x7fa18819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa18659c700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa188105470 0x7fa18819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 -- 172.21.15.19:0/4125515385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa188105470 msgr2=0x7fa18819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa188105470 0x7fa18819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 -- 172.21.15.19:0/4125515385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa188105dc0 msgr2=0x7fa18819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa188105dc0 0x7fa18819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 -- 172.21.15.19:0/4125515385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18819fc00 con 0x7fa18810e8e0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.314+0000 7fa186d9d700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 0x7fa18819f560 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa18000ada0 tx=0x7fa18000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.315+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180017020 con 0x7fa18810e8e0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.315+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18819fef0 con 0x7fa18810e8e0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.315+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa18000cd60 con 0x7fa18810e8e0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.315+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180005280 con 0x7fa18810e8e0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.315+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881a7810 con 0x7fa18810e8e0 2024-05-17T09:20:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.316+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1880512b0 con 0x7fa18810e8e0 2024-05-17T09:20:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.318+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa18000c8d0 con 0x7fa18810e8e0 2024-05-17T09:20:25.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.319+0000 7fa1777fe700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1700b2880 0x7fa1700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:25.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.319+0000 7fa18659c700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1700b2880 0x7fa1700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:25.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.319+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa1800e22d0 con 0x7fa18810e8e0 2024-05-17T09:20:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.320+0000 7fa18659c700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1700b2880 0x7fa1700b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa17800b5c0 tx=0x7fa178005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.323+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa180021020 con 0x7fa18810e8e0 2024-05-17T09:20:25.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.590+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa18819c090 con 0x7fa18810e8e0 2024-05-17T09:20:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.591+0000 7fa1777fe700 1 -- 172.21.15.19:0/4125515385 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fa1800a9c70 con 0x7fa18810e8e0 2024-05-17T09:20:25.591 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:25.591 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:20:25.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1700b2880 msgr2=0x7fa1700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:25.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1700b2880 0x7fa1700b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa17800b5c0 tx=0x7fa178005960 comp rx=0 tx=0).stop 2024-05-17T09:20:25.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 msgr2=0x7fa18819f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 0x7fa18819f560 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa18000ada0 tx=0x7fa18000c330 comp rx=0 tx=0).stop 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 shutdown_connections 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1700b2880 0x7fa1700b4d90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa188105470 0x7fa18819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa18810e8e0 0x7fa18819f560 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 --2- 172.21.15.19:0/4125515385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa188105dc0 0x7fa18819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 >> 172.21.15.19:0/4125515385 conn(0x7fa1880fef20 msgr2=0x7fa1880ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 shutdown_connections 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:25.593+0000 7fa18ca3b700 1 -- 172.21.15.19:0/4125515385 wait complete. 2024-05-17T09:20:25.594 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:20:26.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:26 smithi019 ceph-mon[31620]: pgmap v458: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-17T09:20:26.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:26 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4125515385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:26.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:20:26.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:26 smithi069 ceph-mon[35081]: pgmap v458: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-17T09:20:26.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:26 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4125515385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:26 smithi079 ceph-mon[35502]: pgmap v458: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-17T09:20:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:26 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4125515385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:26.539 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:28.291 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:28 smithi019 ceph-mon[31620]: pgmap v459: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-17T09:20:28.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.412+0000 7fa28810a700 1 -- 172.21.15.19:0/4291303046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa280108730 msgr2=0x7fa280110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.412+0000 7fa28810a700 1 --2- 172.21.15.19:0/4291303046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa280108730 0x7fa280110090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa278009920 tx=0x7fa278009c30 comp rx=0 tx=0).stop 2024-05-17T09:20:28.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 -- 172.21.15.19:0/4291303046 shutdown_connections 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 --2- 172.21.15.19:0/4291303046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2801034b0 0x7fa2801038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 --2- 172.21.15.19:0/4291303046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa280108730 0x7fa280110090 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 --2- 172.21.15.19:0/4291303046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa280107de0 0x7fa2801081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 -- 172.21.15.19:0/4291303046 >> 172.21.15.19:0/4291303046 conn(0x7fa2800fedb0 msgr2=0x7fa2801011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 -- 172.21.15.19:0/4291303046 shutdown_connections 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.413+0000 7fa28810a700 1 -- 172.21.15.19:0/4291303046 wait complete. 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 Processor -- start 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 -- start start 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 0x7fa28019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa280107de0 0x7fa28019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa280108730 0x7fa280199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa280113210 con 0x7fa2801034b0 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa280113090 con 0x7fa280107de0 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa28810a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa280113390 con 0x7fa280108730 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa285ea6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 0x7fa28019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa285ea6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 0x7fa28019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49012/0 (socket says 172.21.15.19:49012) 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa285ea6700 1 -- 172.21.15.19:0/950350706 learned_addr learned my addr 172.21.15.19:0/950350706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa2856a5700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa280107de0 0x7fa28019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa2866a7700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa280108730 0x7fa280199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.414+0000 7fa285ea6700 1 -- 172.21.15.19:0/950350706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa280108730 msgr2=0x7fa280199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa285ea6700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa280108730 0x7fa280199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa285ea6700 1 -- 172.21.15.19:0/950350706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa280107de0 msgr2=0x7fa28019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa285ea6700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa280107de0 0x7fa28019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa285ea6700 1 -- 172.21.15.19:0/950350706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28019a1d0 con 0x7fa2801034b0 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa2866a7700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa280108730 0x7fa280199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa285ea6700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 0x7fa28019f8b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0060b0 tx=0x7fa27c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa276ffd700 1 -- 172.21.15.19:0/950350706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c01a070 con 0x7fa2801034b0 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28019a4c0 con 0x7fa2801034b0 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa276ffd700 1 -- 172.21.15.19:0/950350706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa27c00dd60 con 0x7fa2801034b0 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa276ffd700 1 -- 172.21.15.19:0/950350706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0145e0 con 0x7fa2801034b0 2024-05-17T09:20:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.415+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a6760 con 0x7fa2801034b0 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.419+0000 7fa276ffd700 1 -- 172.21.15.19:0/950350706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa27c01d070 con 0x7fa2801034b0 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.419+0000 7fa276ffd700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa26c0b2880 0x7fa26c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 Processor -- start 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa2856a5700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa26c0b2880 0x7fa26c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa276ffd700 1 -- 172.21.15.19:0/950350706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa27c0e1e70 con 0x7fa2801034b0 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 -- start start 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa2856a5700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa26c0b2880 0x7fa26c0b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa280106900 tx=0x7fa270009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 0x55b98e09fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98df1fe40 0x55b98e0a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e0a05e0 0x55b98e0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b98dfccd70 con 0x55b98e00b430 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b98dfbc910 con 0x55b98e0a05e0 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b98dfc0950 con 0x55b98df1fe40 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa285ea6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 0x55b98e09fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa285ea6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 0x55b98e09fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49016/0 (socket says 172.21.15.19:49016) 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.420+0000 7fa285ea6700 1 -- 172.21.15.19:0/578979829 learned_addr learned my addr 172.21.15.19:0/578979829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa2856a5700 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98df1fe40 0x55b98e0a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa2866a7700 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e0a05e0 0x55b98e0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa285ea6700 1 -- 172.21.15.19:0/578979829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98df1fe40 msgr2=0x55b98e0a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa285ea6700 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98df1fe40 0x55b98e0a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa285ea6700 1 -- 172.21.15.19:0/578979829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e0a05e0 msgr2=0x55b98e0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa285ea6700 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e0a05e0 0x55b98e0aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa285ea6700 1 -- 172.21.15.19:0/578979829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98df65fb0 con 0x55b98e00b430 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa2866a7700 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e0a05e0 0x55b98e0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa285ea6700 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 0x55b98e09fb60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0df500 tx=0x7fa27c0a8950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d5a9316d59f835 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa26bfff700 1 -- 172.21.15.19:0/578979829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0c3070 con 0x55b98e00b430 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa26bfff700 1 -- 172.21.15.19:0/578979829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa27c0ad490 con 0x55b98e00b430 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.421+0000 7fa26bfff700 1 -- 172.21.15.19:0/578979829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0a95a0 con 0x55b98e00b430 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 -- 172.21.15.19:0/578979829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 msgr2=0x55b98e09fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 0x55b98e09fb60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0df500 tx=0x7fa27c0a8950 comp rx=0 tx=0).stop 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 -- 172.21.15.19:0/578979829 shutdown_connections 2024-05-17T09:20:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e0a05e0 0x55b98e0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e00b430 0x55b98e09fb60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/578979829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98df1fe40 0x55b98e0a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 -- 172.21.15.19:0/578979829 >> 172.21.15.19:0/578979829 conn(0x55b98df801c0 msgr2=0x55b98e09c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 -- 172.21.15.19:0/578979829 shutdown_connections 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 -- 172.21.15.19:0/578979829 wait complete. 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.422+0000 7fa296b1fb80 1 -- start start 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 Processor -- start 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 0x55b98e0ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e00b430 0x55b98e0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e0a05e0 0x55b98e0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b98dec2fa0 con 0x55b98df1fe40 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b98df24da0 con 0x55b98e00b430 2024-05-17T09:20:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b98e0af820 con 0x55b98e0a05e0 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa285ea6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 0x55b98e0ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa296b1fb80 10 client.0 _ll_register_callbacks cb 0x55b98df57950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa285ea6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 0x55b98e0ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49026/0 (socket says 172.21.15.19:49026) 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa285ea6700 1 -- 172.21.15.19:0/3620717062 learned_addr learned my addr 172.21.15.19:0/3620717062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa2866a7700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e0a05e0 0x55b98e0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.423+0000 7fa2856a5700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e00b430 0x55b98e0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa285ea6700 1 -- 172.21.15.19:0/3620717062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e0a05e0 msgr2=0x55b98e0acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa285ea6700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e0a05e0 0x55b98e0acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa285ea6700 1 -- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e00b430 msgr2=0x55b98e0aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa285ea6700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e00b430 0x55b98e0aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa285ea6700 1 -- 172.21.15.19:0/3620717062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98df0c8d0 con 0x55b98df1fe40 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa2856a5700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e00b430 0x55b98e0aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa285ea6700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 0x55b98e0ac520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0a8de0 tx=0x7fa27c015be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77496167084537f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa2697fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa2697fa700 1 -- 172.21.15.19:0/3620717062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0c7070 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa2697fa700 1 -- 172.21.15.19:0/3620717062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa27c0b1cb0 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27c0ad490 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa2697fa700 1 -- 172.21.15.19:0/3620717062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa27c0c3040 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa296b1fb80 10 client.15522 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b98df76760 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.424+0000 7fa2697fa700 1 -- 172.21.15.19:0/3620717062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa27c0ad490 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa2697fa700 1 -- 172.21.15.19:0/3620717062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa27c0a7db0 con 0x55b98df1fe40 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa296b1fb80 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa296b1fb80 10 client.15522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:20:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa296b1fb80 20 client.15522 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e0bc5f0 0x55b98e0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa296b1fb80 4 client.15522 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98e06aed0 con 0x55b98e0bc5f0 2024-05-17T09:20:28.431 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:28 smithi069 ceph-mon[35081]: pgmap v459: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-17T09:20:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.425+0000 7fa2856a5700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e0bc5f0 0x55b98e0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.426+0000 7fa2697fa700 10 client.15522 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.426+0000 7fa2856a5700 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e0bc5f0 0x55b98e0bea10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2a0bd6da36932b1 server_cookie=ebbb44e578671ec7 in_seq=0 out_seq=0 2024-05-17T09:20:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.426+0000 7fa2697fa700 1 -- 172.21.15.19:0/3620717062 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b98e06aed0 con 0x55b98e0bc5f0 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.426+0000 7fa2697fa700 10 client.15522 handle_command_reply: tid=0 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.426+0000 7fa296b1fb80 1 client.15522 shutdown 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 msgr2=0x55b98e0ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 0x55b98e0ac520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0a8de0 tx=0x7fa27c015be0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 shutdown_connections 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e0bc5f0 0x55b98e0bea10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e0a05e0 0x55b98e0acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98df1fe40 0x55b98e0ac520 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/3620717062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98e00b430 0x55b98e0aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa2697fa700 0 client.15522 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 >> 172.21.15.19:0/3620717062 conn(0x55b98df801c0 msgr2=0x55b98e093800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 shutdown_connections 2024-05-17T09:20:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 1 -- 172.21.15.19:0/3620717062 wait complete. 2024-05-17T09:20:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.427+0000 7fa296b1fb80 20 client.15522 trim_cache size 0 max 16384 2024-05-17T09:20:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.442+0000 7fa296b1fb80 1 Processor -- start 2024-05-17T09:20:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.442+0000 7fa296b1fb80 1 -- start start 2024-05-17T09:20:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e094230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801b0ae0 con 0x55b98e0a05e0 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801ae370 con 0x55b98df1fe40 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801ac1c0 con 0x55b98e00b430 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49040/0 (socket says 172.21.15.19:49040) 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 -- 172.21.15.19:0/1928237498 learned_addr learned my addr 172.21.15.19:0/1928237498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa285ea6700 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2856a5700 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e094230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 -- 172.21.15.19:0/1928237498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 msgr2=0x55b98e094230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e094230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 -- 172.21.15.19:0/1928237498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 msgr2=0x55b98e0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 -- 172.21.15.19:0/1928237498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98e0a0410 con 0x55b98e0a05e0 2024-05-17T09:20:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.443+0000 7fa2866a7700 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0b8ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa278009670 tx=0x7fa278005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b2561b9fa0c081 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa26b7fe700 1 -- 172.21.15.19:0/1928237498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27800f460 con 0x55b98e0a05e0 2024-05-17T09:20:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa26b7fe700 1 -- 172.21.15.19:0/1928237498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa278005040 con 0x55b98e0a05e0 2024-05-17T09:20:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa26b7fe700 1 -- 172.21.15.19:0/1928237498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278004a40 con 0x55b98e0a05e0 2024-05-17T09:20:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 -- 172.21.15.19:0/1928237498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 msgr2=0x55b98e0b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0b8ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa278009670 tx=0x7fa278005e80 comp rx=0 tx=0).stop 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 -- 172.21.15.19:0/1928237498 shutdown_connections 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0b8ac0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/1928237498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e094230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 -- 172.21.15.19:0/1928237498 >> 172.21.15.19:0/1928237498 conn(0x55b98df801c0 msgr2=0x55b98e0ac300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.444+0000 7fa296b1fb80 1 -- 172.21.15.19:0/1928237498 shutdown_connections 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.445+0000 7fa296b1fb80 1 -- 172.21.15.19:0/1928237498 wait complete. 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.445+0000 7fa296b1fb80 1 -- start start 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.445+0000 7fa296b1fb80 1 Processor -- start 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e0bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801ac830 con 0x55b98e0a05e0 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801adca0 con 0x55b98df1fe40 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801acf00 con 0x55b98e00b430 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0bf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49054/0 (socket says 172.21.15.19:49054) 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 -- 172.21.15.19:0/883051310 learned_addr learned my addr 172.21.15.19:0/883051310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2856a5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e0bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa296b1fb80 10 client.0 _ll_register_callbacks cb 0x55b98df57950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa285ea6700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 -- 172.21.15.19:0/883051310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 msgr2=0x55b98e0bf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e0bf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 -- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 msgr2=0x55b98e0c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 -- 172.21.15.19:0/883051310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98e09cc00 con 0x55b98e0a05e0 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2856a5700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e0bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2866a7700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0bf7e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa278005dc0 tx=0x7fa278005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9991eb850fb58c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2697fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.446+0000 7fa2697fa700 1 -- 172.21.15.19:0/883051310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278032030 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa2697fa700 1 -- 172.21.15.19:0/883051310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa27800f7b0 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa278005040 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa2697fa700 1 -- 172.21.15.19:0/883051310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa27803f050 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 10 client.15528 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b98e126960 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa2697fa700 1 -- 172.21.15.19:0/883051310 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa278005040 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa2697fa700 1 -- 172.21.15.19:0/883051310 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa27800b820 con 0x55b98e0a05e0 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 10 client.15528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 20 client.15528 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e126f60 0x55b98e129380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 4 client.15528 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.447+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b98dfcecc0 con 0x55b98e126f60 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa285ea6700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e126f60 0x55b98e129380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa2697fa700 10 client.15528 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa285ea6700 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e126f60 0x55b98e129380 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74d1430759cbf6bd server_cookie=3a2706f2c7d71a31 in_seq=0 out_seq=0 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa2697fa700 1 -- 172.21.15.19:0/883051310 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b98dfcecc0 con 0x55b98e126f60 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa2697fa700 10 client.15528 handle_command_reply: tid=0 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 client.15528 shutdown 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 msgr2=0x55b98e0bf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0bf7e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa278005dc0 tx=0x7fa278005df0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 shutdown_connections 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55b98e126f60 0x55b98e129380 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa2697fa700 0 client.15528 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b98e0a05e0 0x55b98e0bf7e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.448+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b98df1fe40 0x55b98e0c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa296b1fb80 1 --2- 172.21.15.19:0/883051310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b98e00b430 0x55b98e0bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 >> 172.21.15.19:0/883051310 conn(0x55b98df801c0 msgr2=0x55b98e093f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 shutdown_connections 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa296b1fb80 1 -- 172.21.15.19:0/883051310 wait complete. 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa296b1fb80 20 client.15528 trim_cache size 0 max 16384 2024-05-17T09:20:28.454 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:20:28.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa26c0b2880 msgr2=0x7fa26c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa26c0b2880 0x7fa26c0b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa280106900 tx=0x7fa270009310 comp rx=0 tx=0).stop 2024-05-17T09:20:28.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 msgr2=0x7fa28019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:28.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 0x7fa28019f8b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0060b0 tx=0x7fa27c007f90 comp rx=0 tx=0).stop 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 shutdown_connections 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa26c0b2880 0x7fa26c0b4d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa280108730 0x7fa280199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2801034b0 0x7fa28019f8b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 --2- 172.21.15.19:0/950350706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa280107de0 0x7fa28019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 >> 172.21.15.19:0/950350706 conn(0x7fa2800fedb0 msgr2=0x7fa2801011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 shutdown_connections 2024-05-17T09:20:28.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:28.449+0000 7fa28810a700 1 -- 172.21.15.19:0/950350706 wait complete. 2024-05-17T09:20:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:28 smithi079 ceph-mon[35502]: pgmap v459: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-17T09:20:29.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:20:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:30.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:30 smithi069 ceph-mon[35081]: pgmap v460: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-17T09:20:30.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:30 smithi019 ceph-mon[31620]: pgmap v460: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-17T09:20:30.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:30 smithi079 ceph-mon[35502]: pgmap v460: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-17T09:20:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.484+0000 7f8c5c302700 1 -- 172.21.15.19:0/893474841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54107de0 msgr2=0x7f8c541081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.484+0000 7f8c5c302700 1 --2- 172.21.15.19:0/893474841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54107de0 0x7f8c541081f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c480099c0 tx=0x7f8c48009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 -- 172.21.15.19:0/893474841 shutdown_connections 2024-05-17T09:20:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 --2- 172.21.15.19:0/893474841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c54108730 0x7f8c54110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 --2- 172.21.15.19:0/893474841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c541034b0 0x7f8c541038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 --2- 172.21.15.19:0/893474841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54107de0 0x7f8c541081f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 -- 172.21.15.19:0/893474841 >> 172.21.15.19:0/893474841 conn(0x7f8c540fedb0 msgr2=0x7f8c541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 -- 172.21.15.19:0/893474841 shutdown_connections 2024-05-17T09:20:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 -- 172.21.15.19:0/893474841 wait complete. 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 Processor -- start 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.485+0000 7f8c5c302700 1 -- start start 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5c302700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c541034b0 0x7f8c5419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5c302700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c54107de0 0x7f8c5419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5c302700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 0x7f8c54199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5c302700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541131f0 con 0x7f8c54108730 2024-05-17T09:20:31.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5c302700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113070 con 0x7f8c54107de0 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5c302700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113370 con 0x7f8c541034b0 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 0x7f8c54199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 0x7f8c54199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49076/0 (socket says 172.21.15.19:49076) 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 -- 172.21.15.19:0/2585741916 learned_addr learned my addr 172.21.15.19:0/2585741916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c541034b0 0x7f8c5419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5989d700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c54107de0 0x7f8c5419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 -- 172.21.15.19:0/2585741916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c541034b0 msgr2=0x7f8c5419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c541034b0 0x7f8c5419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 -- 172.21.15.19:0/2585741916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c54107de0 msgr2=0x7f8c5419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c54107de0 0x7f8c5419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 -- 172.21.15.19:0/2585741916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5419a100 con 0x7f8c54108730 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5989d700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c54107de0 0x7f8c5419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.486+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 0x7f8c54199a50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c007f10 tx=0x7f8c4c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.487+0000 7f8c477fe700 1 -- 172.21.15.19:0/2585741916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c018070 con 0x7f8c54108730 2024-05-17T09:20:31.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.487+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5419a3f0 con 0x7f8c54108730 2024-05-17T09:20:31.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.487+0000 7f8c477fe700 1 -- 172.21.15.19:0/2585741916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c4c004020 con 0x7f8c54108730 2024-05-17T09:20:31.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.487+0000 7f8c477fe700 1 -- 172.21.15.19:0/2585741916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0126c0 con 0x7f8c54108730 2024-05-17T09:20:31.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.487+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5406b600 con 0x7f8c54108730 2024-05-17T09:20:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.490+0000 7f8c477fe700 1 -- 172.21.15.19:0/2585741916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8c4c0044f0 con 0x7f8c54108730 2024-05-17T09:20:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.490+0000 7f8c477fe700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c400b2880 0x7f8c400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.490+0000 7f8c6ad17b80 1 Processor -- start 2024-05-17T09:20:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.490+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c400b2880 0x7f8c400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.490+0000 7f8c477fe700 1 -- 172.21.15.19:0/2585741916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8c4c0e47a0 con 0x7f8c54108730 2024-05-17T09:20:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 -- start start 2024-05-17T09:20:31.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5eb7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5eb8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5eb23c0 0x557ef5eb27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ef5de0c90 con 0x557ef5db13d0 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557ef5dd0830 con 0x557ef5d33e40 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557ef5dd4870 con 0x557ef5eb23c0 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c5a09e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5eb7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c5a09e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5eb7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49086/0 (socket says 172.21.15.19:49086) 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c5a09e700 1 -- 172.21.15.19:0/1889269793 learned_addr learned my addr 172.21.15.19:0/1889269793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c400b2880 0x7f8c400b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c5419ab90 tx=0x7f8c50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.491+0000 7f8c5989d700 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5eb8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5eb23c0 0x557ef5eb27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a09e700 1 -- 172.21.15.19:0/1889269793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5eb23c0 msgr2=0x557ef5eb27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5eb23c0 0x557ef5eb27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a09e700 1 -- 172.21.15.19:0/1889269793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 msgr2=0x557ef5eb8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5eb8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a09e700 1 -- 172.21.15.19:0/1889269793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef5d208d0 con 0x557ef5db13d0 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5989d700 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5eb8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5eb7fe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8c50011d10 tx=0x7f8c50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4035cf7d18bc3998 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c44d47700 1 -- 172.21.15.19:0/1889269793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000ebd0 con 0x557ef5db13d0 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c44d47700 1 -- 172.21.15.19:0/1889269793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c500044c0 con 0x557ef5db13d0 2024-05-17T09:20:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.492+0000 7f8c44d47700 1 -- 172.21.15.19:0/1889269793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50026470 con 0x557ef5db13d0 2024-05-17T09:20:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1889269793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 msgr2=0x557ef5eb7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5eb7fe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8c50011d10 tx=0x7f8c50014000 comp rx=0 tx=0).stop 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1889269793 shutdown_connections 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5eb23c0 0x557ef5eb27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5eb7fe0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1889269793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5eb8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1889269793 >> 172.21.15.19:0/1889269793 conn(0x557ef5d72c80 msgr2=0x557ef5eacee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1889269793 shutdown_connections 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.493+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1889269793 wait complete. 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 -- start start 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 Processor -- start 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5ebc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5eb23c0 0x557ef5ebc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ef5cd6fa0 con 0x557ef5db13d0 2024-05-17T09:20:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557ef5d727c0 con 0x557ef5eb23c0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557ef5eb3250 con 0x557ef5d33e40 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c6ad17b80 10 client.0 _ll_register_callbacks cb 0x557ef5d6b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5ebc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5ebc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49090/0 (socket says 172.21.15.19:49090) 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 -- 172.21.15.19:0/3464965166 learned_addr learned my addr 172.21.15.19:0/3464965166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5eb23c0 0x557ef5ebc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 -- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 msgr2=0x557ef5ebbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 -- 172.21.15.19:0/3464965166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5eb23c0 msgr2=0x557ef5ebc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5eb23c0 0x557ef5ebc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.494+0000 7f8c5989d700 1 -- 172.21.15.19:0/3464965166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef5d6f690 con 0x557ef5db13d0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c5989d700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5ebc2c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8c480099c0 tx=0x7f8c480057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea11fe8ede92e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c3dffb700 1 -- 172.21.15.19:0/3464965166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480039c0 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c3dffb700 1 -- 172.21.15.19:0/3464965166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8c48003b60 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c500044c0 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c3dffb700 1 -- 172.21.15.19:0/3464965166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c48030050 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c6ad17b80 10 client.15540 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557ef5ec9740 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c3dffb700 1 -- 172.21.15.19:0/3464965166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c500044c0 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.495+0000 7f8c3dffb700 1 -- 172.21.15.19:0/3464965166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8c4800f690 con 0x557ef5db13d0 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c6ad17b80 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c6ad17b80 10 client.15540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c6ad17b80 20 client.15540 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5ecbe20 0x557ef5ece240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c6ad17b80 4 client.15540 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ef5de2be0 con 0x557ef5ecbe20 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5ecbe20 0x557ef5ece240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c3dffb700 10 client.15540 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.496+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5ecbe20 0x557ef5ece240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbc576daf2e9cdd0 server_cookie=f048d3d0b974b460 in_seq=0 out_seq=0 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c3dffb700 1 -- 172.21.15.19:0/3464965166 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ef5de2be0 con 0x557ef5ecbe20 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c3dffb700 10 client.15540 handle_command_reply: tid=0 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 client.15540 shutdown 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 msgr2=0x557ef5ebc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5ebc2c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8c480099c0 tx=0x7f8c480057d0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 shutdown_connections 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5ecbe20 0x557ef5ece240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5eb23c0 0x557ef5ebc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c3dffb700 0 client.15540 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/3464965166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5db13d0 0x557ef5ebc2c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 >> 172.21.15.19:0/3464965166 conn(0x557ef5d72c80 msgr2=0x557ef5eacea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 shutdown_connections 2024-05-17T09:20:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/3464965166 wait complete. 2024-05-17T09:20:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.497+0000 7f8c6ad17b80 20 client.15540 trim_cache size 0 max 16384 2024-05-17T09:20:31.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 Processor -- start 2024-05-17T09:20:31.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 -- start start 2024-05-17T09:20:31.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5db13d0 0x557ef5ea6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ea74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b0830 con 0x557ef5eb23c0 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541ae0c0 con 0x557ef5db13d0 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541abf10 con 0x557ef5d33e40 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ea74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ea74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49098/0 (socket says 172.21.15.19:49098) 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 -- 172.21.15.19:0/774725215 learned_addr learned my addr 172.21.15.19:0/774725215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5989d700 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5db13d0 0x557ef5ea6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 -- 172.21.15.19:0/774725215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 msgr2=0x557ef5ebc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 -- 172.21.15.19:0/774725215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5db13d0 msgr2=0x557ef5ea6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5db13d0 0x557ef5ea6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a89f700 1 -- 172.21.15.19:0/774725215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef5ebb0a0 con 0x557ef5eb23c0 2024-05-17T09:20:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.513+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ea74a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0ab9a0 tx=0x7f8c4c0a71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3a594d7092e2a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c3ffff700 1 -- 172.21.15.19:0/774725215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0c0e60 con 0x557ef5eb23c0 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c3ffff700 1 -- 172.21.15.19:0/774725215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c4c0ac110 con 0x557ef5eb23c0 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c3ffff700 1 -- 172.21.15.19:0/774725215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c021040 con 0x557ef5eb23c0 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/774725215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 msgr2=0x557ef5ea74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ea74a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0ab9a0 tx=0x7f8c4c0a71a0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/774725215 shutdown_connections 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5d33e40 0x557ef5ebc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ea74a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/774725215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5db13d0 0x557ef5ea6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.514+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/774725215 >> 172.21.15.19:0/774725215 conn(0x557ef5d72c80 msgr2=0x557ef5eb6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:31.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/774725215 shutdown_connections 2024-05-17T09:20:31.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/774725215 wait complete. 2024-05-17T09:20:31.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 -- start start 2024-05-17T09:20:31.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 Processor -- start 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5ec4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5db13d0 0x557ef5ed2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ed3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541ac580 con 0x557ef5eb23c0 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541ad9f0 con 0x557ef5d33e40 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.515+0000 7f8c6ad17b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541acc50 con 0x557ef5db13d0 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c6ad17b80 10 client.0 _ll_register_callbacks cb 0x557ef5d6b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ed3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ed3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49110/0 (socket says 172.21.15.19:49110) 2024-05-17T09:20:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 -- 172.21.15.19:0/1867928954 learned_addr learned my addr 172.21.15.19:0/1867928954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5989d700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5db13d0 0x557ef5ed2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5ec4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 -- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5db13d0 msgr2=0x557ef5ed2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5db13d0 0x557ef5ed2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 -- 172.21.15.19:0/1867928954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 msgr2=0x557ef5ec4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5ec4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 -- 172.21.15.19:0/1867928954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef5ed3aa0 con 0x557ef5eb23c0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c5a89f700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ed3530 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c014040 tx=0x7f8c4c0afeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d730a3c897e25ead server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c3dffb700 1 -- 172.21.15.19:0/1867928954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c070070 con 0x557ef5eb23c0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c3dffb700 1 -- 172.21.15.19:0/1867928954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8c4c0abe80 con 0x557ef5eb23c0 2024-05-17T09:20:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.516+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c0ac110 con 0x557ef5eb23c0 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c3dffb700 1 -- 172.21.15.19:0/1867928954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c021040 con 0x557ef5eb23c0 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 10 client.15546 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557ef5f36500 con 0x557ef5eb23c0 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c3dffb700 1 -- 172.21.15.19:0/1867928954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c4c0ac110 con 0x557ef5eb23c0 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c3dffb700 1 -- 172.21.15.19:0/1867928954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8c4c0b43e0 con 0x557ef5eb23c0 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 10 client.15546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 20 client.15546 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5f36b50 0x557ef5f38f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 4 client.15546 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ef5ddc3c0 con 0x557ef5f36b50 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.517+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5f36b50 0x557ef5f38f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c5a09e700 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5f36b50 0x557ef5f38f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f5dba8ae78e6530 server_cookie=3f98c3de0e643e5a in_seq=0 out_seq=0 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c3dffb700 10 client.15546 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c3dffb700 1 -- 172.21.15.19:0/1867928954 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ef5ddc3c0 con 0x557ef5f36b50 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c3dffb700 10 client.15546 handle_command_reply: tid=0 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 client.15546 shutdown 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 msgr2=0x557ef5ed3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ed3530 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c014040 tx=0x7f8c4c0afeb0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 shutdown_connections 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x557ef5f36b50 0x557ef5f38f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ef5eb23c0 0x557ef5ed3530 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ef5d33e40 0x557ef5ec4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 --2- 172.21.15.19:0/1867928954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ef5db13d0 0x557ef5ed2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 >> 172.21.15.19:0/1867928954 conn(0x557ef5d72c80 msgr2=0x557ef5eb6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c3dffb700 0 client.15546 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 shutdown_connections 2024-05-17T09:20:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 1 -- 172.21.15.19:0/1867928954 wait complete. 2024-05-17T09:20:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.518+0000 7f8c6ad17b80 20 client.15546 trim_cache size 0 max 16384 2024-05-17T09:20:31.525 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c400b2880 msgr2=0x7f8c400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c400b2880 0x7f8c400b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c5419ab90 tx=0x7f8c50009260 comp rx=0 tx=0).stop 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 msgr2=0x7f8c54199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 0x7f8c54199a50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c007f10 tx=0x7f8c4c00f5c0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 shutdown_connections 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8c400b2880 0x7f8c400b4d90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8c541034b0 0x7f8c5419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54108730 0x7f8c54199a50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 --2- 172.21.15.19:0/2585741916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c54107de0 0x7f8c5419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 >> 172.21.15.19:0/2585741916 conn(0x7f8c540fedb0 msgr2=0x7f8c541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 shutdown_connections 2024-05-17T09:20:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:31.519+0000 7f8c5c302700 1 -- 172.21.15.19:0/2585741916 wait complete. 2024-05-17T09:20:32.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:20:32.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:32 smithi069 ceph-mon[35081]: pgmap v461: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-17T09:20:32.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:32 smithi019 ceph-mon[31620]: pgmap v461: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-17T09:20:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:32 smithi079 ceph-mon[35502]: pgmap v461: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-17T09:20:32.620 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:34.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 -- 172.21.15.19:0/3081714729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570104640 msgr2=0x7f4570104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 --2- 172.21.15.19:0/3081714729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570104640 0x7f4570104a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f456c0099c0 tx=0x7f456c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 -- 172.21.15.19:0/3081714729 shutdown_connections 2024-05-17T09:20:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 --2- 172.21.15.19:0/3081714729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4570109870 0x7f45701111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 --2- 172.21.15.19:0/3081714729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570104640 0x7f4570104a50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 --2- 172.21.15.19:0/3081714729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4570108f20 0x7f4570109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 -- 172.21.15.19:0/3081714729 >> 172.21.15.19:0/3081714729 conn(0x7f4570079ef0 msgr2=0x7f457007a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.355+0000 7f4577811700 1 -- 172.21.15.19:0/3081714729 shutdown_connections 2024-05-17T09:20:34.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:34 smithi019 ceph-mon[31620]: pgmap v462: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-17T09:20:34.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.356+0000 7f4577811700 1 -- 172.21.15.19:0/3081714729 wait complete. 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.356+0000 7f4577811700 1 Processor -- start 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.356+0000 7f4577811700 1 -- start start 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4577811700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4570104640 0x7f457019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4577811700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4570108f20 0x7f457019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4577811700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 0x7f457019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4577811700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4570114350 con 0x7f4570109870 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4577811700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45701141d0 con 0x7f4570108f20 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4577811700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f45701144d0 con 0x7f4570104640 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 0x7f457019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f45755ad700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4570104640 0x7f457019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 0x7f457019f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32958/0 (socket says 172.21.15.19:32958) 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 -- 172.21.15.19:0/875241243 learned_addr learned my addr 172.21.15.19:0/875241243 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4574dac700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4570108f20 0x7f457019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 -- 172.21.15.19:0/875241243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4570104640 msgr2=0x7f457019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4570104640 0x7f457019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 -- 172.21.15.19:0/875241243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4570108f20 msgr2=0x7f457019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4570108f20 0x7f457019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 -- 172.21.15.19:0/875241243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457019fb80 con 0x7f4570109870 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.357+0000 7f4575dae700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 0x7f457019f590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4568007d70 tx=0x7f456800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.358+0000 7f45667fc700 1 -- 172.21.15.19:0/875241243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568016020 con 0x7f4570109870 2024-05-17T09:20:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.358+0000 7f45667fc700 1 -- 172.21.15.19:0/875241243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f456800ac90 con 0x7f4570109870 2024-05-17T09:20:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.358+0000 7f45667fc700 1 -- 172.21.15.19:0/875241243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680115e0 con 0x7f4570109870 2024-05-17T09:20:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.358+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457019fe70 con 0x7f4570109870 2024-05-17T09:20:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.358+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a03b0 con 0x7f4570109870 2024-05-17T09:20:34.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.360+0000 7f45667fc700 1 -- 172.21.15.19:0/875241243 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4568019020 con 0x7f4570109870 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.361+0000 7f45667fc700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f455c0b2900 0x7f455c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.361+0000 7f45667fc700 1 -- 172.21.15.19:0/875241243 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f45680e2380 con 0x7f4570109870 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.361+0000 7f45755ad700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f455c0b2900 0x7f455c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 Processor -- start 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f45755ad700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f455c0b2900 0x7f455c0b4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f456c000c00 tx=0x7f456c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 -- start start 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 0x55b344676fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344677520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3446713c0 0x55b3446717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b34459fc90 con 0x55b3445703d0 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b34458f830 con 0x55b3444f2e40 2024-05-17T09:20:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4586226b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b344593870 con 0x55b3446713c0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f45755ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 0x55b344676fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f45755ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 0x55b344676fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32964/0 (socket says 172.21.15.19:32964) 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f45755ad700 1 -- 172.21.15.19:0/74696844 learned_addr learned my addr 172.21.15.19:0/74696844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4574dac700 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344677520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.362+0000 7f4575dae700 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3446713c0 0x55b3446717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f45755ad700 1 -- 172.21.15.19:0/74696844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3446713c0 msgr2=0x55b3446717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f45755ad700 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3446713c0 0x55b3446717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f45755ad700 1 -- 172.21.15.19:0/74696844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 msgr2=0x55b344677520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f45755ad700 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344677520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f45755ad700 1 -- 172.21.15.19:0/74696844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3444df8d0 con 0x55b3445703d0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f45755ad700 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 0x55b344676fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f456c015000 tx=0x7f456c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285ec95dac71ba3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f455b7fe700 1 -- 172.21.15.19:0/74696844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c005690 con 0x55b3445703d0 2024-05-17T09:20:34.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f455b7fe700 1 -- 172.21.15.19:0/74696844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f456c003b10 con 0x55b3445703d0 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.363+0000 7f455b7fe700 1 -- 172.21.15.19:0/74696844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c00fb00 con 0x55b3445703d0 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 -- 172.21.15.19:0/74696844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 msgr2=0x55b344676fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 0x55b344676fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f456c015000 tx=0x7f456c01e000 comp rx=0 tx=0).stop 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 -- 172.21.15.19:0/74696844 shutdown_connections 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3446713c0 0x55b3446717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3445703d0 0x55b344676fe0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 --2- 172.21.15.19:0/74696844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344677520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 -- 172.21.15.19:0/74696844 >> 172.21.15.19:0/74696844 conn(0x55b344531c80 msgr2=0x55b34466bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 -- 172.21.15.19:0/74696844 shutdown_connections 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 -- 172.21.15.19:0/74696844 wait complete. 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.364+0000 7f4586226b80 1 -- start start 2024-05-17T09:20:34.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 Processor -- start 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b34467ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b34467b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b34467b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b344495fa0 con 0x55b3446713c0 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b3445317c0 con 0x55b3445703d0 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b344672190 con 0x55b3444f2e40 2024-05-17T09:20:34.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4586226b80 10 client.0 _ll_register_callbacks cb 0x55b34452a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b34467b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b34467b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32972/0 (socket says 172.21.15.19:32972) 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4575dae700 1 -- 172.21.15.19:0/165623047 learned_addr learned my addr 172.21.15.19:0/165623047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f45755ad700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b34467ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.365+0000 7f4574dac700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b34467b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4575dae700 1 -- 172.21.15.19:0/165623047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 msgr2=0x55b34467ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4575dae700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b34467ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4575dae700 1 -- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 msgr2=0x55b34467b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4575dae700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b34467b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4575dae700 1 -- 172.21.15.19:0/165623047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34452e690 con 0x55b3446713c0 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f45755ad700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b34467ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4575dae700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b34467b700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f45680a6900 tx=0x7f45680a9bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308c9247f77dd5e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4558ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4558ff9700 1 -- 172.21.15.19:0/165623047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680a9d90 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456c003b10 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4558ff9700 1 -- 172.21.15.19:0/165623047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f45680910b0 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4558ff9700 1 -- 172.21.15.19:0/165623047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45680912b0 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4586226b80 10 client.15558 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b344688570 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.366+0000 7f4558ff9700 1 -- 172.21.15.19:0/165623047 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f456c003b10 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4558ff9700 1 -- 172.21.15.19:0/165623047 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f45680bec70 con 0x55b3446713c0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4586226b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4586226b80 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4586226b80 20 client.15558 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4586226b80 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b34468abb0 0x55b34468d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4586226b80 4 client.15558 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3445a1be0 con 0x55b34468abb0 2024-05-17T09:20:34.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f45755ad700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b34468abb0 0x55b34468d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f4558ff9700 10 client.15558 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.367+0000 7f45755ad700 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b34468abb0 0x55b34468d010 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f108a48f18bb59e server_cookie=4759aa46820e9baa in_seq=0 out_seq=0 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4558ff9700 1 -- 172.21.15.19:0/165623047 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b3445a1be0 con 0x55b34468abb0 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4558ff9700 10 client.15558 handle_command_reply: tid=0 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4586226b80 1 client.15558 shutdown 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 msgr2=0x55b34467b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4586226b80 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b34467b700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f45680a6900 tx=0x7f45680a9bd0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 shutdown_connections 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4586226b80 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b34467ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.368+0000 7f4586226b80 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b34468abb0 0x55b34468d010 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4586226b80 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b34467b700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4586226b80 1 --2- 172.21.15.19:0/165623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b34467b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4558ff9700 0 client.15558 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:34.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 >> 172.21.15.19:0/165623047 conn(0x55b344531c80 msgr2=0x55b34466bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 shutdown_connections 2024-05-17T09:20:34.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4586226b80 1 -- 172.21.15.19:0/165623047 wait complete. 2024-05-17T09:20:34.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.369+0000 7f4586226b80 20 client.15558 trim_cache size 0 max 16384 2024-05-17T09:20:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 Processor -- start 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 -- start start 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344686b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3445703d0 0x55b344675ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344676220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b2ae0 con 0x55b3446713c0 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45701b0370 con 0x55b3444f2e40 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.384+0000 7f4586226b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ae1c0 con 0x55b3445703d0 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344676220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344676220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32986/0 (socket says 172.21.15.19:32986) 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 -- 172.21.15.19:0/3743567677 learned_addr learned my addr 172.21.15.19:0/3743567677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f45755ad700 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344686b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4574dac700 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3445703d0 0x55b344675ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 -- 172.21.15.19:0/3743567677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3445703d0 msgr2=0x55b344675ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3445703d0 0x55b344675ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 -- 172.21.15.19:0/3743567677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 msgr2=0x55b344686b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344686b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 -- 172.21.15.19:0/3743567677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3446722d0 con 0x55b3446713c0 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f4575dae700 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344676220 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f45680a56c0 tx=0x7f45680a5bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bbcc952bfc09e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f455affd700 1 -- 172.21.15.19:0/3743567677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680a9460 con 0x55b3446713c0 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f455affd700 1 -- 172.21.15.19:0/3743567677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45680a9d90 con 0x55b3446713c0 2024-05-17T09:20:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.385+0000 7f455affd700 1 -- 172.21.15.19:0/3743567677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680a5270 con 0x55b3446713c0 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 -- 172.21.15.19:0/3743567677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 msgr2=0x55b344676220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344676220 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f45680a56c0 tx=0x7f45680a5bc0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 -- 172.21.15.19:0/3743567677 shutdown_connections 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344676220 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3444f2e40 0x55b344686b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 --2- 172.21.15.19:0/3743567677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3445703d0 0x55b344675ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 -- 172.21.15.19:0/3743567677 >> 172.21.15.19:0/3743567677 conn(0x55b344531c80 msgr2=0x55b344679570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 -- 172.21.15.19:0/3743567677 shutdown_connections 2024-05-17T09:20:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.386+0000 7f4586226b80 1 -- 172.21.15.19:0/3743567677 wait complete. 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 -- start start 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 Processor -- start 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b344692f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b344682d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344683370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ae830 con 0x55b3446713c0 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45701afca0 con 0x55b3445703d0 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f45701aef00 con 0x55b3444f2e40 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4586226b80 10 client.0 _ll_register_callbacks cb 0x55b34452a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344683370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.387+0000 7f4575dae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344683370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32994/0 (socket says 172.21.15.19:32994) 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 -- 172.21.15.19:0/2897542272 learned_addr learned my addr 172.21.15.19:0/2897542272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f45755ad700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b344692f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4574dac700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b344682d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 -- 172.21.15.19:0/2897542272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 msgr2=0x55b344692f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b344692f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 -- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 msgr2=0x55b344682d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b344682d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 -- 172.21.15.19:0/2897542272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3446917c0 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4575dae700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344683370 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4568007d40 tx=0x7f45680a50f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78948b40de68d9b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4558ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4558ff9700 1 -- 172.21.15.19:0/2897542272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680a9460 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4558ff9700 1 -- 172.21.15.19:0/2897542272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f45680910b0 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45680a9d90 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4558ff9700 1 -- 172.21.15.19:0/2897542272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45680ada40 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4586226b80 10 client.15564 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.388+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3446f4880 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4558ff9700 1 -- 172.21.15.19:0/2897542272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45680a9d90 con 0x55b3446713c0 2024-05-17T09:20:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4558ff9700 1 -- 172.21.15.19:0/2897542272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f45680beae0 con 0x55b3446713c0 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4586226b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4586226b80 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4586226b80 20 client.15564 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4586226b80 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b3446f4c90 0x55b3446f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4586226b80 4 client.15564 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b34459b3c0 con 0x55b3446f4c90 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.389+0000 7f45755ad700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b3446f4c90 0x55b3446f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f45755ad700 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b3446f4c90 0x55b3446f70b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec48943470d5792e server_cookie=99272fb152be0673 in_seq=0 out_seq=0 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4558ff9700 10 client.15564 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4558ff9700 1 -- 172.21.15.19:0/2897542272 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b34459b3c0 con 0x55b3446f4c90 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4558ff9700 10 client.15564 handle_command_reply: tid=0 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 client.15564 shutdown 2024-05-17T09:20:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 msgr2=0x55b344683370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344683370 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4568007d40 tx=0x7f45680a50f0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 shutdown_connections 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b3444f2e40 0x55b344692f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b3446f4c90 0x55b3446f70b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4558ff9700 0 client.15564 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b3446713c0 0x55b344683370 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 --2- 172.21.15.19:0/2897542272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3445703d0 0x55b344682d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 >> 172.21.15.19:0/2897542272 conn(0x55b344531c80 msgr2=0x55b344679500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 shutdown_connections 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 1 -- 172.21.15.19:0/2897542272 wait complete. 2024-05-17T09:20:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.390+0000 7f4586226b80 20 client.15564 trim_cache size 0 max 16384 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f455c0b2900 msgr2=0x7f455c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f455c0b2900 0x7f455c0b4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f456c000c00 tx=0x7f456c005fb0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 msgr2=0x7f457019f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 0x7f457019f590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4568007d70 tx=0x7f456800a340 comp rx=0 tx=0).stop 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 shutdown_connections 2024-05-17T09:20:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f455c0b2900 0x7f455c0b4e10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4570104640 0x7f457019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570109870 0x7f457019f590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 --2- 172.21.15.19:0/875241243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4570108f20 0x7f457019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 >> 172.21.15.19:0/875241243 conn(0x7f4570079ef0 msgr2=0x7f4570102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 shutdown_connections 2024-05-17T09:20:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:34.391+0000 7f4577811700 1 -- 172.21.15.19:0/875241243 wait complete. 2024-05-17T09:20:34.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:34 smithi069 ceph-mon[35081]: pgmap v462: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-17T09:20:34.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:34 smithi079 ceph-mon[35502]: pgmap v462: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-17T09:20:34.783 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:20:34.783 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.4 2024-05-17T09:20:34.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 -- 172.21.15.19:0/1201344384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc107090 msgr2=0x7fc5bc1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/1201344384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc107090 0x7fc5bc1094b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009900 tx=0x7fc5a402a410 comp rx=0 tx=0).stop 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 -- 172.21.15.19:0/1201344384 shutdown_connections 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/1201344384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/1201344384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc107090 0x7fc5bc1094b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/1201344384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5bc104730 0x7fc5bc106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 -- 172.21.15.19:0/1201344384 >> 172.21.15.19:0/1201344384 conn(0x7fc5bc0fdd30 msgr2=0x7fc5bc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 -- 172.21.15.19:0/1201344384 shutdown_connections 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.990+0000 7fc5c14e2640 1 -- 172.21.15.19:0/1201344384 wait complete. 2024-05-17T09:20:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 Processor -- start 2024-05-17T09:20:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 -- start start 2024-05-17T09:20:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5bc104730 0x7fc5bc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5bc107090 0x7fc5bc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc10f670 con 0x7fc5bc1099f0 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc10f4f0 con 0x7fc5bc107090 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5c14e2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc10f7f0 con 0x7fc5bc104730 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33020/0 (socket says 172.21.15.19:33020) 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3878101614 learned_addr learned my addr 172.21.15.19:0/3878101614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.991+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5bc107090 0x7fc5bc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5bc104730 0x7fc5bc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3878101614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5bc104730 msgr2=0x7fc5bc19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5bc104730 0x7fc5bc19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3878101614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5bc107090 msgr2=0x7fc5bc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5bc107090 0x7fc5bc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3878101614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc1a1d80 con 0x7fc5bc1099f0 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc1a1710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c970 tx=0x7fc5b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5b3fff640 1 -- 172.21.15.19:0/3878101614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4004230 con 0x7fc5bc1099f0 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc1a2070 con 0x7fc5bc1099f0 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5b3fff640 1 -- 172.21.15.19:0/3878101614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5b4007e40 con 0x7fc5bc1099f0 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5b3fff640 1 -- 172.21.15.19:0/3878101614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4003da0 con 0x7fc5bc1099f0 2024-05-17T09:20:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.992+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc1a2540 con 0x7fc5bc1099f0 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5b3fff640 1 -- 172.21.15.19:0/3878101614 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5b40079b0 con 0x7fc5bc1099f0 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 Processor -- start 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 -- start start 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9058d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb905fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8fc6c30 con 0x55fbb8c8a2f0 2024-05-17T09:20:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8fb67d0 con 0x55fbb8cb31b0 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.995+0000 7fc5c8392740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8fba810 con 0x55fbb8c924d0 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9058d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9058d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33032/0 (socket says 172.21.15.19:33032) 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 -- 172.21.15.19:0/1173585972 learned_addr learned my addr 172.21.15.19:0/1173585972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb905fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 -- 172.21.15.19:0/1173585972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 msgr2=0x55fbb905fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb905fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 -- 172.21.15.19:0/1173585972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 msgr2=0x55fbb90587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 -- 172.21.15.19:0/1173585972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb8ccb210 con 0x55fbb8c8a2f0 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5b3fff640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5a80b28a0 0x7fc5a80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5baffd640 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9058d30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac011470 tx=0x7fc5ac011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18100bf3aac1aade server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5a80b28a0 0x7fc5a80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5b3fff640 1 -- 172.21.15.19:0/3878101614 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5b40e2230 con 0x7fc5bc1099f0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.996+0000 7fc5b17fa640 1 -- 172.21.15.19:0/1173585972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac004d20 con 0x55fbb8c8a2f0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.997+0000 7fc5b17fa640 1 -- 172.21.15.19:0/1173585972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac004050 con 0x55fbb8c8a2f0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.997+0000 7fc5b17fa640 1 -- 172.21.15.19:0/1173585972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00f780 con 0x55fbb8c8a2f0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.997+0000 7fc5c8392740 1 -- 172.21.15.19:0/1173585972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 msgr2=0x55fbb9058d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.997+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5a80b28a0 0x7fc5a80b4db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005760 tx=0x7fc5a4005f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.997+0000 7fc5c8392740 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9058d30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac011470 tx=0x7fc5ac011930 comp rx=0 tx=0).stop 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- 172.21.15.19:0/1173585972 shutdown_connections 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb905fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9058d30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 --2- 172.21.15.19:0/1173585972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- 172.21.15.19:0/1173585972 >> 172.21.15.19:0/1173585972 conn(0x55fbb8ca7300 msgr2=0x55fbb904fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- 172.21.15.19:0/1173585972 shutdown_connections 2024-05-17T09:20:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- 172.21.15.19:0/1173585972 wait complete. 2024-05-17T09:20:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- start start 2024-05-17T09:20:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 Processor -- start 2024-05-17T09:20:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9062000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 0x55fbb9062540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8c9c790 con 0x55fbb8c924d0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8f72890 con 0x55fbb8cb31b0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fbb9060680 con 0x55fbb8c8a2f0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5c8392740 10 client.0 _ll_register_callbacks cb 0x55fbb8f75d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.998+0000 7fc5ba7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 0x55fbb9062540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 0x55fbb9062540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33034/0 (socket says 172.21.15.19:33034) 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 -- 172.21.15.19:0/2068025286 learned_addr learned my addr 172.21.15.19:0/2068025286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9062000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 -- 172.21.15.19:0/2068025286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 msgr2=0x55fbb9062000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9062000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 -- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 msgr2=0x55fbb90694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 -- 172.21.15.19:0/2068025286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb8d97620 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 0x55fbb9062540 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009900 tx=0x7fc5a4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e11692d79f2d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5a27fc640 1 -- 172.21.15.19:0/2068025286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4035040 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5a27fc640 1 -- 172.21.15.19:0/2068025286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5a4031700 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac004050 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5a27fc640 1 -- 172.21.15.19:0/2068025286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5a405d050 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5c8392740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:34.999+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbb906c440 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5a27fc640 1 -- 172.21.15.19:0/2068025286 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac004050 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5a27fc640 1 -- 172.21.15.19:0/2068025286 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc5a4047030 con 0x55fbb8c924d0 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5c8392740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5c8392740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5c8392740 20 client.15576 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8c71d80 0x55fbb906fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5c8392740 4 client.15576 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb8fc8b80 con 0x55fbb8c71d80 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.000+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8c71d80 0x55fbb906fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.001+0000 7fc5a27fc640 10 client.15576 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.001+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8c71d80 0x55fbb906fca0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d563cbaec016d8f6 server_cookie=54613509a23e8e65 in_seq=0 out_seq=0 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5a27fc640 1 -- 172.21.15.19:0/2068025286 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fbb8fc8b80 con 0x55fbb8c71d80 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5a27fc640 10 client.15576 handle_command_reply: tid=0 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 client.15576 shutdown 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 msgr2=0x55fbb9062540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 0x55fbb9062540 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009900 tx=0x7fc5a4031400 comp rx=0 tx=0).stop 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 shutdown_connections 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8c71d80 0x55fbb906fca0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8cb31b0 0x55fbb90694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5a27fc640 0 client.15576 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c924d0 0x55fbb9062540 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2068025286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9062000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 >> 172.21.15.19:0/2068025286 conn(0x55fbb8ca7300 msgr2=0x55fbb9052c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 shutdown_connections 2024-05-17T09:20:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 1 -- 172.21.15.19:0/2068025286 wait complete. 2024-05-17T09:20:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.002+0000 7fc5c8392740 20 client.15576 trim_cache size 0 max 16384 2024-05-17T09:20:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 Processor -- start 2024-05-17T09:20:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 -- start start 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9053240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9053780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb90693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1ad0c0 con 0x55fbb8c71d80 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1aa990 con 0x55fbb8c8a2f0 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5c8392740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1a87e0 con 0x55fbb8c924d0 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9053240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9053240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33036/0 (socket says 172.21.15.19:33036) 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5baffd640 1 -- 172.21.15.19:0/2580042497 learned_addr learned my addr 172.21.15.19:0/2580042497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.018+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9053780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb90693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5baffd640 1 -- 172.21.15.19:0/2580042497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 msgr2=0x55fbb90693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb90693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5baffd640 1 -- 172.21.15.19:0/2580042497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c8a2f0 msgr2=0x55fbb9053780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9053780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5baffd640 1 -- 172.21.15.19:0/2580042497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb9058e10 con 0x55fbb8c71d80 2024-05-17T09:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9053780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9053240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac006010 tx=0x7fc5ac009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65fae91ee009896 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5b0ff9640 1 -- 172.21.15.19:0/2580042497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac030020 con 0x55fbb8c71d80 2024-05-17T09:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5b0ff9640 1 -- 172.21.15.19:0/2580042497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac0046b0 con 0x55fbb8c71d80 2024-05-17T09:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.019+0000 7fc5b0ff9640 1 -- 172.21.15.19:0/2580042497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00f7a0 con 0x55fbb8c71d80 2024-05-17T09:20:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 -- 172.21.15.19:0/2580042497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 msgr2=0x55fbb9053240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9053240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac006010 tx=0x7fc5ac009500 comp rx=0 tx=0).stop 2024-05-17T09:20:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 -- 172.21.15.19:0/2580042497 shutdown_connections 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c924d0 0x55fbb90693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9053780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 --2- 172.21.15.19:0/2580042497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9053240 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 -- 172.21.15.19:0/2580042497 >> 172.21.15.19:0/2580042497 conn(0x55fbb8ca7300 msgr2=0x55fbb9052b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 -- 172.21.15.19:0/2580042497 shutdown_connections 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 -- 172.21.15.19:0/2580042497 wait complete. 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 -- start start 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.020+0000 7fc5c8392740 1 Processor -- start 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9066dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9067310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c924d0 0x55fbb9068240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1a8e50 con 0x55fbb8c71d80 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1aa2c0 con 0x55fbb8c924d0 2024-05-17T09:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1a9520 con 0x55fbb8c8a2f0 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5c8392740 10 client.0 _ll_register_callbacks cb 0x7fc5a4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9066dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9066dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33050/0 (socket says 172.21.15.19:33050) 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 -- 172.21.15.19:0/3530653257 learned_addr learned my addr 172.21.15.19:0/3530653257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c924d0 0x55fbb9068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 -- 172.21.15.19:0/3530653257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 msgr2=0x55fbb9067310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9067310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 -- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c924d0 msgr2=0x55fbb9068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c924d0 0x55fbb9068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 -- 172.21.15.19:0/3530653257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb9068800 con 0x55fbb8c71d80 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c924d0 0x55fbb9068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9066dd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac01b040 tx=0x7fc5ac009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ad6a04bf645122 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.021+0000 7fc5a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5a27fc640 1 -- 172.21.15.19:0/3530653257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac003a30 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac0046b0 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5a27fc640 1 -- 172.21.15.19:0/3530653257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5ac003bd0 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5a27fc640 1 -- 172.21.15.19:0/3530653257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5ac048050 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 10 client.15582 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbb90d8450 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5a27fc640 1 -- 172.21.15.19:0/3530653257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac0046b0 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5a27fc640 1 -- 172.21.15.19:0/3530653257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc5ac02c2f0 con 0x55fbb8c71d80 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 20 client.15582 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8cb31b0 0x55fbb90dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 4 client.15582 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbb8fc2360 con 0x55fbb8cb31b0 2024-05-17T09:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.022+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8cb31b0 0x55fbb90dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.023+0000 7fc5a27fc640 10 client.15582 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.023+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8cb31b0 0x55fbb90dace0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e4628edc76c531 server_cookie=51042d718e779102 in_seq=0 out_seq=0 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.026+0000 7fc5a27fc640 1 -- 172.21.15.19:0/3530653257 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fbb8fc2360 con 0x55fbb8cb31b0 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.026+0000 7fc5a27fc640 10 client.15582 handle_command_reply: tid=0 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.026+0000 7fc5c8392740 1 client.15582 shutdown 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 msgr2=0x55fbb9066dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9066dd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac01b040 tx=0x7fc5ac009c20 comp rx=0 tx=0).stop 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 shutdown_connections 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fbb8cb31b0 0x55fbb90dace0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbb8c924d0 0x55fbb9068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fbb8c8a2f0 0x55fbb9067310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5a27fc640 0 client.15582 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 --2- 172.21.15.19:0/3530653257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8c71d80 0x55fbb9066dd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 >> 172.21.15.19:0/3530653257 conn(0x55fbb8ca7300 msgr2=0x55fbb90532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 shutdown_connections 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 1 -- 172.21.15.19:0/3530653257 wait complete. 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c8392740 20 client.15582 trim_cache size 0 max 16384 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5a80b28a0 msgr2=0x7fc5a80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5a80b28a0 0x7fc5a80b4db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005760 tx=0x7fc5a4005f10 comp rx=0 tx=0).stop 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 msgr2=0x7fc5bc1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc1a1710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c970 tx=0x7fc5b400ce30 comp rx=0 tx=0).stop 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 shutdown_connections 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5a80b28a0 0x7fc5a80b4db0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc1099f0 0x7fc5bc1a1710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5bc107090 0x7fc5bc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 --2- 172.21.15.19:0/3878101614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5bc104730 0x7fc5bc19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.027+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 >> 172.21.15.19:0/3878101614 conn(0x7fc5bc0fdd30 msgr2=0x7fc5bc105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.028+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 shutdown_connections 2024-05-17T09:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:35.028+0000 7fc5c14e2640 1 -- 172.21.15.19:0/3878101614 wait complete. 2024-05-17T09:20:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:36.070 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:20:36.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:20:36.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:36 smithi019 ceph-mon[31620]: pgmap v463: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:20:36.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:36 smithi069 ceph-mon[35081]: pgmap v463: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:20:36.457 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:36 smithi079 ceph-mon[35502]: pgmap v463: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 -- 172.21.15.19:0/3220205984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344104640 msgr2=0x7fa344104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 --2- 172.21.15.19:0/3220205984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344104640 0x7fa344104a50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa3400099c0 tx=0x7fa340009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 -- 172.21.15.19:0/3220205984 shutdown_connections 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 --2- 172.21.15.19:0/3220205984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa344109870 0x7fa3441111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 --2- 172.21.15.19:0/3220205984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344104640 0x7fa344104a50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 --2- 172.21.15.19:0/3220205984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa344108f20 0x7fa344109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 -- 172.21.15.19:0/3220205984 >> 172.21.15.19:0/3220205984 conn(0x7fa344079ef0 msgr2=0x7fa34407a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 -- 172.21.15.19:0/3220205984 shutdown_connections 2024-05-17T09:20:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.175+0000 7fa34bf20700 1 -- 172.21.15.19:0/3220205984 wait complete. 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 Processor -- start 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 -- start start 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa344104640 0x7fa344075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 0x7fa344075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa344109870 0x7fa344076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa344114350 con 0x7fa344108f20 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441141d0 con 0x7fa344109870 2024-05-17T09:20:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa34bf20700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441144d0 con 0x7fa344104640 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa3494bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 0x7fa344075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa3494bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 0x7fa344075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33064/0 (socket says 172.21.15.19:33064) 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.176+0000 7fa3494bb700 1 -- 172.21.15.19:0/2433907272 learned_addr learned my addr 172.21.15.19:0/2433907272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa349cbc700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa344104640 0x7fa344075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa34a4bd700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa344109870 0x7fa344076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa3494bb700 1 -- 172.21.15.19:0/2433907272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa344104640 msgr2=0x7fa344075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa3494bb700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa344104640 0x7fa344075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa3494bb700 1 -- 172.21.15.19:0/2433907272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa344109870 msgr2=0x7fa344076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa3494bb700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa344109870 0x7fa344076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa3494bb700 1 -- 172.21.15.19:0/2433907272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3441a9100 con 0x7fa344108f20 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa3494bb700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 0x7fa344075ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa33400d980 tx=0x7fa33400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334017070 con 0x7fa344108f20 2024-05-17T09:20:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa33400fd20 con 0x7fa344108f20 2024-05-17T09:20:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33400e3b0 con 0x7fa344108f20 2024-05-17T09:20:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3441a93f0 con 0x7fa344108f20 2024-05-17T09:20:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.177+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a98e0 con 0x7fa344108f20 2024-05-17T09:20:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.178+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3440512b0 con 0x7fa344108f20 2024-05-17T09:20:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.183+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa33400f890 con 0x7fa344108f20 2024-05-17T09:20:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.184+0000 7fa33affd700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3300b2900 0x7fa3300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.184+0000 7fa349cbc700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3300b2900 0x7fa3300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.184+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa3340e1ff0 con 0x7fa344108f20 2024-05-17T09:20:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.185+0000 7fa349cbc700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3300b2900 0x7fa3300b4e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa340000c00 tx=0x7fa340005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.187+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3340a99c0 con 0x7fa344108f20 2024-05-17T09:20:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.457+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa344076fe0 con 0x7fa344108f20 2024-05-17T09:20:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.457+0000 7fa33affd700 1 -- 172.21.15.19:0/2433907272 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fa3340a92c0 con 0x7fa344108f20 2024-05-17T09:20:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:37.457 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:20:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3300b2900 msgr2=0x7fa3300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3300b2900 0x7fa3300b4e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa340000c00 tx=0x7fa340005fb0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 msgr2=0x7fa344075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 0x7fa344075ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa33400d980 tx=0x7fa33400dc90 comp rx=0 tx=0).stop 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 shutdown_connections 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3300b2900 0x7fa3300b4e10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa344104640 0x7fa344075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa344109870 0x7fa344076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 --2- 172.21.15.19:0/2433907272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa344108f20 0x7fa344075ff0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 >> 172.21.15.19:0/2433907272 conn(0x7fa344079ef0 msgr2=0x7fa344102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 shutdown_connections 2024-05-17T09:20:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:37.460+0000 7fa34bf20700 1 -- 172.21.15.19:0/2433907272 wait complete. 2024-05-17T09:20:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:20:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:38 smithi069 ceph-mon[35081]: pgmap v464: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:38.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:38 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2433907272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:38.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:38 smithi019 ceph-mon[31620]: pgmap v464: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:38.433 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:38 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2433907272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:38.472 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.4 --await-for 45.0' 2024-05-17T09:20:38.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.4 --await-for 45.0 2024-05-17T09:20:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:38 smithi079 ceph-mon[35502]: pgmap v464: 129 pgs: 129 active+clean; 667 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:38 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2433907272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:38.974 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 -- 172.21.15.19:0/4248215030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 msgr2=0x7f55a4108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 --2- 172.21.15.19:0/4248215030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a4108210 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f558c0099c0 tx=0x7f558c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 -- 172.21.15.19:0/4248215030 shutdown_connections 2024-05-17T09:20:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 --2- 172.21.15.19:0/4248215030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55a4108750 0x7f55a41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 --2- 172.21.15.19:0/4248215030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a41034d0 0x7f55a41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 --2- 172.21.15.19:0/4248215030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a4108210 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 -- 172.21.15.19:0/4248215030 >> 172.21.15.19:0/4248215030 conn(0x7f55a40fef20 msgr2=0x7f55a4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 -- 172.21.15.19:0/4248215030 shutdown_connections 2024-05-17T09:20:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.068+0000 7f55a9de1700 1 -- 172.21.15.19:0/4248215030 wait complete. 2024-05-17T09:20:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 Processor -- start 2024-05-17T09:20:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 -- start start 2024-05-17T09:20:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55a41034d0 0x7f55a419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a4108750 0x7f55a419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4113210 con 0x7f55a4107e00 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4113090 con 0x7f55a4108750 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a9de1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4113390 con 0x7f55a41034d0 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f559affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f559affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a419c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33076/0 (socket says 172.21.15.19:33076) 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f559affd700 1 -- 172.21.15.19:0/2641655680 learned_addr learned my addr 172.21.15.19:0/2641655680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a3fff700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a4108750 0x7f55a419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.069+0000 7f55a37fe700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55a41034d0 0x7f55a419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f559affd700 1 -- 172.21.15.19:0/2641655680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55a41034d0 msgr2=0x7f55a419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f559affd700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55a41034d0 0x7f55a419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f559affd700 1 -- 172.21.15.19:0/2641655680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a4108750 msgr2=0x7f55a419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f559affd700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a4108750 0x7f55a419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f559affd700 1 -- 172.21.15.19:0/2641655680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41a6e90 con 0x7f55a4107e00 2024-05-17T09:20:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f559affd700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a419c110 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f558c00b580 tx=0x7f558c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f55a17fa700 1 -- 172.21.15.19:0/2641655680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c027070 con 0x7f55a4107e00 2024-05-17T09:20:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f55a17fa700 1 -- 172.21.15.19:0/2641655680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f558c00fc00 con 0x7f55a4107e00 2024-05-17T09:20:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f55a17fa700 1 -- 172.21.15.19:0/2641655680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c022cb0 con 0x7f55a4107e00 2024-05-17T09:20:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41a7120 con 0x7f55a4107e00 2024-05-17T09:20:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.070+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41a75d0 con 0x7f55a4107e00 2024-05-17T09:20:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.073+0000 7f55a17fa700 1 -- 172.21.15.19:0/2641655680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f558c02a070 con 0x7f55a4107e00 2024-05-17T09:20:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.074+0000 7f55a17fa700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55900b2900 0x7f55900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.074+0000 7f55a17fa700 1 -- 172.21.15.19:0/2641655680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f558c0e33a0 con 0x7f55a4107e00 2024-05-17T09:20:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.074+0000 7f55a37fe700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55900b2900 0x7f55900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.075+0000 7f55b87f6b80 1 Processor -- start 2024-05-17T09:20:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.075+0000 7f55b87f6b80 1 -- start start 2024-05-17T09:20:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.075+0000 7f55a37fe700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55900b2900 0x7f55900b4e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f559400ac50 tx=0x7f559400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.075+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a0cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.075+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a07630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff03930df0 con 0x55ff03883ec0 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff03920990 con 0x55ff0396f4b0 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff039249d0 con 0x55ff03a07160 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f559affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a0cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a07630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f559affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33080/0 (socket says 172.21.15.19:33080) 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f559affd700 1 -- 172.21.15.19:0/750835442 learned_addr learned my addr 172.21.15.19:0/750835442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a3fff700 1 -- 172.21.15.19:0/750835442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 msgr2=0x55ff03a0cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a3fff700 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a0cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a3fff700 1 -- 172.21.15.19:0/750835442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 msgr2=0x55ff03a0d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a3fff700 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.076+0000 7f55a3fff700 1 -- 172.21.15.19:0/750835442 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff038ca030 con 0x55ff03a07160 2024-05-17T09:20:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.077+0000 7f55a3fff700 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a07630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55a406d110 tx=0x7f559c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39edb172e083e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.077+0000 7f5599ffb700 1 -- 172.21.15.19:0/750835442 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c004bb0 con 0x55ff03a07160 2024-05-17T09:20:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.077+0000 7f5599ffb700 1 -- 172.21.15.19:0/750835442 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f559c004d50 con 0x55ff03a07160 2024-05-17T09:20:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.077+0000 7f5599ffb700 1 -- 172.21.15.19:0/750835442 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c007ba0 con 0x55ff03a07160 2024-05-17T09:20:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 -- 172.21.15.19:0/750835442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 msgr2=0x55ff03a07630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a07630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55a406d110 tx=0x7f559c009ba0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 -- 172.21.15.19:0/750835442 shutdown_connections 2024-05-17T09:20:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a07630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a0cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/750835442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 -- 172.21.15.19:0/750835442 >> 172.21.15.19:0/750835442 conn(0x55ff038e4240 msgr2=0x55ff039fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 -- 172.21.15.19:0/750835442 shutdown_connections 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 -- 172.21.15.19:0/750835442 wait complete. 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.078+0000 7f55b87f6b80 1 -- start start 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 Processor -- start 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a10b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a11060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a115a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff03827020 con 0x55ff03883ec0 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff03888e20 con 0x55ff0396f4b0 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff03a13a30 con 0x55ff03a07160 2024-05-17T09:20:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55b87f6b80 10 client.0 _ll_register_callbacks cb 0x55ff038bb9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a10b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a10b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33096/0 (socket says 172.21.15.19:33096) 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55a37fe700 1 -- 172.21.15.19:0/622412870 learned_addr learned my addr 172.21.15.19:0/622412870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f55a3fff700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.079+0000 7f559affd700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a11060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55a37fe700 1 -- 172.21.15.19:0/622412870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 msgr2=0x55ff03a115a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55a37fe700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a115a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55a37fe700 1 -- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 msgr2=0x55ff03a11060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55a37fe700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a11060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55a37fe700 1 -- 172.21.15.19:0/622412870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff03870950 con 0x55ff03883ec0 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f559affd700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a11060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55a37fe700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a10b20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5594003930 tx=0x7f55940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776e4e6378015603 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f5582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f5582ffd700 1 -- 172.21.15.19:0/622412870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594016070 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f5582ffd700 1 -- 172.21.15.19:0/622412870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5594010840 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c004d50 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f5582ffd700 1 -- 172.21.15.19:0/622412870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5594010a40 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55b87f6b80 10 client.15600 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.080+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff038da7e0 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f5582ffd700 1 -- 172.21.15.19:0/622412870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f559c004d50 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f5582ffd700 1 -- 172.21.15.19:0/622412870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5594027c80 con 0x55ff03883ec0 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f55b87f6b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f55b87f6b80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f55b87f6b80 20 client.15600 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a209c0 0x55ff03a22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f55b87f6b80 4 client.15600 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff039cef50 con 0x55ff03a209c0 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.081+0000 7f559affd700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a209c0 0x55ff03a22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.082+0000 7f559affd700 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a209c0 0x55ff03a22e20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92538ef78e525bae server_cookie=ece1d505094ce790 in_seq=0 out_seq=0 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.082+0000 7f5582ffd700 10 client.15600 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.082+0000 7f5582ffd700 1 -- 172.21.15.19:0/622412870 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f558c084e50 con 0x55ff03a209c0 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.082+0000 7f5582ffd700 10 client.15600 handle_command_reply: tid=0 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 client.15600 shutdown 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 msgr2=0x55ff03a10b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a10b20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5594003930 tx=0x7f55940047c0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 shutdown_connections 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a209c0 0x55ff03a22e20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a10b20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/622412870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a11060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 >> 172.21.15.19:0/622412870 conn(0x55ff038e4240 msgr2=0x55ff039fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.083+0000 7f5582ffd700 0 client.15600 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.084+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 shutdown_connections 2024-05-17T09:20:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.084+0000 7f55b87f6b80 1 -- 172.21.15.19:0/622412870 wait complete. 2024-05-17T09:20:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.084+0000 7f55b87f6b80 20 client.15600 trim_cache size 0 max 16384 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.105+0000 7f55b87f6b80 1 Processor -- start 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 -- start start 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a12e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41b08f0 con 0x55ff03883ec0 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41ae180 con 0x55ff0396f4b0 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41abfd0 con 0x55ff03a07160 2024-05-17T09:20:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33104/0 (socket says 172.21.15.19:33104) 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 -- 172.21.15.19:0/755519422 learned_addr learned my addr 172.21.15.19:0/755519422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a3fff700 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a12e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f559affd700 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 -- 172.21.15.19:0/755519422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 msgr2=0x55ff03a12e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a12e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 -- 172.21.15.19:0/755519422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 msgr2=0x55ff03a06b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a06b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f55a37fe700 1 -- 172.21.15.19:0/755519422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff03a13370 con 0x55ff03883ec0 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.106+0000 7f559affd700 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a06b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.107+0000 7f55a37fe700 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0fa80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f559401e530 tx=0x7f559400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557aa221f66ba21b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.107+0000 7f55997fa700 1 -- 172.21.15.19:0/755519422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594010840 con 0x55ff03883ec0 2024-05-17T09:20:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.107+0000 7f55997fa700 1 -- 172.21.15.19:0/755519422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5594022070 con 0x55ff03883ec0 2024-05-17T09:20:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.107+0000 7f55997fa700 1 -- 172.21.15.19:0/755519422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940276d0 con 0x55ff03883ec0 2024-05-17T09:20:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 -- 172.21.15.19:0/755519422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 msgr2=0x55ff03a0fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0fa80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f559401e530 tx=0x7f559400fa30 comp rx=0 tx=0).stop 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 -- 172.21.15.19:0/755519422 shutdown_connections 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03a07160 0x55ff03a12e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03883ec0 0x55ff03a0fa80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/755519422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a06b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 -- 172.21.15.19:0/755519422 >> 172.21.15.19:0/755519422 conn(0x55ff038e4240 msgr2=0x55ff03a0a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 -- 172.21.15.19:0/755519422 shutdown_connections 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 -- 172.21.15.19:0/755519422 wait complete. 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 -- start start 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.108+0000 7f55b87f6b80 1 Processor -- start 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03883ec0 0x55ff03a18e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 0x55ff03a15c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41ac640 con 0x55ff03a07160 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41adab0 con 0x55ff0396f4b0 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41acd10 con 0x55ff03883ec0 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 0x55ff03a15c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55b87f6b80 10 client.0 _ll_register_callbacks cb 0x55ff038bb9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 0x55ff03a15c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33112/0 (socket says 172.21.15.19:33112) 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 -- 172.21.15.19:0/4171500939 learned_addr learned my addr 172.21.15.19:0/4171500939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a37fe700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03883ec0 0x55ff03a18e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f559affd700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 -- 172.21.15.19:0/4171500939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03883ec0 msgr2=0x55ff03a18e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03883ec0 0x55ff03a18e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 -- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 msgr2=0x55ff03a193c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a193c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.109+0000 7f55a3fff700 1 -- 172.21.15.19:0/4171500939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff03a162c0 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f55a3fff700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 0x55ff03a15c50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55a406d110 tx=0x7f559c008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69f6dbbdf3fb4dc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f5582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f5582ffd700 1 -- 172.21.15.19:0/4171500939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c005590 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f5582ffd700 1 -- 172.21.15.19:0/4171500939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f559c005730 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5594022070 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f5582ffd700 1 -- 172.21.15.19:0/4171500939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f559c005930 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f55b87f6b80 10 client.15606 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff03a8be00 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f5582ffd700 1 -- 172.21.15.19:0/4171500939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5594022070 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.110+0000 7f5582ffd700 1 -- 172.21.15.19:0/4171500939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f559c01f420 con 0x55ff03a07160 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55b87f6b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55b87f6b80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55b87f6b80 20 client.15606 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a8c3a0 0x55ff03a8e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55b87f6b80 4 client.15606 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ff03932d40 con 0x55ff03a8c3a0 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55a37fe700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a8c3a0 0x55ff03a8e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f55a37fe700 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a8c3a0 0x55ff03a8e7c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4570bb6fc2818a8e server_cookie=340f6e6df1f8ee7f in_seq=0 out_seq=0 2024-05-17T09:20:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.111+0000 7f5582ffd700 10 client.15606 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.113+0000 7f5582ffd700 1 -- 172.21.15.19:0/4171500939 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ff03932d40 con 0x55ff03a8c3a0 2024-05-17T09:20:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.113+0000 7f5582ffd700 10 client.15606 handle_command_reply: tid=0 2024-05-17T09:20:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.113+0000 7f55b87f6b80 1 client.15606 shutdown 2024-05-17T09:20:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.113+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 msgr2=0x55ff03a15c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.113+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 0x55ff03a15c50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55a406d110 tx=0x7f559c008f50 comp rx=0 tx=0).stop 2024-05-17T09:20:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 shutdown_connections 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff03883ec0 0x55ff03a18e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ff03a8c3a0 0x55ff03a8e7c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff03a07160 0x55ff03a15c50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 --2- 172.21.15.19:0/4171500939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff0396f4b0 0x55ff03a193c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 >> 172.21.15.19:0/4171500939 conn(0x55ff038e4240 msgr2=0x55ff03a068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f5582ffd700 0 client.15606 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 shutdown_connections 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 1 -- 172.21.15.19:0/4171500939 wait complete. 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55b87f6b80 20 client.15606 trim_cache size 0 max 16384 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:20:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "96e41653-629b-47e2-859e-ae2d388c3013", 2024-05-17T09:20:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:20:40.117 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55900b2900 msgr2=0x7f55900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55a9de1700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55900b2900 0x7f55900b4e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f559400ac50 tx=0x7f559400a380 comp rx=0 tx=0).stop 2024-05-17T09:20:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 msgr2=0x7f55a419c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.114+0000 7f55a9de1700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a419c110 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f558c00b580 tx=0x7f558c004790 comp rx=0 tx=0).stop 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 shutdown_connections 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55900b2900 0x7f55900b4e10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55a41034d0 0x7f55a419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a4108750 0x7f55a419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 --2- 172.21.15.19:0/2641655680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55a4107e00 0x7f55a419c110 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 >> 172.21.15.19:0/2641655680 conn(0x7f55a40fef20 msgr2=0x7f55a41096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 shutdown_connections 2024-05-17T09:20:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:40.115+0000 7f55a9de1700 1 -- 172.21.15.19:0/2641655680 wait complete. 2024-05-17T09:20:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:40 smithi079 ceph-mon[35502]: pgmap v465: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:40.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:40 smithi019 ceph-mon[31620]: pgmap v465: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:40 smithi069 ceph-mon[35081]: pgmap v465: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:41.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:41 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:20:41.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:41 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:20:41.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:41 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:20:41.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:41 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:20:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:41 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:20:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:41 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:20:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:41 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:20:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:41 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:20:41.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:41 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:20:41.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:41 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:20:41.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:41 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:20:41.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:41 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:20:41.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:20:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 -- 172.21.15.19:0/1375821814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 msgr2=0x7f08181038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 --2- 172.21.15.19:0/1375821814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f08181038c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f08080099c0 tx=0x7f0808009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 -- 172.21.15.19:0/1375821814 shutdown_connections 2024-05-17T09:20:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 --2- 172.21.15.19:0/1375821814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818108730 0x7f0818110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 --2- 172.21.15.19:0/1375821814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f08181038c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 --2- 172.21.15.19:0/1375821814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0818107de0 0x7f08181081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 -- 172.21.15.19:0/1375821814 >> 172.21.15.19:0/1375821814 conn(0x7f08180fedb0 msgr2=0x7f08181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:42.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 -- 172.21.15.19:0/1375821814 shutdown_connections 2024-05-17T09:20:42.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.027+0000 7f081debd700 1 -- 172.21.15.19:0/1375821814 wait complete. 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.028+0000 7f081debd700 1 Processor -- start 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 -- start start 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f081810a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818107de0 0x7f081810a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0818108730 0x7f081810abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0818113210 con 0x7f08181034b0 2024-05-17T09:20:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0818113090 con 0x7f0818107de0 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f081debd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0818113390 con 0x7f0818108730 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f08177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f081810a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f08177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f081810a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33128/0 (socket says 172.21.15.19:33128) 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f08177fe700 1 -- 172.21.15.19:0/1013506127 learned_addr learned my addr 172.21.15.19:0/1013506127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f0816ffd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818107de0 0x7f081810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.029+0000 7f0817fff700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0818108730 0x7f081810abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f08177fe700 1 -- 172.21.15.19:0/1013506127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0818108730 msgr2=0x7f081810abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f08177fe700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0818108730 0x7f081810abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f08177fe700 1 -- 172.21.15.19:0/1013506127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818107de0 msgr2=0x7f081810a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f08177fe700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818107de0 0x7f081810a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f08177fe700 1 -- 172.21.15.19:0/1013506127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a9170 con 0x7f08181034b0 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f0816ffd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818107de0 0x7f081810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f08177fe700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f081810a130 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0808000c00 tx=0x7f08080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f0814ff9700 1 -- 172.21.15.19:0/1013506127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808004830 con 0x7f08181034b0 2024-05-17T09:20:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f0814ff9700 1 -- 172.21.15.19:0/1013506127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f080800f7f0 con 0x7f08181034b0 2024-05-17T09:20:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f0814ff9700 1 -- 172.21.15.19:0/1013506127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080053a0 con 0x7f08181034b0 2024-05-17T09:20:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a9400 con 0x7f08181034b0 2024-05-17T09:20:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.030+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a97a0 con 0x7f08181034b0 2024-05-17T09:20:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.034+0000 7f0814ff9700 1 -- 172.21.15.19:0/1013506127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0808029060 con 0x7f08181034b0 2024-05-17T09:20:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.034+0000 7f0814ff9700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f08040b2900 0x7f08040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f0816ffd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f08040b2900 0x7f08040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 Processor -- start 2024-05-17T09:20:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 -- start start 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f0814ff9700 1 -- 172.21.15.19:0/1013506127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f08080e2f80 con 0x7f08181034b0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x560523610630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052360acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56052352ff00 con 0x56052356e5c0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56052351faa0 con 0x56052360a8c0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f08177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x560523610630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560523523ae0 con 0x560523482fd0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f0816ffd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f08040b2900 0x7f08040b4e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08181022c0 tx=0x7f080000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f08177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x560523610630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33132/0 (socket says 172.21.15.19:33132) 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f08177fe700 1 -- 172.21.15.19:0/3006788220 learned_addr learned my addr 172.21.15.19:0/3006788220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.035+0000 7f0816ffd700 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f0817fff700 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052360acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f08177fe700 1 -- 172.21.15.19:0/3006788220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 msgr2=0x560523610bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f08177fe700 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f08177fe700 1 -- 172.21.15.19:0/3006788220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 msgr2=0x56052360acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f08177fe700 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052360acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f08177fe700 1 -- 172.21.15.19:0/3006788220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605234c9140 con 0x56052356e5c0 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f0817fff700 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052360acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f08177fe700 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x560523610630 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f08080a9f80 tx=0x7f08080a9d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64658cdb7c70b92d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f080dffb700 1 -- 172.21.15.19:0/3006788220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080ae630 con 0x56052356e5c0 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f080dffb700 1 -- 172.21.15.19:0/3006788220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08080b3050 con 0x56052356e5c0 2024-05-17T09:20:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.036+0000 7f080dffb700 1 -- 172.21.15.19:0/3006788220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080aa740 con 0x56052356e5c0 2024-05-17T09:20:42.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3006788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 msgr2=0x560523610630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x560523610630 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f08080a9f80 tx=0x7f08080a9d90 comp rx=0 tx=0).stop 2024-05-17T09:20:42.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3006788220 shutdown_connections 2024-05-17T09:20:42.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052360acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x560523610630 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3006788220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3006788220 >> 172.21.15.19:0/3006788220 conn(0x5605234e3350 msgr2=0x5605235fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3006788220 shutdown_connections 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.037+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3006788220 wait complete. 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.038+0000 7f082c8d2b80 1 -- start start 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.038+0000 7f082c8d2b80 1 Processor -- start 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523619e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052356e5c0 0x56052361a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x560523614270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560523426130 con 0x56052360a8c0 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560523487f30 con 0x56052356e5c0 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56052360b490 con 0x560523482fd0 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f0817fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x560523614270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f0817fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x560523614270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33134/0 (socket says 172.21.15.19:33134) 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f0816ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052356e5c0 0x56052361a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f0817fff700 1 -- 172.21.15.19:0/3175085679 learned_addr learned my addr 172.21.15.19:0/3175085679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f082c8d2b80 10 client.0 _ll_register_callbacks cb 0x5605234baae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.039+0000 7f08177fe700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523619e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0816ffd700 1 -- 172.21.15.19:0/3175085679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 msgr2=0x560523619e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0816ffd700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523619e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0816ffd700 1 -- 172.21.15.19:0/3175085679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 msgr2=0x560523614270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0816ffd700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x560523614270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0816ffd700 1 -- 172.21.15.19:0/3175085679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052346fa60 con 0x56052356e5c0 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0817fff700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x560523614270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f0816ffd700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052356e5c0 0x56052361a3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f080000be70 tx=0x7f08000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc2780671a5dddc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f07f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f07f6ffd700 1 -- 172.21.15.19:0/3175085679 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080001f940 con 0x56052356e5c0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08080b3050 con 0x56052356e5c0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.040+0000 7f07f6ffd700 1 -- 172.21.15.19:0/3175085679 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f080001fae0 con 0x56052356e5c0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f07f6ffd700 1 -- 172.21.15.19:0/3175085679 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f080001fce0 con 0x56052356e5c0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 10 client.25325 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605234d98f0 con 0x56052356e5c0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f07f6ffd700 1 -- 172.21.15.19:0/3175085679 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08080b3050 con 0x56052356e5c0 2024-05-17T09:20:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f07f6ffd700 1 -- 172.21.15.19:0/3175085679 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0800026070 con 0x56052356e5c0 2024-05-17T09:20:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 10 client.25325 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 10 client.25325 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:20:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 20 client.25325 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560523624280 0x5605236266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 4 client.25325 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605235ce060 con 0x560523624280 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.041+0000 7f08177fe700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560523624280 0x5605236266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.042+0000 7f07f6ffd700 10 client.25325 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.042+0000 7f08177fe700 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560523624280 0x5605236266d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=973726dceed677f server_cookie=37f67745cc4808f0 in_seq=0 out_seq=0 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.043+0000 7f07f6ffd700 1 -- 172.21.15.19:0/3175085679 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5605235ce060 con 0x560523624280 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.043+0000 7f07f6ffd700 10 client.25325 handle_command_reply: tid=0 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.043+0000 7f082c8d2b80 1 client.25325 shutdown 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.043+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052356e5c0 msgr2=0x56052361a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.043+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052356e5c0 0x56052361a3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f080000be70 tx=0x7f08000047c0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 shutdown_connections 2024-05-17T09:20:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523619e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560523624280 0x5605236266d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x560523614270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/3175085679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052356e5c0 0x56052361a3b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 >> 172.21.15.19:0/3175085679 conn(0x5605234e3350 msgr2=0x5605236010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f07f6ffd700 0 client.25325 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 shutdown_connections 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 1 -- 172.21.15.19:0/3175085679 wait complete. 2024-05-17T09:20:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.044+0000 7f082c8d2b80 20 client.25325 trim_cache size 0 max 16384 2024-05-17T09:20:42.058 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 Processor -- start 2024-05-17T09:20:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 -- start start 2024-05-17T09:20:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x56052362bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052362c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b0910 con 0x56052356e5c0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f08181ae1a0 con 0x56052360a8c0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.071+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08181ac020 con 0x560523482fd0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x56052362bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x56052362bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33136/0 (socket says 172.21.15.19:33136) 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 -- 172.21.15.19:0/977555086 learned_addr learned my addr 172.21.15.19:0/977555086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0817fff700 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052362c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f08177fe700 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 -- 172.21.15.19:0/977555086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 msgr2=0x560523610d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 -- 172.21.15.19:0/977555086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 msgr2=0x56052362c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052362c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 -- 172.21.15.19:0/977555086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605235f6970 con 0x56052356e5c0 2024-05-17T09:20:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f0816ffd700 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x56052362bed0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0800027730 tx=0x7f08000279c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42703de5241f5df1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.072+0000 7f080d7fa700 1 -- 172.21.15.19:0/977555086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080001f240 con 0x56052356e5c0 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.073+0000 7f080d7fa700 1 -- 172.21.15.19:0/977555086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f080001f740 con 0x56052356e5c0 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.073+0000 7f080d7fa700 1 -- 172.21.15.19:0/977555086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000042a0 con 0x56052356e5c0 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.073+0000 7f082c8d2b80 1 -- 172.21.15.19:0/977555086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 msgr2=0x56052362bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.073+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x56052362bed0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0800027730 tx=0x7f08000279c0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.073+0000 7f082c8d2b80 1 -- 172.21.15.19:0/977555086 shutdown_connections 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.073+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560523482fd0 0x560523610d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56052360a8c0 0x56052362c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/977555086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052356e5c0 0x56052362bed0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 -- 172.21.15.19:0/977555086 >> 172.21.15.19:0/977555086 conn(0x5605234e3350 msgr2=0x56052360faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 -- 172.21.15.19:0/977555086 shutdown_connections 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 -- 172.21.15.19:0/977555086 wait complete. 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 -- start start 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.074+0000 7f082c8d2b80 1 Processor -- start 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560523482fd0 0x5605236205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56052356e5c0 0x56052361d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x56052361df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08181ac660 con 0x56052360a8c0 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f08181adad0 con 0x560523482fd0 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08181acd30 con 0x56052356e5c0 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f082c8d2b80 10 client.0 _ll_register_callbacks cb 0x56052361a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x56052361df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x56052361df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33146/0 (socket says 172.21.15.19:33146) 2024-05-17T09:20:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 -- 172.21.15.19:0/1699798522 learned_addr learned my addr 172.21.15.19:0/1699798522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f08177fe700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560523482fd0 0x5605236205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0816ffd700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56052356e5c0 0x56052361d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 -- 172.21.15.19:0/1699798522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56052356e5c0 msgr2=0x56052361d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56052356e5c0 0x56052361d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 -- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560523482fd0 msgr2=0x5605236205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560523482fd0 0x5605236205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.075+0000 7f0817fff700 1 -- 172.21.15.19:0/1699798522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052361e450 con 0x56052360a8c0 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f0817fff700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x56052361df10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0810007bf0 tx=0x7f0810007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59db28a99f2a21e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f07f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:42.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f07f6ffd700 1 -- 172.21.15.19:0/1699798522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810004360 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f07f6ffd700 1 -- 172.21.15.19:0/1699798522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0810004500 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f080001f740 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f07f6ffd700 1 -- 172.21.15.19:0/1699798522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f081002a050 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f082c8d2b80 10 client.15624 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56052368eaa0 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.076+0000 7f07f6ffd700 1 -- 172.21.15.19:0/1699798522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f080001f740 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f07f6ffd700 1 -- 172.21.15.19:0/1699798522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0810018420 con 0x56052360a8c0 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f082c8d2b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f082c8d2b80 10 client.15624 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f082c8d2b80 20 client.15624 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56052368f250 0x560523691670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:42.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f082c8d2b80 4 client.15624 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.4, "await_for": 45.0}] 2024-05-17T09:20:42.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.4, "await_for": 45.0}) -- 0x560523531e50 con 0x56052368f250 2024-05-17T09:20:42.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.077+0000 7f08177fe700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56052368f250 0x560523691670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:42.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.078+0000 7f07f6ffd700 10 client.15624 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:42.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.078+0000 7f08177fe700 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56052368f250 0x560523691670 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4661e14aee95cf79 server_cookie=d618f68e29659ccb in_seq=0 out_seq=0 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f07f6ffd700 1 -- 172.21.15.19:0/1699798522 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560523531e50 con 0x56052368f250 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f07f6ffd700 10 client.15624 handle_command_reply: tid=0 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 client.15624 shutdown 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 msgr2=0x56052361df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x56052361df10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0810007bf0 tx=0x7f0810007c20 comp rx=0 tx=0).stop 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 shutdown_connections 2024-05-17T09:20:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56052368f250 0x560523691670 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56052360a8c0 0x56052361df10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560523482fd0 0x5605236205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 --2- 172.21.15.19:0/1699798522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56052356e5c0 0x56052361d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.087+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 >> 172.21.15.19:0/1699798522 conn(0x5605234e3350 msgr2=0x56052360faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f07f6ffd700 0 client.15624 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 shutdown_connections 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f082c8d2b80 1 -- 172.21.15.19:0/1699798522 wait complete. 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f082c8d2b80 20 client.15624 trim_cache size 0 max 16384 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:42.088 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 2, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "f8acff7d": { 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 2, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.4, 2024-05-17T09:20:42.089 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:20:42.090 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f08040b2900 msgr2=0x7f08040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f08040b2900 0x7f08040b4e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08181022c0 tx=0x7f080000b410 comp rx=0 tx=0).stop 2024-05-17T09:20:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 msgr2=0x7f081810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f081810a130 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0808000c00 tx=0x7f08080047b0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 shutdown_connections 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f08040b2900 0x7f08040b4e10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0818108730 0x7f081810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08181034b0 0x7f081810a130 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 --2- 172.21.15.19:0/1013506127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0818107de0 0x7f081810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.088+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 >> 172.21.15.19:0/1013506127 conn(0x7f08180fedb0 msgr2=0x7f0818106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.089+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 shutdown_connections 2024-05-17T09:20:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:42.089+0000 7f081debd700 1 -- 172.21.15.19:0/1013506127 wait complete. 2024-05-17T09:20:42.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:42 smithi019 ceph-mon[31620]: pgmap v466: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:20:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:42 smithi079 ceph-mon[35502]: pgmap v466: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:20:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:42 smithi069 ceph-mon[35081]: pgmap v466: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:20:43.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:43 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:20:43.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:43 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:20:43.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:43 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 -- 172.21.15.19:0/1633948060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 msgr2=0x7ff34810e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 --2- 172.21.15.19:0/1633948060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34810e380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff330009990 tx=0x7ff330009ca0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 -- 172.21.15.19:0/1633948060 shutdown_connections 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 --2- 172.21.15.19:0/1633948060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff34810e910 0x7ff348110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 --2- 172.21.15.19:0/1633948060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481054e0 0x7ff3481058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 --2- 172.21.15.19:0/1633948060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34810e380 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 -- 172.21.15.19:0/1633948060 >> 172.21.15.19:0/1633948060 conn(0x7ff348078df0 msgr2=0x7ff348079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 -- 172.21.15.19:0/1633948060 shutdown_connections 2024-05-17T09:20:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.179+0000 7ff34cca2700 1 -- 172.21.15.19:0/1633948060 wait complete. 2024-05-17T09:20:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.180+0000 7ff34cca2700 1 Processor -- start 2024-05-17T09:20:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.180+0000 7ff34cca2700 1 -- start start 2024-05-17T09:20:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.180+0000 7ff34cca2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3481054e0 0x7ff34819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.180+0000 7ff34cca2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.180+0000 7ff34cca2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff34810e910 0x7ff34819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.180+0000 7ff34cca2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff348114320 con 0x7ff348105e30 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff34cca2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481141a0 con 0x7ff34810e910 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff34cca2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481144a0 con 0x7ff3481054e0 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47808/0 (socket says 172.21.15.19:47808) 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 -- 172.21.15.19:0/3372932662 learned_addr learned my addr 172.21.15.19:0/3372932662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff34659c700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3481054e0 0x7ff34819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff346d9d700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff34810e910 0x7ff34819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 -- 172.21.15.19:0/3372932662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3481054e0 msgr2=0x7ff34819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3481054e0 0x7ff34819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 -- 172.21.15.19:0/3372932662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff34810e910 msgr2=0x7ff34819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff34810e910 0x7ff34819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 -- 172.21.15.19:0/3372932662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34819fc30 con 0x7ff348105e30 2024-05-17T09:20:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff345d9b700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34819b350 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff3300057c0 tx=0x7ff3300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:44.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff33f7fe700 1 -- 172.21.15.19:0/3372932662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330027070 con 0x7ff348105e30 2024-05-17T09:20:44.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff33f7fe700 1 -- 172.21.15.19:0/3372932662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff33000fb00 con 0x7ff348105e30 2024-05-17T09:20:44.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34819fec0 con 0x7ff348105e30 2024-05-17T09:20:44.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.181+0000 7ff33f7fe700 1 -- 172.21.15.19:0/3372932662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330022b10 con 0x7ff348105e30 2024-05-17T09:20:44.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.182+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a03a0 con 0x7ff348105e30 2024-05-17T09:20:44.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.184+0000 7ff33f7fe700 1 -- 172.21.15.19:0/3372932662 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff33002a070 con 0x7ff348105e30 2024-05-17T09:20:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.185+0000 7ff33f7fe700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3340b2880 0x7ff3340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 Processor -- start 2024-05-17T09:20:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff34659c700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3340b2880 0x7ff3340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff33f7fe700 1 -- 172.21.15.19:0/3372932662 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff3300e36e0 con 0x7ff348105e30 2024-05-17T09:20:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 -- start start 2024-05-17T09:20:44.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff34659c700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3340b2880 0x7ff3340b4d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff33800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d524910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d524eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 0x561d9d51f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d9d44cd70 con 0x561d9d51ecd0 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561d9d43c910 con 0x561d9d48b430 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561d9d440950 con 0x561d9d39fe40 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff346d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 0x561d9d51f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff346d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 0x561d9d51f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47824/0 (socket says 172.21.15.19:47824) 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff346d9d700 1 -- 172.21.15.19:0/2311820125 learned_addr learned my addr 172.21.15.19:0/2311820125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.186+0000 7ff34659c700 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d524910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff345d9b700 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d524eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff346d9d700 1 -- 172.21.15.19:0/2311820125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 msgr2=0x561d9d524eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff346d9d700 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d524eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff346d9d700 1 -- 172.21.15.19:0/2311820125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 msgr2=0x561d9d524910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff346d9d700 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d524910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff346d9d700 1 -- 172.21.15.19:0/2311820125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9d3e5fb0 con 0x561d9d51ecd0 2024-05-17T09:20:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff346d9d700 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 0x561d9d51f1a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff340003fa0 tx=0x7ff340009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3608104bc08097 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff33cff9700 1 -- 172.21.15.19:0/2311820125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340004bb0 con 0x561d9d51ecd0 2024-05-17T09:20:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff33cff9700 1 -- 172.21.15.19:0/2311820125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3400048e0 con 0x561d9d51ecd0 2024-05-17T09:20:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.187+0000 7ff33cff9700 1 -- 172.21.15.19:0/2311820125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340007d20 con 0x561d9d51ecd0 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/2311820125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 msgr2=0x561d9d51f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 0x561d9d51f1a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff340003fa0 tx=0x7ff340009ba0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/2311820125 shutdown_connections 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d51ecd0 0x561d9d51f1a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d524910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/2311820125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d524eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/2311820125 >> 172.21.15.19:0/2311820125 conn(0x561d9d4001c0 msgr2=0x561d9d5197d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/2311820125 shutdown_connections 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.188+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/2311820125 wait complete. 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 -- start start 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 Processor -- start 2024-05-17T09:20:44.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d528570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d528ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d51ecd0 0x561d9d528ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d9d342fa0 con 0x561d9d48b430 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561d9d3a4da0 con 0x561d9d51ecd0 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561d9d52b5a0 con 0x561d9d39fe40 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff35b6b7b80 10 client.0 _ll_register_callbacks cb 0x561d9d3d7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d528ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d528ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47840/0 (socket says 172.21.15.19:47840) 2024-05-17T09:20:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff345d9b700 1 -- 172.21.15.19:0/254154121 learned_addr learned my addr 172.21.15.19:0/254154121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff34659c700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d528570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.189+0000 7ff346d9d700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d51ecd0 0x561d9d528ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff345d9b700 1 -- 172.21.15.19:0/254154121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 msgr2=0x561d9d528570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff345d9b700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d528570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff345d9b700 1 -- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d51ecd0 msgr2=0x561d9d528ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff345d9b700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d51ecd0 0x561d9d528ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff345d9b700 1 -- 172.21.15.19:0/254154121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9d38c8d0 con 0x561d9d48b430 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff34659c700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d528570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff346d9d700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d51ecd0 0x561d9d528ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff345d9b700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d528ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff3300aa6e0 tx=0x7ff3300a6240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57448d1ece090813 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff32dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff32dffb700 1 -- 172.21.15.19:0/254154121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300aa4f0 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff32dffb700 1 -- 172.21.15.19:0/254154121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff3300e33f0 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3400048e0 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff32dffb700 1 -- 172.21.15.19:0/254154121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3300ab0e0 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff35b6b7b80 10 client.15636 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d9d3f6760 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.190+0000 7ff32dffb700 1 -- 172.21.15.19:0/254154121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3400048e0 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff32dffb700 1 -- 172.21.15.19:0/254154121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff3300e2070 con 0x561d9d48b430 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff35b6b7b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff35b6b7b80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff35b6b7b80 20 client.15636 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5383c0 0x561d9d53a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff35b6b7b80 4 client.15636 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:44.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d9d4eaed0 con 0x561d9d5383c0 2024-05-17T09:20:44.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff34659c700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5383c0 0x561d9d53a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff34659c700 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5383c0 0x561d9d53a810 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3af4fa5e6ad5cc server_cookie=f60a8ebd061bc825 in_seq=0 out_seq=0 2024-05-17T09:20:44.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.191+0000 7ff32dffb700 10 client.15636 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff32dffb700 1 -- 172.21.15.19:0/254154121 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561d9d4eaed0 con 0x561d9d5383c0 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff32dffb700 10 client.15636 handle_command_reply: tid=0 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff35b6b7b80 1 client.15636 shutdown 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 msgr2=0x561d9d528ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d528ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff3300aa6e0 tx=0x7ff3300a6240 comp rx=0 tx=0).stop 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 shutdown_connections 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d39fe40 0x561d9d528570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5383c0 0x561d9d53a810 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.192+0000 7ff32dffb700 0 client.15636 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.193+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d51ecd0 0x561d9d528ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.193+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/254154121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d528ab0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.193+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 >> 172.21.15.19:0/254154121 conn(0x561d9d4001c0 msgr2=0x561d9d519790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.193+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 shutdown_connections 2024-05-17T09:20:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.193+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/254154121 wait complete. 2024-05-17T09:20:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.193+0000 7ff35b6b7b80 20 client.15636 trim_cache size 0 max 16384 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 Processor -- start 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 -- start start 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d39fe40 0x561d9d534600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d51e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d53fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b2b10 con 0x561d9d48b430 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b03a0 con 0x561d9d39fe40 2024-05-17T09:20:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481ae1f0 con 0x561d9d51ecd0 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d51e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d51e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47842/0 (socket says 172.21.15.19:47842) 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff345d9b700 1 -- 172.21.15.19:0/3986918732 learned_addr learned my addr 172.21.15.19:0/3986918732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff34659c700 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d39fe40 0x561d9d534600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.214+0000 7ff346d9d700 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d53fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff345d9b700 1 -- 172.21.15.19:0/3986918732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 msgr2=0x561d9d53fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff345d9b700 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d53fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff345d9b700 1 -- 172.21.15.19:0/3986918732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d39fe40 msgr2=0x561d9d534600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff345d9b700 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d39fe40 0x561d9d534600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff345d9b700 1 -- 172.21.15.19:0/3986918732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9d51a2a0 con 0x561d9d48b430 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff34659c700 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d39fe40 0x561d9d534600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff345d9b700 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d51e3d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff3300e39d0 tx=0x7ff330009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e0706b946a8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:44.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff32ffff700 1 -- 172.21.15.19:0/3986918732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300aa330 con 0x561d9d48b430 2024-05-17T09:20:44.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff32ffff700 1 -- 172.21.15.19:0/3986918732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3300aa4f0 con 0x561d9d48b430 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.215+0000 7ff32ffff700 1 -- 172.21.15.19:0/3986918732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300b3320 con 0x561d9d48b430 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/3986918732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 msgr2=0x561d9d51e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d51e3d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff3300e39d0 tx=0x7ff330009710 comp rx=0 tx=0).stop 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/3986918732 shutdown_connections 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d53fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d39fe40 0x561d9d534600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/3986918732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d48b430 0x561d9d51e3d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/3986918732 >> 172.21.15.19:0/3986918732 conn(0x561d9d4001c0 msgr2=0x561d9d51dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/3986918732 shutdown_connections 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.216+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/3986918732 wait complete. 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 -- start start 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 Processor -- start 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 0x561d9d53fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d540000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d540540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481ae860 con 0x561d9d39fe40 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481afcd0 con 0x561d9d48b430 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481aef30 con 0x561d9d51ecd0 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff35b6b7b80 10 client.0 _ll_register_callbacks cb 0x561d9d3d7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff34659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 0x561d9d53fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff34659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 0x561d9d53fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47852/0 (socket says 172.21.15.19:47852) 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff34659c700 1 -- 172.21.15.19:0/76991675 learned_addr learned my addr 172.21.15.19:0/76991675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff345d9b700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d540000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.217+0000 7ff346d9d700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d540540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff34659c700 1 -- 172.21.15.19:0/76991675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 msgr2=0x561d9d540540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff34659c700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d540540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff34659c700 1 -- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 msgr2=0x561d9d540000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff34659c700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d540000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff34659c700 1 -- 172.21.15.19:0/76991675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9d5a1ec0 con 0x561d9d39fe40 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff345d9b700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d540000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff34659c700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 0x561d9d53fac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff338005d90 tx=0x7ff3380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46dd7041cca9d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff32dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff32dffb700 1 -- 172.21.15.19:0/76991675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338011570 con 0x561d9d39fe40 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff32dffb700 1 -- 172.21.15.19:0/76991675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff338011710 con 0x561d9d39fe40 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3300aa4f0 con 0x561d9d39fe40 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff32dffb700 1 -- 172.21.15.19:0/76991675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff338011910 con 0x561d9d39fe40 2024-05-17T09:20:44.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff35b6b7b80 10 client.15642 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:44.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d9d5a30b0 con 0x561d9d39fe40 2024-05-17T09:20:44.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.218+0000 7ff32dffb700 1 -- 172.21.15.19:0/76991675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3300aa4f0 con 0x561d9d39fe40 2024-05-17T09:20:44.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff32dffb700 1 -- 172.21.15.19:0/76991675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff338033b50 con 0x561d9d39fe40 2024-05-17T09:20:44.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff35b6b7b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:44.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff35b6b7b80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:44.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff35b6b7b80 20 client.15642 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5a3760 0x561d9d5a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff35b6b7b80 4 client.15642 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:20:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d9d44ecc0 con 0x561d9d5a3760 2024-05-17T09:20:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff345d9b700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5a3760 0x561d9d5a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff32dffb700 10 client.15642 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.219+0000 7ff345d9b700 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5a3760 0x561d9d5a5b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5626a4422d3ae1b1 server_cookie=9dd02b442452fcb9 in_seq=0 out_seq=0 2024-05-17T09:20:44.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff32dffb700 1 -- 172.21.15.19:0/76991675 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561d9d44ecc0 con 0x561d9d5a3760 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff32dffb700 10 client.15642 handle_command_reply: tid=0 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 client.15642 shutdown 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 msgr2=0x561d9d53fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 0x561d9d53fac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff338005d90 tx=0x7ff3380047c0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 shutdown_connections 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d9d51ecd0 0x561d9d540540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561d9d5a3760 0x561d9d5a5b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d9d39fe40 0x561d9d53fac0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 --2- 172.21.15.19:0/76991675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d9d48b430 0x561d9d540000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 >> 172.21.15.19:0/76991675 conn(0x561d9d4001c0 msgr2=0x561d9d523cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff32dffb700 0 client.15642 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 shutdown_connections 2024-05-17T09:20:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 1 -- 172.21.15.19:0/76991675 wait complete. 2024-05-17T09:20:44.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.220+0000 7ff35b6b7b80 20 client.15642 trim_cache size 0 max 16384 2024-05-17T09:20:44.226 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:20:44.226 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:20:44.226 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:20:44.226 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3340b2880 msgr2=0x7ff3340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3340b2880 0x7ff3340b4d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff33800b410 comp rx=0 tx=0).stop 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 msgr2=0x7ff34819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34819b350 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff3300057c0 tx=0x7ff3300057f0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 shutdown_connections 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3340b2880 0x7ff3340b4d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3481054e0 0x7ff34819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff34810e910 0x7ff34819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 --2- 172.21.15.19:0/3372932662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff348105e30 0x7ff34819b350 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 >> 172.21.15.19:0/3372932662 conn(0x7ff348078df0 msgr2=0x7ff3481001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 shutdown_connections 2024-05-17T09:20:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:44.221+0000 7ff34cca2700 1 -- 172.21.15.19:0/3372932662 wait complete. 2024-05-17T09:20:44.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:44 smithi019 ceph-mon[31620]: pgmap v467: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:44 smithi079 ceph-mon[35502]: pgmap v467: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:44 smithi069 ceph-mon[35081]: pgmap v467: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:44.708 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f8acff7d', quiesce took 9.9 seconds. Will release after: 12.499999999999998 2024-05-17T09:20:45.475 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:20:45.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:20:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:46 smithi019 ceph-mon[31620]: pgmap v468: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:46 smithi079 ceph-mon[35502]: pgmap v468: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:46 smithi069 ceph-mon[35081]: pgmap v468: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 -- 172.21.15.19:0/1193025009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8108730 msgr2=0x7f66c8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 --2- 172.21.15.19:0/1193025009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8108730 0x7f66c8110090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b0d0 tx=0x7f66c000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 -- 172.21.15.19:0/1193025009 shutdown_connections 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 --2- 172.21.15.19:0/1193025009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66c81034b0 0x7f66c81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 --2- 172.21.15.19:0/1193025009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8108730 0x7f66c8110090 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 --2- 172.21.15.19:0/1193025009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8107de0 0x7f66c81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 -- 172.21.15.19:0/1193025009 >> 172.21.15.19:0/1193025009 conn(0x7f66c80fedb0 msgr2=0x7f66c81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 -- 172.21.15.19:0/1193025009 shutdown_connections 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.539+0000 7f66cefed700 1 -- 172.21.15.19:0/1193025009 wait complete. 2024-05-17T09:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 Processor -- start 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 -- start start 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66c81034b0 0x7f66c819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 0x7f66c819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8108730 0x7f66c819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8113210 con 0x7f66c8107de0 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8113090 con 0x7f66c8108730 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66cefed700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8113390 con 0x7f66c81034b0 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 0x7f66c819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.540+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 0x7f66c819b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47856/0 (socket says 172.21.15.19:47856) 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 -- 172.21.15.19:0/49263625 learned_addr learned my addr 172.21.15.19:0/49263625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66ccd89700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66c81034b0 0x7f66c819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66cd58a700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8108730 0x7f66c819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 -- 172.21.15.19:0/49263625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66c81034b0 msgr2=0x7f66c819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66c81034b0 0x7f66c819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 -- 172.21.15.19:0/49263625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8108730 msgr2=0x7f66c819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8108730 0x7f66c819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 -- 172.21.15.19:0/49263625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c819fc20 con 0x7f66c8107de0 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66cd58a700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8108730 0x7f66c819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c7fff700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 0x7f66c819b2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00d980 tx=0x7f66bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc017070 con 0x7f66c8107de0 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66bc009dc0 con 0x7f66c8107de0 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c819ff10 con 0x7f66c8107de0 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.541+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00f3c0 con 0x7f66c8107de0 2024-05-17T09:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.542+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c81a7830 con 0x7f66c8107de0 2024-05-17T09:20:47.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.542+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c80512b0 con 0x7f66c8107de0 2024-05-17T09:20:47.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.544+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f66bc009930 con 0x7f66c8107de0 2024-05-17T09:20:47.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.545+0000 7f66c5ffb700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66b00b2900 0x7f66b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:47.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.545+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f66bc0e1fd0 con 0x7f66c8107de0 2024-05-17T09:20:47.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.545+0000 7f66ccd89700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66b00b2900 0x7f66b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.545+0000 7f66ccd89700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66b00b2900 0x7f66b00b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005950 tx=0x7f66b800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:47.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.548+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66bc0a99c0 con 0x7f66c8107de0 2024-05-17T09:20:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.811+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66c802e5e0 con 0x7f66c8107de0 2024-05-17T09:20:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.812+0000 7f66c5ffb700 1 -- 172.21.15.19:0/49263625 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7f66bc0a92c0 con 0x7f66c8107de0 2024-05-17T09:20:47.811 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:47.812 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66b00b2900 msgr2=0x7f66b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66b00b2900 0x7f66b00b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005950 tx=0x7f66b800a300 comp rx=0 tx=0).stop 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 msgr2=0x7f66c819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 0x7f66c819b2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00d980 tx=0x7f66bc00dc90 comp rx=0 tx=0).stop 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 shutdown_connections 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f66b00b2900 0x7f66b00b4e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66c81034b0 0x7f66c819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c8108730 0x7f66c819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.814+0000 7f66cefed700 1 --2- 172.21.15.19:0/49263625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66c8107de0 0x7f66c819b2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.815+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 >> 172.21.15.19:0/49263625 conn(0x7f66c80fedb0 msgr2=0x7f66c806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.815+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 shutdown_connections 2024-05-17T09:20:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:47.815+0000 7f66cefed700 1 -- 172.21.15.19:0/49263625 wait complete. 2024-05-17T09:20:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:20:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 -- 172.21.15.19:0/1506044405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc103c50 msgr2=0x7fcddc104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/1506044405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc103c50 0x7fcddc104950 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009900 tx=0x7fcdd802a410 comp rx=0 tx=0).stop 2024-05-17T09:20:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 -- 172.21.15.19:0/1506044405 shutdown_connections 2024-05-17T09:20:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/1506044405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc109b90 0x7fcddc10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/1506044405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcddc104e90 0x7fcddc1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/1506044405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc103c50 0x7fcddc104950 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 -- 172.21.15.19:0/1506044405 >> 172.21.15.19:0/1506044405 conn(0x7fcddc0fee60 msgr2=0x7fcddc1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 -- 172.21.15.19:0/1506044405 shutdown_connections 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 -- 172.21.15.19:0/1506044405 wait complete. 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 Processor -- start 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.131+0000 7fcde3d6b640 1 -- start start 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde3d6b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc103c50 0x7fcddc19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde3d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 0x7fcddc19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde3d6b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcddc109b90 0x7fcddc1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde3d6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc10f630 con 0x7fcddc104e90 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde3d6b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc10f4b0 con 0x7fcddc109b90 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde3d6b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc10f7b0 con 0x7fcddc103c50 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 0x7fcddc19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 0x7fcddc19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47870/0 (socket says 172.21.15.19:47870) 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 -- 172.21.15.19:0/2883068691 learned_addr learned my addr 172.21.15.19:0/2883068691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1b07640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc103c50 0x7fcddc19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde2308640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcddc109b90 0x7fcddc1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 -- 172.21.15.19:0/2883068691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc103c50 msgr2=0x7fcddc19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc103c50 0x7fcddc19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 -- 172.21.15.19:0/2883068691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcddc109b90 msgr2=0x7fcddc1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcddc109b90 0x7fcddc1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 -- 172.21.15.19:0/2883068691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc1a1cf0 con 0x7fcddc104e90 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcde1306640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 0x7fcddc19a740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.132+0000 7fcdd2ffd640 1 -- 172.21.15.19:0/2883068691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00cd80 con 0x7fcddc104e90 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.133+0000 7fcdd2ffd640 1 -- 172.21.15.19:0/2883068691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdcc0108c0 con 0x7fcddc104e90 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.133+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc1a1fe0 con 0x7fcddc104e90 2024-05-17T09:20:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.133+0000 7fcdd2ffd640 1 -- 172.21.15.19:0/2883068691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0187c0 con 0x7fcddc104e90 2024-05-17T09:20:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.133+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1a24a0 con 0x7fcddc104e90 2024-05-17T09:20:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 Processor -- start 2024-05-17T09:20:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 -- start start 2024-05-17T09:20:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d1147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d114d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd4e4d0 0x555d5d11bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d5d082c30 con 0x555d5cd4e4d0 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555d5d0727d0 con 0x555d5cd6f1b0 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555d5d076810 con 0x555d5cd462f0 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcde2308640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d1147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcde2308640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d1147b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:57460/0 (socket says 172.21.15.19:57460) 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.136+0000 7fcde2308640 1 -- 172.21.15.19:0/1135807898 learned_addr learned my addr 172.21.15.19:0/1135807898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde1b07640 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d114d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde2308640 1 -- 172.21.15.19:0/1135807898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 msgr2=0x555d5d114d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde2308640 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d114d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde2308640 1 -- 172.21.15.19:0/1135807898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd4e4d0 msgr2=0x555d5d11bc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde2308640 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd4e4d0 0x555d5d11bc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde2308640 1 -- 172.21.15.19:0/1135807898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5cd87210 con 0x555d5cd6f1b0 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde1b07640 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcdd2ffd640 1 -- 172.21.15.19:0/2883068691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcdcc010430 con 0x7fcddc104e90 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcde2308640 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d1147b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009df0 tx=0x7fcdd400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c8cbe00a60427a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcdc7fff640 1 -- 172.21.15.19:0/1135807898 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4003970 con 0x555d5cd6f1b0 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcdc7fff640 1 -- 172.21.15.19:0/1135807898 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd4004030 con 0x555d5cd6f1b0 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.137+0000 7fcdc7fff640 1 -- 172.21.15.19:0/1135807898 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40116a0 con 0x555d5cd6f1b0 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdd2ffd640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc80b2920 0x7fcdc80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcde1b07640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc80b2920 0x7fcdc80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdd2ffd640 1 -- 172.21.15.19:0/2883068691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcdcc0e2940 con 0x7fcddc104e90 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 -- 172.21.15.19:0/1135807898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 msgr2=0x555d5d1147b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d1147b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009df0 tx=0x7fcdd400f730 comp rx=0 tx=0).stop 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 -- 172.21.15.19:0/1135807898 shutdown_connections 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd4e4d0 0x555d5d11bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/1135807898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d1147b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.138+0000 7fcdeac1e740 1 -- 172.21.15.19:0/1135807898 >> 172.21.15.19:0/1135807898 conn(0x555d5cd63300 msgr2=0x555d5d10bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.139+0000 7fcdeac1e740 1 -- 172.21.15.19:0/1135807898 shutdown_connections 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.139+0000 7fcdeac1e740 1 -- 172.21.15.19:0/1135807898 wait complete. 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.139+0000 7fcde1b07640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc80b2920 0x7fcdc80b4e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcdd802a930 tx=0x7fcdd8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.139+0000 7fcdeac1e740 1 -- start start 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.139+0000 7fcdeac1e740 1 Processor -- start 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d11e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd4e4d0 0x555d5d11e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d125560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d5cd58790 con 0x555d5cd462f0 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555d5d02e890 con 0x555d5cd6f1b0 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555d5d11c680 con 0x555d5cd4e4d0 2024-05-17T09:20:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcdeac1e740 10 client.0 _ll_register_callbacks cb 0x555d5d031d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d11e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1306640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd4e4d0 0x555d5d11e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d11e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47886/0 (socket says 172.21.15.19:47886) 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1b07640 1 -- 172.21.15.19:0/303823683 learned_addr learned my addr 172.21.15.19:0/303823683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde2308640 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d125560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1306640 1 -- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 msgr2=0x555d5d125560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1306640 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d125560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1306640 1 -- 172.21.15.19:0/303823683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 msgr2=0x555d5d11e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1306640 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d11e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.140+0000 7fcde1306640 1 -- 172.21.15.19:0/303823683 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5ce53620 con 0x555d5cd4e4d0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcde1306640 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd4e4d0 0x555d5d11e600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0e0400 tx=0x7fcdcc010810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca735c760dac7c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdc57fa640 1 -- 172.21.15.19:0/303823683 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0cae60 con 0x555d5cd4e4d0 2024-05-17T09:20:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdc57fa640 1 -- 172.21.15.19:0/303823683 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcdcc0a5e40 con 0x555d5cd4e4d0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd4004030 con 0x555d5cd4e4d0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdc57fa640 1 -- 172.21.15.19:0/303823683 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc0a9bb0 con 0x555d5cd4e4d0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 10 client.25396 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d5d128500 con 0x555d5cd4e4d0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdc57fa640 1 -- 172.21.15.19:0/303823683 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd4004030 con 0x555d5cd4e4d0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdc57fa640 1 -- 172.21.15.19:0/303823683 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcdcc0a5520 con 0x555d5cd4e4d0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 10 client.25396 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 20 client.25396 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd2dd80 0x555d5d12bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 4 client.25396 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.141+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d5d084b80 con 0x555d5cd2dd80 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.142+0000 7fcde1b07640 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd2dd80 0x555d5d12bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.142+0000 7fcdc57fa640 10 client.25396 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.142+0000 7fcde1b07640 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd2dd80 0x555d5d12bc00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d69f462d8bd96e server_cookie=6f82557bed6ce478 in_seq=0 out_seq=0 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdc57fa640 1 -- 172.21.15.19:0/303823683 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555d5d084b80 con 0x555d5cd2dd80 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdc57fa640 10 client.25396 handle_command_reply: tid=0 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 client.25396 shutdown 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd4e4d0 msgr2=0x555d5d11e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd4e4d0 0x555d5d11e600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0e0400 tx=0x7fcdcc010810 comp rx=0 tx=0).stop 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 shutdown_connections 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd2dd80 0x555d5d12bc00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd6f1b0 0x555d5d125560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdc57fa640 0 client.25396 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd4e4d0 0x555d5d11e600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/303823683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d11e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 >> 172.21.15.19:0/303823683 conn(0x555d5cd63300 msgr2=0x555d5d10ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 shutdown_connections 2024-05-17T09:20:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 1 -- 172.21.15.19:0/303823683 wait complete. 2024-05-17T09:20:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.143+0000 7fcdeac1e740 20 client.25396 trim_cache size 0 max 16384 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 Processor -- start 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 -- start start 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd63300 0x555d5d10be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d125480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d10a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1ad120 con 0x555d5cd462f0 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1aa9f0 con 0x555d5cd2dd80 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1a8840 con 0x555d5cd63300 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcde2308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d10a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcde2308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d10a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47894/0 (socket says 172.21.15.19:47894) 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.159+0000 7fcde2308640 1 -- 172.21.15.19:0/2374408316 learned_addr learned my addr 172.21.15.19:0/2374408316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde1b07640 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd63300 0x555d5d10be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde1306640 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d125480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde2308640 1 -- 172.21.15.19:0/2374408316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd63300 msgr2=0x555d5d10be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde2308640 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd63300 0x555d5d10be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde2308640 1 -- 172.21.15.19:0/2374408316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 msgr2=0x555d5d125480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde2308640 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d125480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde2308640 1 -- 172.21.15.19:0/2374408316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5d10a870 con 0x555d5cd462f0 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde1b07640 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd63300 0x555d5d10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcde2308640 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d10a330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x555d5d115980 tx=0x7fcdd4009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30296ea69f99e454 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcdc77fe640 1 -- 172.21.15.19:0/2374408316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd402b070 con 0x555d5cd462f0 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcdc77fe640 1 -- 172.21.15.19:0/2374408316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd4005040 con 0x555d5cd462f0 2024-05-17T09:20:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.160+0000 7fcdc77fe640 1 -- 172.21.15.19:0/2374408316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40266c0 con 0x555d5cd462f0 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 -- 172.21.15.19:0/2374408316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 msgr2=0x555d5d10a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d10a330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x555d5d115980 tx=0x7fcdd4009b40 comp rx=0 tx=0).stop 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 -- 172.21.15.19:0/2374408316 shutdown_connections 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd462f0 0x555d5d10a330 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d125480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/2374408316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd63300 0x555d5d10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 -- 172.21.15.19:0/2374408316 >> 172.21.15.19:0/2374408316 conn(0x555d5d12bf70 msgr2=0x555d5d10b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 -- 172.21.15.19:0/2374408316 shutdown_connections 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 -- 172.21.15.19:0/2374408316 wait complete. 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 -- start start 2024-05-17T09:20:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.161+0000 7fcdeac1e740 1 Processor -- start 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d122bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d123110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 0x555d5d124040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1a8eb0 con 0x555d5cd63300 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1aa320 con 0x555d5cd2dd80 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1a9580 con 0x555d5cd462f0 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcdeac1e740 10 client.0 _ll_register_callbacks cb 0x7fcdcc0a93b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 0x555d5d124040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 0x555d5d124040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47898/0 (socket says 172.21.15.19:47898) 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 -- 172.21.15.19:0/691678131 learned_addr learned my addr 172.21.15.19:0/691678131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde1306640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d123110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde1b07640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d122bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 -- 172.21.15.19:0/691678131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 msgr2=0x555d5d123110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d123110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 -- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 msgr2=0x555d5d122bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d122bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.162+0000 7fcde2308640 1 -- 172.21.15.19:0/691678131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5d124600 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcde2308640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 0x555d5d124040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400f770 tx=0x7fcdd4004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0321f0f14880e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdc57fa640 1 -- 172.21.15.19:0/691678131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4036060 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdc57fa640 1 -- 172.21.15.19:0/691678131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcdd40051f0 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd4005040 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdc57fa640 1 -- 172.21.15.19:0/691678131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdd402b040 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 10 client.15663 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d5d125890 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdc57fa640 1 -- 172.21.15.19:0/691678131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd4005040 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdc57fa640 1 -- 172.21.15.19:0/691678131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcdd4026a10 con 0x555d5cd63300 2024-05-17T09:20:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 10 client.15663 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 20 client.15663 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd4e4d0 0x555d5d196800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 4 client.15663 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.163+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d5d07e360 con 0x555d5cd4e4d0 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.164+0000 7fcde1b07640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd4e4d0 0x555d5d196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.164+0000 7fcde1b07640 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd4e4d0 0x555d5d196800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fe3578e6db26d2 server_cookie=f89ab93d098793c6 in_seq=0 out_seq=0 2024-05-17T09:20:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.164+0000 7fcdc57fa640 10 client.15663 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdc57fa640 1 -- 172.21.15.19:0/691678131 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d5d07e360 con 0x555d5cd4e4d0 2024-05-17T09:20:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdc57fa640 10 client.15663 handle_command_reply: tid=0 2024-05-17T09:20:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 client.15663 shutdown 2024-05-17T09:20:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 msgr2=0x555d5d124040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 0x555d5d124040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400f770 tx=0x7fcdd4004b70 comp rx=0 tx=0).stop 2024-05-17T09:20:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 shutdown_connections 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d5cd4e4d0 0x555d5d196800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d5cd63300 0x555d5d124040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5cd462f0 0x555d5d123110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdc57fa640 0 client.15663 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 --2- 172.21.15.19:0/691678131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d5cd2dd80 0x555d5d122bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 >> 172.21.15.19:0/691678131 conn(0x555d5d12bf70 msgr2=0x555d5d10adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 shutdown_connections 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 1 -- 172.21.15.19:0/691678131 wait complete. 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.168+0000 7fcdeac1e740 20 client.15663 trim_cache size 0 max 16384 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:20:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:20:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc80b2920 msgr2=0x7fcdc80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc80b2920 0x7fcdc80b4e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcdd802a930 tx=0x7fcdd8031400 comp rx=0 tx=0).stop 2024-05-17T09:20:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 msgr2=0x7fcddc19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 0x7fcddc19a740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 shutdown_connections 2024-05-17T09:20:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcddc109b90 0x7fcddc1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc80b2920 0x7fcdc80b4e30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcddc104e90 0x7fcddc19a740 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 --2- 172.21.15.19:0/2883068691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc103c50 0x7fcddc19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 >> 172.21.15.19:0/2883068691 conn(0x7fcddc0fee60 msgr2=0x7fcddc105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 shutdown_connections 2024-05-17T09:20:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:20:48.169+0000 7fcde3d6b640 1 -- 172.21.15.19:0/2883068691 wait complete. 2024-05-17T09:20:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:48 smithi019 ceph-mon[31620]: pgmap v469: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:48.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:48 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/49263625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:48 smithi079 ceph-mon[35502]: pgmap v469: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:48 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/49263625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:48.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:20:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:48 smithi069 ceph-mon[35081]: pgmap v469: 129 pgs: 129 active+clean; 668 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:48 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/49263625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:20:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:50.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:50 smithi019 ceph-mon[31620]: pgmap v470: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-17T09:20:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:50 smithi079 ceph-mon[35502]: pgmap v470: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-17T09:20:50.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 -- 172.21.15.19:0/2965918453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 msgr2=0x7f65a4110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 --2- 172.21.15.19:0/2965918453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a4110090 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b0d0 tx=0x7f659c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 -- 172.21.15.19:0/2965918453 shutdown_connections 2024-05-17T09:20:50.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 --2- 172.21.15.19:0/2965918453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a41034b0 0x7f65a41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 --2- 172.21.15.19:0/2965918453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a4110090 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 --2- 172.21.15.19:0/2965918453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a4107de0 0x7f65a41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 -- 172.21.15.19:0/2965918453 >> 172.21.15.19:0/2965918453 conn(0x7f65a40fede0 msgr2=0x7f65a4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:50.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 -- 172.21.15.19:0/2965918453 shutdown_connections 2024-05-17T09:20:50.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.560+0000 7f65ab632700 1 -- 172.21.15.19:0/2965918453 wait complete. 2024-05-17T09:20:50.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 Processor -- start 2024-05-17T09:20:50.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 -- start start 2024-05-17T09:20:50.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a41034b0 0x7f65a40ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a4107de0 0x7f65a40ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a40ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4113210 con 0x7f65a4108730 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4113090 con 0x7f65a4107de0 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65ab632700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4113390 con 0x7f65a41034b0 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65a9bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a40ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65a9bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a40ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47910/0 (socket says 172.21.15.19:47910) 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65a9bcf700 1 -- 172.21.15.19:0/854658205 learned_addr learned my addr 172.21.15.19:0/854658205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a4107de0 0x7f65a40ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.561+0000 7f65a93ce700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a41034b0 0x7f65a40ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65a9bcf700 1 -- 172.21.15.19:0/854658205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a41034b0 msgr2=0x7f65a40ff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a41034b0 0x7f65a40ff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65a9bcf700 1 -- 172.21.15.19:0/854658205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a4107de0 msgr2=0x7f65a40ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a4107de0 0x7f65a40ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65a9bcf700 1 -- 172.21.15.19:0/854658205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a41a9000 con 0x7f65a4108730 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a40ffd40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f659c009850 tx=0x7f659c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f659a7fc700 1 -- 172.21.15.19:0/854658205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c01f070 con 0x7f65a4108730 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a41a9290 con 0x7f65a4108730 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f659a7fc700 1 -- 172.21.15.19:0/854658205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f659c005500 con 0x7f65a4108730 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f659a7fc700 1 -- 172.21.15.19:0/854658205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c01b8f0 con 0x7f65a4108730 2024-05-17T09:20:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.562+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a41a9740 con 0x7f65a4108730 2024-05-17T09:20:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.565+0000 7f659a7fc700 1 -- 172.21.15.19:0/854658205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f659c022070 con 0x7f65a4108730 2024-05-17T09:20:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f659a7fc700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65900b2900 0x7f65900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f65ba047b80 1 Processor -- start 2024-05-17T09:20:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f65ba047b80 1 -- start start 2024-05-17T09:20:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f65a93ce700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65900b2900 0x7f65900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f659a7fc700 1 -- 172.21.15.19:0/854658205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f659c00f020 con 0x7f65a4108730 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.566+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99bed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65ba047b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a99b1ad70 con 0x557a99beccd0 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65ba047b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a99b0a910 con 0x557a99a6de40 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65ba047b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557a99b0e950 con 0x557a99b59430 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99bed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a93ce700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65900b2900 0x7f65900b4e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005950 tx=0x7f65a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99bed1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47920/0 (socket says 172.21.15.19:47920) 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 -- 172.21.15.19:0/3000587665 learned_addr learned my addr 172.21.15.19:0/3000587665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a93ce700 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 -- 172.21.15.19:0/3000587665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 msgr2=0x557a99bf2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 -- 172.21.15.19:0/3000587665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 msgr2=0x557a99bf2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 -- 172.21.15.19:0/3000587665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a99ab3fb0 con 0x557a99beccd0 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:20:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99bed1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f659c009900 tx=0x7f659c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3798662a0e55c277 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.567+0000 7f658f7fe700 1 -- 172.21.15.19:0/3000587665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b3390 con 0x557a99beccd0 2024-05-17T09:20:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f658f7fe700 1 -- 172.21.15.19:0/3000587665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f659c0aa780 con 0x557a99beccd0 2024-05-17T09:20:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f658f7fe700 1 -- 172.21.15.19:0/3000587665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b3a40 con 0x557a99beccd0 2024-05-17T09:20:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 -- 172.21.15.19:0/3000587665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 msgr2=0x557a99bed1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99bed1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f659c009900 tx=0x7f659c0099e0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 -- 172.21.15.19:0/3000587665 shutdown_connections 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99bed1a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3000587665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 -- 172.21.15.19:0/3000587665 >> 172.21.15.19:0/3000587665 conn(0x557a99ace1c0 msgr2=0x557a99be77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.568+0000 7f65ba047b80 1 -- 172.21.15.19:0/3000587665 shutdown_connections 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.569+0000 7f65ba047b80 1 -- 172.21.15.19:0/3000587665 wait complete. 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.569+0000 7f65ba047b80 1 -- start start 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.569+0000 7f65ba047b80 1 Processor -- start 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.569+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bfc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.569+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bfc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.569+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99beccd0 0x557a99bf64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65ba047b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a99a10fa0 con 0x557a99b59430 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65ba047b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a99a72da0 con 0x557a99a6de40 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65ba047b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557a99bed9e0 con 0x557a99beccd0 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65ba047b80 10 client.0 _ll_register_callbacks cb 0x557a99aa5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bfc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bfc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47930/0 (socket says 172.21.15.19:47930) 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 -- 172.21.15.19:0/3506719649 learned_addr learned my addr 172.21.15.19:0/3506719649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a93ce700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bfc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99beccd0 0x557a99bf64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 -- 172.21.15.19:0/3506719649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99beccd0 msgr2=0x557a99bf64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99beccd0 0x557a99bf64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 -- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 msgr2=0x557a99bfc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bfc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 -- 172.21.15.19:0/3506719649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a99a5a8d0 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a93ce700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bfc630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6594010a50 tx=0x7f6594010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b3d6730cbca468 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f658cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f658cff9700 1 -- 172.21.15.19:0/3506719649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659400fb00 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f658cff9700 1 -- 172.21.15.19:0/3506719649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f659400fca0 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.570+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c0aa780 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f658cff9700 1 -- 172.21.15.19:0/3506719649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6594024050 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 10 client.15675 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a99ac4760 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f658cff9700 1 -- 172.21.15.19:0/3506719649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f659c0aa780 con 0x557a99b59430 2024-05-17T09:20:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f658cff9700 1 -- 172.21.15.19:0/3506719649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6594003b80 con 0x557a99b59430 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 10 client.15675 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 20 client.15675 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c06230 0x557a99c08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 4 client.15675 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a99bb8ed0 con 0x557a99c06230 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.571+0000 7f65a93ce700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c06230 0x557a99c08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.572+0000 7f65a93ce700 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c06230 0x557a99c08670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=737f94bf2c007aed server_cookie=e81d8c57375ed660 in_seq=0 out_seq=0 2024-05-17T09:20:50.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.572+0000 7f658cff9700 10 client.15675 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:50.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f658cff9700 1 -- 172.21.15.19:0/3506719649 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557a99bb8ed0 con 0x557a99c06230 2024-05-17T09:20:50.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f658cff9700 10 client.15675 handle_command_reply: tid=0 2024-05-17T09:20:50.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 client.15675 shutdown 2024-05-17T09:20:50.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 msgr2=0x557a99bfc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bfc630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6594010a50 tx=0x7f6594010e10 comp rx=0 tx=0).stop 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 shutdown_connections 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c06230 0x557a99c08670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99beccd0 0x557a99bf64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 --2- 172.21.15.19:0/3506719649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bfc630 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 >> 172.21.15.19:0/3506719649 conn(0x557a99ace1c0 msgr2=0x557a99be7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f658cff9700 0 client.15675 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 shutdown_connections 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 1 -- 172.21.15.19:0/3506719649 wait complete. 2024-05-17T09:20:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.573+0000 7f65ba047b80 20 client.15675 trim_cache size 0 max 16384 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 Processor -- start 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 -- start start 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99c0de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b4c90 con 0x557a99beccd0 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b2520 con 0x557a99a6de40 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65ba047b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b0370 con 0x557a99b59430 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99c0de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99c0de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47940/0 (socket says 172.21.15.19:47940) 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 -- 172.21.15.19:0/446413621 learned_addr learned my addr 172.21.15.19:0/446413621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a93ce700 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 -- 172.21.15.19:0/446413621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 msgr2=0x557a99bf5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 -- 172.21.15.19:0/446413621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 msgr2=0x557a99bf4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.589+0000 7f65a9bcf700 1 -- 172.21.15.19:0/446413621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a99be9790 con 0x557a99beccd0 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.590+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99c0de70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f659c006090 tx=0x7f659c00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef25306b28b6614 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.590+0000 7f658effd700 1 -- 172.21.15.19:0/446413621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0ca070 con 0x557a99beccd0 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.590+0000 7f658effd700 1 -- 172.21.15.19:0/446413621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f659c0b3390 con 0x557a99beccd0 2024-05-17T09:20:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.590+0000 7f658effd700 1 -- 172.21.15.19:0/446413621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0af020 con 0x557a99beccd0 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 -- 172.21.15.19:0/446413621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 msgr2=0x557a99c0de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99c0de70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f659c006090 tx=0x7f659c00b9d0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 -- 172.21.15.19:0/446413621 shutdown_connections 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99beccd0 0x557a99c0de70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99a6de40 0x557a99bf4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 --2- 172.21.15.19:0/446413621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99b59430 0x557a99bf5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 -- 172.21.15.19:0/446413621 >> 172.21.15.19:0/446413621 conn(0x557a99ace1c0 msgr2=0x557a99bfc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 -- 172.21.15.19:0/446413621 shutdown_connections 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 -- 172.21.15.19:0/446413621 wait complete. 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 -- start start 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.591+0000 7f65ba047b80 1 Processor -- start 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99a6de40 0x557a99bff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99beccd0 0x557a99c6ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b09e0 con 0x557a99b59430 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b1e50 con 0x557a99beccd0 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65a41b10b0 con 0x557a99a6de40 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 10 client.0 _ll_register_callbacks cb 0x557a99aa5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47952/0 (socket says 172.21.15.19:47952) 2024-05-17T09:20:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 -- 172.21.15.19:0/4165037986 learned_addr learned my addr 172.21.15.19:0/4165037986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a93ce700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99a6de40 0x557a99bff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99beccd0 0x557a99c6ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 -- 172.21.15.19:0/4165037986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99a6de40 msgr2=0x557a99bff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99a6de40 0x557a99bff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 -- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99beccd0 msgr2=0x557a99c6ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99beccd0 0x557a99c6ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 -- 172.21.15.19:0/4165037986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a99c6f2d0 con 0x557a99b59430 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a9bcf700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99beccd0 0x557a99c6ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65a8bcd700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bffe70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x557a99be42f0 tx=0x7f659400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969003103734ba41 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f658cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f658cff9700 1 -- 172.21.15.19:0/4165037986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594005280 con 0x557a99b59430 2024-05-17T09:20:50.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f658cff9700 1 -- 172.21.15.19:0/4165037986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6594005420 con 0x557a99b59430 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.592+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c0b3390 con 0x557a99b59430 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f658cff9700 1 -- 172.21.15.19:0/4165037986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6594037050 con 0x557a99b59430 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 10 client.15681 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a99c70530 con 0x557a99b59430 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f658cff9700 1 -- 172.21.15.19:0/4165037986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f659c0b3390 con 0x557a99b59430 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f658cff9700 1 -- 172.21.15.19:0/4165037986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6594019670 con 0x557a99b59430 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 10 client.15681 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 20 client.15681 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c70be0 0x557a99c73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 4 client.15681 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.593+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a99b1ccc0 con 0x557a99c70be0 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65a93ce700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c70be0 0x557a99c73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65a93ce700 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c70be0 0x557a99c73000 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=927f71700fe57c30 server_cookie=d0dcdd58651fabd0 in_seq=0 out_seq=0 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f658cff9700 10 client.15681 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f658cff9700 1 -- 172.21.15.19:0/4165037986 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a99b1ccc0 con 0x557a99c70be0 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f658cff9700 10 client.15681 handle_command_reply: tid=0 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 client.15681 shutdown 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 msgr2=0x557a99bffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bffe70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x557a99be42f0 tx=0x7f659400af30 comp rx=0 tx=0).stop 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 shutdown_connections 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557a99c70be0 0x557a99c73000 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a99a6de40 0x557a99bff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a99beccd0 0x557a99c6ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 --2- 172.21.15.19:0/4165037986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a99b59430 0x557a99bffe70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 >> 172.21.15.19:0/4165037986 conn(0x557a99ace1c0 msgr2=0x557a99bf1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.594+0000 7f658cff9700 0 client.15681 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 shutdown_connections 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ba047b80 1 -- 172.21.15.19:0/4165037986 wait complete. 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ba047b80 20 client.15681 trim_cache size 0 max 16384 2024-05-17T09:20:50.599 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:20:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65900b2900 msgr2=0x7f65900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65900b2900 0x7f65900b4e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005950 tx=0x7f65a000b410 comp rx=0 tx=0).stop 2024-05-17T09:20:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 msgr2=0x7f65a40ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a40ffd40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f659c009850 tx=0x7f659c007d90 comp rx=0 tx=0).stop 2024-05-17T09:20:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 shutdown_connections 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65900b2900 0x7f65900b4e10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a41034b0 0x7f65a40ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a4108730 0x7f65a40ffd40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 --2- 172.21.15.19:0/854658205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a4107de0 0x7f65a40ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 >> 172.21.15.19:0/854658205 conn(0x7f65a40fede0 msgr2=0x7f65a4075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.595+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 shutdown_connections 2024-05-17T09:20:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:50.596+0000 7f65ab632700 1 -- 172.21.15.19:0/854658205 wait complete. 2024-05-17T09:20:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:50 smithi069 ceph-mon[35081]: pgmap v470: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-17T09:20:51.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:20:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:52.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:52 smithi019 ceph-mon[31620]: pgmap v471: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:52 smithi079 ceph-mon[35502]: pgmap v471: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:52 smithi069 ceph-mon[35081]: pgmap v471: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:53.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.311+0000 7f46fb1a3700 1 -- 172.21.15.19:0/3387138197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f4105e20 msgr2=0x7f46f410e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.311+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/3387138197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f4105e20 0x7f46f410e370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009990 tx=0x7f46e8009ca0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 -- 172.21.15.19:0/3387138197 shutdown_connections 2024-05-17T09:20:53.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/3387138197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f410e900 0x7f46f4110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/3387138197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46f41054d0 0x7f46f41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/3387138197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f4105e20 0x7f46f410e370 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 -- 172.21.15.19:0/3387138197 >> 172.21.15.19:0/3387138197 conn(0x7f46f40fef20 msgr2=0x7f46f4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:53.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 -- 172.21.15.19:0/3387138197 shutdown_connections 2024-05-17T09:20:53.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 -- 172.21.15.19:0/3387138197 wait complete. 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.312+0000 7f46fb1a3700 1 Processor -- start 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 -- start start 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46f41054d0 0x7f46f41a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f4105e20 0x7f46f41a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 0x7f46f419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4114310 con 0x7f46f410e900 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4114190 con 0x7f46f41054d0 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46fb1a3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4114490 con 0x7f46f4105e20 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46f9740700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 0x7f46f419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46f9740700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 0x7f46f419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47978/0 (socket says 172.21.15.19:47978) 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46f9740700 1 -- 172.21.15.19:0/803269859 learned_addr learned my addr 172.21.15.19:0/803269859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46f41054d0 0x7f46f41a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.313+0000 7f46f3fff700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f4105e20 0x7f46f41a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f9740700 1 -- 172.21.15.19:0/803269859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f4105e20 msgr2=0x7f46f41a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f9740700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f4105e20 0x7f46f41a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f9740700 1 -- 172.21.15.19:0/803269859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46f41054d0 msgr2=0x7f46f41a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f9740700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46f41054d0 0x7f46f41a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f9740700 1 -- 172.21.15.19:0/803269859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419b1f0 con 0x7f46f410e900 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f9740700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 0x7f46f419ab50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00ada0 tx=0x7f46ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f1ffb700 1 -- 172.21.15.19:0/803269859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec017020 con 0x7f46f410e900 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f419b4e0 con 0x7f46f410e900 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f1ffb700 1 -- 172.21.15.19:0/803269859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ec00ce10 con 0x7f46f410e900 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46f1ffb700 1 -- 172.21.15.19:0/803269859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec005280 con 0x7f46f410e900 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.314+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f41a7770 con 0x7f46f410e900 2024-05-17T09:20:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.317+0000 7f46f1ffb700 1 -- 172.21.15.19:0/803269859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f46ec00c980 con 0x7f46f410e900 2024-05-17T09:20:53.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.318+0000 7f46f1ffb700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46dc0b2900 0x7f46dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.318+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46dc0b2900 0x7f46dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.318+0000 7f46f1ffb700 1 -- 172.21.15.19:0/803269859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f46ec0e1c30 con 0x7f46f410e900 2024-05-17T09:20:53.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 Processor -- start 2024-05-17T09:20:53.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46dc0b2900 0x7f46dc0b4e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46e400ac50 tx=0x7f46e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 -- start start 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc10f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562bc101fd70 con 0x562bc105e430 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562bc100f910 con 0x562bc10f1cd0 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562bc1013950 con 0x562bc0f72e40 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47984/0 (socket says 172.21.15.19:47984) 2024-05-17T09:20:53.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 -- 172.21.15.19:0/481016779 learned_addr learned my addr 172.21.15.19:0/481016779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f9740700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc10f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 -- 172.21.15.19:0/481016779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 msgr2=0x562bc10f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f3fff700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 -- 172.21.15.19:0/481016779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 msgr2=0x562bc10f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc10f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f8f3f700 1 -- 172.21.15.19:0/481016779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc0fb8fb0 con 0x562bc105e430 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f9740700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc10f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.319+0000 7f46f3fff700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.320+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f7910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f46e400ff90 tx=0x7f46e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d922a9e700f9c0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.320+0000 7f46daffd700 1 -- 172.21.15.19:0/481016779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4016070 con 0x562bc105e430 2024-05-17T09:20:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.320+0000 7f46daffd700 1 -- 172.21.15.19:0/481016779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e4014070 con 0x562bc105e430 2024-05-17T09:20:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.320+0000 7f46daffd700 1 -- 172.21.15.19:0/481016779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4010e40 con 0x562bc105e430 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 -- 172.21.15.19:0/481016779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 msgr2=0x562bc10f7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f7910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f46e400ff90 tx=0x7f46e4012000 comp rx=0 tx=0).stop 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 -- 172.21.15.19:0/481016779 shutdown_connections 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc10f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f7910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/481016779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 -- 172.21.15.19:0/481016779 >> 172.21.15.19:0/481016779 conn(0x562bc0fd31c0 msgr2=0x562bc10ec7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 -- 172.21.15.19:0/481016779 shutdown_connections 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 -- 172.21.15.19:0/481016779 wait complete. 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 -- start start 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.321+0000 7f4709bb8b80 1 Processor -- start 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc105e430 0x562bc10fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 0x562bc10fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562bc0f15fa0 con 0x562bc10f1cd0 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562bc0f77da0 con 0x562bc105e430 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562bc10fe5a0 con 0x562bc0f72e40 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f4709bb8b80 10 client.0 _ll_register_callbacks cb 0x562bc0faa950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 0x562bc10fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 0x562bc10fc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47990/0 (socket says 172.21.15.19:47990) 2024-05-17T09:20:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 -- 172.21.15.19:0/644516776 learned_addr learned my addr 172.21.15.19:0/644516776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f3fff700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc105e430 0x562bc10fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 -- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 msgr2=0x562bc10fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 -- 172.21.15.19:0/644516776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc105e430 msgr2=0x562bc10fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.322+0000 7f46f9740700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc105e430 0x562bc10fbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46f9740700 1 -- 172.21.15.19:0/644516776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc0f5f8d0 con 0x562bc10f1cd0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46f9740700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 0x562bc10fc040 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0a4fb0 tx=0x7f46ec00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea09a5ec987f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46d3fff700 1 -- 172.21.15.19:0/644516776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0a8a40 con 0x562bc10f1cd0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46d3fff700 1 -- 172.21.15.19:0/644516776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f46ec0b1a80 con 0x562bc10f1cd0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e4014070 con 0x562bc10f1cd0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46d3fff700 1 -- 172.21.15.19:0/644516776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46ec0a9630 con 0x562bc10f1cd0 2024-05-17T09:20:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f4709bb8b80 10 client.15693 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562bc0fc9760 con 0x562bc10f1cd0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46d3fff700 1 -- 172.21.15.19:0/644516776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e4014070 con 0x562bc10f1cd0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.323+0000 7f46d3fff700 1 -- 172.21.15.19:0/644516776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f46ec0bf070 con 0x562bc10f1cd0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f4709bb8b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f4709bb8b80 10 client.15693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f4709bb8b80 20 client.15693 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc110b4b0 0x562bc110d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f4709bb8b80 4 client.15693 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bc10bded0 con 0x562bc110b4b0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc110b4b0 0x562bc110d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f46d3fff700 10 client.15693 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.324+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc110b4b0 0x562bc110d8d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4522d3cacead5bc server_cookie=7d4197b24597f9d3 in_seq=0 out_seq=0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f46d3fff700 1 -- 172.21.15.19:0/644516776 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562bc10bded0 con 0x562bc110b4b0 2024-05-17T09:20:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f46d3fff700 10 client.15693 handle_command_reply: tid=0 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f4709bb8b80 1 client.15693 shutdown 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 msgr2=0x562bc10fc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 0x562bc10fc040 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0a4fb0 tx=0x7f46ec00cda0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 shutdown_connections 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc110b4b0 0x562bc110d8d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.325+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc10fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc10f1cd0 0x562bc10fc040 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/644516776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc105e430 0x562bc10fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 >> 172.21.15.19:0/644516776 conn(0x562bc0fd31c0 msgr2=0x562bc10ec790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f46d3fff700 0 client.15693 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 shutdown_connections 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f4709bb8b80 1 -- 172.21.15.19:0/644516776 wait complete. 2024-05-17T09:20:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.326+0000 7f4709bb8b80 20 client.15693 trim_cache size 0 max 16384 2024-05-17T09:20:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.341+0000 7f4709bb8b80 1 Processor -- start 2024-05-17T09:20:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.341+0000 7f4709bb8b80 1 -- start start 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc0f72e40 0x562bc10f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc10f1cd0 0x562bc10f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b2c00 con 0x562bc105e430 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b0490 con 0x562bc0f72e40 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41ae2e0 con 0x562bc10f1cd0 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48006/0 (socket says 172.21.15.19:48006) 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 -- 172.21.15.19:0/883812899 learned_addr learned my addr 172.21.15.19:0/883812899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc0f72e40 0x562bc10f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f9740700 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc10f1cd0 0x562bc10f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 -- 172.21.15.19:0/883812899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc10f1cd0 msgr2=0x562bc10f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc10f1cd0 0x562bc10f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 -- 172.21.15.19:0/883812899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc0f72e40 msgr2=0x562bc10f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc0f72e40 0x562bc10f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 -- 172.21.15.19:0/883812899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc1107680 con 0x562bc105e430 2024-05-17T09:20:53.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.342+0000 7f46f3fff700 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f6ac0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009990 tx=0x7f46e801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1d757667a35e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:53.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.343+0000 7f46da7fc700 1 -- 172.21.15.19:0/883812899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8005330 con 0x562bc105e430 2024-05-17T09:20:53.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.343+0000 7f46da7fc700 1 -- 172.21.15.19:0/883812899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e8027070 con 0x562bc105e430 2024-05-17T09:20:53.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.343+0000 7f46da7fc700 1 -- 172.21.15.19:0/883812899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800bbf0 con 0x562bc105e430 2024-05-17T09:20:53.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 -- 172.21.15.19:0/883812899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 msgr2=0x562bc10f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f6ac0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009990 tx=0x7f46e801a040 comp rx=0 tx=0).stop 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 -- 172.21.15.19:0/883812899 shutdown_connections 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc10f1cd0 0x562bc10f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc0f72e40 0x562bc10f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/883812899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc10f6ac0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 -- 172.21.15.19:0/883812899 >> 172.21.15.19:0/883812899 conn(0x562bc0fd31c0 msgr2=0x562bc10f7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 -- 172.21.15.19:0/883812899 shutdown_connections 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 -- 172.21.15.19:0/883812899 wait complete. 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 -- start start 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.344+0000 7f4709bb8b80 1 Processor -- start 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc1103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc1112500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc1112a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41ae950 con 0x562bc105e430 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41afdc0 con 0x562bc10f1cd0 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41af020 con 0x562bc0f72e40 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f4709bb8b80 10 client.0 _ll_register_callbacks cb 0x562bc0faa950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc1112500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc1112500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48014/0 (socket says 172.21.15.19:48014) 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 -- 172.21.15.19:0/1616986773 learned_addr learned my addr 172.21.15.19:0/1616986773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f9740700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc1112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc1103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 -- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 msgr2=0x562bc1103bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc1103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 -- 172.21.15.19:0/1616986773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 msgr2=0x562bc1112a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc1112a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f3fff700 1 -- 172.21.15.19:0/1616986773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc1112f80 con 0x562bc105e430 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.345+0000 7f46f9740700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc1112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46f3fff700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc1112500 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f46e80098e0 tx=0x7f46e8003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21a92ee74e0f643 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46d3fff700 1 -- 172.21.15.19:0/1616986773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800b870 con 0x562bc105e430 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46d3fff700 1 -- 172.21.15.19:0/1616986773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f46e800ba10 con 0x562bc105e430 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e8027070 con 0x562bc105e430 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46d3fff700 1 -- 172.21.15.19:0/1616986773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46e800bc50 con 0x562bc105e430 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f4709bb8b80 10 client.15699 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562bc11759e0 con 0x562bc105e430 2024-05-17T09:20:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46d3fff700 1 -- 172.21.15.19:0/1616986773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e8027070 con 0x562bc105e430 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.346+0000 7f46d3fff700 1 -- 172.21.15.19:0/1616986773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f46e800f460 con 0x562bc105e430 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f4709bb8b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f4709bb8b80 10 client.15699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f4709bb8b80 20 client.15699 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc1176070 0x562bc1178490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f4709bb8b80 4 client.15699 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:20:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bc1021cc0 con 0x562bc1176070 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc1176070 0x562bc1178490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f46f8f3f700 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc1176070 0x562bc1178490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56121c6c25c8da1a server_cookie=9f21ec2cdbabbac8 in_seq=0 out_seq=0 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.347+0000 7f46d3fff700 10 client.15699 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f46d3fff700 1 -- 172.21.15.19:0/1616986773 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bc1021cc0 con 0x562bc1176070 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f46d3fff700 10 client.15699 handle_command_reply: tid=0 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 client.15699 shutdown 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 msgr2=0x562bc1112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc1112500 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f46e80098e0 tx=0x7f46e8003870 comp rx=0 tx=0).stop 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 shutdown_connections 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x562bc1176070 0x562bc1178490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f46d3fff700 0 client.15699 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562bc0f72e40 0x562bc1103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bc10f1cd0 0x562bc1112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 --2- 172.21.15.19:0/1616986773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562bc105e430 0x562bc1112500 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 >> 172.21.15.19:0/1616986773 conn(0x562bc0fd31c0 msgr2=0x562bc10f7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 shutdown_connections 2024-05-17T09:20:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 1 -- 172.21.15.19:0/1616986773 wait complete. 2024-05-17T09:20:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.348+0000 7f4709bb8b80 20 client.15699 trim_cache size 0 max 16384 2024-05-17T09:20:53.352 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46dc0b2900 msgr2=0x7f46dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46dc0b2900 0x7f46dc0b4e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46e400ac50 tx=0x7f46e400a380 comp rx=0 tx=0).stop 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 msgr2=0x7f46f419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 0x7f46f419ab50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00ada0 tx=0x7f46ec00c330 comp rx=0 tx=0).stop 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 shutdown_connections 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f410e900 0x7f46f419ab50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46f41054d0 0x7f46f41a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46dc0b2900 0x7f46dc0b4e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 --2- 172.21.15.19:0/803269859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f4105e20 0x7f46f41a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 >> 172.21.15.19:0/803269859 conn(0x7f46f40fef20 msgr2=0x7f46f40ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 shutdown_connections 2024-05-17T09:20:53.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:53.349+0000 7f46fb1a3700 1 -- 172.21.15.19:0/803269859 wait complete. 2024-05-17T09:20:54.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:20:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:54.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:54 smithi019 ceph-mon[31620]: pgmap v472: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:54 smithi079 ceph-mon[35502]: pgmap v472: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:54 smithi069 ceph-mon[35081]: pgmap v472: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 -- 172.21.15.19:0/2279992354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8107de0 msgr2=0x7ffad81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 --2- 172.21.15.19:0/2279992354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8107de0 0x7ffad81081f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffac80099c0 tx=0x7ffac8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 -- 172.21.15.19:0/2279992354 shutdown_connections 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 --2- 172.21.15.19:0/2279992354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad8108730 0x7ffad8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 --2- 172.21.15.19:0/2279992354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad81034b0 0x7ffad81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 --2- 172.21.15.19:0/2279992354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8107de0 0x7ffad81081f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 -- 172.21.15.19:0/2279992354 >> 172.21.15.19:0/2279992354 conn(0x7ffad80fedb0 msgr2=0x7ffad81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 -- 172.21.15.19:0/2279992354 shutdown_connections 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.342+0000 7ffae05f3700 1 -- 172.21.15.19:0/2279992354 wait complete. 2024-05-17T09:20:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 Processor -- start 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 -- start start 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 0x7ffad8110a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad8107de0 0x7ffad8110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad8108730 0x7ffad81114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8114360 con 0x7ffad81034b0 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81141e0 con 0x7ffad8108730 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffae05f3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81144e0 con 0x7ffad8107de0 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffade38f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 0x7ffad8110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffade38f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 0x7ffad8110a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41890/0 (socket says 172.21.15.19:41890) 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.343+0000 7ffade38f700 1 -- 172.21.15.19:0/1434320964 learned_addr learned my addr 172.21.15.19:0/1434320964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffadeb90700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad8108730 0x7ffad81114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad8107de0 0x7ffad8110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffade38f700 1 -- 172.21.15.19:0/1434320964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad8107de0 msgr2=0x7ffad8110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffade38f700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad8107de0 0x7ffad8110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffade38f700 1 -- 172.21.15.19:0/1434320964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad8108730 msgr2=0x7ffad81114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffade38f700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad8108730 0x7ffad81114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffade38f700 1 -- 172.21.15.19:0/1434320964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad81a9110 con 0x7ffad81034b0 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffade38f700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 0x7ffad8110a70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffad400d950 tx=0x7ffad400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffacf7fe700 1 -- 172.21.15.19:0/1434320964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4016070 con 0x7ffad81034b0 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad81a9400 con 0x7ffad81034b0 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffacf7fe700 1 -- 172.21.15.19:0/1434320964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffad4009dd0 con 0x7ffad81034b0 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffacf7fe700 1 -- 172.21.15.19:0/1434320964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad400f3c0 con 0x7ffad81034b0 2024-05-17T09:20:56.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.344+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad81a9960 con 0x7ffad81034b0 2024-05-17T09:20:56.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.347+0000 7ffacf7fe700 1 -- 172.21.15.19:0/1434320964 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffad4019070 con 0x7ffad81034b0 2024-05-17T09:20:56.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.348+0000 7ffacf7fe700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffac40b2880 0x7ffac40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.348+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffac40b2880 0x7ffac40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.348+0000 7ffaef008b80 1 Processor -- start 2024-05-17T09:20:56.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.348+0000 7ffacf7fe700 1 -- 172.21.15.19:0/1434320964 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffad40e22a0 con 0x7ffad81034b0 2024-05-17T09:20:56.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 -- start start 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffac40b2880 0x7ffac40b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009990 tx=0x7ffac8005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a96910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2e89bed70 con 0x55d2e8911e40 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2e89ae910 con 0x55d2e89fd430 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaef008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d2e89b2950 con 0x55d2e8a90cd0 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a96eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41900/0 (socket says 172.21.15.19:41900) 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 -- 172.21.15.19:0/1409304008 learned_addr learned my addr 172.21.15.19:0/1409304008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffade38f700 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a96910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffadeb90700 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 -- 172.21.15.19:0/1409304008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 msgr2=0x55d2e8a911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 -- 172.21.15.19:0/1409304008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 msgr2=0x55d2e8a96910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a96910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.349+0000 7ffaddb8e700 1 -- 172.21.15.19:0/1409304008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e8957fb0 con 0x55d2e8911e40 2024-05-17T09:20:56.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.350+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a96eb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffac8015000 tx=0x7ffac801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f2aa0c9929e754 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.350+0000 7ffaccff9700 1 -- 172.21.15.19:0/1409304008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8005610 con 0x55d2e8911e40 2024-05-17T09:20:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.350+0000 7ffaccff9700 1 -- 172.21.15.19:0/1409304008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffac8003b10 con 0x55d2e8911e40 2024-05-17T09:20:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.350+0000 7ffaccff9700 1 -- 172.21.15.19:0/1409304008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800fa50 con 0x55d2e8911e40 2024-05-17T09:20:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 -- 172.21.15.19:0/1409304008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 msgr2=0x55d2e8a96eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a96eb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffac8015000 tx=0x7ffac801d000 comp rx=0 tx=0).stop 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 -- 172.21.15.19:0/1409304008 shutdown_connections 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a96910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1409304008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a96eb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 -- 172.21.15.19:0/1409304008 >> 172.21.15.19:0/1409304008 conn(0x55d2e89721c0 msgr2=0x55d2e8a8b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 -- 172.21.15.19:0/1409304008 shutdown_connections 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 -- 172.21.15.19:0/1409304008 wait complete. 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 -- start start 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.351+0000 7ffaef008b80 1 Processor -- start 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a9a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a9ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a9b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2e88b4fa0 con 0x55d2e8911e40 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2e8916da0 con 0x55d2e89fd430 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d2e8a9d5a0 con 0x55d2e8a90cd0 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaef008b80 10 client.0 _ll_register_callbacks cb 0x55d2e8949950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a9a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a9a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41906/0 (socket says 172.21.15.19:41906) 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 -- 172.21.15.19:0/2700600439 learned_addr learned my addr 172.21.15.19:0/2700600439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a9ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffadeb90700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a9b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 -- 172.21.15.19:0/2700600439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 msgr2=0x55d2e8a9b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a9b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 -- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 msgr2=0x55d2e8a9ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a9ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.352+0000 7ffade38f700 1 -- 172.21.15.19:0/2700600439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e88fe8d0 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffade38f700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a9a600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffad40df930 tx=0x7ffad40a9c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01f1c423477a91b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffac1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffac1ffb700 1 -- 172.21.15.19:0/2700600439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40ad860 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffac1ffb700 1 -- 172.21.15.19:0/2700600439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffad4091070 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac8003b10 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffac1ffb700 1 -- 172.21.15.19:0/2700600439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffad40b2310 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 10 client.15711 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2e8968760 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffac1ffb700 1 -- 172.21.15.19:0/2700600439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffac8003b10 con 0x55d2e8911e40 2024-05-17T09:20:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffac1ffb700 1 -- 172.21.15.19:0/2700600439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffad40b1e80 con 0x55d2e8911e40 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 20 client.15711 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8aac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 4 client.15711 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.353+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2e8a5ced0 con 0x55d2e8aaa3a0 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.354+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8aac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.354+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8aac850 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a21222fa2493c4 server_cookie=3d2ddf54288b8dfb in_seq=0 out_seq=0 2024-05-17T09:20:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.354+0000 7ffac1ffb700 10 client.15711 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffac1ffb700 1 -- 172.21.15.19:0/2700600439 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d2e8a5ced0 con 0x55d2e8aaa3a0 2024-05-17T09:20:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffac1ffb700 10 client.15711 handle_command_reply: tid=0 2024-05-17T09:20:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 client.15711 shutdown 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 msgr2=0x55d2e8a9a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a9a600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffad40df930 tx=0x7ffad40a9c80 comp rx=0 tx=0).stop 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 shutdown_connections 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a9b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8aac850 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8a9a600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffac1ffb700 0 client.15711 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 --2- 172.21.15.19:0/2700600439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e89fd430 0x55d2e8a9ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 >> 172.21.15.19:0/2700600439 conn(0x55d2e89721c0 msgr2=0x55d2e8a8b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.355+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 shutdown_connections 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.356+0000 7ffaef008b80 1 -- 172.21.15.19:0/2700600439 wait complete. 2024-05-17T09:20:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.356+0000 7ffaef008b80 20 client.15711 trim_cache size 0 max 16384 2024-05-17T09:20:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 Processor -- start 2024-05-17T09:20:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 -- start start 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8911e40 0x55d2e8a94c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8a9caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a95cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81b4e10 con 0x55d2e8a90cd0 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81b26a0 con 0x55d2e8911e40 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffaef008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81b04f0 con 0x55d2e89fd430 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffadeb90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a95cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffadeb90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a95cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41920/0 (socket says 172.21.15.19:41920) 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.371+0000 7ffadeb90700 1 -- 172.21.15.19:0/3834977592 learned_addr learned my addr 172.21.15.19:0/3834977592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffade38f700 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8911e40 0x55d2e8a94c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8a9caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffadeb90700 1 -- 172.21.15.19:0/3834977592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 msgr2=0x55d2e8a9caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffadeb90700 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8a9caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffadeb90700 1 -- 172.21.15.19:0/3834977592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8911e40 msgr2=0x55d2e8a94c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffadeb90700 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8911e40 0x55d2e8a94c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffadeb90700 1 -- 172.21.15.19:0/3834977592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e8aa6570 con 0x55d2e8a90cd0 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffade38f700 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8911e40 0x55d2e8a94c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffadeb90700 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a95cf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffad000a500 tx=0x7ffad0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322096a3aa81ec7a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffac3fff700 1 -- 172.21.15.19:0/3834977592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad001b020 con 0x55d2e8a90cd0 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffac3fff700 1 -- 172.21.15.19:0/3834977592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffad00178d0 con 0x55d2e8a90cd0 2024-05-17T09:20:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.372+0000 7ffac3fff700 1 -- 172.21.15.19:0/3834977592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad00168a0 con 0x55d2e8a90cd0 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 -- 172.21.15.19:0/3834977592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 msgr2=0x55d2e8a95cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a95cf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffad000a500 tx=0x7ffad0009650 comp rx=0 tx=0).stop 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 -- 172.21.15.19:0/3834977592 shutdown_connections 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8a90cd0 0x55d2e8a95cf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8911e40 0x55d2e8a94c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 --2- 172.21.15.19:0/3834977592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8a9caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 -- 172.21.15.19:0/3834977592 >> 172.21.15.19:0/3834977592 conn(0x55d2e89721c0 msgr2=0x55d2e8a8fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 -- 172.21.15.19:0/3834977592 shutdown_connections 2024-05-17T09:20:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 -- 172.21.15.19:0/3834977592 wait complete. 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 -- start start 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.373+0000 7ffaef008b80 1 Processor -- start 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8aa2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8ab1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8a90cd0 0x55d2e8ab1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81b0b60 con 0x55d2e8911e40 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81b1fd0 con 0x55d2e8a90cd0 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81b1230 con 0x55d2e89fd430 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaef008b80 10 client.0 _ll_register_callbacks cb 0x55d2e8949950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8aa2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8aa2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41932/0 (socket says 172.21.15.19:41932) 2024-05-17T09:20:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 -- 172.21.15.19:0/1353853643 learned_addr learned my addr 172.21.15.19:0/1353853643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8ab1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffadeb90700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8a90cd0 0x55d2e8ab1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 -- 172.21.15.19:0/1353853643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 msgr2=0x55d2e8ab1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8ab1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 -- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8a90cd0 msgr2=0x55d2e8ab1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8a90cd0 0x55d2e8ab1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 -- 172.21.15.19:0/1353853643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e8ab21c0 con 0x55d2e8911e40 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffadeb90700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8a90cd0 0x55d2e8ab1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffade38f700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8aa2c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffad40a54e0 tx=0x7ffad40a9940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c322c98e353545b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.374+0000 7ffac1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffac1ffb700 1 -- 172.21.15.19:0/1353853643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40ad860 con 0x55d2e8911e40 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffac1ffb700 1 -- 172.21.15.19:0/1353853643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffad40a3bd0 con 0x55d2e8911e40 2024-05-17T09:20:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad00178d0 con 0x55d2e8911e40 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffac1ffb700 1 -- 172.21.15.19:0/1353853643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffad40a9280 con 0x55d2e8911e40 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 10 client.15717 fetch_fsmap learned FSMap version 27 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2e8b14c40 con 0x55d2e8911e40 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffac1ffb700 1 -- 172.21.15.19:0/1353853643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffad00178d0 con 0x55d2e8911e40 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffac1ffb700 1 -- 172.21.15.19:0/1353853643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffad40ca300 con 0x55d2e8911e40 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 20 client.15717 populate_metadata read hostname 'smithi019' 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8b171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 4 client.15717 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.375+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2e89c0cc0 con 0x55d2e8aaa3a0 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8b171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaddb8e700 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8b171e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0331a7ef58123f server_cookie=52487329dbe12324 in_seq=0 out_seq=0 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffac1ffb700 10 client.15717 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffac1ffb700 1 -- 172.21.15.19:0/1353853643 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2e89c0cc0 con 0x55d2e8aaa3a0 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffac1ffb700 10 client.15717 handle_command_reply: tid=0 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 client.15717 shutdown 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 msgr2=0x55d2e8aa2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8aa2c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffad40a54e0 tx=0x7ffad40a9940 comp rx=0 tx=0).stop 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 shutdown_connections 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2e8aaa3a0 0x55d2e8b171e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2e8a90cd0 0x55d2e8ab1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2e8911e40 0x55d2e8aa2c50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffac1ffb700 0 client.15717 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 --2- 172.21.15.19:0/1353853643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2e89fd430 0x55d2e8ab1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 >> 172.21.15.19:0/1353853643 conn(0x55d2e89721c0 msgr2=0x55d2e8a95cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 shutdown_connections 2024-05-17T09:20:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.376+0000 7ffaef008b80 1 -- 172.21.15.19:0/1353853643 wait complete. 2024-05-17T09:20:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffaef008b80 20 client.15717 trim_cache size 0 max 16384 2024-05-17T09:20:56.382 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffac40b2880 msgr2=0x7ffac40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffac40b2880 0x7ffac40b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009990 tx=0x7ffac8005940 comp rx=0 tx=0).stop 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 msgr2=0x7ffad8110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 0x7ffad8110a70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffad400d950 tx=0x7ffad400dc60 comp rx=0 tx=0).stop 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 shutdown_connections 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad8108730 0x7ffad81114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad81034b0 0x7ffad8110a70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffac40b2880 0x7ffac40b4d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 --2- 172.21.15.19:0/1434320964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad8107de0 0x7ffad8110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 >> 172.21.15.19:0/1434320964 conn(0x7ffad80fedb0 msgr2=0x7ffad8109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 shutdown_connections 2024-05-17T09:20:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:56.377+0000 7ffae05f3700 1 -- 172.21.15.19:0/1434320964 wait complete. 2024-05-17T09:20:56.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:56 smithi079 ceph-mon[35502]: pgmap v473: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:56 smithi069 ceph-mon[35081]: pgmap v473: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:56.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:56 smithi019 ceph-mon[31620]: pgmap v473: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:20:57.209 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:20:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:58.141 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:20:58.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:20:58.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:20:58 smithi019 ceph-mon[31620]: pgmap v474: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:20:58 smithi079 ceph-mon[35502]: pgmap v474: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:20:58 smithi069 ceph-mon[35081]: pgmap v474: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:20:59.093 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 -- 172.21.15.19:0/3372898245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105dc0 msgr2=0x7ff13410e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 --2- 172.21.15.19:0/3372898245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105dc0 0x7ff13410e310 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff128009990 tx=0x7ff128009ca0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 -- 172.21.15.19:0/3372898245 shutdown_connections 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 --2- 172.21.15.19:0/3372898245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff13410e8e0 0x7ff134110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 --2- 172.21.15.19:0/3372898245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff134105470 0x7ff134105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 --2- 172.21.15.19:0/3372898245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105dc0 0x7ff13410e310 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 -- 172.21.15.19:0/3372898245 >> 172.21.15.19:0/3372898245 conn(0x7ff1340fef20 msgr2=0x7ff134101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 -- 172.21.15.19:0/3372898245 shutdown_connections 2024-05-17T09:20:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.147+0000 7ff13890c700 1 -- 172.21.15.19:0/3372898245 wait complete. 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.148+0000 7ff13890c700 1 Processor -- start 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 -- start start 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 0x7ff134103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff134105dc0 0x7ff1341036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13410e8e0 0x7ff134103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff134114310 con 0x7ff134105470 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff134114190 con 0x7ff13410e8e0 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13890c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff134114490 con 0x7ff134105dc0 2024-05-17T09:20:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 0x7ff134103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 0x7ff134103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41954/0 (socket says 172.21.15.19:41954) 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 -- 172.21.15.19:0/261704880 learned_addr learned my addr 172.21.15.19:0/261704880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff132d9d700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13410e8e0 0x7ff134103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff131d9b700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff134105dc0 0x7ff1341036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 -- 172.21.15.19:0/261704880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff134105dc0 msgr2=0x7ff1341036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff134105dc0 0x7ff1341036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 -- 172.21.15.19:0/261704880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13410e8e0 msgr2=0x7ff134103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13410e8e0 0x7ff134103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff13259c700 1 -- 172.21.15.19:0/261704880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1341a9070 con 0x7ff134105470 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.149+0000 7ff131d9b700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff134105dc0 0x7ff1341036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.150+0000 7ff13259c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 0x7ff134103160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c9b0 tx=0x7ff12400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.150+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124007b00 con 0x7ff134105470 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.150+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1341a9360 con 0x7ff134105470 2024-05-17T09:20:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.150+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff124007ca0 con 0x7ff134105470 2024-05-17T09:20:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.150+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124018660 con 0x7ff134105470 2024-05-17T09:20:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.150+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341a98a0 con 0x7ff134105470 2024-05-17T09:20:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.151+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1340512b0 con 0x7ff134105470 2024-05-17T09:20:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.152+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff12400f450 con 0x7ff134105470 2024-05-17T09:20:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.153+0000 7ff1237fe700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff11c0b2880 0x7ff11c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:20:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.153+0000 7ff131d9b700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff11c0b2880 0x7ff11c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:20:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.153+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff1240e28c0 con 0x7ff134105470 2024-05-17T09:20:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.154+0000 7ff131d9b700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff11c0b2880 0x7ff11c0b4d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff12800b5c0 tx=0x7ff128005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:20:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.157+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff124014030 con 0x7ff134105470 2024-05-17T09:20:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.435+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff134104690 con 0x7ff134105470 2024-05-17T09:20:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.436+0000 7ff1237fe700 1 -- 172.21.15.19:0/261704880 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7ff1240aa260 con 0x7ff134105470 2024-05-17T09:20:59.436 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:20:59.436 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.438+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff11c0b2880 msgr2=0x7ff11c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.438+0000 7ff13890c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff11c0b2880 0x7ff11c0b4d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff12800b5c0 tx=0x7ff128005960 comp rx=0 tx=0).stop 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.438+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 msgr2=0x7ff134103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.438+0000 7ff13890c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 0x7ff134103160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c9b0 tx=0x7ff12400ccc0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 shutdown_connections 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13410e8e0 0x7ff134103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff134105470 0x7ff134103160 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff11c0b2880 0x7ff11c0b4d90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 --2- 172.21.15.19:0/261704880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff134105dc0 0x7ff1341036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:20:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 >> 172.21.15.19:0/261704880 conn(0x7ff1340fef20 msgr2=0x7ff1340ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:20:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 shutdown_connections 2024-05-17T09:20:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:20:59.439+0000 7ff13890c700 1 -- 172.21.15.19:0/261704880 wait complete. 2024-05-17T09:20:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:21:00.508 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:00 smithi069 ceph-mon[35081]: pgmap v475: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:00.508 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:00 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/261704880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:00.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:00 smithi079 ceph-mon[35502]: pgmap v475: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:00 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/261704880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:00.540 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:00 smithi019 ceph-mon[31620]: pgmap v475: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:00.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:00 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/261704880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:00.858 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id f8acff7d --release --await' 2024-05-17T09:21:00.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id f8acff7d --release --await 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 -- 172.21.15.19:0/414126262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 msgr2=0x7f896010ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 --2- 172.21.15.19:0/414126262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f896010ae80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8958009ca0 tx=0x7f895802a8c0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 -- 172.21.15.19:0/414126262 shutdown_connections 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 --2- 172.21.15.19:0/414126262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f896010ae80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 --2- 172.21.15.19:0/414126262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8960103d60 0x7f8960104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 --2- 172.21.15.19:0/414126262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960102b20 0x7f8960103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 -- 172.21.15.19:0/414126262 >> 172.21.15.19:0/414126262 conn(0x7f89600fdd50 msgr2=0x7f89601001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 -- 172.21.15.19:0/414126262 shutdown_connections 2024-05-17T09:21:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.272+0000 7f8966300640 1 -- 172.21.15.19:0/414126262 wait complete. 2024-05-17T09:21:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 Processor -- start 2024-05-17T09:21:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 -- start start 2024-05-17T09:21:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8960102b20 0x7f896019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960103d60 0x7f896019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f89601a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f896010e530 con 0x7f8960108a60 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f896010e3b0 con 0x7f8960103d60 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8966300640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f896010e6b0 con 0x7f8960102b20 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f896489d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f89601a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f896489d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f89601a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41974/0 (socket says 172.21.15.19:41974) 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f8957fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960103d60 0x7f896019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.273+0000 7f896489d640 1 -- 172.21.15.19:0/2896879795 learned_addr learned my addr 172.21.15.19:0/2896879795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f895ffff640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8960102b20 0x7f896019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f896489d640 1 -- 172.21.15.19:0/2896879795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8960102b20 msgr2=0x7f896019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f896489d640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8960102b20 0x7f896019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f896489d640 1 -- 172.21.15.19:0/2896879795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960103d60 msgr2=0x7f896019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f896489d640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960103d60 0x7f896019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f896489d640 1 -- 172.21.15.19:0/2896879795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89601a1cb0 con 0x7f8960108a60 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f8957fff640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960103d60 0x7f896019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f896489d640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f89601a1640 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8958005e30 tx=0x7f8958003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f895dffb640 1 -- 172.21.15.19:0/2896879795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958032e90 con 0x7f8960108a60 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f895dffb640 1 -- 172.21.15.19:0/2896879795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89580044d0 con 0x7f8960108a60 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f895dffb640 1 -- 172.21.15.19:0/2896879795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958030ce0 con 0x7f8960108a60 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89601a1f40 con 0x7f8960108a60 2024-05-17T09:21:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.274+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89601a23a0 con 0x7f8960108a60 2024-05-17T09:21:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.277+0000 7f896d1b1740 1 Processor -- start 2024-05-17T09:21:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.277+0000 7f895dffb640 1 -- 172.21.15.19:0/2896879795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8958004040 con 0x7f8960108a60 2024-05-17T09:21:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 -- start start 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56087b216c30 con 0x56087af031b0 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56087b2067d0 con 0x56087aeda2f0 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f896d1b1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56087b20a810 con 0x56087aee24d0 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f8957fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f895ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f8957fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41988/0 (socket says 172.21.15.19:41988) 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f8957fff640 1 -- 172.21.15.19:0/908451800 learned_addr learned my addr 172.21.15.19:0/908451800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f8957fff640 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f895ffff640 1 -- 172.21.15.19:0/908451800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 msgr2=0x56087b2ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f895ffff640 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f895ffff640 1 -- 172.21.15.19:0/908451800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 msgr2=0x56087b2a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f895ffff640 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f895ffff640 1 -- 172.21.15.19:0/908451800 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087af1b210 con 0x56087aeda2f0 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.278+0000 7f8957fff640 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f895dffb640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f894c0b28a0 0x7f894c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f895ffff640 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a4920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89601023f0 tx=0x7f89500113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e711ed7cfb7623 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f8957fff640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f894c0b28a0 0x7f894c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f89567fc640 1 -- 172.21.15.19:0/908451800 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950011de0 con 0x56087aeda2f0 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f89567fc640 1 -- 172.21.15.19:0/908451800 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89500128e0 con 0x56087aeda2f0 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f895dffb640 1 -- 172.21.15.19:0/2896879795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8958103ae0 con 0x7f8960108a60 2024-05-17T09:21:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.279+0000 7f89567fc640 1 -- 172.21.15.19:0/908451800 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895000f880 con 0x56087aeda2f0 2024-05-17T09:21:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 -- 172.21.15.19:0/908451800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 msgr2=0x56087b2a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f8957fff640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f894c0b28a0 0x7f894c0b4db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x56087b2a5570 tx=0x7f894800bc10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a4920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89601023f0 tx=0x7f89500113e0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 -- 172.21.15.19:0/908451800 shutdown_connections 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a4920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 --2- 172.21.15.19:0/908451800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 -- 172.21.15.19:0/908451800 >> 172.21.15.19:0/908451800 conn(0x56087aef7300 msgr2=0x56087b29fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 -- 172.21.15.19:0/908451800 shutdown_connections 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 -- 172.21.15.19:0/908451800 wait complete. 2024-05-17T09:21:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 -- start start 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.280+0000 7f896d1b1740 1 Processor -- start 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b2ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56087aeec790 con 0x56087af031b0 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56087b1c2890 con 0x56087aee24d0 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56087b2ac270 con 0x56087aeda2f0 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896d1b1740 10 client.0 _ll_register_callbacks cb 0x56087b1c5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f8957fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b2ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f895ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41994/0 (socket says 172.21.15.19:41994) 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 -- 172.21.15.19:0/3133572441 learned_addr learned my addr 172.21.15.19:0/3133572441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 -- 172.21.15.19:0/3133572441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 msgr2=0x56087b2adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 -- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 msgr2=0x56087b2ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b2ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 -- 172.21.15.19:0/3133572441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087afe7620 con 0x56087af031b0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f896489d640 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2b5180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f89580c6dd0 tx=0x7f89580043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbb5c02f83e8204 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.281+0000 7f893f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f893f7fe640 1 -- 172.21.15.19:0/3133572441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89580cad50 con 0x56087af031b0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f893f7fe640 1 -- 172.21.15.19:0/3133572441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f89580e53d0 con 0x56087af031b0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89500128e0 con 0x56087af031b0 2024-05-17T09:21:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f893f7fe640 1 -- 172.21.15.19:0/3133572441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89580cf100 con 0x56087af031b0 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56087b2b80b0 con 0x56087af031b0 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f893f7fe640 1 -- 172.21.15.19:0/3133572441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89500128e0 con 0x56087af031b0 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f893f7fe640 1 -- 172.21.15.19:0/3133572441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f89580e4020 con 0x56087af031b0 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 20 client.15735 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087aec1d80 0x56087b2bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 4 client.15735 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.282+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56087b218b80 con 0x56087aec1d80 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.283+0000 7f895ffff640 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087aec1d80 0x56087b2bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.283+0000 7f893f7fe640 10 client.15735 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.283+0000 7f895ffff640 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087aec1d80 0x56087b2bb950 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0fdd3fbad02020 server_cookie=ff6b99576495f1ed in_seq=0 out_seq=0 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f893f7fe640 1 -- 172.21.15.19:0/3133572441 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56087b218b80 con 0x56087aec1d80 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f893f7fe640 10 client.15735 handle_command_reply: tid=0 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 client.15735 shutdown 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 msgr2=0x56087b2b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2b5180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f89580c6dd0 tx=0x7f89580043f0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 shutdown_connections 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087aec1d80 0x56087b2bb950 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087af031b0 0x56087b2b5180 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f893f7fe640 0 client.15735 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b2ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 --2- 172.21.15.19:0/3133572441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 >> 172.21.15.19:0/3133572441 conn(0x56087aef7300 msgr2=0x56087b2a2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 shutdown_connections 2024-05-17T09:21:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 1 -- 172.21.15.19:0/3133572441 wait complete. 2024-05-17T09:21:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.284+0000 7f896d1b1740 20 client.15735 trim_cache size 0 max 16384 2024-05-17T09:21:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.299+0000 7f896d1b1740 1 Processor -- start 2024-05-17T09:21:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 -- start start 2024-05-17T09:21:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89601af0f0 con 0x56087aec1d80 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f89601ac9c0 con 0x56087aeda2f0 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896d1b1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f89601aa810 con 0x56087aee24d0 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f895ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f8957fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f895ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41998/0 (socket says 172.21.15.19:41998) 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f895ffff640 1 -- 172.21.15.19:0/1891633446 learned_addr learned my addr 172.21.15.19:0/1891633446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f896489d640 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f8957fff640 1 -- 172.21.15.19:0/1891633446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 msgr2=0x56087b2b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f8957fff640 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f8957fff640 1 -- 172.21.15.19:0/1891633446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 msgr2=0x56087b2a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f8957fff640 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f8957fff640 1 -- 172.21.15.19:0/1891633446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087b2ad6b0 con 0x56087aeda2f0 2024-05-17T09:21:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.300+0000 7f895ffff640 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.301+0000 7f8957fff640 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a3780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x56087b2aed50 tx=0x7f8948011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b72b948442eea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.301+0000 7f8955ffb640 1 -- 172.21.15.19:0/1891633446 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948012530 con 0x56087aeda2f0 2024-05-17T09:21:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.301+0000 7f8955ffb640 1 -- 172.21.15.19:0/1891633446 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8948012b60 con 0x56087aeda2f0 2024-05-17T09:21:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.301+0000 7f8955ffb640 1 -- 172.21.15.19:0/1891633446 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948023980 con 0x56087aeda2f0 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 -- 172.21.15.19:0/1891633446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 msgr2=0x56087b2a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a3780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x56087b2aed50 tx=0x7f8948011000 comp rx=0 tx=0).stop 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 -- 172.21.15.19:0/1891633446 shutdown_connections 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aee24d0 0x56087b2b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aeda2f0 0x56087b2a3780 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 --2- 172.21.15.19:0/1891633446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 -- 172.21.15.19:0/1891633446 >> 172.21.15.19:0/1891633446 conn(0x56087aef7300 msgr2=0x56087b2a2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 -- 172.21.15.19:0/1891633446 shutdown_connections 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 -- 172.21.15.19:0/1891633446 wait complete. 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 -- start start 2024-05-17T09:21:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.302+0000 7f896d1b1740 1 Processor -- start 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b321fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89601aae80 con 0x56087aec1d80 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f89601ac2f0 con 0x56087aee24d0 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f89601ab550 con 0x56087aeda2f0 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896d1b1740 10 client.0 _ll_register_callbacks cb 0x7f89580ca750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42010/0 (socket says 172.21.15.19:42010) 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 -- 172.21.15.19:0/226884132 learned_addr learned my addr 172.21.15.19:0/226884132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f896489d640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b321fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f8957fff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 -- 172.21.15.19:0/226884132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 msgr2=0x56087b2b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 -- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 msgr2=0x56087b321fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b321fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 -- 172.21.15.19:0/226884132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087b322510 con 0x56087aec1d80 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f8957fff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f895ffff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2b2030 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f895001c040 tx=0x7f8950011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdbf58b443ed2f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.303+0000 7f893f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f893f7fe640 1 -- 172.21.15.19:0/226884132 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950005450 con 0x56087aec1d80 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f893f7fe640 1 -- 172.21.15.19:0/226884132 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8950004020 con 0x56087aec1d80 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8948012b60 con 0x56087aec1d80 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f893f7fe640 1 -- 172.21.15.19:0/226884132 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8950039050 con 0x56087aec1d80 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 10 client.15741 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56087b3236d0 con 0x56087aec1d80 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f893f7fe640 1 -- 172.21.15.19:0/226884132 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8948012b60 con 0x56087aec1d80 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f893f7fe640 1 -- 172.21.15.19:0/226884132 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8950004b80 con 0x56087aec1d80 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 20 client.15741 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087af031b0 0x56087b325e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 4 client.15741 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.304+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56087b212360 con 0x56087af031b0 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.305+0000 7f8957fff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087af031b0 0x56087b325e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.305+0000 7f893f7fe640 10 client.15741 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.305+0000 7f8957fff640 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087af031b0 0x56087b325e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6a609b46330271 server_cookie=f626bb0cf8ed8762 in_seq=0 out_seq=0 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f893f7fe640 1 -- 172.21.15.19:0/226884132 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56087b212360 con 0x56087af031b0 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f893f7fe640 10 client.15741 handle_command_reply: tid=0 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 client.15741 shutdown 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 msgr2=0x56087b2b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2b2030 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f895001c040 tx=0x7f8950011d20 comp rx=0 tx=0).stop 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 shutdown_connections 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56087af031b0 0x56087b325e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087aee24d0 0x56087b321fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56087aeda2f0 0x56087b2b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f893f7fe640 0 client.15741 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 --2- 172.21.15.19:0/226884132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087aec1d80 0x56087b2b2030 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 >> 172.21.15.19:0/226884132 conn(0x56087aef7300 msgr2=0x56087b2a3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 shutdown_connections 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 1 -- 172.21.15.19:0/226884132 wait complete. 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.309+0000 7f896d1b1740 20 client.15741 trim_cache size 0 max 16384 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f894c0b28a0 msgr2=0x7f894c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f894c0b28a0 0x7f894c0b4db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x56087b2a5570 tx=0x7f894800bc10 comp rx=0 tx=0).stop 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 msgr2=0x7f89601a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f89601a1640 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8958005e30 tx=0x7f8958003ba0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 shutdown_connections 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f894c0b28a0 0x7f894c0b4db0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8960108a60 0x7f89601a1640 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8960103d60 0x7f896019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 --2- 172.21.15.19:0/2896879795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8960102b20 0x7f896019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 >> 172.21.15.19:0/2896879795 conn(0x7f89600fdd50 msgr2=0x7f89601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 shutdown_connections 2024-05-17T09:21:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:01.310+0000 7f8966300640 1 -- 172.21.15.19:0/2896879795 wait complete. 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 -- 172.21.15.19:0/1863755000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388810e900 msgr2=0x7f3888110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1863755000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388810e900 0x7f3888110de0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f388000b0a0 tx=0x7f388000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 -- 172.21.15.19:0/1863755000 shutdown_connections 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1863755000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f388810e900 0x7f3888110de0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1863755000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38881054d0 0x7f38881058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1863755000 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3888105e20 0x7f388810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 -- 172.21.15.19:0/1863755000 >> 172.21.15.19:0/1863755000 conn(0x7f38880fef20 msgr2=0x7f3888101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 -- 172.21.15.19:0/1863755000 shutdown_connections 2024-05-17T09:21:01.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.342+0000 7f388f0fc700 1 -- 172.21.15.19:0/1863755000 wait complete. 2024-05-17T09:21:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 Processor -- start 2024-05-17T09:21:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 -- start start 2024-05-17T09:21:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 0x7f38881a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3888105e20 0x7f38881a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388810e900 0x7f388819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3888114330 con 0x7f38881054d0 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38881141b0 con 0x7f3888105e20 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.343+0000 7f388f0fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38881144b0 con 0x7f388810e900 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 0x7f38881a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 0x7f38881a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42034/0 (socket says 172.21.15.19:42034) 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 -- 172.21.15.19:0/1624342691 learned_addr learned my addr 172.21.15.19:0/1624342691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f3887fff700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3888105e20 0x7f38881a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388d699700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388810e900 0x7f388819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 -- 172.21.15.19:0/1624342691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388810e900 msgr2=0x7f388819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388810e900 0x7f388819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 -- 172.21.15.19:0/1624342691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3888105e20 msgr2=0x7f38881a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3888105e20 0x7f38881a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 -- 172.21.15.19:0/1624342691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388819b1b0 con 0x7f38881054d0 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f3887fff700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3888105e20 0x7f38881a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f388ce98700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 0x7f38881a0890 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f387800e9a0 tx=0x7f387800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.344+0000 7f3885ffb700 1 -- 172.21.15.19:0/1624342691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387800ca10 con 0x7f38881054d0 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.345+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388819b440 con 0x7f38881054d0 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.345+0000 7f3885ffb700 1 -- 172.21.15.19:0/1624342691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f387800cbb0 con 0x7f38881054d0 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.345+0000 7f3885ffb700 1 -- 172.21.15.19:0/1624342691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878018680 con 0x7f38881054d0 2024-05-17T09:21:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.345+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a7740 con 0x7f38881054d0 2024-05-17T09:21:01.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.348+0000 7f3885ffb700 1 -- 172.21.15.19:0/1624342691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3878010430 con 0x7f38881054d0 2024-05-17T09:21:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f3885ffb700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38700b2880 0x7f38700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 Processor -- start 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 -- start start 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f3887fff700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38700b2880 0x7f38700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f3885ffb700 1 -- 172.21.15.19:0/1624342691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f38780e27c0 con 0x7f38881054d0 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f35a7450 0x55b8f36ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f3529ec0 0x55b8f36ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b8f35d6d10 con 0x55b8f36a8440 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.349+0000 7f389db11b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b8f35c68b0 con 0x55b8f3529ec0 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f389db11b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b8f35ca8f0 con 0x55b8f35a7450 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42042/0 (socket says 172.21.15.19:42042) 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 -- 172.21.15.19:0/1044726178 learned_addr learned my addr 172.21.15.19:0/1044726178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f3887fff700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38700b2880 0x7f38700b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3888102340 tx=0x7f387c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f3887fff700 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f3529ec0 0x55b8f36ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388ce98700 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f35a7450 0x55b8f36ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 -- 172.21.15.19:0/1044726178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f35a7450 msgr2=0x55b8f36ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f35a7450 0x55b8f36ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 -- 172.21.15.19:0/1044726178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f3529ec0 msgr2=0x55b8f36ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f3529ec0 0x55b8f36ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 -- 172.21.15.19:0/1044726178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f3516950 con 0x55b8f36a8440 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f388d699700 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36a8850 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3880009d90 tx=0x7f3880009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b32e33686fc4775 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f386effd700 1 -- 172.21.15.19:0/1044726178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388000bcb0 con 0x55b8f36a8440 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f386effd700 1 -- 172.21.15.19:0/1044726178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f388000be50 con 0x55b8f36a8440 2024-05-17T09:21:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.350+0000 7f386effd700 1 -- 172.21.15.19:0/1044726178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800075c0 con 0x55b8f36a8440 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 -- 172.21.15.19:0/1044726178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 msgr2=0x55b8f36a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36a8850 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3880009d90 tx=0x7f3880009dc0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 -- 172.21.15.19:0/1044726178 shutdown_connections 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f35a7450 0x55b8f36ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36a8850 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 --2- 172.21.15.19:0/1044726178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f3529ec0 0x55b8f36ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 -- 172.21.15.19:0/1044726178 >> 172.21.15.19:0/1044726178 conn(0x55b8f3568d00 msgr2=0x55b8f36a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 -- 172.21.15.19:0/1044726178 shutdown_connections 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.351+0000 7f389db11b80 1 -- 172.21.15.19:0/1044726178 wait complete. 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 -- start start 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 Processor -- start 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 0x55b8f36b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f36a8440 0x55b8f36b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b8f34cd020 con 0x55b8f35a7450 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3568840 con 0x55b8f36a8440 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b8f36a9210 con 0x55b8f3529ec0 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.352+0000 7f389db11b80 10 client.0 _ll_register_callbacks cb 0x55b8f35619a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f3887fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 0x55b8f36b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f3887fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 0x55b8f36b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42056/0 (socket says 172.21.15.19:42056) 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:40532/0 (socket says 172.21.15.19:40532) 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f3887fff700 1 -- 172.21.15.19:0/933322800 learned_addr learned my addr 172.21.15.19:0/933322800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388d699700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f36a8440 0x55b8f36b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 -- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f36a8440 msgr2=0x55b8f36b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f36a8440 0x55b8f36b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 -- 172.21.15.19:0/933322800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 msgr2=0x55b8f36b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 0x55b8f36b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 -- 172.21.15.19:0/933322800 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f3565710 con 0x55b8f3529ec0 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388d699700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f36a8440 0x55b8f36b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f3887fff700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 0x55b8f36b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f388ce98700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36b1c50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38780a6c40 tx=0x7f38780a9f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e047bdff8afa628c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f3867fff700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.353+0000 7f3867fff700 1 -- 172.21.15.19:0/933322800 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38780aa0d0 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388000be50 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f3867fff700 1 -- 172.21.15.19:0/933322800 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f38780cc0d0 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f3867fff700 1 -- 172.21.15.19:0/933322800 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f387806e050 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 10 client.25474 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8f36bf550 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f3867fff700 1 -- 172.21.15.19:0/933322800 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f388000be50 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f3867fff700 1 -- 172.21.15.19:0/933322800 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f387806f240 con 0x55b8f3529ec0 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 20 client.25474 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f36c1be0 0x55b8f36c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 4 client.25474 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.354+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8f35d8c60 con 0x55b8f36c1be0 2024-05-17T09:21:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.355+0000 7f3887fff700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f36c1be0 0x55b8f36c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.355+0000 7f3887fff700 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f36c1be0 0x55b8f36c4000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b1f1a7761af2ef server_cookie=d0bcc4bbef4078f2 in_seq=0 out_seq=0 2024-05-17T09:21:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.355+0000 7f3867fff700 10 client.25474 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f3867fff700 1 -- 172.21.15.19:0/933322800 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8f35d8c60 con 0x55b8f36c1be0 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f3867fff700 10 client.25474 handle_command_reply: tid=0 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 client.25474 shutdown 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 msgr2=0x55b8f36b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36b1c50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38780a6c40 tx=0x7f38780a9f10 comp rx=0 tx=0).stop 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 shutdown_connections 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36b1c50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f36c1be0 0x55b8f36c4000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f36a8440 0x55b8f36b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 --2- 172.21.15.19:0/933322800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f35a7450 0x55b8f36b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f3867fff700 0 client.25474 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 >> 172.21.15.19:0/933322800 conn(0x55b8f3568d00 msgr2=0x55b8f36a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 shutdown_connections 2024-05-17T09:21:01.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 1 -- 172.21.15.19:0/933322800 wait complete. 2024-05-17T09:21:01.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.356+0000 7f389db11b80 20 client.25474 trim_cache size 0 max 16384 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.380+0000 7f389db11b80 1 Processor -- start 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.380+0000 7f389db11b80 1 -- start start 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.380+0000 7f389db11b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f369da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.380+0000 7f389db11b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.380+0000 7f389db11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f389db11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38881b0930 con 0x55b8f36a8440 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f389db11b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38881ae1c0 con 0x55b8f35a7450 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f389db11b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38881ac030 con 0x55b8f3529ec0 2024-05-17T09:21:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f388d699700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51850/0 (socket says 172.21.15.19:51850) 2024-05-17T09:21:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 -- 172.21.15.19:0/4100299884 learned_addr learned my addr 172.21.15.19:0/4100299884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f388d699700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42068/0 (socket says 172.21.15.19:42068) 2024-05-17T09:21:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f388ce98700 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f369da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 -- 172.21.15.19:0/4100299884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 msgr2=0x55b8f369da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f369da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 -- 172.21.15.19:0/4100299884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 msgr2=0x55b8f36c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 -- 172.21.15.19:0/4100299884 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f36be180 con 0x55b8f35a7450 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f388d699700 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.381+0000 7f3887fff700 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36c97c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f387c009fd0 tx=0x7f387c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f1338273182272 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.382+0000 7f386e7fc700 1 -- 172.21.15.19:0/4100299884 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c02a4e0 con 0x55b8f35a7450 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.382+0000 7f386e7fc700 1 -- 172.21.15.19:0/4100299884 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f387c02ab10 con 0x55b8f35a7450 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.382+0000 7f386e7fc700 1 -- 172.21.15.19:0/4100299884 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c01eb40 con 0x55b8f35a7450 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 -- 172.21.15.19:0/4100299884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 msgr2=0x55b8f36c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36c97c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f387c009fd0 tx=0x7f387c014000 comp rx=0 tx=0).stop 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 -- 172.21.15.19:0/4100299884 shutdown_connections 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f369da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 --2- 172.21.15.19:0/4100299884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36c97c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 -- 172.21.15.19:0/4100299884 >> 172.21.15.19:0/4100299884 conn(0x55b8f3568d00 msgr2=0x55b8f36b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 -- 172.21.15.19:0/4100299884 shutdown_connections 2024-05-17T09:21:01.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 -- 172.21.15.19:0/4100299884 wait complete. 2024-05-17T09:21:01.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 -- start start 2024-05-17T09:21:01.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.383+0000 7f389db11b80 1 Processor -- start 2024-05-17T09:21:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38881ac680 con 0x55b8f36a8440 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38881adaf0 con 0x55b8f35a7450 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38881acd50 con 0x55b8f3529ec0 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f389db11b80 10 client.0 _ll_register_callbacks cb 0x55b8f35619a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f388d699700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51860/0 (socket says 172.21.15.19:51860) 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 -- 172.21.15.19:0/4082870626 learned_addr learned my addr 172.21.15.19:0/4082870626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f388ce98700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 -- 172.21.15.19:0/4082870626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 msgr2=0x55b8f36ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 -- 172.21.15.19:0/4082870626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 msgr2=0x55b8f36b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f3887fff700 1 -- 172.21.15.19:0/4082870626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f36b7ff0 con 0x55b8f35a7450 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f388d699700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.384+0000 7f388ce98700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3887fff700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36b7570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f387c02ad70 tx=0x7f387c01e3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600af3a3ddd8c285 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3867fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3867fff700 1 -- 172.21.15.19:0/4082870626 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c02a4e0 con 0x55b8f35a7450 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3867fff700 1 -- 172.21.15.19:0/4082870626 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f387c02a680 con 0x55b8f35a7450 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f387c02ab10 con 0x55b8f35a7450 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3867fff700 1 -- 172.21.15.19:0/4082870626 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f387c040050 con 0x55b8f35a7450 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f389db11b80 10 client.25454 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8f372e2f0 con 0x55b8f35a7450 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3867fff700 1 -- 172.21.15.19:0/4082870626 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f387c01ee50 con 0x55b8f35a7450 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f3867fff700 1 -- 172.21.15.19:0/4082870626 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f387c03b030 con 0x55b8f35a7450 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f389db11b80 10 client.25454 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f389db11b80 10 client.25454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.385+0000 7f389db11b80 20 client.25454 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.386+0000 7f389db11b80 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f372eab0 0x55b8f3730ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.386+0000 7f389db11b80 4 client.25454 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.386+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f387c02ab10 con 0x55b8f372eab0 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.386+0000 7f388ce98700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f372eab0 0x55b8f3730ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.386+0000 7f3867fff700 10 client.25454 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.386+0000 7f388ce98700 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f372eab0 0x55b8f3730ed0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c2f4fbd9b8cf4d server_cookie=e7e48ee47d33fb8d in_seq=0 out_seq=0 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.387+0000 7f3867fff700 1 -- 172.21.15.19:0/4082870626 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f387c02ab10 con 0x55b8f372eab0 2024-05-17T09:21:01.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.387+0000 7f3867fff700 10 client.25454 handle_command_reply: tid=0 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.387+0000 7f389db11b80 1 client.25454 shutdown 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 msgr2=0x55b8f36b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36b7570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f387c02ad70 tx=0x7f387c01e3f0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 shutdown_connections 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8f3529ec0 0x55b8f36ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8f372eab0 0x55b8f3730ed0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8f36a8440 0x55b8f36b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 --2- 172.21.15.19:0/4082870626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8f35a7450 0x55b8f36b7570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f3867fff700 0 client.25454 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 >> 172.21.15.19:0/4082870626 conn(0x55b8f3568d00 msgr2=0x55b8f36a7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 shutdown_connections 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 1 -- 172.21.15.19:0/4082870626 wait complete. 2024-05-17T09:21:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.388+0000 7f389db11b80 20 client.25454 trim_cache size 0 max 16384 2024-05-17T09:21:01.392 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:21:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "33a2970c-1c3d-4fc8-8e1e-d71fd6e513f4", 2024-05-17T09:21:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:21:01.392 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38700b2880 msgr2=0x7f38700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38700b2880 0x7f38700b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3888102340 tx=0x7f387c009260 comp rx=0 tx=0).stop 2024-05-17T09:21:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 msgr2=0x7f38881a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 0x7f38881a0890 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f387800e9a0 tx=0x7f387800ecb0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 shutdown_connections 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f38700b2880 0x7f38700b4d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f388810e900 0x7f388819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38881054d0 0x7f38881a0890 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 --2- 172.21.15.19:0/1624342691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3888105e20 0x7f38881a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 >> 172.21.15.19:0/1624342691 conn(0x7f38880fef20 msgr2=0x7f38880ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.389+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 shutdown_connections 2024-05-17T09:21:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:01.390+0000 7f388f0fc700 1 -- 172.21.15.19:0/1624342691 wait complete. 2024-05-17T09:21:02.042 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:02.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:21:02.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:02 smithi019 ceph-mon[31620]: pgmap v476: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:02.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:02 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:21:02.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:02 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:02.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:02 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:21:02.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:02 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:02 smithi079 ceph-mon[35502]: pgmap v476: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:02 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:21:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:02 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:02 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:21:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:02 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:02 smithi069 ceph-mon[35081]: pgmap v476: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:02 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:21:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:02 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:02 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:21:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:02 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:03.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:03 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:21:03.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:03 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:21:03.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:03 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 -- 172.21.15.19:0/841114679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4109860 msgr2=0x7fc0c41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 --2- 172.21.15.19:0/841114679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4109860 0x7fc0c41111c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00b0d0 tx=0x7fc0bc00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 -- 172.21.15.19:0/841114679 shutdown_connections 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 --2- 172.21.15.19:0/841114679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c4109860 0x7fc0c41111c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 --2- 172.21.15.19:0/841114679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c41045e0 0x7fc0c41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 --2- 172.21.15.19:0/841114679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 0x7fc0c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 -- 172.21.15.19:0/841114679 >> 172.21.15.19:0/841114679 conn(0x7fc0c407af30 msgr2=0x7fc0c407b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 -- 172.21.15.19:0/841114679 shutdown_connections 2024-05-17T09:21:04.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.978+0000 7fc0ca467700 1 -- 172.21.15.19:0/841114679 wait complete. 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.979+0000 7fc0ca467700 1 Processor -- start 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.979+0000 7fc0ca467700 1 -- start start 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0ca467700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 0x7fc0c41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0ca467700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 0x7fc0c41a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0ca467700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4109860 0x7fc0c419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0ca467700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4114340 con 0x7fc0c41045e0 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 0x7fc0c41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0ca467700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41141c0 con 0x7fc0c4109860 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0ca467700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41144c0 con 0x7fc0c4108f10 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 0x7fc0c41a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47202/0 (socket says 172.21.15.19:47202) 2024-05-17T09:21:04.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 -- 172.21.15.19:0/1111443706 learned_addr learned my addr 172.21.15.19:0/1111443706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 0x7fc0c41a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4109860 0x7fc0c419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 -- 172.21.15.19:0/1111443706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 msgr2=0x7fc0c41a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 0x7fc0c41a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 -- 172.21.15.19:0/1111443706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4109860 msgr2=0x7fc0c419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4109860 0x7fc0c419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 -- 172.21.15.19:0/1111443706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c419b1e0 con 0x7fc0c41045e0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 0x7fc0c41a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.980+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 0x7fc0c41a08c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e9a0 tx=0x7fc0b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.981+0000 7fc0c17fa700 1 -- 172.21.15.19:0/1111443706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400cbd0 con 0x7fc0c41045e0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.981+0000 7fc0c17fa700 1 -- 172.21.15.19:0/1111443706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0b400cd70 con 0x7fc0c41045e0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.981+0000 7fc0c17fa700 1 -- 172.21.15.19:0/1111443706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4018680 con 0x7fc0c41045e0 2024-05-17T09:21:04.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.981+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c419b4d0 con 0x7fc0c41045e0 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.981+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c41a7770 con 0x7fc0c41045e0 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.985+0000 7fc0c17fa700 1 -- 172.21.15.19:0/1111443706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc0b4018820 con 0x7fc0c41045e0 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 Processor -- start 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0c17fa700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc0ac0b2900 0x7fc0ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 -- start start 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc0ac0b2900 0x7fc0ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0c17fa700 1 -- 172.21.15.19:0/1111443706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0b40e28e0 con 0x7fc0c41045e0 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cb1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 0x560cc3cab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560cc3bd4e70 con 0x560cc3c13530 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560cc3bc4a10 con 0x560cc3cab430 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.986+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560cc3bc8a50 con 0x560cc3b27f40 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 0x560cc3cab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 0x560cc3cab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54352/0 (socket says 172.21.15.19:54352) 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 -- 172.21.15.19:0/3485035809 learned_addr learned my addr 172.21.15.19:0/3485035809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cb1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc0ac0b2900 0x7fc0ac0b4e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009900 tx=0x7fc0bc009c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 -- 172.21.15.19:0/3485035809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 msgr2=0x560cc3cb1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cb1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 -- 172.21.15.19:0/3485035809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 msgr2=0x560cc3cb1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cb1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 -- 172.21.15.19:0/3485035809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc3b6e0b0 con 0x560cc3cab430 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cb1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.987+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 0x560cc3cab840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00bad0 tx=0x7fc0bc024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350f1926903cb25a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.988+0000 7fc0aa7fc700 1 -- 172.21.15.19:0/3485035809 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc025070 con 0x560cc3cab430 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.988+0000 7fc0aa7fc700 1 -- 172.21.15.19:0/3485035809 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0bc00bde0 con 0x560cc3cab430 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.988+0000 7fc0aa7fc700 1 -- 172.21.15.19:0/3485035809 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc005440 con 0x560cc3cab430 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3485035809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 msgr2=0x560cc3cab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 0x560cc3cab840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00bad0 tx=0x7fc0bc024000 comp rx=0 tx=0).stop 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3485035809 shutdown_connections 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3cab430 0x560cc3cab840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cb1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3485035809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cb1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3485035809 >> 172.21.15.19:0/3485035809 conn(0x560cc3b882c0 msgr2=0x560cc3ca1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3485035809 shutdown_connections 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3485035809 wait complete. 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.989+0000 7fc0d8e7cb80 1 -- start start 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 Processor -- start 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3b27f40 0x560cc3cb4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 0x560cc3cb5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3cab430 0x560cc3cb5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560cc3acb0a0 con 0x560cc3b27f40 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560cc3b2cea0 con 0x560cc3c13530 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 0x560cc3cb5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3b27f40 0x560cc3cb4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 0x560cc3cb5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54366/0 (socket says 172.21.15.19:54366) 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0c37fe700 1 -- 172.21.15.19:0/3837642081 learned_addr learned my addr 172.21.15.19:0/3837642081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560cc3cac210 con 0x560cc3cab430 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0d8e7cb80 10 client.0 _ll_register_callbacks cb 0x560cc3b5fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.990+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3cab430 0x560cc3cb5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0c37fe700 1 -- 172.21.15.19:0/3837642081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3cab430 msgr2=0x560cc3cb5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3cab430 0x560cc3cb5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0c37fe700 1 -- 172.21.15.19:0/3837642081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3b27f40 msgr2=0x560cc3cb4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3b27f40 0x560cc3cb4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0c37fe700 1 -- 172.21.15.19:0/3837642081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc3b149d0 con 0x560cc3c13530 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 0x560cc3cb5430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009ab0 tx=0x7fc0b8009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09b9bd50dbbbc81 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0a37fe700 1 -- 172.21.15.19:0/3837642081 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8004540 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0bc00bde0 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0a37fe700 1 -- 172.21.15.19:0/3837642081 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0b801b440 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0a37fe700 1 -- 172.21.15.19:0/3837642081 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0b8024050 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0d8e7cb80 10 client.25466 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.991+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560cc3b7e860 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0a37fe700 1 -- 172.21.15.19:0/3837642081 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0bc00bde0 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0a37fe700 1 -- 172.21.15.19:0/3837642081 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc0b8019020 con 0x560cc3c13530 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0d8e7cb80 10 client.25466 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0d8e7cb80 10 client.25466 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0d8e7cb80 20 client.25466 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3ca9630 0x560cc3ca9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0d8e7cb80 4 client.25466 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cc3c72fd0 con 0x560cc3ca9630 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.992+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3ca9630 0x560cc3ca9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3ca9630 0x560cc3ca9a60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37535a64255a4a9a server_cookie=3940897212fdcefa in_seq=0 out_seq=0 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0a37fe700 10 client.25466 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0a37fe700 1 -- 172.21.15.19:0/3837642081 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560cc3c72fd0 con 0x560cc3ca9630 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0a37fe700 10 client.25466 handle_command_reply: tid=0 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0d8e7cb80 1 client.25466 shutdown 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 msgr2=0x560cc3cb5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.993+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 0x560cc3cb5430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009ab0 tx=0x7fc0b8009dc0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 shutdown_connections 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3cab430 0x560cc3cb5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3ca9630 0x560cc3ca9a60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3b27f40 0x560cc3cb4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/3837642081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3c13530 0x560cc3cb5430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 >> 172.21.15.19:0/3837642081 conn(0x560cc3b882c0 msgr2=0x560cc3c9bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0a37fe700 0 client.25466 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 shutdown_connections 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/3837642081 wait complete. 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:03.994+0000 7fc0d8e7cb80 20 client.25466 trim_cache size 0 max 16384 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 Processor -- start 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 -- start start 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b882c0 0x560cc3cb0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b27f40 0x560cc3cc1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3c9bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41b09d0 con 0x560cc3c13530 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41ae260 con 0x560cc3b27f40 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41ac0b0 con 0x560cc3b882c0 2024-05-17T09:21:04.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.014+0000 7fc0c8a04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3c9bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3c9bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47230/0 (socket says 172.21.15.19:47230) 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 -- 172.21.15.19:0/1608772289 learned_addr learned my addr 172.21.15.19:0/1608772289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b882c0 0x560cc3cb0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b27f40 0x560cc3cc1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 -- 172.21.15.19:0/1608772289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b882c0 msgr2=0x560cc3cb0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b882c0 0x560cc3cb0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 -- 172.21.15.19:0/1608772289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b27f40 msgr2=0x560cc3cc1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b27f40 0x560cc3cc1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 -- 172.21.15.19:0/1608772289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc3cb3ed0 con 0x560cc3c13530 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3c9bbc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00d010 tx=0x7fc0bc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721b48faf23d46d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0a9ffb700 1 -- 172.21.15.19:0/1608772289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc037070 con 0x560cc3c13530 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0a9ffb700 1 -- 172.21.15.19:0/1608772289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0bc004a50 con 0x560cc3c13530 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.015+0000 7fc0a9ffb700 1 -- 172.21.15.19:0/1608772289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0052f0 con 0x560cc3c13530 2024-05-17T09:21:04.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/1608772289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 msgr2=0x560cc3c9bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3c9bbc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00d010 tx=0x7fc0bc00bf90 comp rx=0 tx=0).stop 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/1608772289 shutdown_connections 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b882c0 0x560cc3cb0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3c9bbc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/1608772289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b27f40 0x560cc3cc1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/1608772289 >> 172.21.15.19:0/1608772289 conn(0x560cc3ca9df0 msgr2=0x560cc3caf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/1608772289 shutdown_connections 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.016+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/1608772289 wait complete. 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 -- start start 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 Processor -- start 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cc0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b882c0 0x560cc3cc0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cc0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41ac720 con 0x560cc3c13530 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41adb90 con 0x560cc3b882c0 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41acdf0 con 0x560cc3b27f40 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0d8e7cb80 10 client.0 _ll_register_callbacks cb 0x560cc3b5fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0c8a04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cc0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.017+0000 7fc0c8a04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cc0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47234/0 (socket says 172.21.15.19:47234) 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 -- 172.21.15.19:0/407617314 learned_addr learned my addr 172.21.15.19:0/407617314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cc0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c37fe700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b882c0 0x560cc3cc0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 -- 172.21.15.19:0/407617314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 msgr2=0x560cc3cc0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cc0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 -- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b882c0 msgr2=0x560cc3cc0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b882c0 0x560cc3cc0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 -- 172.21.15.19:0/407617314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc3d2a620 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cc0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0c8a04700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cc0ad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009a20 tx=0x7fc0bc033700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd57c585e72887 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0a37fe700 1 -- 172.21.15.19:0/407617314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc040030 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0a37fe700 1 -- 172.21.15.19:0/407617314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0bc00f050 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0bc004a50 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0a37fe700 1 -- 172.21.15.19:0/407617314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0bc025040 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0d8e7cb80 10 client.15777 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.018+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560cc3d2b8e0 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0a37fe700 1 -- 172.21.15.19:0/407617314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0bc004a50 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0a37fe700 1 -- 172.21.15.19:0/407617314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc0bc028070 con 0x560cc3c13530 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0d8e7cb80 10 client.15777 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:04.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0d8e7cb80 10 client.15777 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0d8e7cb80 20 client.15777 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3cab430 0x560cc3d2db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0d8e7cb80 4 client.15777 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f8acff7d", "release": true, "await": true}] 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f8acff7d", "release": true, "await": true}) -- 0x560cc3bd6dc0 con 0x560cc3cab430 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.019+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3cab430 0x560cc3d2db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.020+0000 7fc0a37fe700 10 client.15777 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.020+0000 7fc0c3fff700 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3cab430 0x560cc3d2db10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d556e871a414b0a1 server_cookie=b0855520411cff27 in_seq=0 out_seq=0 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.021+0000 7fc0a37fe700 1 -- 172.21.15.19:0/407617314 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560cc3bd6dc0 con 0x560cc3cab430 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.021+0000 7fc0a37fe700 10 client.15777 handle_command_reply: tid=0 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.021+0000 7fc0d8e7cb80 1 client.15777 shutdown 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.021+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 msgr2=0x560cc3cc0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.021+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cc0ad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009a20 tx=0x7fc0bc033700 comp rx=0 tx=0).stop 2024-05-17T09:21:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 shutdown_connections 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560cc3b27f40 0x560cc3cc0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560cc3cab430 0x560cc3d2db10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560cc3c13530 0x560cc3cc0ad0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 --2- 172.21.15.19:0/407617314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560cc3b882c0 0x560cc3cc0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0a37fe700 0 client.15777 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 >> 172.21.15.19:0/407617314 conn(0x560cc3ca9df0 msgr2=0x560cc3c9bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 shutdown_connections 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 1 -- 172.21.15.19:0/407617314 wait complete. 2024-05-17T09:21:04.482 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 4, 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "f8acff7d": { 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4, 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:21:04.483 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.4, 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:04.484 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0d8e7cb80 20 client.15777 trim_cache size 0 max 16384 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc0ac0b2900 msgr2=0x7fc0ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.022+0000 7fc0ca467700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc0ac0b2900 0x7fc0ac0b4e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009900 tx=0x7fc0bc009c80 comp rx=0 tx=0).stop 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 msgr2=0x7fc0c41a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 0x7fc0c41a08c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e9a0 tx=0x7fc0b400ed60 comp rx=0 tx=0).stop 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 shutdown_connections 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4109860 0x7fc0c419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0c41045e0 0x7fc0c41a08c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc0ac0b2900 0x7fc0ac0b4e10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 --2- 172.21.15.19:0/1111443706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0c4108f10 0x7fc0c41a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 >> 172.21.15.19:0/1111443706 conn(0x7fc0c407af30 msgr2=0x7fc0c4101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 shutdown_connections 2024-05-17T09:21:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:04.023+0000 7fc0ca467700 1 -- 172.21.15.19:0/1111443706 wait complete. 2024-05-17T09:21:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:04 smithi079 ceph-mon[35502]: pgmap v477: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-17T09:21:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:04 smithi069 ceph-mon[35081]: pgmap v477: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-17T09:21:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:04 smithi019 ceph-mon[31620]: pgmap v477: 129 pgs: 129 active+clean; 669 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-17T09:21:05.775 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f8acff7d', total seconds elapsed: 31.0 2024-05-17T09:21:05.776 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.4 2024-05-17T09:21:05.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:21:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/3835076048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0107e00 msgr2=0x7fe5f0108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/3835076048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0107e00 0x7fe5f0108210 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80099c0 tx=0x7fe5d8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/3835076048 shutdown_connections 2024-05-17T09:21:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/3835076048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5f0108750 0x7fe5f01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/3835076048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f01034d0 0x7fe5f01038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/3835076048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0107e00 0x7fe5f0108210 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/3835076048 >> 172.21.15.19:0/3835076048 conn(0x7fe5f00fef20 msgr2=0x7fe5f0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/3835076048 shutdown_connections 2024-05-17T09:21:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.306+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/3835076048 wait complete. 2024-05-17T09:21:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.307+0000 7fe5f5bbb700 1 Processor -- start 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.307+0000 7fe5f5bbb700 1 -- start start 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5f5bbb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5f01034d0 0x7fe5f019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5f5bbb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0107e00 0x7fe5f019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5f5bbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 0x7fe5f019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5f5bbb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0113210 con 0x7fe5f0108750 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5f5bbb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0113090 con 0x7fe5f0107e00 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5f5bbb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0113390 con 0x7fe5f01034d0 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 0x7fe5f019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5f01034d0 0x7fe5f019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 0x7fe5f019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47246/0 (socket says 172.21.15.19:47246) 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 -- 172.21.15.19:0/2073036481 learned_addr learned my addr 172.21.15.19:0/2073036481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0107e00 0x7fe5f019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 -- 172.21.15.19:0/2073036481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5f01034d0 msgr2=0x7fe5f019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5f01034d0 0x7fe5f019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 -- 172.21.15.19:0/2073036481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0107e00 msgr2=0x7fe5f019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0107e00 0x7fe5f019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 -- 172.21.15.19:0/2073036481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f019fc40 con 0x7fe5f0108750 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.308+0000 7fe5effff700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 0x7fe5f019f5a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8007ee0 tx=0x7fe5e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.309+0000 7fe5ecff9700 1 -- 172.21.15.19:0/2073036481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800dd20 con 0x7fe5f0108750 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.309+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f019ff30 con 0x7fe5f0108750 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.309+0000 7fe5ecff9700 1 -- 172.21.15.19:0/2073036481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5e8004020 con 0x7fe5f0108750 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.309+0000 7fe5ecff9700 1 -- 172.21.15.19:0/2073036481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8015660 con 0x7fe5f0108750 2024-05-17T09:21:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.309+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f00ff330 con 0x7fe5f0108750 2024-05-17T09:21:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.312+0000 7fe5ecff9700 1 -- 172.21.15.19:0/2073036481 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe5e8015800 con 0x7fe5f0108750 2024-05-17T09:21:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.313+0000 7fe6045d0b80 1 Processor -- start 2024-05-17T09:21:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.313+0000 7fe5ecff9700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5dc0b2900 0x7fe5dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.313+0000 7fe6045d0b80 1 -- start start 2024-05-17T09:21:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.313+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5dc0b2900 0x7fe5dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.313+0000 7fe5ecff9700 1 -- 172.21.15.19:0/2073036481 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe5e80e1f90 con 0x7fe5f0108750 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 0x564fc1c61910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c61eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1c5bcd0 0x564fc1c5c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564fc1b89d70 con 0x564fc1adce40 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564fc1b79910 con 0x564fc1c5bcd0 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564fc1b7d950 con 0x564fc1bc8430 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 0x564fc1c61910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 0x564fc1c61910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:36148/0 (socket says 172.21.15.19:36148) 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2430333038 learned_addr learned my addr 172.21.15.19:0/2430333038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c61eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5effff700 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1c5bcd0 0x564fc1c5c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2430333038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1c5bcd0 msgr2=0x564fc1c5c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1c5bcd0 0x564fc1c5c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2430333038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 msgr2=0x564fc1c61eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c61eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2430333038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc1b22fb0 con 0x564fc1bc8430 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5dc0b2900 0x7fe5dc0b4e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009990 tx=0x7fe5d8005050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.314+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 0x564fc1c61910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000bd70 tx=0x7fe5e00106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bbb7dbe758b32aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.315+0000 7fe5e5ffb700 1 -- 172.21.15.19:0/2430333038 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e001c480 con 0x564fc1bc8430 2024-05-17T09:21:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.315+0000 7fe5e5ffb700 1 -- 172.21.15.19:0/2430333038 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5e001cab0 con 0x564fc1bc8430 2024-05-17T09:21:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.315+0000 7fe5e5ffb700 1 -- 172.21.15.19:0/2430333038 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011880 con 0x564fc1bc8430 2024-05-17T09:21:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2430333038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 msgr2=0x564fc1c61910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 0x564fc1c61910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000bd70 tx=0x7fe5e00106b0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2430333038 shutdown_connections 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1bc8430 0x564fc1c61910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1c5bcd0 0x564fc1c5c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2430333038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c61eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2430333038 >> 172.21.15.19:0/2430333038 conn(0x564fc1b3d1c0 msgr2=0x564fc1c567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2430333038 shutdown_connections 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2430333038 wait complete. 2024-05-17T09:21:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.316+0000 7fe6045d0b80 1 -- start start 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 Processor -- start 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1adce40 0x564fc1c655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c65b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 0x564fc1c66040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a7ffa0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564fc1ae1da0 con 0x564fc1bc8430 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.317+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564fc1c685a0 con 0x564fc1adce40 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe6045d0b80 10 client.0 _ll_register_callbacks cb 0x564fc1b14950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 0x564fc1c66040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 0x564fc1c66040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47250/0 (socket says 172.21.15.19:47250) 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 -- 172.21.15.19:0/1520446962 learned_addr learned my addr 172.21.15.19:0/1520446962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1adce40 0x564fc1c655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c65b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 -- 172.21.15.19:0/1520446962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1adce40 msgr2=0x564fc1c655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1adce40 0x564fc1c655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 -- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 msgr2=0x564fc1c65b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c65b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 -- 172.21.15.19:0/1520446962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc1ac98d0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1adce40 0x564fc1c655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5effff700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 0x564fc1c66040 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80a6410 tx=0x7fe5e80a8990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b883f6f9dd8a1cb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.318+0000 7fe5ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:06.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.319+0000 7fe5ceffd700 1 -- 172.21.15.19:0/1520446962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80a91a0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.319+0000 7fe5ceffd700 1 -- 172.21.15.19:0/1520446962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe5e80a9610 con 0x564fc1c5bcd0 2024-05-17T09:21:06.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.319+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e001cab0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe5ceffd700 1 -- 172.21.15.19:0/1520446962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5e80ad550 con 0x564fc1c5bcd0 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 10 client.15789 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564fc1b33760 con 0x564fc1c5bcd0 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe5ceffd700 1 -- 172.21.15.19:0/1520446962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5e001cab0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe5ceffd700 1 -- 172.21.15.19:0/1520446962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5e8004c60 con 0x564fc1c5bcd0 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 20 client.15789 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1c75280 0x564fc1c776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 4 client.15789 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.320+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc1c27ed0 con 0x564fc1c75280 2024-05-17T09:21:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.321+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1c75280 0x564fc1c776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.321+0000 7fe5ceffd700 10 client.15789 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.321+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1c75280 0x564fc1c776c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54211294be12faf7 server_cookie=c8b27ec4da7a90a7 in_seq=0 out_seq=0 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.322+0000 7fe5ceffd700 1 -- 172.21.15.19:0/1520446962 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564fc1c27ed0 con 0x564fc1c75280 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.322+0000 7fe5ceffd700 10 client.15789 handle_command_reply: tid=0 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.322+0000 7fe6045d0b80 1 client.15789 shutdown 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.322+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 msgr2=0x564fc1c66040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.322+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 0x564fc1c66040 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80a6410 tx=0x7fe5e80a8990 comp rx=0 tx=0).stop 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 shutdown_connections 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1adce40 0x564fc1c655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1c75280 0x564fc1c776c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1c5bcd0 0x564fc1c66040 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/1520446962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c65b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 >> 172.21.15.19:0/1520446962 conn(0x564fc1b3d1c0 msgr2=0x564fc1c56790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe5ceffd700 0 client.15789 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 shutdown_connections 2024-05-17T09:21:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 1 -- 172.21.15.19:0/1520446962 wait complete. 2024-05-17T09:21:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.323+0000 7fe6045d0b80 20 client.15789 trim_cache size 0 max 16384 2024-05-17T09:21:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:06.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 Processor -- start 2024-05-17T09:21:06.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 -- start start 2024-05-17T09:21:06.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c5b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c50800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01b4dc0 con 0x564fc1adce40 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01b2650 con 0x564fc1bc8430 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.344+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01b04a0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c5b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c5b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47256/0 (socket says 172.21.15.19:47256) 2024-05-17T09:21:06.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/147827438 learned_addr learned my addr 172.21.15.19:0/147827438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5effff700 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c50800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/147827438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 msgr2=0x564fc1c50800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c50800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/147827438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 msgr2=0x564fc1c5f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c5f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/147827438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc1c56d00 con 0x564fc1adce40 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c5b6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000bd70 tx=0x7fe5e0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9b7fe0036669f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.345+0000 7fe5e57fa700 1 -- 172.21.15.19:0/147827438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0023040 con 0x564fc1adce40 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.346+0000 7fe5e57fa700 1 -- 172.21.15.19:0/147827438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5e0003680 con 0x564fc1adce40 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.346+0000 7fe5e57fa700 1 -- 172.21.15.19:0/147827438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e001c6e0 con 0x564fc1adce40 2024-05-17T09:21:06.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 -- 172.21.15.19:0/147827438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 msgr2=0x564fc1c5b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c5b6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000bd70 tx=0x7fe5e0005d50 comp rx=0 tx=0).stop 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 -- 172.21.15.19:0/147827438 shutdown_connections 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c50800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c5b6f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/147827438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 -- 172.21.15.19:0/147827438 >> 172.21.15.19:0/147827438 conn(0x564fc1b3d1c0 msgr2=0x564fc1c5f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 -- 172.21.15.19:0/147827438 shutdown_connections 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.347+0000 7fe6045d0b80 1 -- 172.21.15.19:0/147827438 wait complete. 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.348+0000 7fe6045d0b80 1 -- start start 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.348+0000 7fe6045d0b80 1 Processor -- start 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.348+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c6df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c6df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5ef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c6df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47262/0 (socket says 172.21.15.19:47262) 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe6045d0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01b0b10 con 0x564fc1adce40 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01b1f80 con 0x564fc1bc8430 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe6045d0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01b11e0 con 0x564fc1c5bcd0 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe6045d0b80 10 client.0 _ll_register_callbacks cb 0x7fe5e80a9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2879306978 learned_addr learned my addr 172.21.15.19:0/2879306978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5effff700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2879306978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 msgr2=0x564fc1c7cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.349+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c7cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 msgr2=0x564fc1c7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ef7fe700 1 -- 172.21.15.19:0/2879306978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc1c7d270 con 0x564fc1adce40 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ef7fe700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c6df00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe5e002d040 tx=0x7fe5e0005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a5c9e4ae3e9d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ceffd700 1 -- 172.21.15.19:0/2879306978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0023070 con 0x564fc1adce40 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ceffd700 1 -- 172.21.15.19:0/2879306978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe5e0003a30 con 0x564fc1adce40 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e0003680 con 0x564fc1adce40 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe5ceffd700 1 -- 172.21.15.19:0/2879306978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5e0005560 con 0x564fc1adce40 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe6045d0b80 10 client.15795 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.350+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564fc1cdfbf0 con 0x564fc1adce40 2024-05-17T09:21:06.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe5ceffd700 1 -- 172.21.15.19:0/2879306978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5e0003680 con 0x564fc1adce40 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe5ceffd700 1 -- 172.21.15.19:0/2879306978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5e0011440 con 0x564fc1adce40 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe6045d0b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe6045d0b80 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe6045d0b80 20 client.15795 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1ce0040 0x564fc1ce2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe6045d0b80 4 client.15795 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fc1b8bcc0 con 0x564fc1ce0040 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.351+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1ce0040 0x564fc1ce2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe5eeffd700 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1ce0040 0x564fc1ce2460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11388b40699a9576 server_cookie=562d38188cc81f93 in_seq=0 out_seq=0 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe5ceffd700 10 client.15795 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe5ceffd700 1 -- 172.21.15.19:0/2879306978 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564fc1b8bcc0 con 0x564fc1ce0040 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe5ceffd700 10 client.15795 handle_command_reply: tid=0 2024-05-17T09:21:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 client.15795 shutdown 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 msgr2=0x564fc1c6df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c6df00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe5e002d040 tx=0x7fe5e0005160 comp rx=0 tx=0).stop 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 shutdown_connections 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fc1c5bcd0 0x564fc1c7cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564fc1ce0040 0x564fc1ce2460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564fc1adce40 0x564fc1c6df00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 --2- 172.21.15.19:0/2879306978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fc1bc8430 0x564fc1c7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 >> 172.21.15.19:0/2879306978 conn(0x564fc1b3d1c0 msgr2=0x564fc1c5b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.352+0000 7fe5ceffd700 0 client.15795 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 shutdown_connections 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe6045d0b80 1 -- 172.21.15.19:0/2879306978 wait complete. 2024-05-17T09:21:06.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe6045d0b80 20 client.15795 trim_cache size 0 max 16384 2024-05-17T09:21:06.412 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:21:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:21:06.412 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5dc0b2900 msgr2=0x7fe5dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5dc0b2900 0x7fe5dc0b4e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009990 tx=0x7fe5d8005050 comp rx=0 tx=0).stop 2024-05-17T09:21:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 msgr2=0x7fe5f019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 0x7fe5f019f5a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8007ee0 tx=0x7fe5e800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 shutdown_connections 2024-05-17T09:21:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5dc0b2900 0x7fe5dc0b4e10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5f01034d0 0x7fe5f019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f0108750 0x7fe5f019f5a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 --2- 172.21.15.19:0/2073036481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0107e00 0x7fe5f019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.353+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 >> 172.21.15.19:0/2073036481 conn(0x7fe5f00fef20 msgr2=0x7fe5f006c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.354+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 shutdown_connections 2024-05-17T09:21:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:06.354+0000 7fe5f5bbb700 1 -- 172.21.15.19:0/2073036481 wait complete. 2024-05-17T09:21:06.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:06 smithi019 ceph-mon[31620]: pgmap v478: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:06 smithi079 ceph-mon[35502]: pgmap v478: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:06 smithi069 ceph-mon[35081]: pgmap v478: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:08 smithi079 ceph-mon[35502]: pgmap v479: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:08.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:08 smithi019 ceph-mon[31620]: pgmap v479: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:08.535 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:21:08.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:21:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:08 smithi069 ceph-mon[35081]: pgmap v479: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 -- 172.21.15.19:0/1350291069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4108750 msgr2=0x7fa1b41100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 --2- 172.21.15.19:0/1350291069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4108750 0x7fa1b41100b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00b0d0 tx=0x7fa1ac00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 -- 172.21.15.19:0/1350291069 shutdown_connections 2024-05-17T09:21:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 --2- 172.21.15.19:0/1350291069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4108750 0x7fa1b41100b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 --2- 172.21.15.19:0/1350291069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b41034d0 0x7fa1b41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 --2- 172.21.15.19:0/1350291069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1b4107e00 0x7fa1b4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 -- 172.21.15.19:0/1350291069 >> 172.21.15.19:0/1350291069 conn(0x7fa1b40fef20 msgr2=0x7fa1b4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 -- 172.21.15.19:0/1350291069 shutdown_connections 2024-05-17T09:21:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 -- 172.21.15.19:0/1350291069 wait complete. 2024-05-17T09:21:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.047+0000 7fa1bc199700 1 Processor -- start 2024-05-17T09:21:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 -- start start 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1b41034d0 0x7fa1b419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 0x7fa1b419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b4108750 0x7fa1b41a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4113300 con 0x7fa1b4107e00 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4113180 con 0x7fa1b4108750 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1bc199700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4113480 con 0x7fa1b41034d0 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 0x7fa1b419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 0x7fa1b419c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47276/0 (socket says 172.21.15.19:47276) 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 -- 172.21.15.19:0/536660578 learned_addr learned my addr 172.21.15.19:0/536660578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9f35700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1b41034d0 0x7fa1b419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1ba736700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b4108750 0x7fa1b41a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 -- 172.21.15.19:0/536660578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1b41034d0 msgr2=0x7fa1b419bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1b41034d0 0x7fa1b419bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 -- 172.21.15.19:0/536660578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b4108750 msgr2=0x7fa1b41a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b4108750 0x7fa1b41a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.048+0000 7fa1b9734700 1 -- 172.21.15.19:0/536660578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b419c9b0 con 0x7fa1b4107e00 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.049+0000 7fa1b9734700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 0x7fa1b419c100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c8f0 tx=0x7fa1a400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.049+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40078d0 con 0x7fa1b4107e00 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.049+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b41a72a0 con 0x7fa1b4107e00 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.049+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1a400f450 con 0x7fa1b4107e00 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.049+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4018660 con 0x7fa1b4107e00 2024-05-17T09:21:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.049+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b41a77e0 con 0x7fa1b4107e00 2024-05-17T09:21:09.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.050+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b40512b0 con 0x7fa1b4107e00 2024-05-17T09:21:09.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.051+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa1a400f900 con 0x7fa1b4107e00 2024-05-17T09:21:09.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.052+0000 7fa1aaffd700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1a00b2880 0x7fa1a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:09.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.052+0000 7fa1b9f35700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1a00b2880 0x7fa1a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:09.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.052+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa1a40e2400 con 0x7fa1b4107e00 2024-05-17T09:21:09.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.053+0000 7fa1b9f35700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1a00b2880 0x7fa1a00b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009a10 tx=0x7fa1b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:09.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.057+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1a4014030 con 0x7fa1b4107e00 2024-05-17T09:21:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.330+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1b406c680 con 0x7fa1b4107e00 2024-05-17T09:21:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.331+0000 7fa1aaffd700 1 -- 172.21.15.19:0/536660578 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fa1a40a9d10 con 0x7fa1b4107e00 2024-05-17T09:21:09.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:21:09.332 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:21:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1a00b2880 msgr2=0x7fa1a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1a00b2880 0x7fa1a00b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009a10 tx=0x7fa1b0009190 comp rx=0 tx=0).stop 2024-05-17T09:21:09.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 msgr2=0x7fa1b419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 0x7fa1b419c100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c8f0 tx=0x7fa1a400cc00 comp rx=0 tx=0).stop 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 shutdown_connections 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa1a00b2880 0x7fa1a00b4d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1b41034d0 0x7fa1b419bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1b4108750 0x7fa1b41a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 --2- 172.21.15.19:0/536660578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b4107e00 0x7fa1b419c100 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 >> 172.21.15.19:0/536660578 conn(0x7fa1b40fef20 msgr2=0x7fa1b4107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 shutdown_connections 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:09.334+0000 7fa1bc199700 1 -- 172.21.15.19:0/536660578 wait complete. 2024-05-17T09:21:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:21:10.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:10 smithi019 ceph-mon[31620]: pgmap v480: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:21:10.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:10 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/536660578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:10.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:10 smithi079 ceph-mon[35502]: pgmap v480: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:21:10.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:10 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/536660578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:10.571 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.4 --await-for 45.0' 2024-05-17T09:21:10.571 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.4 --await-for 45.0 2024-05-17T09:21:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:10 smithi069 ceph-mon[35081]: pgmap v480: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:21:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:10 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/536660578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:11.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 -- 172.21.15.19:0/2234684992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47281034b0 msgr2=0x7f47281038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 --2- 172.21.15.19:0/2234684992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47281034b0 0x7f47281038c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f471c0099c0 tx=0x7f471c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 -- 172.21.15.19:0/2234684992 shutdown_connections 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 --2- 172.21.15.19:0/2234684992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728108730 0x7f4728110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 --2- 172.21.15.19:0/2234684992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47281034b0 0x7f47281038c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 --2- 172.21.15.19:0/2234684992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4728107de0 0x7f47281081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 -- 172.21.15.19:0/2234684992 >> 172.21.15.19:0/2234684992 conn(0x7f47280fede0 msgr2=0x7f4728101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 -- 172.21.15.19:0/2234684992 shutdown_connections 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.559+0000 7f472f1b2700 1 -- 172.21.15.19:0/2234684992 wait complete. 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 Processor -- start 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 -- start start 2024-05-17T09:21:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47281034b0 0x7f472819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728107de0 0x7f472819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 0x7f472819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4728113210 con 0x7f4728108730 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4728113090 con 0x7f47281034b0 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472f1b2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4728113390 con 0x7f4728107de0 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472d74f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 0x7f472819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472d74f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 0x7f472819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47294/0 (socket says 172.21.15.19:47294) 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472d74f700 1 -- 172.21.15.19:0/252864670 learned_addr learned my addr 172.21.15.19:0/252864670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472cf4e700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47281034b0 0x7f472819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f4727fff700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728107de0 0x7f472819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472d74f700 1 -- 172.21.15.19:0/252864670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728107de0 msgr2=0x7f472819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472d74f700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728107de0 0x7f472819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.560+0000 7f472d74f700 1 -- 172.21.15.19:0/252864670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47281034b0 msgr2=0x7f472819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f472d74f700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47281034b0 0x7f472819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f472d74f700 1 -- 172.21.15.19:0/252864670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472819fc20 con 0x7f4728108730 2024-05-17T09:21:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f4727fff700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728107de0 0x7f472819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f472d74f700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 0x7f472819f580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f472000ef30 tx=0x7f472000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47200092e0 con 0x7f4728108730 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4720004bb0 con 0x7f4728108730 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472819feb0 con 0x7f4728108730 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720004020 con 0x7f4728108730 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.561+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47281a03a0 con 0x7f4728108730 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.562+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47280512b0 con 0x7f4728108730 2024-05-17T09:21:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.564+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f472001b030 con 0x7f4728108730 2024-05-17T09:21:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.565+0000 7f4725ffb700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2880 0x7f47180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.565+0000 7f472cf4e700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2880 0x7f47180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.565+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f47200e20d0 con 0x7f4728108730 2024-05-17T09:21:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.565+0000 7f472cf4e700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2880 0x7f47180b4d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f471c000c00 tx=0x7f471c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:11.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.570+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4720022020 con 0x7f4728108730 2024-05-17T09:21:11.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:11.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.856+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f472802e5e0 con 0x7f4728108730 2024-05-17T09:21:11.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.857+0000 7f4725ffb700 1 -- 172.21.15.19:0/252864670 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7f4720010040 con 0x7f4728108730 2024-05-17T09:21:11.858 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:21:11.858 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2880 msgr2=0x7f47180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2880 0x7f47180b4d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f471c000c00 tx=0x7f471c01a040 comp rx=0 tx=0).stop 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 msgr2=0x7f472819f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 0x7f472819f580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f472000ef30 tx=0x7f472000c3a0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 shutdown_connections 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4728108730 0x7f472819f580 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47281034b0 0x7f472819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2880 0x7f47180b4d90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 --2- 172.21.15.19:0/252864670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4728107de0 0x7f472819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:11.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 >> 172.21.15.19:0/252864670 conn(0x7f47280fede0 msgr2=0x7f4728075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:11.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 shutdown_connections 2024-05-17T09:21:11.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:11.860+0000 7f472f1b2700 1 -- 172.21.15.19:0/252864670 wait complete. 2024-05-17T09:21:11.861 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:21:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:12 smithi079 ceph-mon[35502]: pgmap v481: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:12 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/252864670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:12 smithi069 ceph-mon[35081]: pgmap v481: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:12 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/252864670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:12.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:12 smithi019 ceph-mon[31620]: pgmap v481: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:12.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:12 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/252864670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:12.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:21:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 -- 172.21.15.19:0/1493294578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444108730 msgr2=0x7f9444110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 --2- 172.21.15.19:0/1493294578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444108730 0x7f9444110090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f943c00b0d0 tx=0x7f943c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:14 smithi019 ceph-mon[31620]: pgmap v482: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 -- 172.21.15.19:0/1493294578 shutdown_connections 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 --2- 172.21.15.19:0/1493294578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94441034b0 0x7f94441038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 --2- 172.21.15.19:0/1493294578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444108730 0x7f9444110090 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 --2- 172.21.15.19:0/1493294578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444107de0 0x7f94441081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 -- 172.21.15.19:0/1493294578 >> 172.21.15.19:0/1493294578 conn(0x7f94440fedb0 msgr2=0x7f94441011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 -- 172.21.15.19:0/1493294578 shutdown_connections 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.347+0000 7f9448cce700 1 -- 172.21.15.19:0/1493294578 wait complete. 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.348+0000 7f9448cce700 1 Processor -- start 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.348+0000 7f9448cce700 1 -- start start 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9448cce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94441034b0 0x7f944419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9448cce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 0x7f944419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9448cce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444108730 0x7f944419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9448cce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9444113210 con 0x7f9444107de0 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9448cce700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9444113090 con 0x7f9444108730 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9448cce700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9444113390 con 0x7f94441034b0 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9441d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 0x7f944419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9441d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 0x7f944419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36864/0 (socket says 172.21.15.19:36864) 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9441d9b700 1 -- 172.21.15.19:0/2594491491 learned_addr learned my addr 172.21.15.19:0/2594491491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.349+0000 7f9442d9d700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444108730 0x7f944419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f944259c700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94441034b0 0x7f944419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9441d9b700 1 -- 172.21.15.19:0/2594491491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94441034b0 msgr2=0x7f944419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9441d9b700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94441034b0 0x7f944419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9441d9b700 1 -- 172.21.15.19:0/2594491491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444108730 msgr2=0x7f944419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9441d9b700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444108730 0x7f944419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9441d9b700 1 -- 172.21.15.19:0/2594491491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944419fc20 con 0x7f9444107de0 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9442d9d700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444108730 0x7f944419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9441d9b700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 0x7f944419b2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9438012d80 tx=0x7f943800c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f94337fe700 1 -- 172.21.15.19:0/2594491491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438004830 con 0x7f9444107de0 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944419ff10 con 0x7f9444107de0 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94440ff1c0 con 0x7f9444107de0 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f94337fe700 1 -- 172.21.15.19:0/2594491491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9438005020 con 0x7f9444107de0 2024-05-17T09:21:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.350+0000 7f94337fe700 1 -- 172.21.15.19:0/2594491491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943801a690 con 0x7f9444107de0 2024-05-17T09:21:14.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.354+0000 7f94576e3b80 1 Processor -- start 2024-05-17T09:21:14.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 -- start start 2024-05-17T09:21:14.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94337fe700 1 -- 172.21.15.19:0/2594491491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f943801a830 con 0x7f9444107de0 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f02335f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f023364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f02330580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f0225df00 con 0x559f0229c5c0 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f0224daa0 con 0x559f021b0fd0 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f94576e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f02251ae0 con 0x559f023300b0 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f9442d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f02335f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f9442d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f02335f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36880/0 (socket says 172.21.15.19:36880) 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f9442d9d700 1 -- 172.21.15.19:0/634770761 learned_addr learned my addr 172.21.15.19:0/634770761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f944259c700 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f023364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.355+0000 7f9441d9b700 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f02330580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9442d9d700 1 -- 172.21.15.19:0/634770761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 msgr2=0x559f02330580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9442d9d700 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f02330580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9442d9d700 1 -- 172.21.15.19:0/634770761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 msgr2=0x559f023364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9442d9d700 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f023364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9442d9d700 1 -- 172.21.15.19:0/634770761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f021f7140 con 0x559f0229c5c0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9442d9d700 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f02335f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f943c005af0 tx=0x7f943c005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6892374c8c91d51f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f94337fe700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f942c0b2900 0x7f942c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9430ff9700 1 -- 172.21.15.19:0/634770761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c00be50 con 0x559f0229c5c0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f9430ff9700 1 -- 172.21.15.19:0/634770761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f943c0094e0 con 0x559f0229c5c0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f944259c700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f942c0b2900 0x7f942c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.356+0000 7f94337fe700 1 -- 172.21.15.19:0/2594491491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f94380e62e0 con 0x7f9444107de0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.357+0000 7f9430ff9700 1 -- 172.21.15.19:0/634770761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c0075c0 con 0x559f0229c5c0 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.357+0000 7f94576e3b80 1 -- 172.21.15.19:0/634770761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 msgr2=0x559f02335f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.357+0000 7f94576e3b80 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f02335f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f943c005af0 tx=0x7f943c005b20 comp rx=0 tx=0).stop 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.357+0000 7f944259c700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f942c0b2900 0x7f942c0b4e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x559f0232c5a0 tx=0x7f9434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 -- 172.21.15.19:0/634770761 shutdown_connections 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f02330580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f02335f20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 --2- 172.21.15.19:0/634770761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f023364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 -- 172.21.15.19:0/634770761 >> 172.21.15.19:0/634770761 conn(0x559f02211350 msgr2=0x559f0232ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 -- 172.21.15.19:0/634770761 shutdown_connections 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 -- 172.21.15.19:0/634770761 wait complete. 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 -- start start 2024-05-17T09:21:14.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.358+0000 7f94576e3b80 1 Processor -- start 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f02339b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f02154130 con 0x559f0229c5c0 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f021b5f30 con 0x559f021b0fd0 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f0233cbb0 con 0x559f023300b0 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f94576e3b80 10 client.0 _ll_register_callbacks cb 0x559f021e8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36886/0 (socket says 172.21.15.19:36886) 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 -- 172.21.15.19:0/1208007321 learned_addr learned my addr 172.21.15.19:0/1208007321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f944259c700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f02339b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 -- 172.21.15.19:0/1208007321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 msgr2=0x559f0233a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 -- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 msgr2=0x559f02339b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f02339b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 -- 172.21.15.19:0/1208007321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f0219da60 con 0x559f0229c5c0 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f944259c700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f02339b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233a060 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94380aa7c0 tx=0x7f94380e9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b51dbd2c9399a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.359+0000 7f9429ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f9429ffb700 1 -- 172.21.15.19:0/1208007321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380ad060 con 0x559f0229c5c0 2024-05-17T09:21:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f9429ffb700 1 -- 172.21.15.19:0/1208007321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f94380049d0 con 0x559f0229c5c0 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f022fc060 con 0x559f0229c5c0 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f9429ffb700 1 -- 172.21.15.19:0/1208007321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94380adc50 con 0x559f0229c5c0 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f94576e3b80 10 client.15819 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f022078f0 con 0x559f0229c5c0 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.360+0000 7f9429ffb700 1 -- 172.21.15.19:0/1208007321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559f022fc060 con 0x559f0229c5c0 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f9429ffb700 1 -- 172.21.15.19:0/1208007321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f943801ab20 con 0x559f0229c5c0 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f94576e3b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f94576e3b80 10 client.15819 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f94576e3b80 20 client.15819 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f02349af0 0x559f0234bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f94576e3b80 4 client.15819 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.361+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f0225fe50 con 0x559f02349af0 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.362+0000 7f944259c700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f02349af0 0x559f0234bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.362+0000 7f944259c700 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f02349af0 0x559f0234bf30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0107b5eb3e8bcc9 server_cookie=4b0d6d9c90fa1852 in_seq=0 out_seq=0 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.362+0000 7f9429ffb700 10 client.15819 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f9429ffb700 1 -- 172.21.15.19:0/1208007321 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f0225fe50 con 0x559f02349af0 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f9429ffb700 10 client.15819 handle_command_reply: tid=0 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 client.15819 shutdown 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 msgr2=0x559f0233a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233a060 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94380aa7c0 tx=0x7f94380e9f90 comp rx=0 tx=0).stop 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 shutdown_connections 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f02349af0 0x559f0234bf30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f02339b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1208007321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233a060 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f9429ffb700 0 client.15819 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 >> 172.21.15.19:0/1208007321 conn(0x559f02211350 msgr2=0x559f0232f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 shutdown_connections 2024-05-17T09:21:14.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 1 -- 172.21.15.19:0/1208007321 wait complete. 2024-05-17T09:21:14.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.363+0000 7f94576e3b80 20 client.15819 trim_cache size 0 max 16384 2024-05-17T09:21:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.405+0000 7f94576e3b80 1 Processor -- start 2024-05-17T09:21:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.405+0000 7f94576e3b80 1 -- start start 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.405+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f021b0fd0 0x559f02325680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f0229c5c0 0x559f02335390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f023300b0 0x559f02345c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f944259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f021b0fd0 0x559f02325680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f94576e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94240114e0 con 0x559f021b0fd0 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f94576e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f942400ed70 con 0x559f023300b0 2024-05-17T09:21:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f944259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f021b0fd0 0x559f02325680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36892/0 (socket says 172.21.15.19:36892) 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f94576e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f942400cbc0 con 0x559f0229c5c0 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f944259c700 1 -- 172.21.15.19:0/1045724224 learned_addr learned my addr 172.21.15.19:0/1045724224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f0229c5c0 0x559f02335390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9442d9d700 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f023300b0 0x559f02345c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9441d9b700 1 -- 172.21.15.19:0/1045724224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f023300b0 msgr2=0x559f02345c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f023300b0 0x559f02345c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9441d9b700 1 -- 172.21.15.19:0/1045724224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f021b0fd0 msgr2=0x559f02325680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f021b0fd0 0x559f02325680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9441d9b700 1 -- 172.21.15.19:0/1045724224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f0232b280 con 0x559f0229c5c0 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.406+0000 7f9442d9d700 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f023300b0 0x559f02345c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.407+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f0229c5c0 0x559f02335390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f94380a95d0 tx=0x7f94380e6670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cb7825cd4c1f70d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.407+0000 7f942bfff700 1 -- 172.21.15.19:0/1045724224 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380b63b0 con 0x559f0229c5c0 2024-05-17T09:21:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.407+0000 7f942bfff700 1 -- 172.21.15.19:0/1045724224 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94380ad550 con 0x559f0229c5c0 2024-05-17T09:21:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.407+0000 7f942bfff700 1 -- 172.21.15.19:0/1045724224 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380aca10 con 0x559f0229c5c0 2024-05-17T09:21:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 -- 172.21.15.19:0/1045724224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f0229c5c0 msgr2=0x559f02335390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f0229c5c0 0x559f02335390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f94380a95d0 tx=0x7f94380e6670 comp rx=0 tx=0).stop 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 -- 172.21.15.19:0/1045724224 shutdown_connections 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f023300b0 0x559f02345c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f021b0fd0 0x559f02325680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1045724224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f0229c5c0 0x559f02335390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 -- 172.21.15.19:0/1045724224 >> 172.21.15.19:0/1045724224 conn(0x559f02211350 msgr2=0x559f02333ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 -- 172.21.15.19:0/1045724224 shutdown_connections 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.408+0000 7f94576e3b80 1 -- 172.21.15.19:0/1045724224 wait complete. 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.409+0000 7f94576e3b80 1 -- start start 2024-05-17T09:21:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.409+0000 7f94576e3b80 1 Processor -- start 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f0233ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f942400d230 con 0x559f0229c5c0 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f942400e6a0 con 0x559f021b0fd0 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f942400d900 con 0x559f023300b0 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f944259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f0233ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f94576e3b80 10 client.0 _ll_register_callbacks cb 0x559f021e8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f944259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f0233ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35216/0 (socket says 172.21.15.19:35216) 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f944259c700 1 -- 172.21.15.19:0/1388601317 learned_addr learned my addr 172.21.15.19:0/1388601317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.410+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f9442d9d700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f944259c700 1 -- 172.21.15.19:0/1388601317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 msgr2=0x559f0233f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f944259c700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f944259c700 1 -- 172.21.15.19:0/1388601317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 msgr2=0x559f0233f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f944259c700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f944259c700 1 -- 172.21.15.19:0/1388601317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f023b4800 con 0x559f021b0fd0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f944259c700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f0233ee30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f94340117c0 tx=0x7f9434015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33661624190a50e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f9429ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.411+0000 7f9429ffb700 1 -- 172.21.15.19:0/1388601317 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340298c0 con 0x559f021b0fd0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.412+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94380ad550 con 0x559f021b0fd0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.412+0000 7f9429ffb700 1 -- 172.21.15.19:0/1388601317 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9434029a60 con 0x559f021b0fd0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.412+0000 7f9429ffb700 1 -- 172.21.15.19:0/1388601317 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f943401d450 con 0x559f021b0fd0 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.412+0000 7f94576e3b80 10 client.25517 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.412+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f023b5aa0 con 0x559f021b0fd0 2024-05-17T09:21:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.412+0000 7f9429ffb700 1 -- 172.21.15.19:0/1388601317 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94380ad550 con 0x559f021b0fd0 2024-05-17T09:21:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f9429ffb700 1 -- 172.21.15.19:0/1388601317 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9434030070 con 0x559f021b0fd0 2024-05-17T09:21:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f94576e3b80 10 client.25517 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f94576e3b80 10 client.25517 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:21:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f94576e3b80 20 client.25517 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f023b6280 0x559f023b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f94576e3b80 4 client.25517 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.4, "await_for": 45.0}] 2024-05-17T09:21:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.4, "await_for": 45.0}) -- 0x559f02259630 con 0x559f023b6280 2024-05-17T09:21:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.413+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f023b6280 0x559f023b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.414+0000 7f9441d9b700 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f023b6280 0x559f023b86a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4606ef63caba117 server_cookie=290bae1cd7799723 in_seq=0 out_seq=0 2024-05-17T09:21:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.414+0000 7f9429ffb700 10 client.25517 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f9429ffb700 1 -- 172.21.15.19:0/1388601317 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559f02259630 con 0x559f023b6280 2024-05-17T09:21:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f9429ffb700 10 client.25517 handle_command_reply: tid=0 2024-05-17T09:21:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 client.25517 shutdown 2024-05-17T09:21:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 msgr2=0x559f0233ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f0233ee30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f94340117c0 tx=0x7f9434015040 comp rx=0 tx=0).stop 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 shutdown_connections 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f023300b0 0x559f0233f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f023b6280 0x559f023b86a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f021b0fd0 0x559f0233ee30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 --2- 172.21.15.19:0/1388601317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f0229c5c0 0x559f0233f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 >> 172.21.15.19:0/1388601317 conn(0x559f02211350 msgr2=0x559f02333ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f9429ffb700 0 client.25517 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.425+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 shutdown_connections 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.426+0000 7f94576e3b80 1 -- 172.21.15.19:0/1388601317 wait complete. 2024-05-17T09:21:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.426+0000 7f94576e3b80 20 client.25517 trim_cache size 0 max 16384 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 6, 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "6a72cbc5": { 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 6, 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:21:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.4, 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:14.430 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.426+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f942c0b2900 msgr2=0x7f942c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.426+0000 7f9448cce700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f942c0b2900 0x7f942c0b4e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x559f0232c5a0 tx=0x7f9434009260 comp rx=0 tx=0).stop 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.426+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 msgr2=0x7f944419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.426+0000 7f9448cce700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 0x7f944419b2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9438012d80 tx=0x7f943800c5b0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 shutdown_connections 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f942c0b2900 0x7f942c0b4e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94441034b0 0x7f944419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9444108730 0x7f944419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 --2- 172.21.15.19:0/2594491491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9444107de0 0x7f944419b2f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 >> 172.21.15.19:0/2594491491 conn(0x7f94440fedb0 msgr2=0x7f944406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 shutdown_connections 2024-05-17T09:21:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:14.427+0000 7f9448cce700 1 -- 172.21.15.19:0/2594491491 wait complete. 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.443+0000 7ff23456c640 1 -- 172.21.15.19:0/1018393393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1043b0 msgr2=0x7ff22c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.443+0000 7ff23456c640 1 --2- 172.21.15.19:0/1018393393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1043b0 0x7ff22c1067d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff228009900 tx=0x7ff22802a410 comp rx=0 tx=0).stop 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 -- 172.21.15.19:0/1018393393 shutdown_connections 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 --2- 172.21.15.19:0/1018393393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c1098d0 0x7ff22c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 --2- 172.21.15.19:0/1018393393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff22c106da0 0x7ff22c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 --2- 172.21.15.19:0/1018393393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1043b0 0x7ff22c1067d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 -- 172.21.15.19:0/1018393393 >> 172.21.15.19:0/1018393393 conn(0x7ff22c0fdd30 msgr2=0x7ff22c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 -- 172.21.15.19:0/1018393393 shutdown_connections 2024-05-17T09:21:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 -- 172.21.15.19:0/1018393393 wait complete. 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.444+0000 7ff23456c640 1 Processor -- start 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 -- start start 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff22c1043b0 0x7ff22c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c106da0 0x7ff22c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 0x7ff22c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10f5e0 con 0x7ff22c1098d0 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10f460 con 0x7ff22c106da0 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff23456c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10f760 con 0x7ff22c1043b0 2024-05-17T09:21:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff232b09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 0x7ff22c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff232b09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 0x7ff22c1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36922/0 (socket says 172.21.15.19:36922) 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff232b09640 1 -- 172.21.15.19:0/607748345 learned_addr learned my addr 172.21.15.19:0/607748345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff231b07640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c106da0 0x7ff22c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff232308640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff22c1043b0 0x7ff22c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff231b07640 1 -- 172.21.15.19:0/607748345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff22c1043b0 msgr2=0x7ff22c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff231b07640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff22c1043b0 0x7ff22c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff231b07640 1 -- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 msgr2=0x7ff22c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff231b07640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 0x7ff22c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff231b07640 1 -- 172.21.15.19:0/607748345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c1a1d50 con 0x7ff22c106da0 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.445+0000 7ff232b09640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 0x7ff22c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.446+0000 7ff231b07640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c106da0 0x7ff22c19a750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00e9f0 tx=0x7ff21c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.446+0000 7ff2237fe640 1 -- 172.21.15.19:0/607748345 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00ccc0 con 0x7ff22c106da0 2024-05-17T09:21:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.446+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c1a1fe0 con 0x7ff22c106da0 2024-05-17T09:21:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.446+0000 7ff2237fe640 1 -- 172.21.15.19:0/607748345 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff21c005020 con 0x7ff22c106da0 2024-05-17T09:21:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.446+0000 7ff2237fe640 1 -- 172.21.15.19:0/607748345 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00ccc0 con 0x7ff22c106da0 2024-05-17T09:21:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.446+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1a2480 con 0x7ff22c106da0 2024-05-17T09:21:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 Processor -- start 2024-05-17T09:21:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 -- start start 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff2237fe640 1 -- 172.21.15.19:0/607748345 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff21c0047a0 con 0x7ff22c106da0 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56436a52bc30 con 0x56436a1f74d0 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56436a51b7d0 con 0x56436a1ef2f0 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.449+0000 7ff23b419740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56436a51f810 con 0x56436a2181b0 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff231b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232308640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff231b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36938/0 (socket says 172.21.15.19:36938) 2024-05-17T09:21:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff231b07640 1 -- 172.21.15.19:0/1582579007 learned_addr learned my addr 172.21.15.19:0/1582579007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 -- 172.21.15.19:0/1582579007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 msgr2=0x56436a5bdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5bdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 -- 172.21.15.19:0/1582579007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 msgr2=0x56436a5c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 -- 172.21.15.19:0/1582579007 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436a230210 con 0x56436a2181b0 2024-05-17T09:21:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.450+0000 7ff232b09640 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5bd7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2240117e0 tx=0x7ff224011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a53a5a161efa0308 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.451+0000 7ff220bfe640 1 -- 172.21.15.19:0/1582579007 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224011310 con 0x56436a2181b0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.451+0000 7ff2237fe640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2180b2ca0 0x7ff2180b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.451+0000 7ff220bfe640 1 -- 172.21.15.19:0/1582579007 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff22400f9a0 con 0x56436a2181b0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.451+0000 7ff220bfe640 1 -- 172.21.15.19:0/1582579007 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224005630 con 0x56436a2181b0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.451+0000 7ff232308640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2180b2ca0 0x7ff2180b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.451+0000 7ff2237fe640 1 -- 172.21.15.19:0/607748345 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff21c00cac0 con 0x7ff22c106da0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff232308640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2180b2ca0 0x7ff2180b51b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff228005ab0 tx=0x7ff228000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 -- 172.21.15.19:0/1582579007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 msgr2=0x56436a5bd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5bd7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2240117e0 tx=0x7ff224011ca0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 -- 172.21.15.19:0/1582579007 shutdown_connections 2024-05-17T09:21:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 --2- 172.21.15.19:0/1582579007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5bd7b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 -- 172.21.15.19:0/1582579007 >> 172.21.15.19:0/1582579007 conn(0x56436a20c300 msgr2=0x56436a5b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 -- 172.21.15.19:0/1582579007 shutdown_connections 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 -- 172.21.15.19:0/1582579007 wait complete. 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 -- start start 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.452+0000 7ff23b419740 1 Processor -- start 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56436a201790 con 0x56436a1f74d0 2024-05-17T09:21:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56436a4d7890 con 0x56436a1ef2f0 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56436a5c5680 con 0x56436a2181b0 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff23b419740 10 client.0 _ll_register_callbacks cb 0x56436a4dad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff231b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff232b09640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff231b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36948/0 (socket says 172.21.15.19:36948) 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff232b09640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46144/0 (socket says 172.21.15.19:46144) 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff231b07640 1 -- 172.21.15.19:0/4090647629 learned_addr learned my addr 172.21.15.19:0/4090647629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff232308640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.453+0000 7ff231b07640 1 -- 172.21.15.19:0/4090647629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 msgr2=0x56436a5ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff231b07640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff231b07640 1 -- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 msgr2=0x56436a5c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff231b07640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5c6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff231b07640 1 -- 172.21.15.19:0/4090647629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436a2fc620 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff232b09640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff231b07640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c7510 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff21c004650 tx=0x7ff21c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d91afd9d91a2578 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff215ffb640 1 -- 172.21.15.19:0/4090647629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c01c070 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff215ffb640 1 -- 172.21.15.19:0/4090647629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff21c005490 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22400f9a0 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff215ffb640 1 -- 172.21.15.19:0/4090647629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff21c0a8920 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff23b419740 10 client.15837 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56436a5d1470 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.454+0000 7ff215ffb640 1 -- 172.21.15.19:0/4090647629 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff22400f9a0 con 0x56436a1f74d0 2024-05-17T09:21:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff215ffb640 1 -- 172.21.15.19:0/4090647629 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff21c0ad120 con 0x56436a1f74d0 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff23b419740 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff23b419740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff23b419740 20 client.15837 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff23b419740 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a1d6d80 0x56436a5d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff23b419740 4 client.15837 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56436a52db80 con 0x56436a1d6d80 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.455+0000 7ff232308640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a1d6d80 0x56436a5d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.456+0000 7ff215ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.456+0000 7ff232308640 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a1d6d80 0x56436a5d4b60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=133df909465a2ea3 server_cookie=b0e9274672813089 in_seq=0 out_seq=0 2024-05-17T09:21:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.456+0000 7ff215ffb640 1 -- 172.21.15.19:0/4090647629 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56436a52db80 con 0x56436a1d6d80 2024-05-17T09:21:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.456+0000 7ff215ffb640 10 client.15837 handle_command_reply: tid=0 2024-05-17T09:21:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.456+0000 7ff23b419740 1 client.15837 shutdown 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 msgr2=0x56436a5c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c7510 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff21c004650 tx=0x7ff21c00cf00 comp rx=0 tx=0).stop 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 shutdown_connections 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a1d6d80 0x56436a5d4b60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a2181b0 0x56436a5ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff215ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1f74d0 0x56436a5c7510 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 --2- 172.21.15.19:0/4090647629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 >> 172.21.15.19:0/4090647629 conn(0x56436a20c300 msgr2=0x56436a5b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 shutdown_connections 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 1 -- 172.21.15.19:0/4090647629 wait complete. 2024-05-17T09:21:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.457+0000 7ff23b419740 20 client.15837 trim_cache size 0 max 16384 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.473+0000 7ff23b419740 1 Processor -- start 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.473+0000 7ff23b419740 1 -- start start 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff23b419740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1d6d80 0x56436a5b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff23b419740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 0x56436a5ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff23b419740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1f74d0 0x56436a5ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff23b419740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ad150 con 0x56436a1ef2f0 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 0x56436a5ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 0x56436a5ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36950/0 (socket says 172.21.15.19:36950) 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 -- 172.21.15.19:0/3451454178 learned_addr learned my addr 172.21.15.19:0/3451454178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1aaa20 con 0x56436a1f74d0 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1a8870 con 0x56436a1d6d80 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff232308640 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1d6d80 0x56436a5b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff232b09640 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1f74d0 0x56436a5ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 -- 172.21.15.19:0/3451454178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1d6d80 msgr2=0x56436a5b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1d6d80 0x56436a5b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 -- 172.21.15.19:0/3451454178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1f74d0 msgr2=0x56436a5ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1f74d0 0x56436a5ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.474+0000 7ff231b07640 1 -- 172.21.15.19:0/3451454178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436a5bd340 con 0x56436a1ef2f0 2024-05-17T09:21:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.475+0000 7ff231b07640 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 0x56436a5ce390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0a6420 tx=0x7ff21c0ad6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2f0b9c048c8861 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.475+0000 7ff217fff640 1 -- 172.21.15.19:0/3451454178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c01c070 con 0x56436a1ef2f0 2024-05-17T09:21:14.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.475+0000 7ff217fff640 1 -- 172.21.15.19:0/3451454178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff21c0a98b0 con 0x56436a1ef2f0 2024-05-17T09:21:14.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.475+0000 7ff217fff640 1 -- 172.21.15.19:0/3451454178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00ccc0 con 0x56436a1ef2f0 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 msgr2=0x56436a5ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 0x56436a5ce390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0a6420 tx=0x7ff21c0ad6a0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 shutdown_connections 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1f74d0 0x56436a5ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1ef2f0 0x56436a5ce390 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 --2- 172.21.15.19:0/3451454178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1d6d80 0x56436a5b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 >> 172.21.15.19:0/3451454178 conn(0x56436a20c300 msgr2=0x56436a5bd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 shutdown_connections 2024-05-17T09:21:14.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 -- 172.21.15.19:0/3451454178 wait complete. 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 -- start start 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.476+0000 7ff23b419740 1 Processor -- start 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 0x56436a5cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1f74d0 0x56436a63b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 0x56436a5cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1a8ee0 con 0x56436a1d6d80 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1aa350 con 0x56436a1ef2f0 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1a95b0 con 0x56436a1f74d0 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 0x56436a5cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36952/0 (socket says 172.21.15.19:36952) 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 -- 172.21.15.19:0/4074689211 learned_addr learned my addr 172.21.15.19:0/4074689211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff23b419740 10 client.0 _ll_register_callbacks cb 0x7ff21c0e5510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232b09640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1f74d0 0x56436a63b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff231b07640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 -- 172.21.15.19:0/4074689211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1f74d0 msgr2=0x56436a63b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1f74d0 0x56436a63b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 -- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 msgr2=0x56436a5cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232308640 1 -- 172.21.15.19:0/4074689211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436a5cee70 con 0x56436a1d6d80 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.477+0000 7ff232b09640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1f74d0 0x56436a63b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff232308640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 0x56436a5cb970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff228052fd0 tx=0x7ff228005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9d2dd9a7e8f1cb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff215ffb640 1 -- 172.21.15.19:0/4074689211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228035040 con 0x56436a1d6d80 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff215ffb640 1 -- 172.21.15.19:0/4074689211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff228052610 con 0x56436a1d6d80 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21c0a98b0 con 0x56436a1d6d80 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff215ffb640 1 -- 172.21.15.19:0/4074689211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff228062050 con 0x56436a1d6d80 2024-05-17T09:21:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56436a63c760 con 0x56436a1d6d80 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff215ffb640 1 -- 172.21.15.19:0/4074689211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff21c0a98b0 con 0x56436a1d6d80 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff215ffb640 1 -- 172.21.15.19:0/4074689211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff228044040 con 0x56436a1d6d80 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 20 client.15843 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a2181b0 0x56436a63f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 4 client.15843 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.478+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56436a527360 con 0x56436a2181b0 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.479+0000 7ff231b07640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a2181b0 0x56436a63f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.479+0000 7ff215ffb640 10 client.15843 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.479+0000 7ff231b07640 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a2181b0 0x56436a63f0a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e16c7e05af311 server_cookie=eeff4910383215e9 in_seq=0 out_seq=0 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.483+0000 7ff215ffb640 1 -- 172.21.15.19:0/4074689211 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56436a527360 con 0x56436a2181b0 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.483+0000 7ff215ffb640 10 client.15843 handle_command_reply: tid=0 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.483+0000 7ff23b419740 1 client.15843 shutdown 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.483+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 msgr2=0x56436a5cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.483+0000 7ff23b419740 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 0x56436a5cb970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff228052fd0 tx=0x7ff228005fd0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.483+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 shutdown_connections 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56436a2181b0 0x56436a63f0a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56436a1f74d0 0x56436a63b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56436a1ef2f0 0x56436a5cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff215ffb640 0 client.15843 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 --2- 172.21.15.19:0/4074689211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56436a1d6d80 0x56436a5cb970 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 >> 172.21.15.19:0/4074689211 conn(0x56436a20c300 msgr2=0x56436a5bbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 shutdown_connections 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 1 -- 172.21.15.19:0/4074689211 wait complete. 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23b419740 20 client.15843 trim_cache size 0 max 16384 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:21:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:21:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2180b2ca0 msgr2=0x7ff2180b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2180b2ca0 0x7ff2180b51b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff228005ab0 tx=0x7ff228000bc0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c106da0 msgr2=0x7ff22c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c106da0 0x7ff22c19a750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00e9f0 tx=0x7ff21c00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 shutdown_connections 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2180b2ca0 0x7ff2180b51b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff22c1098d0 0x7ff22c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c106da0 0x7ff22c19a750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 --2- 172.21.15.19:0/607748345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff22c1043b0 0x7ff22c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.484+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 >> 172.21.15.19:0/607748345 conn(0x7ff22c0fdd30 msgr2=0x7ff22c1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.485+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 shutdown_connections 2024-05-17T09:21:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:14.485+0000 7ff23456c640 1 -- 172.21.15.19:0/607748345 wait complete. 2024-05-17T09:21:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:14 smithi079 ceph-mon[35502]: pgmap v482: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:14 smithi069 ceph-mon[35081]: pgmap v482: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:15.362 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a72cbc5', quiesce took 9.6 seconds. Will release after: 12.799999999999999 2024-05-17T09:21:15.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:21:15.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:21:15.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:21:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:16 smithi079 ceph-mon[35502]: pgmap v483: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:21:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:21:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:21:16.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:16 smithi019 ceph-mon[31620]: pgmap v483: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:16.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:21:16.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:21:16.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:21:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:16 smithi069 ceph-mon[35081]: pgmap v483: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:21:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:21:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:21:16.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 -- 172.21.15.19:0/1044904605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 msgr2=0x7f80841100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 --2- 172.21.15.19:0/1044904605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841100d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f807c00b0d0 tx=0x7f807c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 -- 172.21.15.19:0/1044904605 shutdown_connections 2024-05-17T09:21:16.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 --2- 172.21.15.19:0/1044904605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841100d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 --2- 172.21.15.19:0/1044904605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8084103540 0x7f8084103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 --2- 172.21.15.19:0/1044904605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8084107e20 0x7f8084108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 -- 172.21.15.19:0/1044904605 >> 172.21.15.19:0/1044904605 conn(0x7f8084078df0 msgr2=0x7f8084079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 -- 172.21.15.19:0/1044904605 shutdown_connections 2024-05-17T09:21:16.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 -- 172.21.15.19:0/1044904605 wait complete. 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.949+0000 7f808c216700 1 Processor -- start 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 -- start start 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8084103540 0x7f808419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8084107e20 0x7f808419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112180 con 0x7f8084108770 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112000 con 0x7f8084103540 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808c216700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112300 con 0x7f8084107e20 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f80897b1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8084107e20 0x7f808419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36962/0 (socket says 172.21.15.19:36962) 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 -- 172.21.15.19:0/982007689 learned_addr learned my addr 172.21.15.19:0/982007689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f8089fb2700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8084103540 0x7f808419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 -- 172.21.15.19:0/982007689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8084107e20 msgr2=0x7f808419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8084107e20 0x7f808419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 -- 172.21.15.19:0/982007689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8084103540 msgr2=0x7f808419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8084103540 0x7f808419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.950+0000 7f808a7b3700 1 -- 172.21.15.19:0/982007689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8084198f00 con 0x7f8084108770 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.951+0000 7f808a7b3700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841988c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c020 tx=0x7f807c00c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.951+0000 7f807affd700 1 -- 172.21.15.19:0/982007689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c026020 con 0x7f8084108770 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.951+0000 7f807affd700 1 -- 172.21.15.19:0/982007689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807c0094f0 con 0x7f8084108770 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.951+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8084199190 con 0x7f8084108770 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.951+0000 7f807affd700 1 -- 172.21.15.19:0/982007689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c021730 con 0x7f8084108770 2024-05-17T09:21:16.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.951+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841996d0 con 0x7f8084108770 2024-05-17T09:21:16.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.954+0000 7f807affd700 1 -- 172.21.15.19:0/982007689 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f807c029020 con 0x7f8084108770 2024-05-17T09:21:16.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.955+0000 7f809ac2bb80 1 Processor -- start 2024-05-17T09:21:16.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.955+0000 7f809ac2bb80 1 -- start start 2024-05-17T09:21:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f807affd700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f80700b2880 0x7f80700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648061633d0 0x564806269fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 0x5648062647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 0x5648062647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 0x5648062647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36964/0 (socket says 172.21.15.19:36964) 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f80897b1700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f80700b2880 0x7f80700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 -- 172.21.15.19:0/1001859137 learned_addr learned my addr 172.21.15.19:0/1001859137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564806192c90 con 0x5648062643c0 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f807affd700 1 -- 172.21.15.19:0/982007689 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f807c0e7e90 con 0x7f8084108770 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564806182830 con 0x5648060e5e40 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564806186870 con 0x5648061633d0 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648061633d0 0x564806269fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f80897b1700 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 -- 172.21.15.19:0/1001859137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648061633d0 msgr2=0x564806269fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648061633d0 0x564806269fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 -- 172.21.15.19:0/1001859137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 msgr2=0x56480626a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.956+0000 7f808a7b3700 1 -- 172.21.15.19:0/1001859137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648060d28d0 con 0x5648062643c0 2024-05-17T09:21:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.957+0000 7f808a7b3700 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 0x5648062647d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f807c0aee90 tx=0x7f807c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a839841c8f3d7f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.957+0000 7f80897b1700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f80700b2880 0x7f80700b4d90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x564806269aa0 tx=0x7f807400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.957+0000 7f806ffff700 1 -- 172.21.15.19:0/1001859137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0ae900 con 0x5648062643c0 2024-05-17T09:21:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.957+0000 7f806ffff700 1 -- 172.21.15.19:0/1001859137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807c0aeca0 con 0x5648062643c0 2024-05-17T09:21:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.957+0000 7f806ffff700 1 -- 172.21.15.19:0/1001859137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c016300 con 0x5648062643c0 2024-05-17T09:21:16.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 msgr2=0x5648062647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 0x5648062647d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f807c0aee90 tx=0x7f807c032000 comp rx=0 tx=0).stop 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 shutdown_connections 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648061633d0 0x564806269fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648062643c0 0x5648062647d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1001859137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 >> 172.21.15.19:0/1001859137 conn(0x564806124c80 msgr2=0x56480625eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 shutdown_connections 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.958+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1001859137 wait complete. 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 -- start start 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 Processor -- start 2024-05-17T09:21:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 0x564806273690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648061633d0 0x564806273bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x56480626da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564806088fa0 con 0x5648060e5e40 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648061247c0 con 0x5648061633d0 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564806265010 con 0x5648062643c0 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f809ac2bb80 10 client.0 _ll_register_callbacks cb 0x56480611d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.959+0000 7f8089fb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 0x564806273690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 0x564806273690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36970/0 (socket says 172.21.15.19:36970) 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 -- 172.21.15.19:0/1016384693 learned_addr learned my addr 172.21.15.19:0/1016384693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f80897b1700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648061633d0 0x564806273bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f808a7b3700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x56480626da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 -- 172.21.15.19:0/1016384693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 msgr2=0x56480626da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x56480626da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 -- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648061633d0 msgr2=0x564806273bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648061633d0 0x564806273bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 -- 172.21.15.19:0/1016384693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564806121690 con 0x5648060e5e40 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f80897b1700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648061633d0 0x564806273bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 0x564806273690 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f80800096b0 tx=0x7f80800099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807c2c542e961b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f806d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f806d7fa700 1 -- 172.21.15.19:0/1016384693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808001d020 con 0x5648060e5e40 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f806d7fa700 1 -- 172.21.15.19:0/1016384693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8080019920 con 0x5648060e5e40 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.960+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807c0aeca0 con 0x5648060e5e40 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f806d7fa700 1 -- 172.21.15.19:0/1016384693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8080028050 con 0x5648060e5e40 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 10 client.15855 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56480627b310 con 0x5648060e5e40 2024-05-17T09:21:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f806d7fa700 1 -- 172.21.15.19:0/1016384693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807c0aeca0 con 0x5648060e5e40 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f806d7fa700 1 -- 172.21.15.19:0/1016384693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8080018aa0 con 0x5648060e5e40 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 10 client.15855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 20 client.15855 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x56480627d9f0 0x56480627fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 4 client.15855 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.961+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564806194be0 con 0x56480627d9f0 2024-05-17T09:21:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.962+0000 7f80897b1700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x56480627d9f0 0x56480627fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.962+0000 7f80897b1700 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x56480627d9f0 0x56480627fe10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd317f0ac454cd8f server_cookie=db9a5211729e1f4e in_seq=0 out_seq=0 2024-05-17T09:21:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.962+0000 7f806d7fa700 10 client.15855 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f806d7fa700 1 -- 172.21.15.19:0/1016384693 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564806194be0 con 0x56480627d9f0 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f806d7fa700 10 client.15855 handle_command_reply: tid=0 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 client.15855 shutdown 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 msgr2=0x564806273690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 0x564806273690 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f80800096b0 tx=0x7f80800099c0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 shutdown_connections 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x56480627d9f0 0x56480627fe10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x56480626da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648060e5e40 0x564806273690 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1016384693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648061633d0 0x564806273bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f806d7fa700 0 client.15855 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 >> 172.21.15.19:0/1016384693 conn(0x564806124c80 msgr2=0x56480625eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 shutdown_connections 2024-05-17T09:21:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1016384693 wait complete. 2024-05-17T09:21:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.963+0000 7f809ac2bb80 20 client.15855 trim_cache size 0 max 16384 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.979+0000 7f809ac2bb80 1 Processor -- start 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.979+0000 7f809ac2bb80 1 -- start start 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806273ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x5648062692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80841b0890 con 0x5648061633d0 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80841ae120 con 0x5648060e5e40 2024-05-17T09:21:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f80841abf70 con 0x5648062643c0 2024-05-17T09:21:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f80897b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806273ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f80897b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806273ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36978/0 (socket says 172.21.15.19:36978) 2024-05-17T09:21:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f80897b1700 1 -- 172.21.15.19:0/298774426 learned_addr learned my addr 172.21.15.19:0/298774426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f808a7b3700 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x5648062692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f8089fb2700 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f808a7b3700 1 -- 172.21.15.19:0/298774426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 msgr2=0x56480626c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f808a7b3700 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f808a7b3700 1 -- 172.21.15.19:0/298774426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 msgr2=0x564806273ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f808a7b3700 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806273ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f808a7b3700 1 -- 172.21.15.19:0/298774426 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564806258c80 con 0x5648062643c0 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.980+0000 7f80897b1700 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806273ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.981+0000 7f808a7b3700 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x5648062692f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f807c0aee90 tx=0x7f807c006160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b56fe59cb4ea63f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.981+0000 7f806f7fe700 1 -- 172.21.15.19:0/298774426 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0af890 con 0x5648062643c0 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.981+0000 7f806f7fe700 1 -- 172.21.15.19:0/298774426 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807c0b3540 con 0x5648062643c0 2024-05-17T09:21:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.981+0000 7f806f7fe700 1 -- 172.21.15.19:0/298774426 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c033350 con 0x5648062643c0 2024-05-17T09:21:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 -- 172.21.15.19:0/298774426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 msgr2=0x5648062692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x5648062692f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f807c0aee90 tx=0x7f807c006160 comp rx=0 tx=0).stop 2024-05-17T09:21:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 -- 172.21.15.19:0/298774426 shutdown_connections 2024-05-17T09:21:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648062643c0 0x5648062692f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648060e5e40 0x56480626c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/298774426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806273ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 -- 172.21.15.19:0/298774426 >> 172.21.15.19:0/298774426 conn(0x564806124c80 msgr2=0x564806268be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 -- 172.21.15.19:0/298774426 shutdown_connections 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 -- 172.21.15.19:0/298774426 wait complete. 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 -- start start 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.982+0000 7f809ac2bb80 1 Processor -- start 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648060e5e40 0x564806279910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806279e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648062643c0 0x564806276e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80841ac5e0 con 0x5648061633d0 2024-05-17T09:21:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80841ada50 con 0x5648062643c0 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f80841accb0 con 0x5648060e5e40 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f809ac2bb80 10 client.0 _ll_register_callbacks cb 0x56480611d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806279e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806279e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36980/0 (socket says 172.21.15.19:36980) 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 -- 172.21.15.19:0/1810095934 learned_addr learned my addr 172.21.15.19:0/1810095934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648060e5e40 0x564806279910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f808a7b3700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648062643c0 0x564806276e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 -- 172.21.15.19:0/1810095934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648060e5e40 msgr2=0x564806279910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648060e5e40 0x564806279910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 -- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648062643c0 msgr2=0x564806276e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648062643c0 0x564806276e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.983+0000 7f80897b1700 1 -- 172.21.15.19:0/1810095934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648062774a0 con 0x5648061633d0 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f808a7b3700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648062643c0 0x564806276e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f80897b1700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806279e50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f808406d130 tx=0x7f80740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29e6ff6b51b4bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f806d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f806d7fa700 1 -- 172.21.15.19:0/1810095934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074031060 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f806d7fa700 1 -- 172.21.15.19:0/1810095934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f807400f450 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807c0b3540 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f806d7fa700 1 -- 172.21.15.19:0/1810095934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8074038050 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f809ac2bb80 10 client.15861 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.984+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648062e7e10 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f806d7fa700 1 -- 172.21.15.19:0/1810095934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f807c0b3540 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f806d7fa700 1 -- 172.21.15.19:0/1810095934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8074012570 con 0x5648061633d0 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f809ac2bb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f809ac2bb80 10 client.15861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f809ac2bb80 20 client.15861 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5648062e8530 0x5648062ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f809ac2bb80 4 client.15861 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56480618e3c0 con 0x5648062e8530 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.985+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5648062e8530 0x5648062ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f806d7fa700 10 client.15861 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f8089fb2700 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5648062e8530 0x5648062ea950 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84597f6d0e15efcf server_cookie=58fb0fceb1aae0b5 in_seq=0 out_seq=0 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f806d7fa700 1 -- 172.21.15.19:0/1810095934 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56480618e3c0 con 0x5648062e8530 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f806d7fa700 10 client.15861 handle_command_reply: tid=0 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 client.15861 shutdown 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 msgr2=0x564806279e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806279e50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f808406d130 tx=0x7f80740047c0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 shutdown_connections 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5648062e8530 0x5648062ea950 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648060e5e40 0x564806279910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648062643c0 0x564806276e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 --2- 172.21.15.19:0/1810095934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648061633d0 0x564806279e50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 >> 172.21.15.19:0/1810095934 conn(0x564806124c80 msgr2=0x564806258f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.986+0000 7f806d7fa700 0 client.15861 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 shutdown_connections 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f809ac2bb80 1 -- 172.21.15.19:0/1810095934 wait complete. 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f809ac2bb80 20 client.15861 trim_cache size 0 max 16384 2024-05-17T09:21:16.992 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:21:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f80700b2880 msgr2=0x7f80700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f808c216700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f80700b2880 0x7f80700b4d90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x564806269aa0 tx=0x7f807400a380 comp rx=0 tx=0).stop 2024-05-17T09:21:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 msgr2=0x7f80841988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.987+0000 7f808c216700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841988c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c020 tx=0x7f807c00c050 comp rx=0 tx=0).stop 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 shutdown_connections 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8084108770 0x7f80841988c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8084103540 0x7f808419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f80700b2880 0x7f80700b4d90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 --2- 172.21.15.19:0/982007689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8084107e20 0x7f808419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 >> 172.21.15.19:0/982007689 conn(0x7f8084078df0 msgr2=0x7f8084100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 shutdown_connections 2024-05-17T09:21:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:16.988+0000 7f808c216700 1 -- 172.21.15.19:0/982007689 wait complete. 2024-05-17T09:21:17.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:21:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:18.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:18 smithi019 ceph-mon[31620]: pgmap v484: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:18 smithi069 ceph-mon[35081]: pgmap v484: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:18 smithi079 ceph-mon[35502]: pgmap v484: 129 pgs: 129 active+clean; 670 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 -- 172.21.15.19:0/1784893188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c68108730 msgr2=0x7f9c68110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/1784893188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c68108730 0x7f9c68110090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9c600099c0 tx=0x7f9c60009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 -- 172.21.15.19:0/1784893188 shutdown_connections 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/1784893188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c68108730 0x7f9c68110090 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/1784893188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c681034b0 0x7f9c681038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/1784893188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c68107de0 0x7f9c681081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 -- 172.21.15.19:0/1784893188 >> 172.21.15.19:0/1784893188 conn(0x7f9c680fedb0 msgr2=0x7f9c681011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 -- 172.21.15.19:0/1784893188 shutdown_connections 2024-05-17T09:21:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.877+0000 7f9c6d13b700 1 -- 172.21.15.19:0/1784893188 wait complete. 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.878+0000 7f9c6d13b700 1 Processor -- start 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.878+0000 7f9c6d13b700 1 -- start start 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6d13b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 0x7f9c681a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6d13b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c68107de0 0x7f9c681a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6d13b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c68108730 0x7f9c6819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6d13b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68113210 con 0x7f9c681034b0 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6d13b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68113090 con 0x7f9c68108730 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6d13b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68113390 con 0x7f9c68107de0 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 0x7f9c681a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 0x7f9c681a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36994/0 (socket says 172.21.15.19:36994) 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 -- 172.21.15.19:0/3983691752 learned_addr learned my addr 172.21.15.19:0/3983691752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6659c700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c68107de0 0x7f9c681a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c6759e700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c68108730 0x7f9c6819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 -- 172.21.15.19:0/3983691752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c68107de0 msgr2=0x7f9c681a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c68107de0 0x7f9c681a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 -- 172.21.15.19:0/3983691752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c68108730 msgr2=0x7f9c6819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c68108730 0x7f9c6819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.879+0000 7f9c66d9d700 1 -- 172.21.15.19:0/3983691752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6819b0e0 con 0x7f9c681034b0 2024-05-17T09:21:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.880+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 0x7f9c681a0880 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800ef10 tx=0x7f9c5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.880+0000 7f9c5ffff700 1 -- 172.21.15.19:0/3983691752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800c9c0 con 0x7f9c681034b0 2024-05-17T09:21:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.880+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6819b370 con 0x7f9c681034b0 2024-05-17T09:21:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.880+0000 7f9c5ffff700 1 -- 172.21.15.19:0/3983691752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c58019070 con 0x7f9c681034b0 2024-05-17T09:21:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.880+0000 7f9c5ffff700 1 -- 172.21.15.19:0/3983691752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58014480 con 0x7f9c681034b0 2024-05-17T09:21:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.880+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c681a7710 con 0x7f9c681034b0 2024-05-17T09:21:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.883+0000 7f9c5ffff700 1 -- 172.21.15.19:0/3983691752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c58012050 con 0x7f9c681034b0 2024-05-17T09:21:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.883+0000 7f9c5ffff700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c540b2880 0x7f9c540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.884+0000 7f9c6659c700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c540b2880 0x7f9c540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.884+0000 7f9c5ffff700 1 -- 172.21.15.19:0/3983691752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9c580e24f0 con 0x7f9c681034b0 2024-05-17T09:21:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.884+0000 7f9c6659c700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c540b2880 0x7f9c540b4d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9c68109900 tx=0x7f9c5000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.885+0000 7f9c7bb50b80 1 Processor -- start 2024-05-17T09:21:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.885+0000 7f9c7bb50b80 1 -- start start 2024-05-17T09:21:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8896d13d0 0x55a8897d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 0x55a8897d8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8897d23c0 0x55a8897d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a889700c90 con 0x55a889653e40 2024-05-17T09:21:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8896f0830 con 0x55a8897d23c0 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8896f4870 con 0x55a8896d13d0 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c6659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 0x55a8897d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c6659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 0x55a8897d8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36998/0 (socket says 172.21.15.19:36998) 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c6659c700 1 -- 172.21.15.19:0/3846246083 learned_addr learned my addr 172.21.15.19:0/3846246083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8896d13d0 0x55a8897d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.886+0000 7f9c6759e700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8897d23c0 0x55a8897d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6659c700 1 -- 172.21.15.19:0/3846246083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8896d13d0 msgr2=0x55a8897d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6659c700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8896d13d0 0x55a8897d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6659c700 1 -- 172.21.15.19:0/3846246083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8897d23c0 msgr2=0x55a8897d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6659c700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8897d23c0 0x55a8897d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6659c700 1 -- 172.21.15.19:0/3846246083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8896408d0 con 0x55a889653e40 2024-05-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8896d13d0 0x55a8897d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6759e700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8897d23c0 0x55a8897d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c6659c700 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 0x55a8897d8520 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9c500044d0 tx=0x7f9c50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc2e460e136646a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c5d7fa700 1 -- 172.21.15.19:0/3846246083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50016070 con 0x55a889653e40 2024-05-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c5d7fa700 1 -- 172.21.15.19:0/3846246083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c50014070 con 0x55a889653e40 2024-05-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.887+0000 7f9c5d7fa700 1 -- 172.21.15.19:0/3846246083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50010b90 con 0x55a889653e40 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/3846246083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 msgr2=0x55a8897d8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 0x55a8897d8520 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9c500044d0 tx=0x7f9c50012000 comp rx=0 tx=0).stop 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/3846246083 shutdown_connections 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8896d13d0 0x55a8897d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8897d23c0 0x55a8897d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/3846246083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a889653e40 0x55a8897d8520 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/3846246083 >> 172.21.15.19:0/3846246083 conn(0x55a889692c80 msgr2=0x55a8897ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/3846246083 shutdown_connections 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.888+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/3846246083 wait complete. 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 -- start start 2024-05-17T09:21:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 Processor -- start 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 0x55a8897dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8896d13d0 0x55a8897dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8897d23c0 0x55a8897dc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8895f6fa0 con 0x55a8896d13d0 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8896927c0 con 0x55a889653e40 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8897d3250 con 0x55a8897d23c0 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.889+0000 7f9c7bb50b80 10 client.0 _ll_register_callbacks cb 0x55a88968b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 0x55a8897dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 0x55a8897dbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35316/0 (socket says 172.21.15.19:35316) 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 -- 172.21.15.19:0/1068077323 learned_addr learned my addr 172.21.15.19:0/1068077323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c6659c700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8896d13d0 0x55a8897dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c6759e700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8897d23c0 0x55a8897dc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 -- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8897d23c0 msgr2=0x55a8897dc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8897d23c0 0x55a8897dc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 -- 172.21.15.19:0/1068077323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8896d13d0 msgr2=0x55a8897dc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8896d13d0 0x55a8897dc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 -- 172.21.15.19:0/1068077323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88968f690 con 0x55a889653e40 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c6659c700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8896d13d0 0x55a8897dc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 0x55a8897dbc40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c580dfc90 tx=0x7f9c580a9ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff32235dec9b2b13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.890+0000 7f9c4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/1068077323 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c580cc0b0 con 0x55a889653e40 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/1068077323 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9c580910b0 con 0x55a889653e40 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c50014070 con 0x55a889653e40 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/1068077323 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c580a9270 con 0x55a889653e40 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c7bb50b80 10 client.25565 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8897e95d0 con 0x55a889653e40 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/1068077323 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c50014070 con 0x55a889653e40 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/1068077323 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9c580bebb0 con 0x55a889653e40 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.891+0000 7f9c7bb50b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.892+0000 7f9c7bb50b80 10 client.25565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.892+0000 7f9c7bb50b80 20 client.25565 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.892+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a8897ee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.892+0000 7f9c7bb50b80 4 client.25565 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.892+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a889702be0 con 0x55a8897ebc10 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.892+0000 7f9c6659c700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a8897ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.893+0000 7f9c4e7fc700 10 client.25565 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.893+0000 7f9c6659c700 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a8897ee030 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2ba458dfb739029 server_cookie=2d5b15a3b8679d95 in_seq=0 out_seq=0 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.893+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/1068077323 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a889702be0 con 0x55a8897ebc10 2024-05-17T09:21:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.893+0000 7f9c4e7fc700 10 client.25565 handle_command_reply: tid=0 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.893+0000 7f9c7bb50b80 1 client.25565 shutdown 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 msgr2=0x55a8897dbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 0x55a8897dbc40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c580dfc90 tx=0x7f9c580a9ca0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 shutdown_connections 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a8897ee030 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8897d23c0 0x55a8897dc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c4e7fc700 0 client.25565 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a889653e40 0x55a8897dbc40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/1068077323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8896d13d0 0x55a8897dc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 >> 172.21.15.19:0/1068077323 conn(0x55a889692c80 msgr2=0x55a8897ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 shutdown_connections 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.894+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/1068077323 wait complete. 2024-05-17T09:21:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.895+0000 7f9c7bb50b80 20 client.25565 trim_cache size 0 max 16384 2024-05-17T09:21:19.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.925+0000 7f9c7bb50b80 1 Processor -- start 2024-05-17T09:21:19.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.925+0000 7f9c7bb50b80 1 -- start start 2024-05-17T09:21:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897dc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681b2ae0 con 0x55a8897d23c0 2024-05-17T09:21:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681b0370 con 0x55a8896d13d0 2024-05-17T09:21:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681ae1c0 con 0x55a889653e40 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c6759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c6759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37010/0 (socket says 172.21.15.19:37010) 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c6759e700 1 -- 172.21.15.19:0/2616859910 learned_addr learned my addr 172.21.15.19:0/2616859910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897dc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.926+0000 7f9c6659c700 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c6759e700 1 -- 172.21.15.19:0/2616859910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 msgr2=0x55a8897dc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c6759e700 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897dc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c6759e700 1 -- 172.21.15.19:0/2616859910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 msgr2=0x55a8897e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c6759e700 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c6759e700 1 -- 172.21.15.19:0/2616859910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8897dca40 con 0x55a8897d23c0 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c6759e700 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897c7670 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9c600099c0 tx=0x7f9c60005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616df98d707ef5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c5cff9700 1 -- 172.21.15.19:0/2616859910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60005460 con 0x55a8897d23c0 2024-05-17T09:21:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c5cff9700 1 -- 172.21.15.19:0/2616859910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c60005600 con 0x55a8897d23c0 2024-05-17T09:21:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.927+0000 7f9c5cff9700 1 -- 172.21.15.19:0/2616859910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000bda0 con 0x55a8897d23c0 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2616859910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 msgr2=0x55a8897c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897c7670 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9c600099c0 tx=0x7f9c60005040 comp rx=0 tx=0).stop 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2616859910 shutdown_connections 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897c7670 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2616859910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.928+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2616859910 >> 172.21.15.19:0/2616859910 conn(0x55a889692c80 msgr2=0x55a8897c6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.929+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2616859910 shutdown_connections 2024-05-17T09:21:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.929+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2616859910 wait complete. 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.929+0000 7f9c7bb50b80 1 -- start start 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.929+0000 7f9c7bb50b80 1 Processor -- start 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681ae830 con 0x55a8897d23c0 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681afca0 con 0x55a8896d13d0 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681aef00 con 0x55a889653e40 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c7bb50b80 10 client.0 _ll_register_callbacks cb 0x55a88968b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c6759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897e41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46248/0 (socket says 172.21.15.19:46248) 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 -- 172.21.15.19:0/2984944466 learned_addr learned my addr 172.21.15.19:0/2984944466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c6659c700 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 -- 172.21.15.19:0/2984944466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 msgr2=0x55a8897d8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897d8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 -- 172.21.15.19:0/2984944466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 msgr2=0x55a8897d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897d8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 -- 172.21.15.19:0/2984944466 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8897d9000 con 0x55a889653e40 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c66d9d700 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897e41b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c580a9460 tx=0x7f9c58009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b355750c93e46e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.930+0000 7f9c4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/2984944466 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58091380 con 0x55a889653e40 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/2984944466 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9c580a5d50 con 0x55a889653e40 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c60005600 con 0x55a889653e40 2024-05-17T09:21:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/2984944466 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c580cc0b0 con 0x55a889653e40 2024-05-17T09:21:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 10 client.25585 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8898560a0 con 0x55a889653e40 2024-05-17T09:21:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/2984944466 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c60005600 con 0x55a889653e40 2024-05-17T09:21:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/2984944466 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9c58005c00 con 0x55a889653e40 2024-05-17T09:21:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 10 client.25585 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 10 client.25585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:21:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 20 client.25585 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a889858630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 4 client.25585 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.931+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8896fc3c0 con 0x55a8897ebc10 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.932+0000 7f9c6659c700 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a889858630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.932+0000 7f9c4e7fc700 10 client.25585 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.932+0000 7f9c6659c700 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a889858630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ab74f9b26bb0983 server_cookie=936016684540566c in_seq=0 out_seq=0 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.932+0000 7f9c4e7fc700 1 -- 172.21.15.19:0/2984944466 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8896fc3c0 con 0x55a8897ebc10 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.932+0000 7f9c4e7fc700 10 client.25585 handle_command_reply: tid=0 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.932+0000 7f9c7bb50b80 1 client.25585 shutdown 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 msgr2=0x55a8897e41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897e41b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c580a9460 tx=0x7f9c58009990 comp rx=0 tx=0).stop 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 shutdown_connections 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x55a8897ebc10 0x55a889858630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a889653e40 0x55a8897e41b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8897d23c0 0x55a8897d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 --2- 172.21.15.19:0/2984944466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8896d13d0 0x55a8897d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 >> 172.21.15.19:0/2984944466 conn(0x55a889692c80 msgr2=0x55a8897d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c4e7fc700 0 client.25585 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 shutdown_connections 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 1 -- 172.21.15.19:0/2984944466 wait complete. 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c7bb50b80 20 client.25585 trim_cache size 0 max 16384 2024-05-17T09:21:19.937 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c540b2880 msgr2=0x7f9c540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c540b2880 0x7f9c540b4d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9c68109900 tx=0x7f9c5000a380 comp rx=0 tx=0).stop 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 msgr2=0x7f9c681a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.933+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 0x7f9c681a0880 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800ef10 tx=0x7f9c5800c3a0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 shutdown_connections 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c68108730 0x7f9c6819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c681034b0 0x7f9c681a0880 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c540b2880 0x7f9c540b4d90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 --2- 172.21.15.19:0/3983691752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c68107de0 0x7f9c681a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 >> 172.21.15.19:0/3983691752 conn(0x7f9c680fedb0 msgr2=0x7f9c6806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 shutdown_connections 2024-05-17T09:21:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:19.934+0000 7f9c6d13b700 1 -- 172.21.15.19:0/3983691752 wait complete. 2024-05-17T09:21:20.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:20 smithi019 ceph-mon[31620]: pgmap v485: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:20 smithi069 ceph-mon[35081]: pgmap v485: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:20 smithi079 ceph-mon[35502]: pgmap v485: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:20.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:21:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:22.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:22 smithi019 ceph-mon[31620]: pgmap v486: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:22 smithi069 ceph-mon[35081]: pgmap v486: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:22 smithi079 ceph-mon[35502]: pgmap v486: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 -- 172.21.15.19:0/2360234409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac41034b0 msgr2=0x7f5ac41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/2360234409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac41034b0 0x7f5ac41038c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40099c0 tx=0x7f5ab4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 -- 172.21.15.19:0/2360234409 shutdown_connections 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/2360234409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ac4108730 0x7f5ac4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/2360234409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac41034b0 0x7f5ac41038c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/2360234409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ac4107de0 0x7f5ac41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 -- 172.21.15.19:0/2360234409 >> 172.21.15.19:0/2360234409 conn(0x7f5ac40fedb0 msgr2=0x7f5ac41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:22.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 -- 172.21.15.19:0/2360234409 shutdown_connections 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.933+0000 7f5ac8c28700 1 -- 172.21.15.19:0/2360234409 wait complete. 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 Processor -- start 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 -- start start 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ac41034b0 0x7f5ac419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ac4107de0 0x7f5ac419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 0x7f5ac419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4113260 con 0x7f5ac4108730 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41130e0 con 0x7f5ac41034b0 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.934+0000 7f5ac8c28700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41133e0 con 0x7f5ac4107de0 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 0x7f5ac419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 0x7f5ac419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37028/0 (socket says 172.21.15.19:37028) 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 -- 172.21.15.19:0/3199266006 learned_addr learned my addr 172.21.15.19:0/3199266006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac259c700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ac41034b0 0x7f5ac419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ac4107de0 0x7f5ac419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 -- 172.21.15.19:0/3199266006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ac4107de0 msgr2=0x7f5ac419c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ac4107de0 0x7f5ac419c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 -- 172.21.15.19:0/3199266006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ac41034b0 msgr2=0x7f5ac419bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ac41034b0 0x7f5ac419bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 -- 172.21.15.19:0/3199266006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac41a6f30 con 0x7f5ac4108730 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac2d9d700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 0x7f5ac419c680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ada0 tx=0x7f5abc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ab37fe700 1 -- 172.21.15.19:0/3199266006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc017020 con 0x7f5ac4108730 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac41a7220 con 0x7f5ac4108730 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ab37fe700 1 -- 172.21.15.19:0/3199266006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5abc00cd60 con 0x7f5ac4108730 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.935+0000 7f5ab37fe700 1 -- 172.21.15.19:0/3199266006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc005280 con 0x7f5ac4108730 2024-05-17T09:21:22.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.936+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac41a7710 con 0x7f5ac4108730 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.939+0000 7f5ab37fe700 1 -- 172.21.15.19:0/3199266006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5abc005420 con 0x7f5ac4108730 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.939+0000 7f5ad763db80 1 Processor -- start 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 -- start start 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a403fecc90 con 0x55a4040c27d0 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a403fdc830 con 0x55a403f3fe40 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ad763db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a403fe0870 con 0x55a403fbd3d0 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ac1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ac1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37044/0 (socket says 172.21.15.19:37044) 2024-05-17T09:21:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/2196371865 learned_addr learned my addr 172.21.15.19:0/2196371865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ab37fe700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5aac0b2880 0x7f5aac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.940+0000 7f5ac259c700 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac2d9d700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5aac0b2880 0x7f5aac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/2196371865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 msgr2=0x55a4040c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/2196371865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 msgr2=0x55a4040c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/2196371865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a403f2c8d0 con 0x55a4040c27d0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ab37fe700 1 -- 172.21.15.19:0/3199266006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5abc0e1d30 con 0x7f5ac4108730 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040c2be0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55a4040baba0 tx=0x7f5ab8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff25b5490caedb9a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ab0ff9700 1 -- 172.21.15.19:0/2196371865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8012dd0 con 0x55a4040c27d0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ac2d9d700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5aac0b2880 0x7f5aac0b4d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ad70 tx=0x7f5abc06e650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ab0ff9700 1 -- 172.21.15.19:0/2196371865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab8004cc0 con 0x55a4040c27d0 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.941+0000 7f5ab0ff9700 1 -- 172.21.15.19:0/2196371865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab800f3f0 con 0x55a4040c27d0 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 -- 172.21.15.19:0/2196371865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 msgr2=0x55a4040c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040c2be0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55a4040baba0 tx=0x7f5ab8005d20 comp rx=0 tx=0).stop 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 -- 172.21.15.19:0/2196371865 shutdown_connections 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040c2be0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2196371865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 -- 172.21.15.19:0/2196371865 >> 172.21.15.19:0/2196371865 conn(0x55a403f7ec80 msgr2=0x55a4040b8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 -- 172.21.15.19:0/2196371865 shutdown_connections 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.942+0000 7f5ad763db80 1 -- 172.21.15.19:0/2196371865 wait complete. 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 -- start start 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 Processor -- start 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a403ee2fa0 con 0x55a4040c27d0 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a403f7e7c0 con 0x55a403f3fe40 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a4040c35c0 con 0x55a403fbd3d0 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ad763db80 10 client.0 _ll_register_callbacks cb 0x55a403f77920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ac1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ac1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37048/0 (socket says 172.21.15.19:37048) 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.943+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/814650478 learned_addr learned my addr 172.21.15.19:0/814650478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac259c700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/814650478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 msgr2=0x55a4040cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 msgr2=0x55a4040cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/814650478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a403f7b690 con 0x55a4040c27d0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040ccb80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800b510 tx=0x7f5ab8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aa8e908243da4a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5aa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/814650478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8004750 con 0x55a4040c27d0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/814650478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5ab801b040 con 0x55a4040c27d0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab8004cc0 con 0x55a4040c27d0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/814650478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ab80048f0 con 0x55a4040c27d0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ad763db80 10 client.15891 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.944+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4040d99f0 con 0x55a4040c27d0 2024-05-17T09:21:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/814650478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab8004cc0 con 0x55a4040c27d0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/814650478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ab8003960 con 0x55a4040c27d0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ad763db80 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ad763db80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ad763db80 20 client.15891 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ad763db80 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a4040dbec0 0x55a4040de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ad763db80 4 client.15891 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a403feebe0 con 0x55a4040dbec0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.945+0000 7f5ac259c700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a4040dbec0 0x55a4040de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.946+0000 7f5aa9ffb700 10 client.15891 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.946+0000 7f5ac259c700 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a4040dbec0 0x55a4040de2e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2692471e00da326 server_cookie=9e744fbcd26f31b6 in_seq=0 out_seq=0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/814650478 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a403feebe0 con 0x55a4040dbec0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5aa9ffb700 10 client.15891 handle_command_reply: tid=0 2024-05-17T09:21:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 client.15891 shutdown 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 msgr2=0x55a4040ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040ccb80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800b510 tx=0x7f5ab8021040 comp rx=0 tx=0).stop 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 shutdown_connections 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a4040dbec0 0x55a4040de2e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4040c27d0 0x55a4040ccb80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 --2- 172.21.15.19:0/814650478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403fbd3d0 0x55a4040cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 >> 172.21.15.19:0/814650478 conn(0x55a403f7ec80 msgr2=0x55a4040b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5aa9ffb700 0 client.15891 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 shutdown_connections 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 1 -- 172.21.15.19:0/814650478 wait complete. 2024-05-17T09:21:22.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.947+0000 7f5ad763db80 20 client.15891 trim_cache size 0 max 16384 2024-05-17T09:21:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 Processor -- start 2024-05-17T09:21:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 -- start start 2024-05-17T09:21:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403f3fe40 0x55a4040b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4040d8030 0x55a4040c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41b0a50 con 0x55a403fbd3d0 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ae2e0 con 0x55a4040d8030 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ad763db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ac130 con 0x55a403f3fe40 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.963+0000 7f5ac259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37050/0 (socket says 172.21.15.19:37050) 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 -- 172.21.15.19:0/2469439783 learned_addr learned my addr 172.21.15.19:0/2469439783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403f3fe40 0x55a4040b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4040d8030 0x55a4040c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 -- 172.21.15.19:0/2469439783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403f3fe40 msgr2=0x55a4040b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403f3fe40 0x55a4040b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 -- 172.21.15.19:0/2469439783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4040d8030 msgr2=0x55a4040c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4040d8030 0x55a4040c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 -- 172.21.15.19:0/2469439783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4040ce9a0 con 0x55a403fbd3d0 2024-05-17T09:21:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5ac259c700 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040c6820 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40058e0 tx=0x7f5ab4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1463597abbac752 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5aabfff700 1 -- 172.21.15.19:0/2469439783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400b990 con 0x55a403fbd3d0 2024-05-17T09:21:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5aabfff700 1 -- 172.21.15.19:0/2469439783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab4003680 con 0x55a403fbd3d0 2024-05-17T09:21:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.964+0000 7f5aabfff700 1 -- 172.21.15.19:0/2469439783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400f6c0 con 0x55a403fbd3d0 2024-05-17T09:21:22.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 -- 172.21.15.19:0/2469439783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 msgr2=0x55a4040c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040c6820 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40058e0 tx=0x7f5ab4005040 comp rx=0 tx=0).stop 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 -- 172.21.15.19:0/2469439783 shutdown_connections 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a403f3fe40 0x55a4040b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4040d8030 0x55a4040c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 --2- 172.21.15.19:0/2469439783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040c6820 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 -- 172.21.15.19:0/2469439783 >> 172.21.15.19:0/2469439783 conn(0x55a403f7ec80 msgr2=0x55a4040c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.965+0000 7f5ad763db80 1 -- 172.21.15.19:0/2469439783 shutdown_connections 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 -- 172.21.15.19:0/2469439783 wait complete. 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 -- start start 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 Processor -- start 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a4040d8030 0x55a4040d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ac7a0 con 0x55a403fbd3d0 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41adc10 con 0x55a403f3fe40 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ace70 con 0x55a4040d8030 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.966+0000 7f5ad763db80 10 client.0 _ll_register_callbacks cb 0x55a403f77920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a4040d8030 0x55a4040d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a4040d8030 0x55a4040d1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46294/0 (socket says 172.21.15.19:46294) 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac1d9b700 1 -- 172.21.15.19:0/1257804115 learned_addr learned my addr 172.21.15.19:0/1257804115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 -- 172.21.15.19:0/1257804115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a4040d8030 msgr2=0x55a4040d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a4040d8030 0x55a4040d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 -- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 msgr2=0x55a4040d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 -- 172.21.15.19:0/1257804115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4040d2410 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ac259c700 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040d1990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098e0 tx=0x7f5ab4005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f6267c30db589a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5aa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/1257804115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4003c80 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab4003680 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.967+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/1257804115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5ab400b730 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/1257804115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ab400b950 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 10 client.15897 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a404146390 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/1257804115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab4003680 con 0x55a403fbd3d0 2024-05-17T09:21:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/1257804115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ab4038a20 con 0x55a403fbd3d0 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 20 client.15897 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a404146990 0x55a404148db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 4 client.15897 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a403fe83c0 con 0x55a404146990 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.968+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a404146990 0x55a404148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ac1d9b700 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a404146990 0x55a404148db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f3003a9d5f6a4e server_cookie=60830c3df526ad84 in_seq=0 out_seq=0 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5aa9ffb700 10 client.15897 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5aa9ffb700 1 -- 172.21.15.19:0/1257804115 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a403fe83c0 con 0x55a404146990 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5aa9ffb700 10 client.15897 handle_command_reply: tid=0 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 client.15897 shutdown 2024-05-17T09:21:22.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 msgr2=0x55a4040d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040d1990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098e0 tx=0x7f5ab4005620 comp rx=0 tx=0).stop 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 shutdown_connections 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a4040d8030 0x55a4040d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a404146990 0x55a404148db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5aa9ffb700 0 client.15897 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a403f3fe40 0x55a4040d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 --2- 172.21.15.19:0/1257804115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a403fbd3d0 0x55a4040d1990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.969+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 >> 172.21.15.19:0/1257804115 conn(0x55a403f7ec80 msgr2=0x55a4040c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 shutdown_connections 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ad763db80 1 -- 172.21.15.19:0/1257804115 wait complete. 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ad763db80 20 client.15897 trim_cache size 0 max 16384 2024-05-17T09:21:22.979 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:21:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5aac0b2880 msgr2=0x7f5aac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5aac0b2880 0x7f5aac0b4d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ad70 tx=0x7f5abc06e650 comp rx=0 tx=0).stop 2024-05-17T09:21:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 msgr2=0x7f5ac419c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 0x7f5ac419c680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ada0 tx=0x7f5abc00c330 comp rx=0 tx=0).stop 2024-05-17T09:21:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 shutdown_connections 2024-05-17T09:21:22.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ac4108730 0x7f5ac419c680 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ac41034b0 0x7f5ac419bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5aac0b2880 0x7f5aac0b4d90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 --2- 172.21.15.19:0/3199266006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ac4107de0 0x7f5ac419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 >> 172.21.15.19:0/3199266006 conn(0x7f5ac40fedb0 msgr2=0x7f5ac41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.970+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 shutdown_connections 2024-05-17T09:21:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:22.971+0000 7f5ac8c28700 1 -- 172.21.15.19:0/3199266006 wait complete. 2024-05-17T09:21:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:24 smithi069 ceph-mon[35081]: pgmap v487: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:24 smithi019 ceph-mon[31620]: pgmap v487: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:24 smithi079 ceph-mon[35502]: pgmap v487: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:24.774 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:21:24.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:21:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:26.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:26 smithi019 ceph-mon[31620]: pgmap v488: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:26 smithi069 ceph-mon[35081]: pgmap v488: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:26 smithi079 ceph-mon[35502]: pgmap v488: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 -- 172.21.15.19:0/3399203564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca41034b0 msgr2=0x7fcca41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/3399203564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca41034b0 0x7fcca41038c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcc940099c0 tx=0x7fcc94009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 -- 172.21.15.19:0/3399203564 shutdown_connections 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/3399203564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca4108730 0x7fcca4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/3399203564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca41034b0 0x7fcca41038c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/3399203564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcca4107de0 0x7fcca41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 -- 172.21.15.19:0/3399203564 >> 172.21.15.19:0/3399203564 conn(0x7fcca40fede0 msgr2=0x7fcca4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 -- 172.21.15.19:0/3399203564 shutdown_connections 2024-05-17T09:21:27.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.022+0000 7fccaa1f7700 1 -- 172.21.15.19:0/3399203564 wait complete. 2024-05-17T09:21:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 Processor -- start 2024-05-17T09:21:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 -- start start 2024-05-17T09:21:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca41034b0 0x7fcca419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 0x7fcca419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcca4108730 0x7fcca41a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4113260 con 0x7fcca4107de0 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41130e0 con 0x7fcca41034b0 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fccaa1f7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41133e0 con 0x7fcca4108730 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.023+0000 7fcc9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 0x7fcca419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 0x7fcca419c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37776/0 (socket says 172.21.15.19:37776) 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 -- 172.21.15.19:0/1297618584 learned_addr learned my addr 172.21.15.19:0/1297618584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca41034b0 0x7fcca419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcca4108730 0x7fcca41a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 -- 172.21.15.19:0/1297618584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcca4108730 msgr2=0x7fcca41a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcca4108730 0x7fcca41a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 -- 172.21.15.19:0/1297618584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca41034b0 msgr2=0x7fcca419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca41034b0 0x7fcca419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 -- 172.21.15.19:0/1297618584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca419c970 con 0x7fcca4107de0 2024-05-17T09:21:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca41034b0 0x7fcca419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcc9affd700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 0x7fcca419c0c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e970 tx=0x7fcc8c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.024+0000 7fcca17fa700 1 -- 172.21.15.19:0/1297618584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c004a30 con 0x7fcca4107de0 2024-05-17T09:21:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.025+0000 7fcca17fa700 1 -- 172.21.15.19:0/1297618584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc8c0054b0 con 0x7fcca4107de0 2024-05-17T09:21:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.025+0000 7fcca17fa700 1 -- 172.21.15.19:0/1297618584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00cc20 con 0x7fcca4107de0 2024-05-17T09:21:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.025+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41999d0 con 0x7fcca4107de0 2024-05-17T09:21:27.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.025+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca4199ec0 con 0x7fcca4107de0 2024-05-17T09:21:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.030+0000 7fcca17fa700 1 -- 172.21.15.19:0/1297618584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcc8c004fa0 con 0x7fcca4107de0 2024-05-17T09:21:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.030+0000 7fccb8c0cb80 1 Processor -- start 2024-05-17T09:21:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.030+0000 7fccb8c0cb80 1 -- start start 2024-05-17T09:21:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dce0ec0 0x557c0de655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca17fa700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc900b2880 0x7fcc900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de5f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de65060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de65060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56794/0 (socket says 172.21.15.19:56794) 2024-05-17T09:21:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dce0ec0 0x557c0de655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dce0ec0 0x557c0de655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37782/0 (socket says 172.21.15.19:37782) 2024-05-17T09:21:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca17fa700 1 -- 172.21.15.19:0/1297618584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcc8c0e26e0 con 0x7fcca4107de0 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca3fff700 1 -- 172.21.15.19:0/497802010 learned_addr learned my addr 172.21.15.19:0/497802010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c0dd8dd10 con 0x557c0dce0ec0 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c0dd7d8b0 con 0x557c0de5f440 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557c0dd818f0 con 0x557c0dd5e450 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcca3fff700 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de5f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.031+0000 7fcc9affd700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc900b2880 0x7fcc900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcca3fff700 1 -- 172.21.15.19:0/497802010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 msgr2=0x557c0de5f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcca3fff700 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de5f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcca3fff700 1 -- 172.21.15.19:0/497802010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dce0ec0 msgr2=0x557c0de655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcca3fff700 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dce0ec0 0x557c0de655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcca3fff700 1 -- 172.21.15.19:0/497802010 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0dccd950 con 0x557c0dd5e450 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcca3fff700 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de65060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c090 tx=0x7fcc9c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b1ec5edf665705 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcc9affd700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc900b2880 0x7fcc900b4d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0a9650 tx=0x7fcc8c004ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcc99ffb700 1 -- 172.21.15.19:0/497802010 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c020d90 con 0x557c0dd5e450 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcc99ffb700 1 -- 172.21.15.19:0/497802010 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9c003a00 con 0x557c0dd5e450 2024-05-17T09:21:27.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.032+0000 7fcc99ffb700 1 -- 172.21.15.19:0/497802010 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0041c0 con 0x557c0dd5e450 2024-05-17T09:21:27.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 msgr2=0x557c0de65060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de65060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c090 tx=0x7fcc9c00b040 comp rx=0 tx=0).stop 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 shutdown_connections 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de65060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de5f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/497802010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dce0ec0 0x557c0de655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 >> 172.21.15.19:0/497802010 conn(0x557c0dd1fd00 msgr2=0x557c0de59f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 shutdown_connections 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/497802010 wait complete. 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.034+0000 7fccb8c0cb80 1 -- start start 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 Processor -- start 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de68db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 0x557c0de69830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c0dc84020 con 0x557c0de5f440 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c0dd1f840 con 0x557c0dce0ec0 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557c0de60270 con 0x557c0dd5e450 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fccb8c0cb80 10 client.0 _ll_register_callbacks cb 0x557c0dd189a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 0x557c0de69830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 0x557c0de69830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37798/0 (socket says 172.21.15.19:37798) 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fcca3fff700 1 -- 172.21.15.19:0/1287055529 learned_addr learned my addr 172.21.15.19:0/1287055529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de68db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.035+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcca3fff700 1 -- 172.21.15.19:0/1287055529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 msgr2=0x557c0de692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcca3fff700 1 -- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 msgr2=0x557c0de68db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de68db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcca3fff700 1 -- 172.21.15.19:0/1287055529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0dd1c710 con 0x557c0de5f440 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 0x557c0de69830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c014cc0 tx=0x7fcc9c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4f56a4c6462e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcc82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1287055529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0092e0 con 0x557c0de5f440 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1287055529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcc9c02e040 con 0x557c0de5f440 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.036+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c003a00 con 0x557c0de5f440 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1287055529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c007500 con 0x557c0de5f440 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 10 client.15909 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c0de766e0 con 0x557c0de5f440 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1287055529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9c003a00 con 0x557c0de5f440 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1287055529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcc9c023040 con 0x557c0de5f440 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 20 client.15909 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0de78bb0 0x557c0de7afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 4 client.15909 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c0dd8fc60 con 0x557c0de78bb0 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.037+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0de78bb0 0x557c0de7afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.039+0000 7fcc82ffd700 10 client.15909 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.039+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0de78bb0 0x557c0de7afd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761f60e8b3aaffc3 server_cookie=65fd291c87392899 in_seq=0 out_seq=0 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.039+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1287055529 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557c0dd8fc60 con 0x557c0de78bb0 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.039+0000 7fcc82ffd700 10 client.15909 handle_command_reply: tid=0 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.039+0000 7fccb8c0cb80 1 client.15909 shutdown 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 msgr2=0x557c0de69830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 0x557c0de69830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c014cc0 tx=0x7fcc9c006000 comp rx=0 tx=0).stop 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 shutdown_connections 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0de78bb0 0x557c0de7afd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0de5f440 0x557c0de69830 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de68db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1287055529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dd5e450 0x557c0de692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 >> 172.21.15.19:0/1287055529 conn(0x557c0dd1fd00 msgr2=0x557c0de59f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fcc82ffd700 0 client.15909 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 shutdown_connections 2024-05-17T09:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1287055529 wait complete. 2024-05-17T09:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.040+0000 7fccb8c0cb80 20 client.15909 trim_cache size 0 max 16384 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 Processor -- start 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 -- start start 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de64490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de54a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0de5f440 0x557c0de68580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b0b60 con 0x557c0dd5e450 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41ae3f0 con 0x557c0dce0ec0 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fccb8c0cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41ac240 con 0x557c0de5f440 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.075+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de64490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de64490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55528/0 (socket says 172.21.15.19:55528) 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 -- 172.21.15.19:0/1688959195 learned_addr learned my addr 172.21.15.19:0/1688959195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de54a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0de5f440 0x557c0de68580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 -- 172.21.15.19:0/1688959195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0de5f440 msgr2=0x557c0de68580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0de5f440 0x557c0de68580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 -- 172.21.15.19:0/1688959195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 msgr2=0x557c0de54a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de54a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 -- 172.21.15.19:0/1688959195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0de5bf20 con 0x557c0dce0ec0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de54a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.076+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de64490 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c014c50 tx=0x7fcc9c020300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966157bb480ab47f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.077+0000 7fcc997fa700 1 -- 172.21.15.19:0/1688959195 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c03a040 con 0x557c0dce0ec0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.077+0000 7fcc997fa700 1 -- 172.21.15.19:0/1688959195 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9c003a00 con 0x557c0dce0ec0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.077+0000 7fcc997fa700 1 -- 172.21.15.19:0/1688959195 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c025da0 con 0x557c0dce0ec0 2024-05-17T09:21:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1688959195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 msgr2=0x557c0de64490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de64490 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c014c50 tx=0x7fcc9c020300 comp rx=0 tx=0).stop 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1688959195 shutdown_connections 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0de5f440 0x557c0de68580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0dce0ec0 0x557c0de64490 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1688959195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de54a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1688959195 >> 172.21.15.19:0/1688959195 conn(0x557c0dd1fd00 msgr2=0x557c0de62970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1688959195 shutdown_connections 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1688959195 wait complete. 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.078+0000 7fccb8c0cb80 1 -- start start 2024-05-17T09:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fccb8c0cb80 1 Processor -- start 2024-05-17T09:21:27.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dce0ec0 0x557c0de70df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fccb8c0cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dce0ec0 0x557c0de70df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dce0ec0 0x557c0de70df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56814/0 (socket says 172.21.15.19:56814) 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fcca37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fcca37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de71330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37822/0 (socket says 172.21.15.19:37822) 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fcca3fff700 1 -- 172.21.15.19:0/1821999737 learned_addr learned my addr 172.21.15.19:0/1821999737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.079+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41ac8b0 con 0x557c0dd5e450 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41add20 con 0x557c0de5f440 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41acf80 con 0x557c0dce0ec0 2024-05-17T09:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fcca37fe700 1 -- 172.21.15.19:0/1821999737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dce0ec0 msgr2=0x557c0de70df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dce0ec0 0x557c0de70df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fcca37fe700 1 -- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 msgr2=0x557c0de6ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de6ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fcca37fe700 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0de71a10 con 0x557c0dd5e450 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fccb8c0cb80 10 client.0 _ll_register_callbacks cb 0x557c0dd189a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.080+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c003a00 con 0x557c0dd5e450 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fcca37fe700 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de71330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400b580 tx=0x7fcc9401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c99a699e4e5087 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fcc82ffd700 10 client.15915 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1821999737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940052b0 con 0x557c0dd5e450 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1821999737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcc94005450 con 0x557c0dd5e450 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1821999737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc940056b0 con 0x557c0dd5e450 2024-05-17T09:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fccb8c0cb80 10 client.15915 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.081+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=27}) -- 0x557c0dee38b0 con 0x557c0dd5e450 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1821999737 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcc940037c0 con 0x557c0dd5e450 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fccb8c0cb80 10 client.15915 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fccb8c0cb80 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fccb8c0cb80 20 client.15915 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0dee3e70 0x557c0dee6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fccb8c0cb80 4 client.15915 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557c0dd89440 con 0x557c0dee3e70 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.082+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0dee3e70 0x557c0dee6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.083+0000 7fcca3fff700 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0dee3e70 0x557c0dee6380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9c4357efa33924 server_cookie=943925b396d223f8 in_seq=0 out_seq=0 2024-05-17T09:21:27.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.083+0000 7fcc82ffd700 10 client.15915 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.084+0000 7fcc82ffd700 1 -- 172.21.15.19:0/1821999737 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557c0dd89440 con 0x557c0dee3e70 2024-05-17T09:21:27.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.084+0000 7fcc82ffd700 10 client.15915 handle_command_reply: tid=0 2024-05-17T09:21:27.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.084+0000 7fccb8c0cb80 1 client.15915 shutdown 2024-05-17T09:21:27.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 msgr2=0x557c0de71330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de71330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400b580 tx=0x7fcc9401a040 comp rx=0 tx=0).stop 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 shutdown_connections 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c0dce0ec0 0x557c0de70df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c0dee3e70 0x557c0dee6380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c0de5f440 0x557c0de6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 --2- 172.21.15.19:0/1821999737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c0dd5e450 0x557c0de71330 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 >> 172.21.15.19:0/1821999737 conn(0x557c0dd1fd00 msgr2=0x557c0de54770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fcc82ffd700 0 client.15915 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 shutdown_connections 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 1 -- 172.21.15.19:0/1821999737 wait complete. 2024-05-17T09:21:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.085+0000 7fccb8c0cb80 20 client.15915 trim_cache size 0 max 16384 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ad0b47c5-a929-417e-8888-0c527966ca7b", 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc900b2880 msgr2=0x7fcc900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc900b2880 0x7fcc900b4d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0a9650 tx=0x7fcc8c004ea0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 msgr2=0x7fcca419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 0x7fcca419c0c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e970 tx=0x7fcc8c00ec80 comp rx=0 tx=0).stop 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 shutdown_connections 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcc900b2880 0x7fcc900b4d90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcca4108730 0x7fcca41a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcca41034b0 0x7fcca419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 --2- 172.21.15.19:0/1297618584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca4107de0 0x7fcca419c0c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 >> 172.21.15.19:0/1297618584 conn(0x7fcca40fede0 msgr2=0x7fcca4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 shutdown_connections 2024-05-17T09:21:27.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:27.086+0000 7fccaa1f7700 1 -- 172.21.15.19:0/1297618584 wait complete. 2024-05-17T09:21:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.626+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1160169151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 msgr2=0x7f9ed8104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.626+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1160169151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8104190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009860 tx=0x7f9ecc02a340 comp rx=0 tx=0).stop 2024-05-17T09:21:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1160169151 shutdown_connections 2024-05-17T09:21:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1160169151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ed8108a80 0x7f9ed810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1160169151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8104190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1160169151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8102b40 0x7f9ed8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1160169151 >> 172.21.15.19:0/1160169151 conn(0x7f9ed80fdd30 msgr2=0x7f9ed81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1160169151 shutdown_connections 2024-05-17T09:21:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1160169151 wait complete. 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.627+0000 7f9eddfc7640 1 Processor -- start 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 -- start start 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ed8102b40 0x7f9ed8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 0x7f9ed81a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 0x7f9ed81a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed810f610 con 0x7f9ed8108a80 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed810f490 con 0x7f9ed8103d80 2024-05-17T09:21:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9eddfc7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed810f790 con 0x7f9ed8102b40 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 0x7f9ed81a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37836/0 (socket says 172.21.15.19:37836) 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 -- 172.21.15.19:0/1210361833 learned_addr learned my addr 172.21.15.19:0/1210361833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ed8102b40 0x7f9ed8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 -- 172.21.15.19:0/1210361833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ed8102b40 msgr2=0x7f9ed8197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ed8102b40 0x7f9ed8197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 -- 172.21.15.19:0/1210361833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 msgr2=0x7f9ed8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.628+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.629+0000 7f9ed7fff640 1 -- 172.21.15.19:0/1210361833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed81a80f0 con 0x7f9ed8108a80 2024-05-17T09:21:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.629+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.629+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 0x7f9ed81a7a80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000eb10 tx=0x7f9ed000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.629+0000 7f9ed4ff9640 1 -- 172.21.15.19:0/1210361833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0004b20 con 0x7f9ed8108a80 2024-05-17T09:21:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.629+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed81a83e0 con 0x7f9ed8108a80 2024-05-17T09:21:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.629+0000 7f9ed4ff9640 1 -- 172.21.15.19:0/1210361833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ed0004cc0 con 0x7f9ed8108a80 2024-05-17T09:21:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.630+0000 7f9ed4ff9640 1 -- 172.21.15.19:0/1210361833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0010690 con 0x7f9ed8108a80 2024-05-17T09:21:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.630+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed81a8900 con 0x7f9ed8108a80 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ed4ff9640 1 -- 172.21.15.19:0/1210361833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9ed0005020 con 0x7f9ed8108a80 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 Processor -- start 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 -- start start 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7582f0 0x55e88cb22920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e88ca94c30 con 0x55e88c7604d0 2024-05-17T09:21:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e88ca847d0 con 0x55e88c7582f0 2024-05-17T09:21:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.634+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e88ca88810 con 0x55e88c7811b0 2024-05-17T09:21:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb29820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37844/0 (socket says 172.21.15.19:37844) 2024-05-17T09:21:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed6ffd640 1 -- 172.21.15.19:0/2250274941 learned_addr learned my addr 172.21.15.19:0/2250274941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7582f0 0x55e88cb22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed77fe640 1 -- 172.21.15.19:0/2250274941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 msgr2=0x55e88cb223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed77fe640 1 -- 172.21.15.19:0/2250274941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 msgr2=0x55e88cb29820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb29820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed77fe640 1 -- 172.21.15.19:0/2250274941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88c799210 con 0x55e88c7582f0 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb29820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.635+0000 7f9ed4ff9640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec00b2920 0x7f9ec00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.636+0000 7f9ed4ff9640 1 -- 172.21.15.19:0/1210361833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9ed00e2ba0 con 0x7f9ed8108a80 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.636+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7582f0 0x55e88cb22920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009530 tx=0x7f9ec8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251e305f2d7b41d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.636+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec00b2920 0x7f9ec00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.636+0000 7f9ebdffb640 1 -- 172.21.15.19:0/2250274941 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8005020 con 0x55e88c7582f0 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.636+0000 7f9ebdffb640 1 -- 172.21.15.19:0/2250274941 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ec80047a0 con 0x55e88c7582f0 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.636+0000 7f9ebdffb640 1 -- 172.21.15.19:0/2250274941 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8019690 con 0x55e88c7582f0 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/2250274941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7582f0 msgr2=0x55e88cb22920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7582f0 0x55e88cb22920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009530 tx=0x7f9ec8011810 comp rx=0 tx=0).stop 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec00b2920 0x7f9ec00b4e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc031000 tx=0x7f9ecc005730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/2250274941 shutdown_connections 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb29820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7582f0 0x55e88cb22920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/2250274941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/2250274941 >> 172.21.15.19:0/2250274941 conn(0x55e88c775300 msgr2=0x55e88cb1dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/2250274941 shutdown_connections 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.637+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/2250274941 wait complete. 2024-05-17T09:21:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.638+0000 7f9ee4e7a740 1 -- start start 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.638+0000 7f9ee4e7a740 1 Processor -- start 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.638+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 0x55e88cb2bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.638+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.638+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7811b0 0x55e88cb33190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e88c76a790 con 0x55e88c7582f0 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e88ca40890 con 0x55e88c7811b0 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e88cb2a270 con 0x55e88c7604d0 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ee4e7a740 10 client.0 _ll_register_callbacks cb 0x55e88ca43d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 0x55e88cb2bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 0x55e88cb2bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37858/0 (socket says 172.21.15.19:37858) 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 -- 172.21.15.19:0/1191635283 learned_addr learned my addr 172.21.15.19:0/1191635283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7811b0 0x55e88cb33190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 -- 172.21.15.19:0/1191635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 msgr2=0x55e88cb2c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb2c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 -- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7811b0 msgr2=0x55e88cb33190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7811b0 0x55e88cb33190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.639+0000 7f9ed77fe640 1 -- 172.21.15.19:0/1191635283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88c865620 con 0x55e88c7582f0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 0x55e88cb2bcf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba12a88155e2a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/1191635283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8029070 con 0x55e88c7582f0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/1191635283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9ec80053a0 con 0x55e88c7582f0 2024-05-17T09:21:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec80047a0 con 0x55e88c7582f0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/1191635283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ec8035050 con 0x55e88c7582f0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9ee4e7a740 10 client.15927 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e88cb360c0 con 0x55e88c7582f0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.640+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/1191635283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ec80238c0 con 0x55e88c7582f0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/1191635283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ec80195c0 con 0x55e88c7582f0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ee4e7a740 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ee4e7a740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ee4e7a740 20 client.15927 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88c73fd80 0x55e88cb394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ee4e7a740 4 client.15927 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9ec80047a0 con 0x55e88c73fd80 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.641+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88c73fd80 0x55e88cb394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.642+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88c73fd80 0x55e88cb394e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8fb79d62820022 server_cookie=f987a4488a4b6f3 in_seq=0 out_seq=0 2024-05-17T09:21:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.642+0000 7f9eb6ffd640 10 client.15927 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.642+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/1191635283 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f9ec80047a0 con 0x55e88c73fd80 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.642+0000 7f9eb6ffd640 10 client.15927 handle_command_reply: tid=0 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.642+0000 7f9ee4e7a740 1 client.15927 shutdown 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 msgr2=0x55e88cb2bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 0x55e88cb2bcf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8004dc0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 shutdown_connections 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88c73fd80 0x55e88cb394e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7811b0 0x55e88cb33190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb2c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/1191635283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7582f0 0x55e88cb2bcf0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 >> 172.21.15.19:0/1191635283 conn(0x55e88c775300 msgr2=0x55e88cb20c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9eb6ffd640 0 client.15927 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 shutdown_connections 2024-05-17T09:21:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/1191635283 wait complete. 2024-05-17T09:21:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.643+0000 7f9ee4e7a740 20 client.15927 trim_cache size 0 max 16384 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.660+0000 7f9ee4e7a740 1 Processor -- start 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.660+0000 7f9ee4e7a740 1 -- start start 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb2b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81b5790 con 0x55e88c7604d0 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81b3060 con 0x55e88c7151c0 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81b0eb0 con 0x55e88c7811b0 2024-05-17T09:21:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb268b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37874/0 (socket says 172.21.15.19:37874) 2024-05-17T09:21:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 -- 172.21.15.19:0/3090173055 learned_addr learned my addr 172.21.15.19:0/3090173055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb2b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 -- 172.21.15.19:0/3090173055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 msgr2=0x55e88cb330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb330b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 -- 172.21.15.19:0/3090173055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 msgr2=0x55e88cb2b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb2b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.661+0000 7f9ed77fe640 1 -- 172.21.15.19:0/3090173055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88cb21240 con 0x55e88c7604d0 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.662+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb268b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8011850 tx=0x7f9ec8005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d1577aa649499d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.662+0000 7f9ebd7fa640 1 -- 172.21.15.19:0/3090173055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8029070 con 0x55e88c7604d0 2024-05-17T09:21:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.662+0000 7f9ebd7fa640 1 -- 172.21.15.19:0/3090173055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ec8032070 con 0x55e88c7604d0 2024-05-17T09:21:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.662+0000 7f9ebd7fa640 1 -- 172.21.15.19:0/3090173055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8005600 con 0x55e88c7604d0 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3090173055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 msgr2=0x55e88cb268b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb268b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8011850 tx=0x7f9ec8005c70 comp rx=0 tx=0).stop 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3090173055 shutdown_connections 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb2b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7811b0 0x55e88cb330b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3090173055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7604d0 0x55e88cb268b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.663+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3090173055 >> 172.21.15.19:0/3090173055 conn(0x55e88c775300 msgr2=0x55e88cb20b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.664+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3090173055 shutdown_connections 2024-05-17T09:21:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.664+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3090173055 wait complete. 2024-05-17T09:21:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.664+0000 7f9ee4e7a740 1 -- start start 2024-05-17T09:21:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.664+0000 7f9ee4e7a740 1 Processor -- start 2024-05-17T09:21:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.665+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb31a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb31f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ee4e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7811b0 0x55e88cb324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81b1520 con 0x55e88c7811b0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81b2990 con 0x55e88c7151c0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ee4e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81b1bf0 con 0x55e88c7604d0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb31f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb31f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56862/0 (socket says 172.21.15.19:56862) 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 -- 172.21.15.19:0/3535541264 learned_addr learned my addr 172.21.15.19:0/3535541264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb31a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ee4e7a740 10 client.0 _ll_register_callbacks cb 0x7f9ec8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7811b0 0x55e88cb324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 -- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 msgr2=0x55e88cb31a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb31a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 -- 172.21.15.19:0/3535541264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7811b0 msgr2=0x55e88cb324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7811b0 0x55e88cb324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 -- 172.21.15.19:0/3535541264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88cb9fba0 con 0x55e88c7604d0 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed7fff640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7811b0 0x55e88cb324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.666+0000 7f9ed6ffd640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb31f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc034000 tx=0x7f9ecc005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0df55714967b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/3535541264 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc037040 con 0x55e88c7604d0 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/3535541264 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9ecc005240 con 0x55e88c7604d0 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec8032070 con 0x55e88c7604d0 2024-05-17T09:21:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/3535541264 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ecc005440 con 0x55e88c7604d0 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9ee4e7a740 10 client.25636 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e88cba0eb0 con 0x55e88c7604d0 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.667+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/3535541264 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ec8032070 con 0x55e88c7604d0 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/3535541264 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ecc03d040 con 0x55e88c7604d0 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ee4e7a740 10 client.25636 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ee4e7a740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ee4e7a740 20 client.25636 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88cba18e0 0x55e88cba3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ee4e7a740 4 client.25636 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e88ca96b80 con 0x55e88cba18e0 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.668+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88cba18e0 0x55e88cba3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.669+0000 7f9eb6ffd640 10 client.25636 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.669+0000 7f9ed77fe640 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88cba18e0 0x55e88cba3d00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2122b78a583fb0 server_cookie=3d81bd3d2b57eeab in_seq=0 out_seq=0 2024-05-17T09:21:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9eb6ffd640 1 -- 172.21.15.19:0/3535541264 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e88ca96b80 con 0x55e88cba18e0 2024-05-17T09:21:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9eb6ffd640 10 client.25636 handle_command_reply: tid=0 2024-05-17T09:21:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 client.25636 shutdown 2024-05-17T09:21:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 msgr2=0x55e88cb31f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb31f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc034000 tx=0x7f9ecc005c90 comp rx=0 tx=0).stop 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 shutdown_connections 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e88cba18e0 0x55e88cba3d00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88c7811b0 0x55e88cb324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e88c7604d0 0x55e88cb31f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 --2- 172.21.15.19:0/3535541264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e88c7151c0 0x55e88cb31a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 >> 172.21.15.19:0/3535541264 conn(0x55e88c775300 msgr2=0x55e88cb20b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.673+0000 7f9eb6ffd640 0 client.25636 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 shutdown_connections 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9ee4e7a740 1 -- 172.21.15.19:0/3535541264 wait complete. 2024-05-17T09:21:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9ee4e7a740 20 client.25636 trim_cache size 0 max 16384 2024-05-17T09:21:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:21:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:21:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:21:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec00b2920 msgr2=0x7f9ec00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec00b2920 0x7f9ec00b4e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc031000 tx=0x7f9ecc005730 comp rx=0 tx=0).stop 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 msgr2=0x7f9ed81a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.674+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 0x7f9ed81a7a80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000eb10 tx=0x7f9ed000c2d0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 shutdown_connections 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ec00b2920 0x7f9ec00b4e30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed8108a80 0x7f9ed81a7a80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed8103d80 0x7f9ed8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 --2- 172.21.15.19:0/1210361833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ed8102b40 0x7f9ed8197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 >> 172.21.15.19:0/1210361833 conn(0x7f9ed80fdd30 msgr2=0x7f9ed80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 shutdown_connections 2024-05-17T09:21:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:27.675+0000 7f9eddfc7640 1 -- 172.21.15.19:0/1210361833 wait complete. 2024-05-17T09:21:27.865 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:21:28.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:21:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:28.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:28 smithi019 ceph-mon[31620]: pgmap v489: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:28.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:28 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:21:28.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:28 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:28.525 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:28 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:21:28.525 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:28 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:28.525 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:28 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:21:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:28 smithi069 ceph-mon[35081]: pgmap v489: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:28 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:21:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:28 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:28 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:21:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:28 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:28 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:21:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:28 smithi079 ceph-mon[35502]: pgmap v489: 129 pgs: 129 active+clean; 671 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:28 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:21:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:28 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:28 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:21:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:28 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:28 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:21:29.036 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:30.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:30 smithi019 ceph-mon[31620]: pgmap v490: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-17T09:21:30.574 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:30 smithi079 ceph-mon[35502]: pgmap v490: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-17T09:21:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:30 smithi069 ceph-mon[35081]: pgmap v490: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-17T09:21:31.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 -- 172.21.15.19:0/3344822622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c1034b0 msgr2=0x7f695c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3344822622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c1034b0 0x7f695c1038c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f694c0099c0 tx=0x7f694c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 -- 172.21.15.19:0/3344822622 shutdown_connections 2024-05-17T09:21:31.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3344822622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c108730 0x7f695c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3344822622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c1034b0 0x7f695c1038c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3344822622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c107de0 0x7f695c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 -- 172.21.15.19:0/3344822622 >> 172.21.15.19:0/3344822622 conn(0x7f695c0fede0 msgr2=0x7f695c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 -- 172.21.15.19:0/3344822622 shutdown_connections 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.281+0000 7f6961fbc700 1 -- 172.21.15.19:0/3344822622 wait complete. 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.282+0000 7f6961fbc700 1 Processor -- start 2024-05-17T09:21:31.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.282+0000 7f6961fbc700 1 -- start start 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.283+0000 7f6961fbc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c1034b0 0x7f695c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.283+0000 7f6961fbc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c107de0 0x7f695c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.283+0000 7f6961fbc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 0x7f695c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 0x7f695c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 0x7f695c19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37906/0 (socket says 172.21.15.19:37906) 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695bfff700 1 -- 172.21.15.19:0/3682354361 learned_addr learned my addr 172.21.15.19:0/3682354361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f695c113230 con 0x7f695c108730 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c1034b0 0x7f695c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1130b0 con 0x7f695c107de0 2024-05-17T09:21:31.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1133b0 con 0x7f695c1034b0 2024-05-17T09:21:31.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c107de0 0x7f695c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695b7fe700 1 -- 172.21.15.19:0/3682354361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c107de0 msgr2=0x7f695c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c107de0 0x7f695c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695b7fe700 1 -- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 msgr2=0x7f695c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 0x7f695c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695b7fe700 1 -- 172.21.15.19:0/3682354361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c19fc20 con 0x7f695c1034b0 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.284+0000 7f695bfff700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 0x7f695c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.285+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c1034b0 0x7f695c19ad50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f694c0098e0 tx=0x7f694c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.285+0000 7f69597fa700 1 -- 172.21.15.19:0/3682354361 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c026070 con 0x7f695c1034b0 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.285+0000 7f69597fa700 1 -- 172.21.15.19:0/3682354361 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694c00f460 con 0x7f695c1034b0 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.285+0000 7f69597fa700 1 -- 172.21.15.19:0/3682354361 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c009e60 con 0x7f695c1034b0 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.285+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c19feb0 con 0x7f695c1034b0 2024-05-17T09:21:31.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.285+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1a0340 con 0x7f695c1034b0 2024-05-17T09:21:31.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.289+0000 7f69597fa700 1 -- 172.21.15.19:0/3682354361 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f694c029070 con 0x7f695c1034b0 2024-05-17T09:21:31.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.290+0000 7f69709d1b80 1 Processor -- start 2024-05-17T09:21:31.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.290+0000 7f69709d1b80 1 -- start start 2024-05-17T09:21:31.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.290+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd356910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd356eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd3511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69709d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eedd27ed70 con 0x55eedd350cd0 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69597fa700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69480b2900 0x7f69480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69709d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55eedd26e910 con 0x55eedd2bd430 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69709d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55eedd272950 con 0x55eedd1d1e40 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f695b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd356eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f6952ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd3511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f6952ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd3511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37918/0 (socket says 172.21.15.19:37918) 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f6952ffd700 1 -- 172.21.15.19:0/3987630205 learned_addr learned my addr 172.21.15.19:0/3987630205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f69597fa700 1 -- 172.21.15.19:0/3682354361 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f694c0e2cb0 con 0x7f695c1034b0 2024-05-17T09:21:31.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.291+0000 7f695bfff700 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd356910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f695bfff700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69480b2900 0x7f69480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6952ffd700 1 -- 172.21.15.19:0/3987630205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 msgr2=0x55eedd356eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd356eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6952ffd700 1 -- 172.21.15.19:0/3987630205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 msgr2=0x55eedd356910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd356910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6952ffd700 1 -- 172.21.15.19:0/3987630205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eedd217fb0 con 0x55eedd350cd0 2024-05-17T09:21:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f695bfff700 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd356910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd3511a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6944010d10 tx=0x7f69440098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29731ebd64028918 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6951ffb700 1 -- 172.21.15.19:0/3987630205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944020070 con 0x55eedd350cd0 2024-05-17T09:21:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6951ffb700 1 -- 172.21.15.19:0/3987630205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694401bad0 con 0x55eedd350cd0 2024-05-17T09:21:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.292+0000 7f6951ffb700 1 -- 172.21.15.19:0/3987630205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694401aa80 con 0x55eedd350cd0 2024-05-17T09:21:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.293+0000 7f695bfff700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69480b2900 0x7f69480b4e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55eedd3563d0 tx=0x7f6954004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.293+0000 7f69709d1b80 1 -- 172.21.15.19:0/3987630205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 msgr2=0x55eedd3511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.293+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd3511a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6944010d10 tx=0x7f69440098a0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 -- 172.21.15.19:0/3987630205 shutdown_connections 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd3511a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd356910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3987630205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd356eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 -- 172.21.15.19:0/3987630205 >> 172.21.15.19:0/3987630205 conn(0x55eedd2321c0 msgr2=0x55eedd34b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 -- 172.21.15.19:0/3987630205 shutdown_connections 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 -- 172.21.15.19:0/3987630205 wait complete. 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 -- start start 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.294+0000 7f69709d1b80 1 Processor -- start 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd35a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd35aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd35afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eedd174fa0 con 0x55eedd350cd0 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55eedd1d6da0 con 0x55eedd2bd430 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55eedd35d5a0 con 0x55eedd1d1e40 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f69709d1b80 10 client.0 _ll_register_callbacks cb 0x55eedd209950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f695b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd35a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f695bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd35afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f695bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd35afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37922/0 (socket says 172.21.15.19:37922) 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f695bfff700 1 -- 172.21.15.19:0/3814806138 learned_addr learned my addr 172.21.15.19:0/3814806138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.295+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd35aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f695b7fe700 1 -- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 msgr2=0x55eedd35aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd35aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f695b7fe700 1 -- 172.21.15.19:0/3814806138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 msgr2=0x55eedd35afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd35afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f695b7fe700 1 -- 172.21.15.19:0/3814806138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eedd1be8d0 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f695b7fe700 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd35a540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f694c0a9cb0 tx=0x7f694c0a9a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f25f9d9afd58b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.296+0000 7f693affd700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.297+0000 7f693affd700 1 -- 172.21.15.19:0/3814806138 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c0aa6b0 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.297+0000 7f693affd700 1 -- 172.21.15.19:0/3814806138 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f694c0a65a0 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.297+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694401bad0 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.297+0000 7f693affd700 1 -- 172.21.15.19:0/3814806138 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f694c0ae360 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.297+0000 7f69709d1b80 10 client.25648 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.297+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eedd228760 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f693affd700 1 -- 172.21.15.19:0/3814806138 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694401bad0 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f693affd700 1 -- 172.21.15.19:0/3814806138 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f694c0a5920 con 0x55eedd1d1e40 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f69709d1b80 10 client.25648 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f69709d1b80 10 client.25648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f69709d1b80 20 client.25648 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:31.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd372960 0x55eedd374e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f69709d1b80 4 client.25648 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:31.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.298+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eedd31ced0 con 0x55eedd372960 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.299+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd372960 0x55eedd374e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.299+0000 7f6952ffd700 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd372960 0x55eedd374e10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9fb8c095b631964 server_cookie=f446786dac02a4da in_seq=0 out_seq=0 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.299+0000 7f693affd700 10 client.25648 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f693affd700 1 -- 172.21.15.19:0/3814806138 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eedd31ced0 con 0x55eedd372960 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f693affd700 10 client.25648 handle_command_reply: tid=0 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 client.25648 shutdown 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 msgr2=0x55eedd35a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd35a540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f694c0a9cb0 tx=0x7f694c0a9a30 comp rx=0 tx=0).stop 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 shutdown_connections 2024-05-17T09:21:31.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd35a540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd372960 0x55eedd374e10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd35afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 --2- 172.21.15.19:0/3814806138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd2bd430 0x55eedd35aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f693affd700 0 client.25648 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 >> 172.21.15.19:0/3814806138 conn(0x55eedd2321c0 msgr2=0x55eedd34b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.300+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 shutdown_connections 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.301+0000 7f69709d1b80 1 -- 172.21.15.19:0/3814806138 wait complete. 2024-05-17T09:21:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.301+0000 7f69709d1b80 20 client.25648 trim_cache size 0 max 16384 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 Processor -- start 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 -- start start 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 0x55eedd359540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd350cd0 0x55eedd345800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1b07b0 con 0x55eedd2bd430 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ae040 con 0x55eedd350cd0 2024-05-17T09:21:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f69709d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1abeb0 con 0x55eedd1d1e40 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f6952ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 0x55eedd359540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f6952ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 0x55eedd359540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37938/0 (socket says 172.21.15.19:37938) 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f6952ffd700 1 -- 172.21.15.19:0/937603995 learned_addr learned my addr 172.21.15.19:0/937603995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.325+0000 7f695bfff700 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd350cd0 0x55eedd345800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f695b7fe700 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f6952ffd700 1 -- 172.21.15.19:0/937603995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 msgr2=0x55eedd36ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f6952ffd700 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f6952ffd700 1 -- 172.21.15.19:0/937603995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd350cd0 msgr2=0x55eedd345800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f6952ffd700 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd350cd0 0x55eedd345800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f6952ffd700 1 -- 172.21.15.19:0/937603995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eedd35b330 con 0x55eedd2bd430 2024-05-17T09:21:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f6952ffd700 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 0x55eedd359540 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6944010ce0 tx=0x7f694401a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15fc00d24f7e412 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f69517fa700 1 -- 172.21.15.19:0/937603995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944020040 con 0x55eedd2bd430 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f69517fa700 1 -- 172.21.15.19:0/937603995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69440054d0 con 0x55eedd2bd430 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.326+0000 7f69517fa700 1 -- 172.21.15.19:0/937603995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440038e0 con 0x55eedd2bd430 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 -- 172.21.15.19:0/937603995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 msgr2=0x55eedd359540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 0x55eedd359540 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6944010ce0 tx=0x7f694401a5e0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 -- 172.21.15.19:0/937603995 shutdown_connections 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd350cd0 0x55eedd345800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 --2- 172.21.15.19:0/937603995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd2bd430 0x55eedd359540 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.327+0000 7f69709d1b80 1 -- 172.21.15.19:0/937603995 >> 172.21.15.19:0/937603995 conn(0x55eedd2321c0 msgr2=0x55eedd358e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.328+0000 7f69709d1b80 1 -- 172.21.15.19:0/937603995 shutdown_connections 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.328+0000 7f69709d1b80 1 -- 172.21.15.19:0/937603995 wait complete. 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.328+0000 7f69709d1b80 1 -- start start 2024-05-17T09:21:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.328+0000 7f69709d1b80 1 Processor -- start 2024-05-17T09:21:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f69709d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd36b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f6952ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd36b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f6952ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd36b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37954/0 (socket says 172.21.15.19:37954) 2024-05-17T09:21:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f6952ffd700 1 -- 172.21.15.19:0/1955189662 learned_addr learned my addr 172.21.15.19:0/1955189662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f695b7fe700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd367fd0 0x55eedd3683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ac500 con 0x55eedd350cd0 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ad970 con 0x55eedd367fd0 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.329+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1acbd0 con 0x55eedd1d1e40 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f695bfff700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd367fd0 0x55eedd3683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f69709d1b80 10 client.0 _ll_register_callbacks cb 0x55eedd209950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f6952ffd700 1 -- 172.21.15.19:0/1955189662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 msgr2=0x55eedd36b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f6952ffd700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f6952ffd700 1 -- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd367fd0 msgr2=0x55eedd3683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f6952ffd700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd367fd0 0x55eedd3683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f6952ffd700 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eedd368af0 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f6952ffd700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd36b740 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f69440098e0 tx=0x7f6944009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1425d32c2dfee509 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f693affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f693affd700 1 -- 172.21.15.19:0/1955189662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944020070 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f693affd700 1 -- 172.21.15.19:0/1955189662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f694401b440 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.330+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69440054d0 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f693affd700 1 -- 172.21.15.19:0/1955189662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6944042050 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f69709d1b80 10 client.15951 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eedd3de210 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f693affd700 1 -- 172.21.15.19:0/1955189662 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69440054d0 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f693affd700 1 -- 172.21.15.19:0/1955189662 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6944003bb0 con 0x55eedd350cd0 2024-05-17T09:21:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f69709d1b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f69709d1b80 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f69709d1b80 20 client.15951 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.331+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd3de600 0x55eedd3e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f69709d1b80 4 client.15951 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eedd280cc0 con 0x55eedd3de600 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f695b7fe700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd3de600 0x55eedd3e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f695b7fe700 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd3de600 0x55eedd3e0a20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f01176da580c4d server_cookie=db71be68a35436f0 in_seq=0 out_seq=0 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f693affd700 10 client.15951 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f693affd700 1 -- 172.21.15.19:0/1955189662 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eedd280cc0 con 0x55eedd3de600 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f693affd700 10 client.15951 handle_command_reply: tid=0 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.332+0000 7f69709d1b80 1 client.15951 shutdown 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 msgr2=0x55eedd36b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd36b740 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f69440098e0 tx=0x7f6944009d20 comp rx=0 tx=0).stop 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 shutdown_connections 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eedd1d1e40 0x55eedd36b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55eedd3de600 0x55eedd3e0a20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eedd367fd0 0x55eedd3683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f693affd700 0 client.15951 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 --2- 172.21.15.19:0/1955189662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eedd350cd0 0x55eedd36b740 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 >> 172.21.15.19:0/1955189662 conn(0x55eedd2321c0 msgr2=0x55eedd34fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 shutdown_connections 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 1 -- 172.21.15.19:0/1955189662 wait complete. 2024-05-17T09:21:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.333+0000 7f69709d1b80 20 client.15951 trim_cache size 0 max 16384 2024-05-17T09:21:31.339 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:31.339 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:21:31.339 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:21:31.339 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69480b2900 msgr2=0x7f69480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69480b2900 0x7f69480b4e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55eedd3563d0 tx=0x7f6954004b40 comp rx=0 tx=0).stop 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c1034b0 msgr2=0x7f695c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c1034b0 0x7f695c19ad50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f694c0098e0 tx=0x7f694c0045e0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 shutdown_connections 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69480b2900 0x7f69480b4e10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f695c1034b0 0x7f695c19ad50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f695c108730 0x7f695c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 --2- 172.21.15.19:0/3682354361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c107de0 0x7f695c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 >> 172.21.15.19:0/3682354361 conn(0x7f695c0fede0 msgr2=0x7f695c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 shutdown_connections 2024-05-17T09:21:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.334+0000 7f6961fbc700 1 -- 172.21.15.19:0/3682354361 wait complete. 2024-05-17T09:21:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.968+0000 7fe28daf8700 1 -- 172.21.15.19:0/91888582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2881034d0 msgr2=0x7fe2881038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.968+0000 7fe28daf8700 1 --2- 172.21.15.19:0/91888582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2881034d0 0x7fe2881038e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe278009990 tx=0x7fe278009ca0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.968+0000 7fe28daf8700 1 -- 172.21.15.19:0/91888582 shutdown_connections 2024-05-17T09:21:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.969+0000 7fe28daf8700 1 --2- 172.21.15.19:0/91888582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288108750 0x7fe2881100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.969+0000 7fe28daf8700 1 --2- 172.21.15.19:0/91888582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2881034d0 0x7fe2881038e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.969+0000 7fe28daf8700 1 --2- 172.21.15.19:0/91888582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe288108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.969+0000 7fe28daf8700 1 -- 172.21.15.19:0/91888582 >> 172.21.15.19:0/91888582 conn(0x7fe2880fef40 msgr2=0x7fe288101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.969+0000 7fe28daf8700 1 -- 172.21.15.19:0/91888582 shutdown_connections 2024-05-17T09:21:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.969+0000 7fe28daf8700 1 -- 172.21.15.19:0/91888582 wait complete. 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.970+0000 7fe28daf8700 1 Processor -- start 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.970+0000 7fe28daf8700 1 -- start start 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.970+0000 7fe28daf8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2881034d0 0x7fe28819d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.970+0000 7fe28daf8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe28819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe28daf8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe288108750 0x7fe2881a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe28daf8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe288112190 con 0x7fe2881034d0 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe28daf8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe288112010 con 0x7fe288108750 2024-05-17T09:21:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe28daf8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe288112310 con 0x7fe288107e00 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe28819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe287fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe288108750 0x7fe2881a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe28819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56944/0 (socket says 172.21.15.19:56944) 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 -- 172.21.15.19:0/2259370795 learned_addr learned my addr 172.21.15.19:0/2259370795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 -- 172.21.15.19:0/2259370795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe288108750 msgr2=0x7fe2881a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe288108750 0x7fe2881a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 -- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2881034d0 msgr2=0x7fe28819d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2881034d0 0x7fe28819d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe286ffd700 1 -- 172.21.15.19:0/2259370795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2881a1fa0 con 0x7fe288107e00 2024-05-17T09:21:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.971+0000 7fe287fff700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe288108750 0x7fe2881a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.972+0000 7fe286ffd700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe28819d710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b550 tx=0x7fe27c00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.972+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c037020 con 0x7fe288107e00 2024-05-17T09:21:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.972+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe27c033ad0 con 0x7fe288107e00 2024-05-17T09:21:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.972+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c032b50 con 0x7fe288107e00 2024-05-17T09:21:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.973+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2881a2290 con 0x7fe288107e00 2024-05-17T09:21:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.973+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2881a2720 con 0x7fe288107e00 2024-05-17T09:21:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.974+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2880512b0 con 0x7fe288107e00 2024-05-17T09:21:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.978+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe27c03a020 con 0x7fe288107e00 2024-05-17T09:21:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.978+0000 7fe284ff9700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2700b2980 0x7fe2700b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.979+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe27c1041c0 con 0x7fe288107e00 2024-05-17T09:21:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.979+0000 7fe2877fe700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2700b2980 0x7fe2700b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.979+0000 7fe2877fe700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2700b2980 0x7fe2700b4e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2780037c0 tx=0x7fe278019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:31.980+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe27c041020 con 0x7fe288107e00 2024-05-17T09:21:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.401+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe28819e430 con 0x7fe288107e00 2024-05-17T09:21:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.402+0000 7fe284ff9700 1 -- 172.21.15.19:0/2259370795 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7fe28819e430 con 0x7fe288107e00 2024-05-17T09:21:32.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:21:32.403 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:21:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2700b2980 msgr2=0x7fe2700b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2700b2980 0x7fe2700b4e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2780037c0 tx=0x7fe278019040 comp rx=0 tx=0).stop 2024-05-17T09:21:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 msgr2=0x7fe28819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe28819d710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b550 tx=0x7fe27c00b860 comp rx=0 tx=0).stop 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 shutdown_connections 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe288108750 0x7fe2881a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2881034d0 0x7fe28819d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2700b2980 0x7fe2700b4e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 --2- 172.21.15.19:0/2259370795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288107e00 0x7fe28819d710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 >> 172.21.15.19:0/2259370795 conn(0x7fe2880fef40 msgr2=0x7fe288101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 shutdown_connections 2024-05-17T09:21:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:32.406+0000 7fe28daf8700 1 -- 172.21.15.19:0/2259370795 wait complete. 2024-05-17T09:21:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:21:32.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:21:32.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:21:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:32 smithi069 ceph-mon[35081]: pgmap v491: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:32 smithi019 ceph-mon[31620]: pgmap v491: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:32 smithi079 ceph-mon[35502]: pgmap v491: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:33.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 6a72cbc5 --release --await' 2024-05-17T09:21:33.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 6a72cbc5 --release --await 2024-05-17T09:21:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:33.305 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:33 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2259370795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:33 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2259370795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:33.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:33 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2259370795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:34.576 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:34 smithi019 ceph-mon[31620]: pgmap v492: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:34.637 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:34 smithi069 ceph-mon[35081]: pgmap v492: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:34 smithi079 ceph-mon[35502]: pgmap v492: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:36.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:36 smithi019 ceph-mon[31620]: pgmap v493: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 -- 172.21.15.19:0/1193049678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc105e20 msgr2=0x7f78dc10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/1193049678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc105e20 0x7f78dc10e370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009990 tx=0x7f78d0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 -- 172.21.15.19:0/1193049678 shutdown_connections 2024-05-17T09:21:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/1193049678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78dc10e900 0x7f78dc110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/1193049678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc1054d0 0x7f78dc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/1193049678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc105e20 0x7f78dc10e370 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 -- 172.21.15.19:0/1193049678 >> 172.21.15.19:0/1193049678 conn(0x7f78dc0fef20 msgr2=0x7f78dc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 -- 172.21.15.19:0/1193049678 shutdown_connections 2024-05-17T09:21:36.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.504+0000 7f78e0fbf700 1 -- 172.21.15.19:0/1193049678 wait complete. 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.505+0000 7f78e0fbf700 1 Processor -- start 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.505+0000 7f78e0fbf700 1 -- start start 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.505+0000 7f78e0fbf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 0x7f78dc1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.505+0000 7f78e0fbf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc105e20 0x7f78dc103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.505+0000 7f78e0fbf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78dc10e900 0x7f78dc103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 0x7f78dc1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78e0fbf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114310 con 0x7f78dc1054d0 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78e0fbf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114190 con 0x7f78dc105e20 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 0x7f78dc1031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48832/0 (socket says 172.21.15.19:48832) 2024-05-17T09:21:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 -- 172.21.15.19:0/12435679 learned_addr learned my addr 172.21.15.19:0/12435679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:36.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114490 con 0x7f78dc10e900 2024-05-17T09:21:36.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78d9d9b700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc105e20 0x7f78dc103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78dad9d700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78dc10e900 0x7f78dc103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 -- 172.21.15.19:0/12435679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78dc10e900 msgr2=0x7f78dc103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78dc10e900 0x7f78dc103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 -- 172.21.15.19:0/12435679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc105e20 msgr2=0x7f78dc103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc105e20 0x7f78dc103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 -- 172.21.15.19:0/12435679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc1a9080 con 0x7f78dc1054d0 2024-05-17T09:21:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.506+0000 7f78da59c700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 0x7f78dc1031c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00d950 tx=0x7f78cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.507+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc016070 con 0x7f78dc1054d0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.507+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc1a9370 con 0x7f78dc1054d0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.507+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78cc00fd20 con 0x7f78dc1054d0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.507+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc1a98d0 con 0x7f78dc1054d0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.507+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc020060 con 0x7f78dc1054d0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.508+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78dc0512b0 con 0x7f78dc1054d0 2024-05-17T09:21:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.510+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f78cc019070 con 0x7f78dc1054d0 2024-05-17T09:21:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.511+0000 7f78cb7fe700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78c40b2900 0x7f78c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.511+0000 7f78d9d9b700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78c40b2900 0x7f78c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.511+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f78cc0e2000 con 0x7f78dc1054d0 2024-05-17T09:21:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.512+0000 7f78d9d9b700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78c40b2900 0x7f78c40b4e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f78d00098e0 tx=0x7f78d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.516+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78cc0a9970 con 0x7f78dc1054d0 2024-05-17T09:21:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:36 smithi069 ceph-mon[35081]: pgmap v493: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.700+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/3123522732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c1034c0 msgr2=0x7f9e9c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.700+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/3123522732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c1038d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0099c0 tx=0x7f9e8c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/3123522732 shutdown_connections 2024-05-17T09:21:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/3123522732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 0x7f9e9c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/3123522732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c1038d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/3123522732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c107df0 0x7f9e9c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/3123522732 >> 172.21.15.19:0/3123522732 conn(0x7f9e9c0fef10 msgr2=0x7f9e9c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/3123522732 shutdown_connections 2024-05-17T09:21:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.701+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/3123522732 wait complete. 2024-05-17T09:21:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.702+0000 7f9ea1d1e700 1 Processor -- start 2024-05-17T09:21:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.703+0000 7f9ea1d1e700 1 -- start start 2024-05-17T09:21:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.703+0000 7f9ea1d1e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.703+0000 7f9ea1d1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 0x7f9e9c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.703+0000 7f9ea1d1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 0x7f9e9c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 0x7f9e9c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 0x7f9e9c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48846/0 (socket says 172.21.15.19:48846) 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35326/0 (socket says 172.21.15.19:35326) 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 0x7f9e9c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 0x7f9e9c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:60606/0 (socket says 172.21.15.19:60606) 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9e9affd700 1 -- 172.21.15.19:0/994306599 learned_addr learned my addr 172.21.15.19:0/994306599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1121d0 con 0x7f9e9c107df0 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c112050 con 0x7f9e9c108740 2024-05-17T09:21:36.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.704+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c112350 con 0x7f9e9c1034c0 2024-05-17T09:21:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.705+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/994306599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 msgr2=0x7f9e9c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.705+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 0x7f9e9c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.705+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 msgr2=0x7f9e9c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.705+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 0x7f9e9c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.705+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/994306599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c1990e0 con 0x7f9e9c1034c0 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.705+0000 7f9e9affd700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 0x7f9e9c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.706+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c19e810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009990 tx=0x7f9e8c00dbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.706+0000 7f9e98ff9700 1 -- 172.21.15.19:0/994306599 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c003db0 con 0x7f9e9c1034c0 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.706+0000 7f9e98ff9700 1 -- 172.21.15.19:0/994306599 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e8c0043e0 con 0x7f9e9c1034c0 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.706+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9c199370 con 0x7f9e9c1034c0 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.706+0000 7f9e98ff9700 1 -- 172.21.15.19:0/994306599 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c01de10 con 0x7f9e9c1034c0 2024-05-17T09:21:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.706+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9c199870 con 0x7f9e9c1034c0 2024-05-17T09:21:36.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.709+0000 7f9e98ff9700 1 -- 172.21.15.19:0/994306599 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9e8c00c040 con 0x7f9e9c1034c0 2024-05-17T09:21:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.710+0000 7f9e98ff9700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9e840b2900 0x7f9e840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.710+0000 7f9e98ff9700 1 -- 172.21.15.19:0/994306599 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9e8c0e6ad0 con 0x7f9e9c1034c0 2024-05-17T09:21:36.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.711+0000 7f9e9affd700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9e840b2900 0x7f9e840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.711+0000 7f9e9affd700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9e840b2900 0x7f9e840b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c107dc0 tx=0x7f9e90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.712+0000 7f9eb0733b80 1 Processor -- start 2024-05-17T09:21:36.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.712+0000 7f9eb0733b80 1 -- start start 2024-05-17T09:21:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee2bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9e9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9e9b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee2bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9e9b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee2bce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35336/0 (socket says 172.21.15.19:35336) 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9e9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2bd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48856/0 (socket says 172.21.15.19:48856) 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee2b7020 0x5624ee2b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5624ee1e4e70 con 0x5624ee137f40 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.713+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/3582936280 learned_addr learned my addr 172.21.15.19:0/3582936280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624ee1d4a10 con 0x5624ee2b7020 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee2b7020 0x5624ee2b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5624ee1d8a50 con 0x5624ee223530 2024-05-17T09:21:36.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/3582936280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee2b7020 msgr2=0x5624ee2b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee2b7020 0x5624ee2b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/3582936280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 msgr2=0x5624ee2bd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2bd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/3582936280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ee17e0b0 con 0x5624ee223530 2024-05-17T09:21:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9affd700 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.714+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee2bce90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0add60 tx=0x7f9e8c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62acde24fbdd725 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.715+0000 7f9e81ffb700 1 -- 172.21.15.19:0/3582936280 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0addd0 con 0x5624ee223530 2024-05-17T09:21:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.715+0000 7f9e81ffb700 1 -- 172.21.15.19:0/3582936280 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e8c0b2180 con 0x5624ee223530 2024-05-17T09:21:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.715+0000 7f9e81ffb700 1 -- 172.21.15.19:0/3582936280 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0ae030 con 0x5624ee223530 2024-05-17T09:21:36.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 msgr2=0x5624ee2bce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee2bce90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0add60 tx=0x7f9e8c009310 comp rx=0 tx=0).stop 2024-05-17T09:21:36.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 shutdown_connections 2024-05-17T09:21:36.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee2bce90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee2b7020 0x5624ee2b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/3582936280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 >> 172.21.15.19:0/3582936280 conn(0x5624ee1982c0 msgr2=0x5624ee2b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 shutdown_connections 2024-05-17T09:21:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.716+0000 7f9eb0733b80 1 -- 172.21.15.19:0/3582936280 wait complete. 2024-05-17T09:21:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.717+0000 7f9eb0733b80 1 -- start start 2024-05-17T09:21:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.717+0000 7f9eb0733b80 1 Processor -- start 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.717+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee2ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 0x5624ee2cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5624ee0db0a0 con 0x5624ee223530 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624ee13cea0 con 0x5624ee137f40 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5624ee1d8a50 con 0x5624ee2b7020 2024-05-17T09:21:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9eb0733b80 10 client.0 _ll_register_callbacks cb 0x5624ee16fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 0x5624ee2cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 0x5624ee2cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48862/0 (socket says 172.21.15.19:48862) 2024-05-17T09:21:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 -- 172.21.15.19:0/4266744687 learned_addr learned my addr 172.21.15.19:0/4266744687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee2ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 -- 172.21.15.19:0/4266744687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 msgr2=0x5624ee2c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 -- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 msgr2=0x5624ee2ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee2ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9affd700 1 -- 172.21.15.19:0/4266744687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ee1249d0 con 0x5624ee223530 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.718+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee2ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9e9affd700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 0x5624ee2cf260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000ee50 tx=0x7f9e900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f636f2ab4e0bba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9e7affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9e7affd700 1 -- 172.21.15.19:0/4266744687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001dd00 con 0x5624ee223530 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9e7affd700 1 -- 172.21.15.19:0/4266744687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9e90004830 con 0x5624ee223530 2024-05-17T09:21:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e8c0b2180 con 0x5624ee223530 2024-05-17T09:21:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9e7affd700 1 -- 172.21.15.19:0/4266744687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e90042050 con 0x5624ee223530 2024-05-17T09:21:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9eb0733b80 10 client.15969 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.719+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624ee18e860 con 0x5624ee223530 2024-05-17T09:21:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.720+0000 7f9e7affd700 1 -- 172.21.15.19:0/4266744687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e8c0b2180 con 0x5624ee223530 2024-05-17T09:21:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.720+0000 7f9e7affd700 1 -- 172.21.15.19:0/4266744687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9e90037420 con 0x5624ee223530 2024-05-17T09:21:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9eb0733b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:36.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9eb0733b80 10 client.15969 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:21:36.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9eb0733b80 20 client.15969 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:36.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee2c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9eb0733b80 4 client.15969 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624ee282fd0 con 0x5624ee2bff40 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.721+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee2c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.722+0000 7f9e7affd700 10 client.15969 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.722+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee2c2360 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62eeacfcbbb6a41 server_cookie=35ce87033cca8c01 in_seq=0 out_seq=0 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9e7affd700 1 -- 172.21.15.19:0/4266744687 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5624ee282fd0 con 0x5624ee2bff40 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9e7affd700 10 client.15969 handle_command_reply: tid=0 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 client.15969 shutdown 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 msgr2=0x5624ee2cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 0x5624ee2cf260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000ee50 tx=0x7f9e900047c0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 shutdown_connections 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee2c2360 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9e7affd700 0 client.15969 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee2ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4266744687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee223530 0x5624ee2cf260 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 >> 172.21.15.19:0/4266744687 conn(0x5624ee1982c0 msgr2=0x5624ee2b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 shutdown_connections 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.723+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4266744687 wait complete. 2024-05-17T09:21:36.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.724+0000 7f9eb0733b80 20 client.15969 trim_cache size 0 max 16384 2024-05-17T09:21:36.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.745+0000 7f9eb0733b80 1 Processor -- start 2024-05-17T09:21:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.745+0000 7f9eb0733b80 1 -- start start 2024-05-17T09:21:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee223530 0x5624ee2ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1b0990 con 0x5624ee137f40 2024-05-17T09:21:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1ae220 con 0x5624ee223530 2024-05-17T09:21:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9eb0733b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1ac070 con 0x5624ee2b7020 2024-05-17T09:21:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.746+0000 7f9e9bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2bace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35360/0 (socket says 172.21.15.19:35360) 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 -- 172.21.15.19:0/4031508282 learned_addr learned my addr 172.21.15.19:0/4031508282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9affd700 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee223530 0x5624ee2ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 -- 172.21.15.19:0/4031508282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee223530 msgr2=0x5624ee2ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee223530 0x5624ee2ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 -- 172.21.15.19:0/4031508282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 msgr2=0x5624ee2c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.747+0000 7f9e9bfff700 1 -- 172.21.15.19:0/4031508282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ee2c9540 con 0x5624ee2b7020 2024-05-17T09:21:36.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.748+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2bace0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e94003d50 tx=0x7f9e94009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1825fbf5eaea1188 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.748+0000 7f9e817fa700 1 -- 172.21.15.19:0/4031508282 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940048c0 con 0x5624ee2b7020 2024-05-17T09:21:36.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.748+0000 7f9e817fa700 1 -- 172.21.15.19:0/4031508282 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e9401b070 con 0x5624ee2b7020 2024-05-17T09:21:36.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.748+0000 7f9e817fa700 1 -- 172.21.15.19:0/4031508282 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94017670 con 0x5624ee2b7020 2024-05-17T09:21:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.749+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4031508282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 msgr2=0x5624ee2bace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.749+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2bace0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e94003d50 tx=0x7f9e94009920 comp rx=0 tx=0).stop 2024-05-17T09:21:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.750+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4031508282 shutdown_connections 2024-05-17T09:21:36.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.750+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee2b7020 0x5624ee2bace0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.750+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee137f40 0x5624ee2c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.750+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/4031508282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee223530 0x5624ee2ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.750+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4031508282 >> 172.21.15.19:0/4031508282 conn(0x5624ee1982c0 msgr2=0x5624ee2c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.750+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4031508282 shutdown_connections 2024-05-17T09:21:36.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.751+0000 7f9eb0733b80 1 -- 172.21.15.19:0/4031508282 wait complete. 2024-05-17T09:21:36.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.751+0000 7f9eb0733b80 1 -- start start 2024-05-17T09:21:36.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.752+0000 7f9eb0733b80 1 Processor -- start 2024-05-17T09:21:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.753+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee3344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.753+0000 7f9eb0733b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee334a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.753+0000 7f9e9b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee3344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.753+0000 7f9e9b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee3344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:60646/0 (socket says 172.21.15.19:60646) 2024-05-17T09:21:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.753+0000 7f9e9b7fe700 1 -- 172.21.15.19:0/2808290780 learned_addr learned my addr 172.21.15.19:0/2808290780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.754+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee2b7020 0x5624ee32e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.754+0000 7f9e9affd700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee334a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.754+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee2b7020 0x5624ee32e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.754+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1ac6e0 con 0x5624ee2b7020 2024-05-17T09:21:36.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.755+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1adb50 con 0x5624ee137f40 2024-05-17T09:21:36.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.755+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1acdb0 con 0x5624ee223530 2024-05-17T09:21:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.755+0000 7f9eb0733b80 10 client.0 _ll_register_callbacks cb 0x5624ee16fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.755+0000 7f9e9affd700 1 -- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 msgr2=0x5624ee3344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.755+0000 7f9e9affd700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee3344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.755+0000 7f9e9affd700 1 -- 172.21.15.19:0/2808290780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee2b7020 msgr2=0x5624ee32e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.756+0000 7f9e9affd700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee2b7020 0x5624ee32e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.756+0000 7f9e9affd700 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ee32ef40 con 0x5624ee223530 2024-05-17T09:21:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.756+0000 7f9e9bfff700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee2b7020 0x5624ee32e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.756+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9401b070 con 0x5624ee223530 2024-05-17T09:21:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9e9affd700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee334a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e90037840 tx=0x7f9e900376d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0610d9fe60a367c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9e7affd700 10 client.25681 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9e7affd700 1 -- 172.21.15.19:0/2808290780 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001d9d0 con 0x5624ee223530 2024-05-17T09:21:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9e7affd700 1 -- 172.21.15.19:0/2808290780 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9e90004830 con 0x5624ee223530 2024-05-17T09:21:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9e7affd700 1 -- 172.21.15.19:0/2808290780 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e90004a50 con 0x5624ee223530 2024-05-17T09:21:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9eb0733b80 10 client.25681 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.757+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5624ee33bae0 con 0x5624ee223530 2024-05-17T09:21:36.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9e7affd700 1 -- 172.21.15.19:0/2808290780 <== mon.1 v2:172.21.15.79:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9e90004bf0 con 0x5624ee223530 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9eb0733b80 10 client.25681 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9eb0733b80 10 client.25681 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9eb0733b80 20 client.25681 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee33e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9eb0733b80 4 client.25681 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a72cbc5", "release": true, "await": true}] 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.758+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a72cbc5", "release": true, "await": true}) -- 0x5624ee1e6dc0 con 0x5624ee2bff40 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.759+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee33e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.759+0000 7f9e9b7fe700 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee33e040 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426f1c850fb79830 server_cookie=bba713fc3d88ca2a in_seq=0 out_seq=0 2024-05-17T09:21:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.759+0000 7f9e7affd700 10 client.25681 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9e7affd700 1 -- 172.21.15.19:0/2808290780 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5624ee1e6dc0 con 0x5624ee2bff40 2024-05-17T09:21:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9e7affd700 10 client.25681 handle_command_reply: tid=0 2024-05-17T09:21:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9eb0733b80 1 client.25681 shutdown 2024-05-17T09:21:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 msgr2=0x5624ee334a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee334a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e90037840 tx=0x7f9e900376d0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 shutdown_connections 2024-05-17T09:21:36.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.761+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624ee2bff40 0x5624ee33e040 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624ee2b7020 0x5624ee32e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624ee137f40 0x5624ee3344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 27, 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 8, 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "6a72cbc5": { 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 8, 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:21:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.4, 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:36.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:21:36.767 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:36.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 1 --2- 172.21.15.19:0/2808290780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624ee223530 0x5624ee334a10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 >> 172.21.15.19:0/2808290780 conn(0x5624ee1982c0 msgr2=0x5624ee2bb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9e7affd700 0 client.25681 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 shutdown_connections 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 1 -- 172.21.15.19:0/2808290780 wait complete. 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.762+0000 7f9eb0733b80 20 client.25681 trim_cache size 0 max 16384 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.764+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9e840b2900 msgr2=0x7f9e840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.764+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9e840b2900 0x7f9e840b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c107dc0 tx=0x7f9e90009260 comp rx=0 tx=0).stop 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 msgr2=0x7f9e9c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c19e810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009990 tx=0x7f9e8c00dbd0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 shutdown_connections 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9e840b2900 0x7f9e840b4e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e9c1034c0 0x7f9e9c19e810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e9c108740 0x7f9e9c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 --2- 172.21.15.19:0/994306599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e9c107df0 0x7f9e9c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 >> 172.21.15.19:0/994306599 conn(0x7f9e9c0fef10 msgr2=0x7f9e9c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 shutdown_connections 2024-05-17T09:21:36.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.765+0000 7f9ea1d1e700 1 -- 172.21.15.19:0/994306599 wait complete. 2024-05-17T09:21:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:36 smithi079 ceph-mon[35502]: pgmap v493: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:36.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.827+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78dc02e5e0 con 0x7f78dc1054d0 2024-05-17T09:21:36.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.828+0000 7f78cb7fe700 1 -- 172.21.15.19:0/12435679 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7f78cc0a9270 con 0x7f78dc1054d0 2024-05-17T09:21:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:21:36.829 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:21:36.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.830+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78c40b2900 msgr2=0x7f78c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.830+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78c40b2900 0x7f78c40b4e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f78d00098e0 tx=0x7f78d0005960 comp rx=0 tx=0).stop 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.830+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 msgr2=0x7f78dc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.830+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 0x7f78dc1031c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00d950 tx=0x7f78cc00dc60 comp rx=0 tx=0).stop 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 shutdown_connections 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f78c40b2900 0x7f78c40b4e10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78dc10e900 0x7f78dc103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78dc1054d0 0x7f78dc1031c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 --2- 172.21.15.19:0/12435679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc105e20 0x7f78dc103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 >> 172.21.15.19:0/12435679 conn(0x7f78dc0fef20 msgr2=0x7f78dc0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 shutdown_connections 2024-05-17T09:21:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:36.831+0000 7f78e0fbf700 1 -- 172.21.15.19:0/12435679 wait complete. 2024-05-17T09:21:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:21:37.511 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a72cbc5', total seconds elapsed: 31.7 2024-05-17T09:21:37.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:37 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/12435679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:37.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:37 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/12435679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:37 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/12435679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:21:38.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:21:38.418 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:38.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:38 smithi019 ceph-mon[31620]: pgmap v494: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:38 smithi069 ceph-mon[35081]: pgmap v494: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:38 smithi079 ceph-mon[35502]: pgmap v494: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.307+0000 7fe574755700 1 -- 172.21.15.19:0/3787230221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c1034b0 msgr2=0x7fe56c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.307+0000 7fe574755700 1 --2- 172.21.15.19:0/3787230221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c1034b0 0x7fe56c1038c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe5680099c0 tx=0x7fe568009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.308+0000 7fe574755700 1 -- 172.21.15.19:0/3787230221 shutdown_connections 2024-05-17T09:21:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.308+0000 7fe574755700 1 --2- 172.21.15.19:0/3787230221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c108730 0x7fe56c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.308+0000 7fe574755700 1 --2- 172.21.15.19:0/3787230221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c1034b0 0x7fe56c1038c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.308+0000 7fe574755700 1 --2- 172.21.15.19:0/3787230221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe56c107de0 0x7fe56c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.308+0000 7fe574755700 1 -- 172.21.15.19:0/3787230221 >> 172.21.15.19:0/3787230221 conn(0x7fe56c0fedb0 msgr2=0x7fe56c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.309+0000 7fe574755700 1 -- 172.21.15.19:0/3787230221 shutdown_connections 2024-05-17T09:21:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.309+0000 7fe574755700 1 -- 172.21.15.19:0/3787230221 wait complete. 2024-05-17T09:21:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 Processor -- start 2024-05-17T09:21:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 -- start start 2024-05-17T09:21:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1034b0 0x7fe56c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe56c107de0 0x7fe56c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 0x7fe56c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c113260 con 0x7fe56c108730 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1130e0 con 0x7fe56c107de0 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe574755700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1133e0 con 0x7fe56c1034b0 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe572cf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 0x7fe56c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe572cf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 0x7fe56c1998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48902/0 (socket says 172.21.15.19:48902) 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.310+0000 7fe572cf2700 1 -- 172.21.15.19:0/2836029872 learned_addr learned my addr 172.21.15.19:0/2836029872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe571cf0700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe56c107de0 0x7fe56c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe572cf2700 1 -- 172.21.15.19:0/2836029872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1034b0 msgr2=0x7fe56c19f680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:21:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe572cf2700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1034b0 0x7fe56c19f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe572cf2700 1 -- 172.21.15.19:0/2836029872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe56c107de0 msgr2=0x7fe56c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe572cf2700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe56c107de0 0x7fe56c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe572cf2700 1 -- 172.21.15.19:0/2836029872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c199f50 con 0x7fe56c108730 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe572cf2700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 0x7fe56c1998a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe564007ee0 tx=0x7fe56400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.311+0000 7fe5637fe700 1 -- 172.21.15.19:0/2836029872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400dbd0 con 0x7fe56c108730 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.312+0000 7fe5637fe700 1 -- 172.21.15.19:0/2836029872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe56400dd70 con 0x7fe56c108730 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.312+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c19a240 con 0x7fe56c108730 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.312+0000 7fe5637fe700 1 -- 172.21.15.19:0/2836029872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640058a0 con 0x7fe56c108730 2024-05-17T09:21:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.312+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c19a6c0 con 0x7fe56c108730 2024-05-17T09:21:40.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.316+0000 7fe5637fe700 1 -- 172.21.15.19:0/2836029872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe56401a020 con 0x7fe56c108730 2024-05-17T09:21:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.318+0000 7fe5637fe700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5580b2900 0x7fe5580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.318+0000 7fe5637fe700 1 -- 172.21.15.19:0/2836029872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe5640e2ac0 con 0x7fe56c108730 2024-05-17T09:21:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.318+0000 7fe5724f1700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5580b2900 0x7fe5580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.318+0000 7fe58316ab80 1 Processor -- start 2024-05-17T09:21:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.319+0000 7fe5724f1700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5580b2900 0x7fe5580b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5680058e0 tx=0x7fe568000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.319+0000 7fe58316ab80 1 -- start start 2024-05-17T09:21:40.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.319+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e350d3d0 0x5612e36183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.319+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e3618930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.319+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e36127d0 0x5612e3612be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.319+0000 7fe571cf0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e3618930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e3618930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48908/0 (socket says 172.21.15.19:48908) 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe5724f1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e350d3d0 0x5612e36183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe5724f1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e350d3d0 0x5612e36183f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35396/0 (socket says 172.21.15.19:35396) 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 -- 172.21.15.19:0/991836509 learned_addr learned my addr 172.21.15.19:0/991836509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5612e353cc90 con 0x5612e348fe40 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5612e352c830 con 0x5612e36127d0 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5612e3530870 con 0x5612e350d3d0 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 -- 172.21.15.19:0/991836509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e350d3d0 msgr2=0x5612e36183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e350d3d0 0x5612e36183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 -- 172.21.15.19:0/991836509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e36127d0 msgr2=0x5612e3612be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:21:40.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e36127d0 0x5612e3612be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.320+0000 7fe571cf0700 1 -- 172.21.15.19:0/991836509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e347c8d0 con 0x5612e348fe40 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.321+0000 7fe571cf0700 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e3618930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00de20 tx=0x7fe55c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fb29b34cf13e89 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.321+0000 7fe560ff9700 1 -- 172.21.15.19:0/991836509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c009ce0 con 0x5612e348fe40 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.321+0000 7fe560ff9700 1 -- 172.21.15.19:0/991836509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe55c0044c0 con 0x5612e348fe40 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.321+0000 7fe560ff9700 1 -- 172.21.15.19:0/991836509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c005020 con 0x5612e348fe40 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 msgr2=0x5612e3618930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e3618930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00de20 tx=0x7fe55c009580 comp rx=0 tx=0).stop 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 shutdown_connections 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e350d3d0 0x5612e36183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e36127d0 0x5612e3612be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 --2- 172.21.15.19:0/991836509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e3618930 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 >> 172.21.15.19:0/991836509 conn(0x5612e34cec80 msgr2=0x5612e3608ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.322+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 shutdown_connections 2024-05-17T09:21:40.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.323+0000 7fe58316ab80 1 -- 172.21.15.19:0/991836509 wait complete. 2024-05-17T09:21:40.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.323+0000 7fe58316ab80 1 -- start start 2024-05-17T09:21:40.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.323+0000 7fe58316ab80 1 Processor -- start 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.323+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e361c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.323+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e361c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e36127d0 0x5612e361caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe58316ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5612e3432fa0 con 0x5612e36127d0 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe58316ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5612e34ce7c0 con 0x5612e350d3d0 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe58316ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5612e3613660 con 0x5612e348fe40 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe58316ab80 10 client.0 _ll_register_callbacks cb 0x5612e34c7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe572cf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e36127d0 0x5612e361caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe572cf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e36127d0 0x5612e361caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48912/0 (socket says 172.21.15.19:48912) 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe571cf0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e361c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe572cf2700 1 -- 172.21.15.19:0/3098764261 learned_addr learned my addr 172.21.15.19:0/3098764261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe5724f1700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e361c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe571cf0700 1 -- 172.21.15.19:0/3098764261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 msgr2=0x5612e361c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe571cf0700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e361c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe571cf0700 1 -- 172.21.15.19:0/3098764261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e36127d0 msgr2=0x5612e361caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe571cf0700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e36127d0 0x5612e361caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe571cf0700 1 -- 172.21.15.19:0/3098764261 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e34cb690 con 0x5612e350d3d0 2024-05-17T09:21:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.324+0000 7fe5724f1700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e361c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe571cf0700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e361c5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5612e3617eb0 tx=0x7fe55c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c53dfab3bb3d932 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe555ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe555ffb700 1 -- 172.21.15.19:0/3098764261 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c025020 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c0044c0 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe555ffb700 1 -- 172.21.15.19:0/3098764261 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe55c004950 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe555ffb700 1 -- 172.21.15.19:0/3098764261 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe55c030050 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe58316ab80 10 client.25676 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612e360b3a0 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe555ffb700 1 -- 172.21.15.19:0/3098764261 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe55c0044c0 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.325+0000 7fe555ffb700 1 -- 172.21.15.19:0/3098764261 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe55c017ac0 con 0x5612e350d3d0 2024-05-17T09:21:40.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe58316ab80 10 client.25676 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:40.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe58316ab80 10 client.25676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:21:40.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe58316ab80 20 client.25676 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e360d8d0 0x5612e360dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe58316ab80 4 client.25676 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e353ebe0 con 0x5612e360d8d0 2024-05-17T09:21:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe5724f1700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e360d8d0 0x5612e360dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe5724f1700 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e360d8d0 0x5612e360dce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d6634aecc833495 server_cookie=62355f33b3268c5e in_seq=0 out_seq=0 2024-05-17T09:21:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.326+0000 7fe555ffb700 10 client.25676 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.327+0000 7fe555ffb700 1 -- 172.21.15.19:0/3098764261 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5612e353ebe0 con 0x5612e360d8d0 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.327+0000 7fe555ffb700 10 client.25676 handle_command_reply: tid=0 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.327+0000 7fe58316ab80 1 client.25676 shutdown 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.327+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 msgr2=0x5612e361c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e361c5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5612e3617eb0 tx=0x7fe55c005270 comp rx=0 tx=0).stop 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 shutdown_connections 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e360d8d0 0x5612e360dce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe555ffb700 0 client.25676 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e361c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e36127d0 0x5612e361caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3098764261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e361c5b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 >> 172.21.15.19:0/3098764261 conn(0x5612e34cec80 msgr2=0x5612e3608ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 shutdown_connections 2024-05-17T09:21:40.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 1 -- 172.21.15.19:0/3098764261 wait complete. 2024-05-17T09:21:40.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.328+0000 7fe58316ab80 20 client.25676 trim_cache size 0 max 16384 2024-05-17T09:21:40.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 Processor -- start 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 -- start start 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e3603990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e3628260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 0x5612e3617840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe550011430 con 0x5612e360d8d0 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe55000ecc0 con 0x5612e350d3d0 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe58316ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe55000cb10 con 0x5612e348fe40 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe572cf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 0x5612e3617840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe572cf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 0x5612e3617840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48924/0 (socket says 172.21.15.19:48924) 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe572cf2700 1 -- 172.21.15.19:0/3328861177 learned_addr learned my addr 172.21.15.19:0/3328861177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.344+0000 7fe571cf0700 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e3628260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe5724f1700 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e3603990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe572cf2700 1 -- 172.21.15.19:0/3328861177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 msgr2=0x5612e3603990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe572cf2700 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e3603990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe572cf2700 1 -- 172.21.15.19:0/3328861177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 msgr2=0x5612e3628260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe572cf2700 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e3628260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe572cf2700 1 -- 172.21.15.19:0/3328861177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e360a0b0 con 0x5612e360d8d0 2024-05-17T09:21:40.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe571cf0700 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e3628260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:40.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.345+0000 7fe572cf2700 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 0x5612e3617840 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe5640a9a30 tx=0x7fe5640a9680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc36925054bb86b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.346+0000 7fe557fff700 1 -- 172.21.15.19:0/3328861177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640a9840 con 0x5612e360d8d0 2024-05-17T09:21:40.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.346+0000 7fe557fff700 1 -- 172.21.15.19:0/3328861177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5640c0070 con 0x5612e360d8d0 2024-05-17T09:21:40.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.346+0000 7fe557fff700 1 -- 172.21.15.19:0/3328861177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564004d30 con 0x5612e360d8d0 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.346+0000 7fe58316ab80 1 -- 172.21.15.19:0/3328861177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 msgr2=0x5612e3617840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.346+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 0x5612e3617840 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe5640a9a30 tx=0x7fe5640a9680 comp rx=0 tx=0).stop 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 -- 172.21.15.19:0/3328861177 shutdown_connections 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e348fe40 0x5612e3603990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e360d8d0 0x5612e3617840 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 --2- 172.21.15.19:0/3328861177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e3628260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 -- 172.21.15.19:0/3328861177 >> 172.21.15.19:0/3328861177 conn(0x5612e34cec80 msgr2=0x5612e3615d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 -- 172.21.15.19:0/3328861177 shutdown_connections 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 -- 172.21.15.19:0/3328861177 wait complete. 2024-05-17T09:21:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 -- start start 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 Processor -- start 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e362c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.347+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e362ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe58316ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e360d8d0 0x5612e362d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe58316ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe55000d180 con 0x5612e348fe40 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe58316ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe55000e5f0 con 0x5612e350d3d0 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe58316ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe55000d850 con 0x5612e360d8d0 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe58316ab80 10 client.0 _ll_register_callbacks cb 0x5612e34c7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e362c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e362c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48934/0 (socket says 172.21.15.19:48934) 2024-05-17T09:21:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 -- 172.21.15.19:0/4151896763 learned_addr learned my addr 172.21.15.19:0/4151896763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe572cf2700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e360d8d0 0x5612e362d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe571cf0700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e362ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 -- 172.21.15.19:0/4151896763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e360d8d0 msgr2=0x5612e362d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e360d8d0 0x5612e362d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 -- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 msgr2=0x5612e362ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e362ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 -- 172.21.15.19:0/4151896763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e3690aa0 con 0x5612e348fe40 2024-05-17T09:21:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe571cf0700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e362ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe5724f1700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e362c8d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19a9e0 tx=0x7fe56800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc7c5c58bf39ac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe555ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.348+0000 7fe555ffb700 1 -- 172.21.15.19:0/4151896763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56800b9a0 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5640c0070 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe555ffb700 1 -- 172.21.15.19:0/4151896763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe56800bb40 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe555ffb700 1 -- 172.21.15.19:0/4151896763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe568045050 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 10 client.15990 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612e3691ce0 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe555ffb700 1 -- 172.21.15.19:0/4151896763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5640c0070 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe555ffb700 1 -- 172.21.15.19:0/4151896763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe56803a410 con 0x5612e348fe40 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 10 client.15990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 20 client.15990 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:40.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e36127d0 0x5612e3694430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 4 client.15990 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612e35383c0 con 0x5612e36127d0 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.349+0000 7fe571cf0700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e36127d0 0x5612e3694430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.350+0000 7fe555ffb700 10 client.15990 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.350+0000 7fe571cf0700 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e36127d0 0x5612e3694430 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6908a470c8a6ce00 server_cookie=7504f117daceb937 in_seq=0 out_seq=0 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe555ffb700 1 -- 172.21.15.19:0/4151896763 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612e35383c0 con 0x5612e36127d0 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe555ffb700 10 client.15990 handle_command_reply: tid=0 2024-05-17T09:21:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 client.15990 shutdown 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 msgr2=0x5612e362c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e362c8d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19a9e0 tx=0x7fe56800b540 comp rx=0 tx=0).stop 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 shutdown_connections 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x5612e36127d0 0x5612e3694430 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5612e360d8d0 0x5612e362d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612e348fe40 0x5612e362c8d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 --2- 172.21.15.19:0/4151896763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612e350d3d0 0x5612e362ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 >> 172.21.15.19:0/4151896763 conn(0x5612e34cec80 msgr2=0x5612e36036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe555ffb700 0 client.15990 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 shutdown_connections 2024-05-17T09:21:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 1 -- 172.21.15.19:0/4151896763 wait complete. 2024-05-17T09:21:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.351+0000 7fe58316ab80 20 client.15990 trim_cache size 0 max 16384 2024-05-17T09:21:40.355 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:21:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.353+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5580b2900 msgr2=0x7fe5580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.353+0000 7fe574755700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5580b2900 0x7fe5580b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5680058e0 tx=0x7fe568000bc0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.353+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 msgr2=0x7fe56c1998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.353+0000 7fe574755700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 0x7fe56c1998a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe564007ee0 tx=0x7fe56400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 shutdown_connections 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe5580b2900 0x7fe5580b4e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1034b0 0x7fe56c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56c108730 0x7fe56c1998a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 --2- 172.21.15.19:0/2836029872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe56c107de0 0x7fe56c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 >> 172.21.15.19:0/2836029872 conn(0x7fe56c0fedb0 msgr2=0x7fe56c107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 shutdown_connections 2024-05-17T09:21:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:40.354+0000 7fe574755700 1 -- 172.21.15.19:0/2836029872 wait complete. 2024-05-17T09:21:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:40 smithi069 ceph-mon[35081]: pgmap v495: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:21:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:40 smithi019 ceph-mon[31620]: pgmap v495: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:21:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:40 smithi079 ceph-mon[35502]: pgmap v495: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:21:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 -- 172.21.15.19:0/3678336243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4108a60 msgr2=0x7f35e410ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 --2- 172.21.15.19:0/3678336243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4108a60 0x7f35e410ae80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009be0 tx=0x7f35dc02a6c0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 -- 172.21.15.19:0/3678336243 shutdown_connections 2024-05-17T09:21:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 --2- 172.21.15.19:0/3678336243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4108a60 0x7f35e410ae80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 --2- 172.21.15.19:0/3678336243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4103d60 0x7f35e4104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 --2- 172.21.15.19:0/3678336243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e4102b20 0x7f35e4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 -- 172.21.15.19:0/3678336243 >> 172.21.15.19:0/3678336243 conn(0x7f35e40fdd50 msgr2=0x7f35e41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 -- 172.21.15.19:0/3678336243 shutdown_connections 2024-05-17T09:21:40.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.830+0000 7f35e8e30640 1 -- 172.21.15.19:0/3678336243 wait complete. 2024-05-17T09:21:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.831+0000 7f35e8e30640 1 Processor -- start 2024-05-17T09:21:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.831+0000 7f35e8e30640 1 -- start start 2024-05-17T09:21:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.831+0000 7f35e8e30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 0x7f35e4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.831+0000 7f35e259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 0x7f35e4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 0x7f35e4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48944/0 (socket says 172.21.15.19:48944) 2024-05-17T09:21:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e8e30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4103d60 0x7f35e41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 -- 172.21.15.19:0/2001222353 learned_addr learned my addr 172.21.15.19:0/2001222353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e4108a60 0x7f35e419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35e410d3f0 con 0x7f35e4102b20 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35e410d270 con 0x7f35e4103d60 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35e410d570 con 0x7f35e4108a60 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e4108a60 0x7f35e419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4103d60 0x7f35e41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 -- 172.21.15.19:0/2001222353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e4108a60 msgr2=0x7f35e419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e4108a60 0x7f35e419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 -- 172.21.15.19:0/2001222353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4103d60 msgr2=0x7f35e41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4103d60 0x7f35e41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.832+0000 7f35e259c640 1 -- 172.21.15.19:0/2001222353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e419fa90 con 0x7f35e4102b20 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.833+0000 7f35e259c640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 0x7f35e4197fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d40071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.833+0000 7f35d37fe640 1 -- 172.21.15.19:0/2001222353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4004020 con 0x7f35e4102b20 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.833+0000 7f35d37fe640 1 -- 172.21.15.19:0/2001222353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d400f4b0 con 0x7f35e4102b20 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.833+0000 7f35d37fe640 1 -- 172.21.15.19:0/2001222353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400e660 con 0x7f35e4102b20 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.833+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e419fd80 con 0x7f35e4102b20 2024-05-17T09:21:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.833+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41a01c0 con 0x7f35e4102b20 2024-05-17T09:21:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.837+0000 7f35efdbd740 1 Processor -- start 2024-05-17T09:21:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 -- start start 2024-05-17T09:21:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 0x556b0f1cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee012f0 0x556b0f1cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee094d0 0x556b0f1d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b0f13dc30 con 0x556b0ee2a1b0 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556b0f12d7d0 con 0x556b0ee094d0 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35efdbd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556b0f131810 con 0x556b0ee012f0 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35d37fe640 1 -- 172.21.15.19:0/2001222353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f35d4021020 con 0x7f35e4102b20 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 0x556b0f1cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 0x556b0f1cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48954/0 (socket says 172.21.15.19:48954) 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35e2d9d640 1 -- 172.21.15.19:0/84580244 learned_addr learned my addr 172.21.15.19:0/84580244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee094d0 0x556b0f1d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.838+0000 7f35e259c640 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee012f0 0x556b0f1cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35e2d9d640 1 -- 172.21.15.19:0/84580244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee012f0 msgr2=0x556b0f1cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee012f0 0x556b0f1cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35e2d9d640 1 -- 172.21.15.19:0/84580244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee094d0 msgr2=0x556b0f1d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee094d0 0x556b0f1d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35e2d9d640 1 -- 172.21.15.19:0/84580244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0ee42210 con 0x556b0ee2a1b0 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 0x556b0f1cb3a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02acb0 tx=0x7f35dc030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7054a52548fba8cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35d0ff9640 1 -- 172.21.15.19:0/84580244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc041030 con 0x556b0ee2a1b0 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.839+0000 7f35d37fe640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f35cc0b2920 0x7f35cc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.840+0000 7f35d37fe640 1 -- 172.21.15.19:0/2001222353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f35d40e5a70 con 0x7f35e4102b20 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.840+0000 7f35d0ff9640 1 -- 172.21.15.19:0/84580244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35dc037040 con 0x556b0ee2a1b0 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.840+0000 7f35d0ff9640 1 -- 172.21.15.19:0/84580244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc036040 con 0x556b0ee2a1b0 2024-05-17T09:21:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.840+0000 7f35e259c640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f35cc0b2920 0x7f35cc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 -- 172.21.15.19:0/84580244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 msgr2=0x556b0f1cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35e259c640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f35cc0b2920 0x7f35cc0b4e30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35d40a8d00 tx=0x7f35d4007960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 0x556b0f1cb3a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02acb0 tx=0x7f35dc030fe0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 -- 172.21.15.19:0/84580244 shutdown_connections 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee094d0 0x556b0f1d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee012f0 0x556b0f1cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 --2- 172.21.15.19:0/84580244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee2a1b0 0x556b0f1cb3a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 -- 172.21.15.19:0/84580244 >> 172.21.15.19:0/84580244 conn(0x556b0ee1e300 msgr2=0x556b0f1c6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 -- 172.21.15.19:0/84580244 shutdown_connections 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 -- 172.21.15.19:0/84580244 wait complete. 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.841+0000 7f35efdbd740 1 -- start start 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 Processor -- start 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1d4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48964/0 (socket says 172.21.15.19:48964) 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 -- 172.21.15.19:0/2164113508 learned_addr learned my addr 172.21.15.19:0/2164113508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 0x556b0f1dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b0ee13790 con 0x556b0ee012f0 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556b0f0e9890 con 0x556b0ee2a1b0 2024-05-17T09:21:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556b0f1d3210 con 0x556b0ee094d0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 -- 172.21.15.19:0/2164113508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 msgr2=0x556b0f1d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 -- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 msgr2=0x556b0f1dc1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 0x556b0f1dc1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35e2d9d640 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0ef0e620 con 0x556b0ee012f0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.842+0000 7f35efdbd740 10 client.0 _ll_register_callbacks cb 0x556b0f0ecd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1d4d30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f35dc034040 tx=0x7f35dc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d9f571afa46f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc037040 con 0x556b0ee012f0 2024-05-17T09:21:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35c9ffb640 1 -- 172.21.15.19:0/2164113508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc036050 con 0x556b0ee012f0 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35c9ffb640 1 -- 172.21.15.19:0/2164113508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f35dc031440 con 0x556b0ee012f0 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35c9ffb640 1 -- 172.21.15.19:0/2164113508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35dc041040 con 0x556b0ee012f0 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35efdbd740 10 client.16002 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.843+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556b0f1def80 con 0x556b0ee012f0 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35c9ffb640 1 -- 172.21.15.19:0/2164113508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f35dc03c040 con 0x556b0ee012f0 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35efdbd740 10 client.16002 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35efdbd740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35efdbd740 20 client.16002 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0ede8d80 0x556b0f1e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35efdbd740 4 client.16002 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b0f13fb80 con 0x556b0ede8d80 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.844+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0ede8d80 0x556b0f1e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.845+0000 7f35c9ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.845+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0ede8d80 0x556b0f1e22e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeba4b2e56564df4 server_cookie=9c365fae7ab05fb8 in_seq=0 out_seq=0 2024-05-17T09:21:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.845+0000 7f35c9ffb640 1 -- 172.21.15.19:0/2164113508 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b0f13fb80 con 0x556b0ede8d80 2024-05-17T09:21:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.845+0000 7f35c9ffb640 10 client.16002 handle_command_reply: tid=0 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.845+0000 7f35efdbd740 1 client.16002 shutdown 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 msgr2=0x556b0f1d4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1d4d30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f35dc034040 tx=0x7f35dc02a6c0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 shutdown_connections 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0ede8d80 0x556b0f1e22e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 0x556b0f1dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35c9ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2164113508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1d4d30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 >> 172.21.15.19:0/2164113508 conn(0x556b0ee1e300 msgr2=0x556b0f1c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 shutdown_connections 2024-05-17T09:21:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 1 -- 172.21.15.19:0/2164113508 wait complete. 2024-05-17T09:21:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.846+0000 7f35efdbd740 20 client.16002 trim_cache size 0 max 16384 2024-05-17T09:21:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 Processor -- start 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 -- start start 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee012f0 0x556b0f1e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 0x556b0f1dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee2a1b0 0x556b0f1e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41ad000 con 0x556b0ee094d0 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41aa8d0 con 0x556b0ee012f0 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35efdbd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41a8720 con 0x556b0ee2a1b0 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 0x556b0f1dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 0x556b0f1dc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48970/0 (socket says 172.21.15.19:48970) 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 -- 172.21.15.19:0/2229053463 learned_addr learned my addr 172.21.15.19:0/2229053463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee012f0 0x556b0f1e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee2a1b0 0x556b0f1e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 -- 172.21.15.19:0/2229053463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee2a1b0 msgr2=0x556b0f1e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee2a1b0 0x556b0f1e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 -- 172.21.15.19:0/2229053463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee012f0 msgr2=0x556b0f1e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee012f0 0x556b0f1e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e1d9b640 1 -- 172.21.15.19:0/2229053463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0f1d2690 con 0x556b0ee094d0 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.877+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee2a1b0 0x556b0f1e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.878+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 0x556b0f1dc160 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f35e4198ff0 tx=0x7f35d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1291d5480082d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.878+0000 7f35cbfff640 1 -- 172.21.15.19:0/2229053463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8003d60 con 0x556b0ee094d0 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.878+0000 7f35cbfff640 1 -- 172.21.15.19:0/2229053463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d801e070 con 0x556b0ee094d0 2024-05-17T09:21:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.878+0000 7f35cbfff640 1 -- 172.21.15.19:0/2229053463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8004d10 con 0x556b0ee094d0 2024-05-17T09:21:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 -- 172.21.15.19:0/2229053463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 msgr2=0x556b0f1dc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 0x556b0f1dc160 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f35e4198ff0 tx=0x7f35d8009260 comp rx=0 tx=0).stop 2024-05-17T09:21:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 -- 172.21.15.19:0/2229053463 shutdown_connections 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee2a1b0 0x556b0f1e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee094d0 0x556b0f1dc160 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 --2- 172.21.15.19:0/2229053463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee012f0 0x556b0f1e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 -- 172.21.15.19:0/2229053463 >> 172.21.15.19:0/2229053463 conn(0x556b0ee1e300 msgr2=0x556b0f1ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 -- 172.21.15.19:0/2229053463 shutdown_connections 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 -- 172.21.15.19:0/2229053463 wait complete. 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 -- start start 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.879+0000 7f35efdbd740 1 Processor -- start 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 0x556b0f251190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1db3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48976/0 (socket says 172.21.15.19:48976) 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41a8d90 con 0x556b0ee012f0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 -- 172.21.15.19:0/3267600537 learned_addr learned my addr 172.21.15.19:0/3267600537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41aa200 con 0x556b0ee2a1b0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41a9460 con 0x556b0ee094d0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35efdbd740 10 client.0 _ll_register_callbacks cb 0x7f35dc030c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 -- 172.21.15.19:0/3267600537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 msgr2=0x556b0f1db910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1db910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 -- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 msgr2=0x556b0f251190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 0x556b0f251190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.880+0000 7f35e2d9d640 1 -- 172.21.15.19:0/3267600537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0f251750 con 0x556b0ee012f0 2024-05-17T09:21:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35e2d9d640 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1db3d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0099a0 tx=0x7f35dc030860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c532d88a072bb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35c9ffb640 1 -- 172.21.15.19:0/3267600537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc036040 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35c9ffb640 1 -- 172.21.15.19:0/3267600537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f35dc003970 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d801e070 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35c9ffb640 1 -- 172.21.15.19:0/3267600537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35dc06b050 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35efdbd740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b0f252950 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35c9ffb640 1 -- 172.21.15.19:0/3267600537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d801e070 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.881+0000 7f35c9ffb640 1 -- 172.21.15.19:0/3267600537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f35dc03c040 con 0x556b0ee012f0 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35efdbd740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35efdbd740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35efdbd740 20 client.16008 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35efdbd740 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0edbe1c0 0x556b0f255310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35efdbd740 4 client.16008 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:21:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b0f139360 con 0x556b0edbe1c0 2024-05-17T09:21:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0edbe1c0 0x556b0f255310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35c9ffb640 10 client.16008 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.882+0000 7f35e1d9b640 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0edbe1c0 0x556b0f255310 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beaaec0494319937 server_cookie=50a9a654a128ab43 in_seq=0 out_seq=0 2024-05-17T09:21:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.886+0000 7f35c9ffb640 1 -- 172.21.15.19:0/3267600537 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b0f139360 con 0x556b0edbe1c0 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.886+0000 7f35c9ffb640 10 client.16008 handle_command_reply: tid=0 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.886+0000 7f35efdbd740 1 client.16008 shutdown 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 msgr2=0x556b0f1db3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1db3d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0099a0 tx=0x7f35dc030860 comp rx=0 tx=0).stop 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 shutdown_connections 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556b0edbe1c0 0x556b0f255310 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556b0ee2a1b0 0x556b0f251190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556b0ee094d0 0x556b0f1db910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 --2- 172.21.15.19:0/3267600537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b0ee012f0 0x556b0f1db3d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35c9ffb640 0 client.16008 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 >> 172.21.15.19:0/3267600537 conn(0x556b0ee1e300 msgr2=0x556b0f1d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 shutdown_connections 2024-05-17T09:21:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 1 -- 172.21.15.19:0/3267600537 wait complete. 2024-05-17T09:21:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.887+0000 7f35efdbd740 20 client.16008 trim_cache size 0 max 16384 2024-05-17T09:21:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:21:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:21:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f35cc0b2920 msgr2=0x7f35cc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f35cc0b2920 0x7f35cc0b4e30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35d40a8d00 tx=0x7f35d4007960 comp rx=0 tx=0).stop 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 msgr2=0x7f35e4197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 0x7f35e4197fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d40071c0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 shutdown_connections 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e4108a60 0x7f35e419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35e4103d60 0x7f35e41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f35cc0b2920 0x7f35cc0b4e30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 --2- 172.21.15.19:0/2001222353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35e4102b20 0x7f35e4197fa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 >> 172.21.15.19:0/2001222353 conn(0x7f35e40fdd50 msgr2=0x7f35e4104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 shutdown_connections 2024-05-17T09:21:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:40.888+0000 7f35e8e30640 1 -- 172.21.15.19:0/2001222353 wait complete. 2024-05-17T09:21:41.067 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:21:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:42 smithi069 ceph-mon[35081]: pgmap v496: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:42 smithi019 ceph-mon[31620]: pgmap v496: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:42 smithi079 ceph-mon[35502]: pgmap v496: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-17T09:21:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:44 smithi069 ceph-mon[35081]: pgmap v497: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T09:21:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:44 smithi019 ceph-mon[31620]: pgmap v497: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T09:21:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:44 smithi079 ceph-mon[35502]: pgmap v497: 129 pgs: 129 active+clean; 672 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T09:21:46.631 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:46 smithi019 ceph-mon[31620]: pgmap v498: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:46 smithi069 ceph-mon[35081]: pgmap v498: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:46 smithi079 ceph-mon[35502]: pgmap v498: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:46.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.803+0000 7fa66b0e3700 1 -- 172.21.15.19:0/2634012152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 msgr2=0x7fa66410f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.803+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/2634012152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa66410f880 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00d330 tx=0x7fa65c00d640 comp rx=0 tx=0).stop 2024-05-17T09:21:46.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.803+0000 7fa66b0e3700 1 -- 172.21.15.19:0/2634012152 shutdown_connections 2024-05-17T09:21:46.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.803+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/2634012152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa66410f880 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.803+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/2634012152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6641089e0 0x7fa664108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.803+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/2634012152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6641029b0 0x7fa664102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.804+0000 7fa66b0e3700 1 -- 172.21.15.19:0/2634012152 >> 172.21.15.19:0/2634012152 conn(0x7fa6640fe6e0 msgr2=0x7fa664100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.804+0000 7fa66b0e3700 1 -- 172.21.15.19:0/2634012152 shutdown_connections 2024-05-17T09:21:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.804+0000 7fa66b0e3700 1 -- 172.21.15.19:0/2634012152 wait complete. 2024-05-17T09:21:46.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.805+0000 7fa66b0e3700 1 Processor -- start 2024-05-17T09:21:46.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.805+0000 7fa66b0e3700 1 -- start start 2024-05-17T09:21:46.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa66b0e3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6641029b0 0x7fa664198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa66b0e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa664198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa66b0e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6641089e0 0x7fa6641991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa66b0e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641121e0 con 0x7fa664103300 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa66b0e3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa664112060 con 0x7fa6641089e0 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa66b0e3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa664112360 con 0x7fa6641029b0 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa663fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa664198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa663fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa664198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54462/0 (socket says 172.21.15.19:54462) 2024-05-17T09:21:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa663fff700 1 -- 172.21.15.19:0/1048735625 learned_addr learned my addr 172.21.15.19:0/1048735625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa668e7f700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6641029b0 0x7fa664198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.806+0000 7fa669680700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6641089e0 0x7fa6641991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa663fff700 1 -- 172.21.15.19:0/1048735625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6641029b0 msgr2=0x7fa664198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa663fff700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6641029b0 0x7fa664198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa663fff700 1 -- 172.21.15.19:0/1048735625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6641089e0 msgr2=0x7fa6641991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa663fff700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6641089e0 0x7fa6641991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa663fff700 1 -- 172.21.15.19:0/1048735625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6641ad2e0 con 0x7fa664103300 2024-05-17T09:21:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa663fff700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa664198c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa65000e8b0 tx=0x7fa65000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa661ffb700 1 -- 172.21.15.19:0/1048735625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650004a30 con 0x7fa664103300 2024-05-17T09:21:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6641ad570 con 0x7fa664103300 2024-05-17T09:21:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.807+0000 7fa661ffb700 1 -- 172.21.15.19:0/1048735625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa65000cc40 con 0x7fa664103300 2024-05-17T09:21:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.808+0000 7fa661ffb700 1 -- 172.21.15.19:0/1048735625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500054e0 con 0x7fa664103300 2024-05-17T09:21:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.808+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641adb00 con 0x7fa664103300 2024-05-17T09:21:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.810+0000 7fa661ffb700 1 -- 172.21.15.19:0/1048735625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa650004c60 con 0x7fa664103300 2024-05-17T09:21:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.811+0000 7fa661ffb700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6540b2980 0x7fa6540b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.811+0000 7fa661ffb700 1 -- 172.21.15.19:0/1048735625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6500e2340 con 0x7fa664103300 2024-05-17T09:21:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.812+0000 7fa668e7f700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6540b2980 0x7fa6540b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.812+0000 7fa668e7f700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6540b2980 0x7fa6540b4e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa664103af0 tx=0x7fa658011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.813+0000 7fa679af8b80 1 Processor -- start 2024-05-17T09:21:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 -- start start 2024-05-17T09:21:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8db25e40 0x563a8dcaa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8dca43c0 0x563a8dca47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a8dbd2c90 con 0x563a8dba33d0 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a8dbc2830 con 0x563a8dca43c0 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa679af8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563a8dbc6870 con 0x563a8db25e40 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa668e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa668e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54470/0 (socket says 172.21.15.19:54470) 2024-05-17T09:21:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa668e7f700 1 -- 172.21.15.19:0/2643592322 learned_addr learned my addr 172.21.15.19:0/2643592322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa663fff700 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8db25e40 0x563a8dcaa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.814+0000 7fa669680700 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8dca43c0 0x563a8dca47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa668e7f700 1 -- 172.21.15.19:0/2643592322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8db25e40 msgr2=0x563a8dcaa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa668e7f700 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8db25e40 0x563a8dcaa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa668e7f700 1 -- 172.21.15.19:0/2643592322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8dca43c0 msgr2=0x563a8dca47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa668e7f700 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8dca43c0 0x563a8dca47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa668e7f700 1 -- 172.21.15.19:0/2643592322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8db128d0 con 0x563a8dba33d0 2024-05-17T09:21:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa668e7f700 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca9fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa658004330 tx=0x7fa658015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a6880c03c3396b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa64effd700 1 -- 172.21.15.19:0/2643592322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65802a070 con 0x563a8dba33d0 2024-05-17T09:21:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa64effd700 1 -- 172.21.15.19:0/2643592322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa65800fc00 con 0x563a8dba33d0 2024-05-17T09:21:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.815+0000 7fa64effd700 1 -- 172.21.15.19:0/2643592322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658025d40 con 0x563a8dba33d0 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.816+0000 7fa679af8b80 1 -- 172.21.15.19:0/2643592322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 msgr2=0x563a8dca9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.816+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca9fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa658004330 tx=0x7fa658015000 comp rx=0 tx=0).stop 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 -- 172.21.15.19:0/2643592322 shutdown_connections 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8dca43c0 0x563a8dca47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca9fe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2643592322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8db25e40 0x563a8dcaa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 -- 172.21.15.19:0/2643592322 >> 172.21.15.19:0/2643592322 conn(0x563a8db64c80 msgr2=0x563a8dc9eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:46.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 -- 172.21.15.19:0/2643592322 shutdown_connections 2024-05-17T09:21:46.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 -- 172.21.15.19:0/2643592322 wait complete. 2024-05-17T09:21:46.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.817+0000 7fa679af8b80 1 -- start start 2024-05-17T09:21:46.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.818+0000 7fa679af8b80 1 Processor -- start 2024-05-17T09:21:46.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.818+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcadd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.818+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa679af8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a8dac8fa0 con 0x563a8dca43c0 2024-05-17T09:21:46.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa679af8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a8db647c0 con 0x563a8db25e40 2024-05-17T09:21:46.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa679af8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563a8dca5250 con 0x563a8dba33d0 2024-05-17T09:21:46.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa669680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcadd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa679af8b80 10 client.0 _ll_register_callbacks cb 0x563a8db5d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcadd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58408/0 (socket says 172.21.15.19:58408) 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 -- 172.21.15.19:0/1371493696 learned_addr learned my addr 172.21.15.19:0/1371493696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa663fff700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 -- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 msgr2=0x563a8dcae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 -- 172.21.15.19:0/1371493696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 msgr2=0x563a8dcae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa668e7f700 1 -- 172.21.15.19:0/1371493696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8db61690 con 0x563a8db25e40 2024-05-17T09:21:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.819+0000 7fa669680700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:46.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa668e7f700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcadd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa65800af30 tx=0x7fa658025720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2486cab788ee652 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:46.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa643fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:46.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa643fff700 1 -- 172.21.15.19:0/1371493696 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65802a040 con 0x563a8db25e40 2024-05-17T09:21:46.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65800fc00 con 0x563a8db25e40 2024-05-17T09:21:46.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa643fff700 1 -- 172.21.15.19:0/1371493696 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa658025940 con 0x563a8db25e40 2024-05-17T09:21:46.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa643fff700 1 -- 172.21.15.19:0/1371493696 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa658025b40 con 0x563a8db25e40 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa679af8b80 10 client.25706 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a8dcbb620 con 0x563a8db25e40 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa643fff700 1 -- 172.21.15.19:0/1371493696 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa65800fc00 con 0x563a8db25e40 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.820+0000 7fa643fff700 1 -- 172.21.15.19:0/1371493696 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa658036030 con 0x563a8db25e40 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa679af8b80 10 client.25706 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa679af8b80 10 client.25706 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa679af8b80 20 client.25706 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa679af8b80 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dcbdbb0 0x563a8dcbffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa679af8b80 4 client.25706 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a8dbd4be0 con 0x563a8dcbdbb0 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.821+0000 7fa663fff700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dcbdbb0 0x563a8dcbffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.822+0000 7fa643fff700 10 client.25706 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.822+0000 7fa663fff700 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dcbdbb0 0x563a8dcbffd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb0736d34126e0ac server_cookie=72131cbd413d0489 in_seq=0 out_seq=0 2024-05-17T09:21:46.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.822+0000 7fa643fff700 1 -- 172.21.15.19:0/1371493696 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a8dbd4be0 con 0x563a8dcbdbb0 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.822+0000 7fa643fff700 10 client.25706 handle_command_reply: tid=0 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 client.25706 shutdown 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 msgr2=0x563a8dcadd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcadd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa65800af30 tx=0x7fa658025720 comp rx=0 tx=0).stop 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 shutdown_connections 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dcbdbb0 0x563a8dcbffd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcadd30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa643fff700 0 client.25706 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 --2- 172.21.15.19:0/1371493696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.823+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 >> 172.21.15.19:0/1371493696 conn(0x563a8db64c80 msgr2=0x563a8dc9eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.824+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 shutdown_connections 2024-05-17T09:21:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.824+0000 7fa679af8b80 1 -- 172.21.15.19:0/1371493696 wait complete. 2024-05-17T09:21:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.824+0000 7fa679af8b80 20 client.25706 trim_cache size 0 max 16384 2024-05-17T09:21:46.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 Processor -- start 2024-05-17T09:21:46.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 -- start start 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcacc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dca43c0 0x563a8dcb9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b9050 con 0x563a8dba33d0 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b68e0 con 0x563a8db25e40 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa679af8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b4730 con 0x563a8dca43c0 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa663fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa668e7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcacc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa663fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54484/0 (socket says 172.21.15.19:54484) 2024-05-17T09:21:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.840+0000 7fa663fff700 1 -- 172.21.15.19:0/3742802897 learned_addr learned my addr 172.21.15.19:0/3742802897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa668e7f700 1 -- 172.21.15.19:0/3742802897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dca43c0 msgr2=0x563a8dcb9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa668e7f700 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dca43c0 0x563a8dcb9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa668e7f700 1 -- 172.21.15.19:0/3742802897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 msgr2=0x563a8dca8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa668e7f700 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa668e7f700 1 -- 172.21.15.19:0/3742802897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8dca0ea0 con 0x563a8db25e40 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa663fff700 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa668e7f700 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcacc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6580049d0 tx=0x7fa658004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a0a0db9ac28351 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa64e7fc700 1 -- 172.21.15.19:0/3742802897 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65802a040 con 0x563a8db25e40 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa64e7fc700 1 -- 172.21.15.19:0/3742802897 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa65800fc00 con 0x563a8db25e40 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.841+0000 7fa64e7fc700 1 -- 172.21.15.19:0/3742802897 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65800d4f0 con 0x563a8db25e40 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 -- 172.21.15.19:0/3742802897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 msgr2=0x563a8dcacc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcacc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6580049d0 tx=0x7fa658004570 comp rx=0 tx=0).stop 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 -- 172.21.15.19:0/3742802897 shutdown_connections 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dca43c0 0x563a8dcb9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcacc90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 --2- 172.21.15.19:0/3742802897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dba33d0 0x563a8dca8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.842+0000 7fa679af8b80 1 -- 172.21.15.19:0/3742802897 >> 172.21.15.19:0/3742802897 conn(0x563a8db64c80 msgr2=0x563a8dc98f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:46.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.843+0000 7fa679af8b80 1 -- 172.21.15.19:0/3742802897 shutdown_connections 2024-05-17T09:21:46.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.843+0000 7fa679af8b80 1 -- 172.21.15.19:0/3742802897 wait complete. 2024-05-17T09:21:46.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.843+0000 7fa679af8b80 1 -- start start 2024-05-17T09:21:46.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.843+0000 7fa679af8b80 1 Processor -- start 2024-05-17T09:21:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcbffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcc0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcc0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b4da0 con 0x563a8dca43c0 2024-05-17T09:21:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b6210 con 0x563a8db25e40 2024-05-17T09:21:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b5470 con 0x563a8dba33d0 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa679af8b80 10 client.0 _ll_register_callbacks cb 0x563a8db5d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa669680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcc0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa668e7f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcbffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa669680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcc0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54490/0 (socket says 172.21.15.19:54490) 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa669680700 1 -- 172.21.15.19:0/2838359743 learned_addr learned my addr 172.21.15.19:0/2838359743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.844+0000 7fa663fff700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcc0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa668e7f700 1 -- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 msgr2=0x563a8dcc0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa668e7f700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcc0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa668e7f700 1 -- 172.21.15.19:0/2838359743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 msgr2=0x563a8dcc0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa668e7f700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcc0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa668e7f700 1 -- 172.21.15.19:0/2838359743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8dd26510 con 0x563a8db25e40 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa669680700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcc0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa668e7f700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcbffd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa65803e000 tx=0x7fa65800fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6a1477dd24bb28 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa643fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa643fff700 1 -- 172.21.15.19:0/2838359743 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65801e070 con 0x563a8db25e40 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa643fff700 1 -- 172.21.15.19:0/2838359743 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa65800dc20 con 0x563a8db25e40 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.845+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65800fc00 con 0x563a8db25e40 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa643fff700 1 -- 172.21.15.19:0/2838359743 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa65802a040 con 0x563a8db25e40 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 10 client.25712 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a8dd27710 con 0x563a8db25e40 2024-05-17T09:21:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa643fff700 1 -- 172.21.15.19:0/2838359743 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa65800de60 con 0x563a8db25e40 2024-05-17T09:21:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa643fff700 1 -- 172.21.15.19:0/2838359743 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa65803e030 con 0x563a8db25e40 2024-05-17T09:21:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 10 client.25712 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 10 client.25712 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 20 client.25712 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dd27c80 0x563a8dd2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 4 client.25712 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.846+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa65800fc00 con 0x563a8dd27c80 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.847+0000 7fa663fff700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dd27c80 0x563a8dd2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.847+0000 7fa643fff700 10 client.25712 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.847+0000 7fa663fff700 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dd27c80 0x563a8dd2a0a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=656f35e53abbb53 server_cookie=6cf665ab9541de16 in_seq=0 out_seq=0 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa643fff700 1 -- 172.21.15.19:0/2838359743 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa65800fc00 con 0x563a8dd27c80 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa643fff700 10 client.25712 handle_command_reply: tid=0 2024-05-17T09:21:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 client.25712 shutdown 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 msgr2=0x563a8dcbffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcbffd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa65803e000 tx=0x7fa65800fe40 comp rx=0 tx=0).stop 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 shutdown_connections 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x563a8dd27c80 0x563a8dd2a0a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a8dca43c0 0x563a8dcc0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa643fff700 0 client.25712 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a8db25e40 0x563a8dcbffd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 --2- 172.21.15.19:0/2838359743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a8dba33d0 0x563a8dcc0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 >> 172.21.15.19:0/2838359743 conn(0x563a8db64c80 msgr2=0x563a8dca8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 shutdown_connections 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.848+0000 7fa679af8b80 1 -- 172.21.15.19:0/2838359743 wait complete. 2024-05-17T09:21:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.849+0000 7fa679af8b80 20 client.25712 trim_cache size 0 max 16384 2024-05-17T09:21:46.852 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:21:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.849+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6540b2980 msgr2=0x7fa6540b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.849+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6540b2980 0x7fa6540b4e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa664103af0 tx=0x7fa658011040 comp rx=0 tx=0).stop 2024-05-17T09:21:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.849+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 msgr2=0x7fa664198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.849+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa664198c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa65000e8b0 tx=0x7fa65000ebc0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 shutdown_connections 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6540b2980 0x7fa6540b4e90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6641029b0 0x7fa664198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6641089e0 0x7fa6641991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 --2- 172.21.15.19:0/1048735625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664103300 0x7fa664198c80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 >> 172.21.15.19:0/1048735625 conn(0x7fa6640fe6e0 msgr2=0x7fa664106b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 shutdown_connections 2024-05-17T09:21:46.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:46.850+0000 7fa66b0e3700 1 -- 172.21.15.19:0/1048735625 wait complete. 2024-05-17T09:21:47.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:21:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:48.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:48 smithi019 ceph-mon[31620]: pgmap v499: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:48 smithi069 ceph-mon[35081]: pgmap v499: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:48 smithi079 ceph-mon[35502]: pgmap v499: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.176+0000 7f4219e88700 1 -- 172.21.15.19:0/1619823346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 msgr2=0x7f4214108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.176+0000 7f4219e88700 1 --2- 172.21.15.19:0/1619823346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f4214108220 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f42080099c0 tx=0x7f4208009cd0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 -- 172.21.15.19:0/1619823346 shutdown_connections 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 --2- 172.21.15.19:0/1619823346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f4214103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 --2- 172.21.15.19:0/1619823346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 0x7f42141100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 --2- 172.21.15.19:0/1619823346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f4214108220 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 -- 172.21.15.19:0/1619823346 >> 172.21.15.19:0/1619823346 conn(0x7f4214078e40 msgr2=0x7f4214079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 -- 172.21.15.19:0/1619823346 shutdown_connections 2024-05-17T09:21:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.177+0000 7f4219e88700 1 -- 172.21.15.19:0/1619823346 wait complete. 2024-05-17T09:21:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 Processor -- start 2024-05-17T09:21:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 -- start start 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f421419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f421419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 0x7f42141a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42141121a0 con 0x7f4214107e10 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4214112020 con 0x7f4214108760 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.178+0000 7f4219e88700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4214112320 con 0x7f4214103530 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f4212ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f421419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f4213fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 0x7f42141a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f421419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f421419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37040/0 (socket says 172.21.15.19:37040) 2024-05-17T09:21:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f4213fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 0x7f42141a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58440/0 (socket says 172.21.15.19:58440) 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 -- 172.21.15.19:0/3741251838 learned_addr learned my addr 172.21.15.19:0/3741251838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 -- 172.21.15.19:0/3741251838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 msgr2=0x7f42141a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 0x7f42141a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 -- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 msgr2=0x7f421419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f421419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 -- 172.21.15.19:0/3741251838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42141a1df0 con 0x7f4214103530 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f4212ffd700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f421419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.179+0000 7f42137fe700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f421419cf20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f420400e9a0 tx=0x7f420400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.180+0000 7f4210ff9700 1 -- 172.21.15.19:0/3741251838 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400cbd0 con 0x7f4214103530 2024-05-17T09:21:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.180+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42141a20e0 con 0x7f4214103530 2024-05-17T09:21:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.180+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42141a9a00 con 0x7f4214103530 2024-05-17T09:21:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.181+0000 7f4210ff9700 1 -- 172.21.15.19:0/3741251838 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f420400cd70 con 0x7f4214103530 2024-05-17T09:21:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.181+0000 7f4210ff9700 1 -- 172.21.15.19:0/3741251838 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204018680 con 0x7f4214103530 2024-05-17T09:21:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.184+0000 7f4210ff9700 1 -- 172.21.15.19:0/3741251838 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4204018820 con 0x7f4214103530 2024-05-17T09:21:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.185+0000 7f4210ff9700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41fc0b2980 0x7f41fc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.185+0000 7f4212ffd700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41fc0b2980 0x7f41fc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.186+0000 7f4210ff9700 1 -- 172.21.15.19:0/3741251838 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f42040e26f0 con 0x7f4214103530 2024-05-17T09:21:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.186+0000 7f4212ffd700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41fc0b2980 0x7f41fc0b4e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f4208003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.186+0000 7f422889db80 1 Processor -- start 2024-05-17T09:21:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.186+0000 7f422889db80 1 -- start start 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f422889db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602952413d0 0x560295347fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f422889db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295348520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f422889db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x5602953427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f422889db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560295270c90 con 0x5602952413d0 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f422889db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560295260830 con 0x5602953423c0 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f422889db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560295264870 con 0x5602951c3e40 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f4212ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295348520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f4212ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295348520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37050/0 (socket says 172.21.15.19:37050) 2024-05-17T09:21:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f4212ffd700 1 -- 172.21.15.19:0/3808836503 learned_addr learned my addr 172.21.15.19:0/3808836503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.187+0000 7f4213fff700 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x5602953427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f4212ffd700 1 -- 172.21.15.19:0/3808836503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 msgr2=0x5602953427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f4212ffd700 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x5602953427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f4212ffd700 1 -- 172.21.15.19:0/3808836503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602952413d0 msgr2=0x560295347fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f4212ffd700 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602952413d0 0x560295347fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f4212ffd700 1 -- 172.21.15.19:0/3808836503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602951b08d0 con 0x5602951c3e40 2024-05-17T09:21:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f4212ffd700 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295348520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4208015000 tx=0x7f420801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94cf307909b45229 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f41f9ffb700 1 -- 172.21.15.19:0/3808836503 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208005670 con 0x5602951c3e40 2024-05-17T09:21:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f41f9ffb700 1 -- 172.21.15.19:0/3808836503 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4208003c50 con 0x5602951c3e40 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.188+0000 7f41f9ffb700 1 -- 172.21.15.19:0/3808836503 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420802f8a0 con 0x5602951c3e40 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 -- 172.21.15.19:0/3808836503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 msgr2=0x560295348520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295348520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4208015000 tx=0x7f420801d000 comp rx=0 tx=0).stop 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 -- 172.21.15.19:0/3808836503 shutdown_connections 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x5602953427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602952413d0 0x560295347fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 --2- 172.21.15.19:0/3808836503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295348520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 -- 172.21.15.19:0/3808836503 >> 172.21.15.19:0/3808836503 conn(0x560295202c80 msgr2=0x56029533cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 -- 172.21.15.19:0/3808836503 shutdown_connections 2024-05-17T09:21:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.190+0000 7f422889db80 1 -- 172.21.15.19:0/3808836503 wait complete. 2024-05-17T09:21:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.191+0000 7f422889db80 1 -- start start 2024-05-17T09:21:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.191+0000 7f422889db80 1 Processor -- start 2024-05-17T09:21:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.191+0000 7f422889db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x56029534bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f422889db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x56029534c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f422889db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x56029534c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f422889db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560295166fa0 con 0x5602953423c0 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f422889db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5602952027c0 con 0x5602952413d0 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f422889db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560295357f60 con 0x5602951c3e40 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f422889db80 10 client.0 _ll_register_callbacks cb 0x5602951fb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f42137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x56029534bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x56029534c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x56029534c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54526/0 (socket says 172.21.15.19:54526) 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 -- 172.21.15.19:0/4231089913 learned_addr learned my addr 172.21.15.19:0/4231089913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4212ffd700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x56029534c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 -- 172.21.15.19:0/4231089913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 msgr2=0x56029534bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x56029534bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 -- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 msgr2=0x56029534c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x56029534c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.192+0000 7f4213fff700 1 -- 172.21.15.19:0/4231089913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602951ff690 con 0x5602953423c0 2024-05-17T09:21:50.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f4212ffd700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x56029534c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:50.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f4213fff700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x56029534c7b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f420c009990 tx=0x7f420c009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29597481ba7a9fa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f41f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f41f2ffd700 1 -- 172.21.15.19:0/4231089913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c01d070 con 0x5602953423c0 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f41f2ffd700 1 -- 172.21.15.19:0/4231089913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f420c004bb0 con 0x5602953423c0 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4208003c50 con 0x5602953423c0 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f41f2ffd700 1 -- 172.21.15.19:0/4231089913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f420c028050 con 0x5602953423c0 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f422889db80 10 client.16032 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.193+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560295359970 con 0x5602953423c0 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f41f2ffd700 1 -- 172.21.15.19:0/4231089913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4208003c50 con 0x5602953423c0 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f41f2ffd700 1 -- 172.21.15.19:0/4231089913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f420c017bf0 con 0x5602953423c0 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f422889db80 10 client.16032 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f422889db80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f422889db80 20 client.16032 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f422889db80 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x56029535e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f422889db80 4 client.16032 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560295272be0 con 0x56029535c2d0 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.194+0000 7f42137fe700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x56029535e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.195+0000 7f41f2ffd700 10 client.16032 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.196+0000 7f42137fe700 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x56029535e720 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa72fc843c789ed6 server_cookie=b0589986d02f381a in_seq=0 out_seq=0 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f41f2ffd700 1 -- 172.21.15.19:0/4231089913 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560295272be0 con 0x56029535c2d0 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f41f2ffd700 10 client.16032 handle_command_reply: tid=0 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 client.16032 shutdown 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 msgr2=0x56029534c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x56029534c7b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f420c009990 tx=0x7f420c009d50 comp rx=0 tx=0).stop 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 shutdown_connections 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x56029534bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x56029535e720 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f41f2ffd700 0 client.16032 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x56029534c7b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 --2- 172.21.15.19:0/4231089913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x56029534c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.197+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 >> 172.21.15.19:0/4231089913 conn(0x560295202c80 msgr2=0x56029533cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.198+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 shutdown_connections 2024-05-17T09:21:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.198+0000 7f422889db80 1 -- 172.21.15.19:0/4231089913 wait complete. 2024-05-17T09:21:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.198+0000 7f422889db80 20 client.16032 trim_cache size 0 max 16384 2024-05-17T09:21:50.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.228+0000 7f422889db80 1 Processor -- start 2024-05-17T09:21:50.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.228+0000 7f422889db80 1 -- start start 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f422889db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 0x560295347210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f422889db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602952413d0 0x560295337670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f422889db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x56029534c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f422889db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4200011400 con 0x5602951c3e40 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f42137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 0x560295347210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f42137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 0x560295347210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54536/0 (socket says 172.21.15.19:54536) 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.229+0000 7f42137fe700 1 -- 172.21.15.19:0/1988782080 learned_addr learned my addr 172.21.15.19:0/1988782080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f420000ec90 con 0x5602953423c0 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4213fff700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x56029534c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4212ffd700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602952413d0 0x560295337670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f420000cae0 con 0x5602952413d0 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4212ffd700 1 -- 172.21.15.19:0/1988782080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 msgr2=0x56029534c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4212ffd700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x56029534c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4212ffd700 1 -- 172.21.15.19:0/1988782080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 msgr2=0x560295347210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4212ffd700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 0x560295347210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.230+0000 7f4212ffd700 1 -- 172.21.15.19:0/1988782080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602953352f0 con 0x5602952413d0 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.231+0000 7f4213fff700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x56029534c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.231+0000 7f42137fe700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 0x560295347210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.231+0000 7f4212ffd700 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602952413d0 0x560295337670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x56029533da60 tx=0x7f4208009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a830ec75a8853f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.231+0000 7f41f97fa700 1 -- 172.21.15.19:0/1988782080 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208005670 con 0x5602952413d0 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.231+0000 7f41f97fa700 1 -- 172.21.15.19:0/1988782080 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f420802f480 con 0x5602952413d0 2024-05-17T09:21:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.231+0000 7f41f97fa700 1 -- 172.21.15.19:0/1988782080 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420800bbc0 con 0x5602952413d0 2024-05-17T09:21:50.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.233+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602952413d0 msgr2=0x560295337670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.233+0000 7f422889db80 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602952413d0 0x560295337670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x56029533da60 tx=0x7f4208009910 comp rx=0 tx=0).stop 2024-05-17T09:21:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 shutdown_connections 2024-05-17T09:21:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602953423c0 0x56029534c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602951c3e40 0x560295347210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 --2- 172.21.15.19:0/1988782080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602952413d0 0x560295337670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 >> 172.21.15.19:0/1988782080 conn(0x560295202c80 msgr2=0x560295336f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 shutdown_connections 2024-05-17T09:21:50.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 -- 172.21.15.19:0/1988782080 wait complete. 2024-05-17T09:21:50.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.234+0000 7f422889db80 1 -- start start 2024-05-17T09:21:50.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 Processor -- start 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295362ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x560295363410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x560295363950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f420000d150 con 0x5602953423c0 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f420000e5c0 con 0x5602952413d0 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f420000d820 con 0x5602951c3e40 2024-05-17T09:21:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.235+0000 7f422889db80 10 client.0 _ll_register_callbacks cb 0x5602951fb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295362ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f4212ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x560295363410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295362ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37080/0 (socket says 172.21.15.19:37080) 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 -- 172.21.15.19:0/252729612 learned_addr learned my addr 172.21.15.19:0/252729612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f4213fff700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x560295363950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 -- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 msgr2=0x560295363410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x560295363410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 -- 172.21.15.19:0/252729612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 msgr2=0x560295363950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x560295363950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f42137fe700 1 -- 172.21.15.19:0/252729612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560295348890 con 0x5602951c3e40 2024-05-17T09:21:50.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f4212ffd700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x560295363410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:50.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.236+0000 7f4213fff700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x560295363950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:50.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f42137fe700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295362ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42040dfd50 tx=0x7f42040a9070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee3246fe94e9c2e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:50.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f41f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:50.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f41f2ffd700 1 -- 172.21.15.19:0/252729612 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040aa000 con 0x5602951c3e40 2024-05-17T09:21:50.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f41f2ffd700 1 -- 172.21.15.19:0/252729612 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f42040cc0c0 con 0x5602951c3e40 2024-05-17T09:21:50.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420802f480 con 0x5602951c3e40 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f41f2ffd700 1 -- 172.21.15.19:0/252729612 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f42040a9900 con 0x5602951c3e40 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f422889db80 10 client.25738 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.237+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560295354080 con 0x5602951c3e40 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f41f2ffd700 1 -- 172.21.15.19:0/252729612 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f420802f480 con 0x5602951c3e40 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f41f2ffd700 1 -- 172.21.15.19:0/252729612 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f42040b22d0 con 0x5602951c3e40 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f422889db80 10 client.25738 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f422889db80 10 client.25738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f422889db80 20 client.25738 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f422889db80 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x5602953c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f422889db80 4 client.25738 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.238+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56029526c3c0 con 0x56029535c2d0 2024-05-17T09:21:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.239+0000 7f4212ffd700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x5602953c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.239+0000 7f4212ffd700 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x5602953c8dc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac18b2f462cedec server_cookie=f9ef76b4c8fac1d4 in_seq=0 out_seq=0 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.239+0000 7f41f2ffd700 10 client.25738 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.239+0000 7f41f2ffd700 1 -- 172.21.15.19:0/252729612 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56029526c3c0 con 0x56029535c2d0 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.239+0000 7f41f2ffd700 10 client.25738 handle_command_reply: tid=0 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.239+0000 7f422889db80 1 client.25738 shutdown 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 msgr2=0x560295362ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295362ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42040dfd50 tx=0x7f42040a9070 comp rx=0 tx=0).stop 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 shutdown_connections 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602951c3e40 0x560295362ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56029535c2d0 0x5602953c8dc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602953423c0 0x560295363950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 --2- 172.21.15.19:0/252729612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602952413d0 0x560295363410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 >> 172.21.15.19:0/252729612 conn(0x560295202c80 msgr2=0x5602953413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f41f2ffd700 0 client.25738 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 shutdown_connections 2024-05-17T09:21:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 1 -- 172.21.15.19:0/252729612 wait complete. 2024-05-17T09:21:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.240+0000 7f422889db80 20 client.25738 trim_cache size 0 max 16384 2024-05-17T09:21:50.243 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41fc0b2980 msgr2=0x7f41fc0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41fc0b2980 0x7f41fc0b4e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f4208003680 comp rx=0 tx=0).stop 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 msgr2=0x7f421419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f421419cf20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f420400e9a0 tx=0x7f420400ecb0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 shutdown_connections 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41fc0b2980 0x7f41fc0b4e90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4214103530 0x7f421419cf20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4214108760 0x7f42141a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 --2- 172.21.15.19:0/3741251838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4214107e10 0x7f421419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 >> 172.21.15.19:0/3741251838 conn(0x7f4214078e40 msgr2=0x7f42141017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.241+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 shutdown_connections 2024-05-17T09:21:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:50.242+0000 7f4219e88700 1 -- 172.21.15.19:0/3741251838 wait complete. 2024-05-17T09:21:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:50 smithi069 ceph-mon[35081]: pgmap v500: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:50 smithi019 ceph-mon[31620]: pgmap v500: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:50 smithi079 ceph-mon[35502]: pgmap v500: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:21:52.103 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:21:52.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:21:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:52 smithi019 ceph-mon[31620]: pgmap v501: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:52 smithi069 ceph-mon[35081]: pgmap v501: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:52.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:52 smithi079 ceph-mon[35502]: pgmap v501: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 -- 172.21.15.19:0/1079390413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148102b40 msgr2=0x7ff148103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 --2- 172.21.15.19:0/1079390413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148102b40 0x7ff148103840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13802a410 comp rx=0 tx=0).stop 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 -- 172.21.15.19:0/1079390413 shutdown_connections 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 --2- 172.21.15.19:0/1079390413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148108a80 0x7ff14810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 --2- 172.21.15.19:0/1079390413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148103d80 0x7ff148104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 --2- 172.21.15.19:0/1079390413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148102b40 0x7ff148103840 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 -- 172.21.15.19:0/1079390413 >> 172.21.15.19:0/1079390413 conn(0x7ff1480fdd30 msgr2=0x7ff1481001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 -- 172.21.15.19:0/1079390413 shutdown_connections 2024-05-17T09:21:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.033+0000 7ff14db0d640 1 -- 172.21.15.19:0/1079390413 wait complete. 2024-05-17T09:21:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.034+0000 7ff14db0d640 1 Processor -- start 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.034+0000 7ff14db0d640 1 -- start start 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff14db0d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148102b40 0x7ff148199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff14db0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148103d80 0x7ff1481996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff14db0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 0x7ff1481a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff14db0d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810e520 con 0x7ff148108a80 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff14db0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810e3a0 con 0x7ff148103d80 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff14db0d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810e6a0 con 0x7ff148102b40 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 0x7ff1481a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 0x7ff1481a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44624/0 (socket says 172.21.15.19:44624) 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 -- 172.21.15.19:0/2599787678 learned_addr learned my addr 172.21.15.19:0/2599787678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff1477fe640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148102b40 0x7ff148199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff146ffd640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148103d80 0x7ff1481996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 -- 172.21.15.19:0/2599787678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148102b40 msgr2=0x7ff148199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148102b40 0x7ff148199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 -- 172.21.15.19:0/2599787678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148103d80 msgr2=0x7ff1481996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148103d80 0x7ff1481996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff147fff640 1 -- 172.21.15.19:0/2599787678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14806a750 con 0x7ff148108a80 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff1477fe640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148102b40 0x7ff148199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.035+0000 7ff146ffd640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148103d80 0x7ff1481996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.036+0000 7ff147fff640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 0x7ff1481a0630 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff14000eb10 tx=0x7ff14000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.036+0000 7ff144ff9640 1 -- 172.21.15.19:0/2599787678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004b20 con 0x7ff148108a80 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.036+0000 7ff144ff9640 1 -- 172.21.15.19:0/2599787678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff140004cc0 con 0x7ff148108a80 2024-05-17T09:21:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.036+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14806aa40 con 0x7ff148108a80 2024-05-17T09:21:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.036+0000 7ff144ff9640 1 -- 172.21.15.19:0/2599787678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140010690 con 0x7ff148108a80 2024-05-17T09:21:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.036+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14806af30 con 0x7ff148108a80 2024-05-17T09:21:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.040+0000 7ff1549c4740 1 Processor -- start 2024-05-17T09:21:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.040+0000 7ff144ff9640 1 -- 172.21.15.19:0/2599787678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff140010830 con 0x7ff148108a80 2024-05-17T09:21:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.040+0000 7ff1549c4740 1 -- start start 2024-05-17T09:21:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.040+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a55ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.040+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a55ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.040+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a523d4d0 0x5568a5606820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff1549c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568a5571c30 con 0x5568a52352f0 2024-05-17T09:21:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff1549c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5568a55617d0 con 0x5568a523d4d0 2024-05-17T09:21:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff1549c4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5568a5565810 con 0x5568a525e1b0 2024-05-17T09:21:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff1477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a55ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff1477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a55ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44636/0 (socket says 172.21.15.19:44636) 2024-05-17T09:21:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff1477fe640 1 -- 172.21.15.19:0/819587334 learned_addr learned my addr 172.21.15.19:0/819587334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.041+0000 7ff146ffd640 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a55ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff146ffd640 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a523d4d0 0x5568a5606820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff1477fe640 1 -- 172.21.15.19:0/819587334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 msgr2=0x5568a55ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff1477fe640 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a55ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff1477fe640 1 -- 172.21.15.19:0/819587334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a523d4d0 msgr2=0x5568a5606820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff1477fe640 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a523d4d0 0x5568a5606820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff1477fe640 1 -- 172.21.15.19:0/819587334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a5276210 con 0x5568a52352f0 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff1477fe640 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a55ff920 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff1380047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7745ef3ac3a41aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff144ff9640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff1300b2920 0x7ff1300b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff12dffb640 1 -- 172.21.15.19:0/819587334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138035030 con 0x5568a52352f0 2024-05-17T09:21:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.042+0000 7ff12dffb640 1 -- 172.21.15.19:0/819587334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff138040070 con 0x5568a52352f0 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.043+0000 7ff12dffb640 1 -- 172.21.15.19:0/819587334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138003850 con 0x5568a52352f0 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.043+0000 7ff146ffd640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff1300b2920 0x7ff1300b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.043+0000 7ff144ff9640 1 -- 172.21.15.19:0/2599787678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff1400e2a50 con 0x7ff148108a80 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.043+0000 7ff1549c4740 1 -- 172.21.15.19:0/819587334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 msgr2=0x5568a55ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.043+0000 7ff1549c4740 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a55ff920 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff1380047f0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.043+0000 7ff146ffd640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff1300b2920 0x7ff1300b4e30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5568a5600570 tx=0x7ff13c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 -- 172.21.15.19:0/819587334 shutdown_connections 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a523d4d0 0x5568a5606820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a55ff920 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 --2- 172.21.15.19:0/819587334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a55ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 -- 172.21.15.19:0/819587334 >> 172.21.15.19:0/819587334 conn(0x5568a5252300 msgr2=0x5568a55fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 -- 172.21.15.19:0/819587334 shutdown_connections 2024-05-17T09:21:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 -- 172.21.15.19:0/819587334 wait complete. 2024-05-17T09:21:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 -- start start 2024-05-17T09:21:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.044+0000 7ff1549c4740 1 Processor -- start 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a52352f0 0x5568a5608c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a56091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a5610120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568a5247790 con 0x5568a523d4d0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5568a551d890 con 0x5568a52352f0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5568a5607210 con 0x5568a525e1b0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1549c4740 10 client.0 _ll_register_callbacks cb 0x5568a5520d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff146ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a56091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff146ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a56091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44648/0 (socket says 172.21.15.19:44648) 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff146ffd640 1 -- 172.21.15.19:0/38622697 learned_addr learned my addr 172.21.15.19:0/38622697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff147fff640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a5610120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff1477fe640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a52352f0 0x5568a5608c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff147fff640 1 -- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a52352f0 msgr2=0x5568a5608c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff147fff640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a52352f0 0x5568a5608c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.045+0000 7ff147fff640 1 -- 172.21.15.19:0/38622697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 msgr2=0x5568a56091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff147fff640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a56091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff147fff640 1 -- 172.21.15.19:0/38622697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a5342620 con 0x5568a525e1b0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff146ffd640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a56091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff147fff640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a5610120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1400a6f30 tx=0x7ff1400a96d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3fb70f731811a72 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff126ffd640 1 -- 172.21.15.19:0/38622697 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400c1070 con 0x5568a525e1b0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff126ffd640 1 -- 172.21.15.19:0/38622697 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff1400b28a0 con 0x5568a525e1b0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff138040070 con 0x5568a525e1b0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.046+0000 7ff126ffd640 1 -- 172.21.15.19:0/38622697 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1400aa3c0 con 0x5568a525e1b0 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 10 client.25750 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568a5613120 con 0x5568a525e1b0 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff126ffd640 1 -- 172.21.15.19:0/38622697 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff138040070 con 0x5568a525e1b0 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff126ffd640 1 -- 172.21.15.19:0/38622697 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff140010b20 con 0x5568a525e1b0 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 10 client.25750 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 20 client.25750 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a521cd80 0x5568a5616850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 4 client.25750 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.047+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a5573b80 con 0x5568a521cd80 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.048+0000 7ff1477fe640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a521cd80 0x5568a5616850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.048+0000 7ff126ffd640 10 client.25750 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.048+0000 7ff1477fe640 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a521cd80 0x5568a5616850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebebf67ef15c01d8 server_cookie=4becb0fe90feb98b in_seq=0 out_seq=0 2024-05-17T09:21:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff126ffd640 1 -- 172.21.15.19:0/38622697 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5568a5573b80 con 0x5568a521cd80 2024-05-17T09:21:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff126ffd640 10 client.25750 handle_command_reply: tid=0 2024-05-17T09:21:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 client.25750 shutdown 2024-05-17T09:21:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 msgr2=0x5568a5610120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a5610120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1400a6f30 tx=0x7ff1400a96d0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 shutdown_connections 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a521cd80 0x5568a5616850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a525e1b0 0x5568a5610120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff126ffd640 0 client.25750 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a56091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 --2- 172.21.15.19:0/38622697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a52352f0 0x5568a5608c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 >> 172.21.15.19:0/38622697 conn(0x5568a5252300 msgr2=0x5568a55fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 shutdown_connections 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 1 -- 172.21.15.19:0/38622697 wait complete. 2024-05-17T09:21:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.049+0000 7ff1549c4740 20 client.25750 trim_cache size 0 max 16384 2024-05-17T09:21:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.065+0000 7ff1549c4740 1 Processor -- start 2024-05-17T09:21:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.065+0000 7ff1549c4740 1 -- start start 2024-05-17T09:21:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.065+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a56038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.065+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a52352f0 0x5568a5610040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.065+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a55fe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff1549c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481b1510 con 0x5568a523d4d0 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff1549c4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481aede0 con 0x5568a521cd80 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff1549c4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481acc30 con 0x5568a52352f0 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a55fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a55fe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44650/0 (socket says 172.21.15.19:44650) 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 -- 172.21.15.19:0/1262413202 learned_addr learned my addr 172.21.15.19:0/1262413202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff1477fe640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a56038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff146ffd640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a52352f0 0x5568a5610040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 -- 172.21.15.19:0/1262413202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a52352f0 msgr2=0x5568a5610040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a52352f0 0x5568a5610040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 -- 172.21.15.19:0/1262413202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 msgr2=0x5568a56038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a56038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 -- 172.21.15.19:0/1262413202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a56005a0 con 0x5568a523d4d0 2024-05-17T09:21:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff146ffd640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a52352f0 0x5568a5610040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff1477fe640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a56038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:21:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.066+0000 7ff147fff640 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a55fe240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff1400a99c0 tx=0x7ff140005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206582d44f4af1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.067+0000 7ff12d7fa640 1 -- 172.21.15.19:0/1262413202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400c1070 con 0x5568a523d4d0 2024-05-17T09:21:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.067+0000 7ff12d7fa640 1 -- 172.21.15.19:0/1262413202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1400a9cc0 con 0x5568a523d4d0 2024-05-17T09:21:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.067+0000 7ff12d7fa640 1 -- 172.21.15.19:0/1262413202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400a5450 con 0x5568a523d4d0 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 -- 172.21.15.19:0/1262413202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 msgr2=0x5568a55fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a55fe240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff1400a99c0 tx=0x7ff140005710 comp rx=0 tx=0).stop 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 -- 172.21.15.19:0/1262413202 shutdown_connections 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a523d4d0 0x5568a55fe240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a52352f0 0x5568a5610040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 --2- 172.21.15.19:0/1262413202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a56038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 -- 172.21.15.19:0/1262413202 >> 172.21.15.19:0/1262413202 conn(0x5568a5252300 msgr2=0x5568a55fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 -- 172.21.15.19:0/1262413202 shutdown_connections 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 -- 172.21.15.19:0/1262413202 wait complete. 2024-05-17T09:21:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.068+0000 7ff1549c4740 1 -- start start 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 Processor -- start 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a560efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a560f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff146ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a560f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff146ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a560f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44666/0 (socket says 172.21.15.19:44666) 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a523d4d0 0x5568a567cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff146ffd640 1 -- 172.21.15.19:0/3088842883 learned_addr learned my addr 172.21.15.19:0/3088842883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ad2a0 con 0x5568a52352f0 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ae710 con 0x5568a521cd80 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ad970 con 0x5568a523d4d0 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.069+0000 7ff1477fe640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a560efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff147fff640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a523d4d0 0x5568a567cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff146ffd640 1 -- 172.21.15.19:0/3088842883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a523d4d0 msgr2=0x5568a567cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff146ffd640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a523d4d0 0x5568a567cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff146ffd640 1 -- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 msgr2=0x5568a560efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff146ffd640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a560efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff146ffd640 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a567d450 con 0x5568a52352f0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff1477fe640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a560efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff1549c4740 10 client.0 _ll_register_callbacks cb 0x7ff1400a5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff146ffd640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a560f4f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff13c021480 tx=0x7ff13c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d708673f7546c3ea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1400a9cc0 con 0x5568a52352f0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.070+0000 7ff126ffd640 1 -- 172.21.15.19:0/3088842883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c005020 con 0x5568a52352f0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff126ffd640 1 -- 172.21.15.19:0/3088842883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff13c01a070 con 0x5568a52352f0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff126ffd640 1 -- 172.21.15.19:0/3088842883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff13c0051c0 con 0x5568a52352f0 2024-05-17T09:21:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 10 client.16056 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5568a567e540 con 0x5568a52352f0 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff126ffd640 1 -- 172.21.15.19:0/3088842883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff13c005360 con 0x5568a52352f0 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 10 client.16056 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 20 client.16056 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a525e1b0 0x5568a5680f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 4 client.16056 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568a556d360 con 0x5568a525e1b0 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.071+0000 7ff1477fe640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a525e1b0 0x5568a5680f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.072+0000 7ff1477fe640 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a525e1b0 0x5568a5680f10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b908acb9ece7e6c1 server_cookie=93485714977b0c0b in_seq=0 out_seq=0 2024-05-17T09:21:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.072+0000 7ff126ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.075+0000 7ff126ffd640 1 -- 172.21.15.19:0/3088842883 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5568a556d360 con 0x5568a525e1b0 2024-05-17T09:21:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.075+0000 7ff126ffd640 10 client.16056 handle_command_reply: tid=0 2024-05-17T09:21:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 client.16056 shutdown 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 msgr2=0x5568a560f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a560f4f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff13c021480 tx=0x7ff13c005f90 comp rx=0 tx=0).stop 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 shutdown_connections 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5568a525e1b0 0x5568a5680f10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5568a523d4d0 0x5568a567cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568a52352f0 0x5568a560f4f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff126ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 --2- 172.21.15.19:0/3088842883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568a521cd80 0x5568a560efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 >> 172.21.15.19:0/3088842883 conn(0x5568a5252300 msgr2=0x5568a56027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 shutdown_connections 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 1 -- 172.21.15.19:0/3088842883 wait complete. 2024-05-17T09:21:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.076+0000 7ff1549c4740 20 client.16056 trim_cache size 0 max 16384 2024-05-17T09:21:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:21:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:21:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:21:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff1300b2920 msgr2=0x7ff1300b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff1300b2920 0x7ff1300b4e30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5568a5600570 tx=0x7ff13c013040 comp rx=0 tx=0).stop 2024-05-17T09:21:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 msgr2=0x7ff1481a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 0x7ff1481a0630 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff14000eb10 tx=0x7ff14000c2d0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 shutdown_connections 2024-05-17T09:21:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff1300b2920 0x7ff1300b4e30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff148108a80 0x7ff1481a0630 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff148103d80 0x7ff1481996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 --2- 172.21.15.19:0/2599787678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff148102b40 0x7ff148199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 >> 172.21.15.19:0/2599787678 conn(0x7ff1480fdd30 msgr2=0x7ff148101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 shutdown_connections 2024-05-17T09:21:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:21:54.077+0000 7ff14db0d640 1 -- 172.21.15.19:0/2599787678 wait complete. 2024-05-17T09:21:54.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:54 smithi019 ceph-mon[31620]: pgmap v502: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 -- 172.21.15.19:0/560232716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac1029b0 msgr2=0x7ff4ac102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/560232716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac1029b0 0x7ff4ac102dc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000aaa0 tx=0x7ff4a000adb0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 -- 172.21.15.19:0/560232716 shutdown_connections 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/560232716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac103300 0x7ff4ac10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/560232716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac1089e0 0x7ff4ac108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/560232716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac1029b0 0x7ff4ac102dc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 -- 172.21.15.19:0/560232716 >> 172.21.15.19:0/560232716 conn(0x7ff4ac0fe6e0 msgr2=0x7ff4ac100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.552+0000 7ff4b0d20700 1 -- 172.21.15.19:0/560232716 shutdown_connections 2024-05-17T09:21:54.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.553+0000 7ff4b0d20700 1 -- 172.21.15.19:0/560232716 wait complete. 2024-05-17T09:21:54.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.554+0000 7ff4b0d20700 1 Processor -- start 2024-05-17T09:21:54.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.554+0000 7ff4b0d20700 1 -- start start 2024-05-17T09:21:54.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.554+0000 7ff4b0d20700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac1029b0 0x7ff4ac19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.554+0000 7ff4b0d20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 0x7ff4ac19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 0x7ff4ac19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 0x7ff4ac19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44680/0 (socket says 172.21.15.19:44680) 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4b0d20700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac1089e0 0x7ff4ac198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/2789375184 learned_addr learned my addr 172.21.15.19:0/2789375184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1121a0 con 0x7ff4ac103300 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac112020 con 0x7ff4ac1029b0 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac1029b0 0x7ff4ac19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac112320 con 0x7ff4ac1089e0 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac1089e0 0x7ff4ac198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4aad9d700 1 -- 172.21.15.19:0/2789375184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac1029b0 msgr2=0x7ff4ac19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac1029b0 0x7ff4ac19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.555+0000 7ff4aad9d700 1 -- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 msgr2=0x7ff4ac19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 0x7ff4ac19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff4aad9d700 1 -- 172.21.15.19:0/2789375184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac1990b0 con 0x7ff4ac1089e0 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 0x7ff4ac19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac1089e0 0x7ff4ac198a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400de20 tx=0x7ff4a400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff49b7fe700 1 -- 172.21.15.19:0/2789375184 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a401b070 con 0x7ff4ac1089e0 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff49b7fe700 1 -- 172.21.15.19:0/2789375184 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4a400ebb0 con 0x7ff4ac1089e0 2024-05-17T09:21:54.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff49b7fe700 1 -- 172.21.15.19:0/2789375184 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40145a0 con 0x7ff4ac1089e0 2024-05-17T09:21:54.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac1993a0 con 0x7ff4ac1089e0 2024-05-17T09:21:54.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.556+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1a5670 con 0x7ff4ac1089e0 2024-05-17T09:21:54.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.561+0000 7ff4bf735b80 1 Processor -- start 2024-05-17T09:21:54.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.562+0000 7ff49b7fe700 1 -- 172.21.15.19:0/2789375184 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff4a401e070 con 0x7ff4ac1089e0 2024-05-17T09:21:54.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.562+0000 7ff4bf735b80 1 -- start start 2024-05-17T09:21:54.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 0x55b145d05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145b80ec0 0x55b145d055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 0x55b145d05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 0x55b145d05060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44696/0 (socket says 172.21.15.19:44696) 2024-05-17T09:21:54.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff49b7fe700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4940b2980 0x7ff4940b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/3223357626 learned_addr learned my addr 172.21.15.19:0/3223357626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145cff440 0x55b145cff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.563+0000 7ff49b7fe700 1 -- 172.21.15.19:0/2789375184 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff4a40e3b10 con 0x7ff4ac1089e0 2024-05-17T09:21:54.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b145c2dd10 con 0x55b145bfe450 2024-05-17T09:21:54.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b145c1d8b0 con 0x55b145cff440 2024-05-17T09:21:54.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b145c218f0 con 0x55b145b80ec0 2024-05-17T09:21:54.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145b80ec0 0x55b145d055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/3223357626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145b80ec0 msgr2=0x55b145d055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145b80ec0 0x55b145d055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/3223357626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145cff440 msgr2=0x55b145cff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145cff440 0x55b145cff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/3223357626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b145b6d950 con 0x55b145bfe450 2024-05-17T09:21:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.564+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4940b2980 0x7ff4940b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.565+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 0x55b145d05060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000a7b0 tx=0x7ff4a000a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db24a62519c5546 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.565+0000 7ff498ff9700 1 -- 172.21.15.19:0/3223357626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0027040 con 0x55b145bfe450 2024-05-17T09:21:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.565+0000 7ff498ff9700 1 -- 172.21.15.19:0/3223357626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4a002eae0 con 0x55b145bfe450 2024-05-17T09:21:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.565+0000 7ff498ff9700 1 -- 172.21.15.19:0/3223357626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a001b040 con 0x55b145bfe450 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 msgr2=0x55b145d05060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 0x55b145d05060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000a7b0 tx=0x7ff4a000a7e0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 shutdown_connections 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145cff440 0x55b145cff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145bfe450 0x55b145d05060 secure :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000a7b0 tx=0x7ff4a000a7e0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3223357626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145b80ec0 0x55b145d055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.566+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 >> 172.21.15.19:0/3223357626 conn(0x55b145bbfd00 msgr2=0x55b145cf9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.567+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4940b2980 0x7ff4940b4e90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002d000 tx=0x7ff4a002e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.567+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 shutdown_connections 2024-05-17T09:21:54.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.568+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3223357626 wait complete. 2024-05-17T09:21:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.568+0000 7ff4bf735b80 1 -- start start 2024-05-17T09:21:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.568+0000 7ff4bf735b80 1 Processor -- start 2024-05-17T09:21:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.568+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d08e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 0x55b145d093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 0x55b145d093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 0x55b145d093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44698/0 (socket says 172.21.15.19:44698) 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/2645461211 learned_addr learned my addr 172.21.15.19:0/2645461211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4bf735b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b145b24020 con 0x55b145cff440 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b145bbf840 con 0x55b145b80ec0 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b145d002d0 con 0x55b145d098f0 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4bf735b80 10 client.0 _ll_register_callbacks cb 0x55b145bb89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/2645461211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 msgr2=0x55b145d14ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d14ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 msgr2=0x55b145d08e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d08e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 -- 172.21.15.19:0/2645461211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b145bbc710 con 0x55b145cff440 2024-05-17T09:21:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 0x55b145d093b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c690 tx=0x7ff4a0005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e87cb92d06e790e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.569+0000 7ff491ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff491ffb700 1 -- 172.21.15.19:0/2645461211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0027030 con 0x55b145cff440 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff491ffb700 1 -- 172.21.15.19:0/2645461211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff4a001b050 con 0x55b145cff440 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a002eae0 con 0x55b145cff440 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff491ffb700 1 -- 172.21.15.19:0/2645461211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4a0044050 con 0x55b145cff440 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff4bf735b80 10 client.16068 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:54.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b145d16cb0 con 0x55b145cff440 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.570+0000 7ff491ffb700 1 -- 172.21.15.19:0/2645461211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4a000ee20 con 0x55b145cff440 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff491ffb700 1 -- 172.21.15.19:0/2645461211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4a0036070 con 0x55b145cff440 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4bf735b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4bf735b80 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4bf735b80 20 client.16068 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d19180 0x55b145d1b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4bf735b80 4 client.16068 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4a002eae0 con 0x55b145d19180 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.571+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d19180 0x55b145d1b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.572+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d19180 0x55b145d1b5a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b8380239ebe8a0 server_cookie=e4478cf319d2d389 in_seq=0 out_seq=0 2024-05-17T09:21:54.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.572+0000 7ff491ffb700 10 client.16068 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.572+0000 7ff491ffb700 1 -- 172.21.15.19:0/2645461211 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7ff4a002eae0 con 0x55b145d19180 2024-05-17T09:21:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.572+0000 7ff491ffb700 10 client.16068 handle_command_reply: tid=0 2024-05-17T09:21:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.572+0000 7ff4bf735b80 1 client.16068 shutdown 2024-05-17T09:21:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 msgr2=0x55b145d093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 0x55b145d093b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c690 tx=0x7ff4a0005790 comp rx=0 tx=0).stop 2024-05-17T09:21:54.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 shutdown_connections 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d19180 0x55b145d1b5a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff491ffb700 0 client.16068 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d08e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/2645461211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145cff440 0x55b145d093b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 >> 172.21.15.19:0/2645461211 conn(0x55b145bbfd00 msgr2=0x55b145cf9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 shutdown_connections 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 1 -- 172.21.15.19:0/2645461211 wait complete. 2024-05-17T09:21:54.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.573+0000 7ff4bf735b80 20 client.16068 trim_cache size 0 max 16384 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 Processor -- start 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 -- start start 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d14e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d098f0 0x55b145d033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d19180 0x55b145d1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1b08e0 con 0x55b145d098f0 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ae170 con 0x55b145b80ec0 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4bf735b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1abfc0 con 0x55b145d19180 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.610+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d14e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d14e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34068/0 (socket says 172.21.15.19:34068) 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aa59c700 1 -- 172.21.15.19:0/1341863566 learned_addr learned my addr 172.21.15.19:0/1341863566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d19180 0x55b145d1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 -- 172.21.15.19:0/1341863566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 msgr2=0x55b145d14e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d14e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 -- 172.21.15.19:0/1341863566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d098f0 msgr2=0x55b145d033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d098f0 0x55b145d033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 -- 172.21.15.19:0/1341863566 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b145cf2370 con 0x55b145d19180 2024-05-17T09:21:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.611+0000 7ff4aad9d700 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d19180 0x55b145d1e5d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55b145d0bcf0 tx=0x7ff4a40aa560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a7e04d504e7b794 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.612+0000 7ff493fff700 1 -- 172.21.15.19:0/1341863566 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40c1020 con 0x55b145d19180 2024-05-17T09:21:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.612+0000 7ff493fff700 1 -- 172.21.15.19:0/1341863566 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4a40aad20 con 0x55b145d19180 2024-05-17T09:21:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.612+0000 7ff493fff700 1 -- 172.21.15.19:0/1341863566 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40b36f0 con 0x55b145d19180 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 -- 172.21.15.19:0/1341863566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d19180 msgr2=0x55b145d1e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d19180 0x55b145d1e5d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55b145d0bcf0 tx=0x7ff4a40aa560 comp rx=0 tx=0).stop 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 -- 172.21.15.19:0/1341863566 shutdown_connections 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d19180 0x55b145d1e5d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145b80ec0 0x55b145d14e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/1341863566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d098f0 0x55b145d033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.613+0000 7ff4bf735b80 1 -- 172.21.15.19:0/1341863566 >> 172.21.15.19:0/1341863566 conn(0x55b145bbfd00 msgr2=0x55b145cfeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.614+0000 7ff4bf735b80 1 -- 172.21.15.19:0/1341863566 shutdown_connections 2024-05-17T09:21:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.614+0000 7ff4bf735b80 1 -- 172.21.15.19:0/1341863566 wait complete. 2024-05-17T09:21:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.614+0000 7ff4bf735b80 1 -- start start 2024-05-17T09:21:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.614+0000 7ff4bf735b80 1 Processor -- start 2024-05-17T09:21:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d7d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145d19180 0x55b145d7db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d779e0 0x55b145d77df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ac630 con 0x55b145d779e0 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d779e0 0x55b145d77df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145d19180 0x55b145d7db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d7d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d779e0 0x55b145d77df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44716/0 (socket says 172.21.15.19:44716) 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4a9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145d19180 0x55b145d7db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34080/0 (socket says 172.21.15.19:34080) 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aad9d700 1 -- 172.21.15.19:0/3426469722 learned_addr learned my addr 172.21.15.19:0/3426469722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1adaa0 con 0x55b145d19180 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1acd00 con 0x55b145d098f0 2024-05-17T09:21:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4bf735b80 10 client.0 _ll_register_callbacks cb 0x55b145bb89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aa59c700 1 -- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145d19180 msgr2=0x55b145d7db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145d19180 0x55b145d7db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aa59c700 1 -- 172.21.15.19:0/3426469722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d779e0 msgr2=0x55b145d77df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.615+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d779e0 0x55b145d77df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff4aa59c700 1 -- 172.21.15.19:0/3426469722 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b145d78460 con 0x55b145d098f0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff4aa59c700 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d7d620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b145d16c80 tx=0x7ff49c020680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3927b505d925b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff491ffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff491ffb700 1 -- 172.21.15.19:0/3426469722 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c010460 con 0x55b145d098f0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff491ffb700 1 -- 172.21.15.19:0/3426469722 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff49c010600 con 0x55b145d098f0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a40aad20 con 0x55b145d098f0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.616+0000 7ff491ffb700 1 -- 172.21.15.19:0/3426469722 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff49c032050 con 0x55b145d098f0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.617+0000 7ff4bf735b80 10 client.25768 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.617+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b145d85110 con 0x55b145d098f0 2024-05-17T09:21:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.617+0000 7ff491ffb700 1 -- 172.21.15.19:0/3426469722 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff49c010c90 con 0x55b145d098f0 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff491ffb700 1 -- 172.21.15.19:0/3426469722 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff49c029070 con 0x55b145d098f0 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4bf735b80 10 client.25768 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4bf735b80 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4bf735b80 20 client.25768 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d857d0 0x55b145d87bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4bf735b80 4 client.25768 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff4a40aad20 con 0x55b145d857d0 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.618+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d857d0 0x55b145d87bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.619+0000 7ff4a9d9b700 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d857d0 0x55b145d87bf0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a4b2de1524fca0 server_cookie=b57e30e98984675a in_seq=0 out_seq=0 2024-05-17T09:21:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.619+0000 7ff491ffb700 10 client.25768 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.620+0000 7ff491ffb700 1 -- 172.21.15.19:0/3426469722 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff4a40aad20 con 0x55b145d857d0 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.620+0000 7ff491ffb700 10 client.25768 handle_command_reply: tid=0 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.620+0000 7ff4bf735b80 1 client.25768 shutdown 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.620+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 msgr2=0x55b145d7d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.620+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d7d620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b145d16c80 tx=0x7ff49c020680 comp rx=0 tx=0).stop 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 shutdown_connections 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b145d098f0 0x55b145d7d620 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b145d16c80 tx=0x7ff49c020680 comp rx=0 tx=0).stop 2024-05-17T09:21:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b145d857d0 0x55b145d87bf0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff491ffb700 0 client.25768 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b145d779e0 0x55b145d77df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 --2- 172.21.15.19:0/3426469722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b145d19180 0x55b145d7db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 >> 172.21.15.19:0/3426469722 conn(0x55b145bbfd00 msgr2=0x55b145d0a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 shutdown_connections 2024-05-17T09:21:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 1 -- 172.21.15.19:0/3426469722 wait complete. 2024-05-17T09:21:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.621+0000 7ff4bf735b80 20 client.25768 trim_cache size 0 max 16384 2024-05-17T09:21:54.623 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:54.623 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:21:54.623 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "7b7e65eb-2402-410f-8ef7-045107fb4d5f", 2024-05-17T09:21:54.623 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:21:54.623 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4940b2980 msgr2=0x7ff4940b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4940b2980 0x7ff4940b4e90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002d000 tx=0x7ff4a002e000 comp rx=0 tx=0).stop 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac1089e0 msgr2=0x7ff4ac198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac1089e0 0x7ff4ac198a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400de20 tx=0x7ff4a400bef0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 shutdown_connections 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4940b2980 0x7ff4940b4e90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4ac1089e0 0x7ff4ac198a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac1029b0 0x7ff4ac19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 --2- 172.21.15.19:0/2789375184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac103300 0x7ff4ac19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 >> 172.21.15.19:0/2789375184 conn(0x7ff4ac0fe6e0 msgr2=0x7ff4ac107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 shutdown_connections 2024-05-17T09:21:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:54.623+0000 7ff4b0d20700 1 -- 172.21.15.19:0/2789375184 wait complete. 2024-05-17T09:21:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:54 smithi069 ceph-mon[35081]: pgmap v502: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:54 smithi079 ceph-mon[35502]: pgmap v502: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:55.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:21:55.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:55 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:21:55.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:55 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:55.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:55 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:21:55.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:55 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:55.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:55 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:21:55.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:55 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:55.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:55 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:21:55.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:55 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:55.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:55 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:21:55.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:55 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:55.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:55 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:21:55.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:55 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:21:56.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:56 smithi019 ceph-mon[31620]: pgmap v503: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:56 smithi069 ceph-mon[35081]: pgmap v503: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:56 smithi079 ceph-mon[35502]: pgmap v503: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:21:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3572508371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc1029b0 msgr2=0x7f20cc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3572508371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc1029b0 0x7f20cc102dc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009990 tx=0x7f20bc009ca0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3572508371 shutdown_connections 2024-05-17T09:21:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3572508371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20cc1089e0 0x7f20cc108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3572508371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20cc103300 0x7f20cc10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3572508371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc1029b0 0x7f20cc102dc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.963+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3572508371 >> 172.21.15.19:0/3572508371 conn(0x7f20cc0fe6e0 msgr2=0x7f20cc100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.964+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3572508371 shutdown_connections 2024-05-17T09:21:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.964+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3572508371 wait complete. 2024-05-17T09:21:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.964+0000 7f20d3dfd700 1 Processor -- start 2024-05-17T09:21:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.964+0000 7f20d3dfd700 1 -- start start 2024-05-17T09:21:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d3dfd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20cc1029b0 0x7f20cc19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d3dfd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 0x7f20cc19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d3dfd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20cc1089e0 0x7f20cc19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d3dfd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc112180 con 0x7f20cc103300 2024-05-17T09:21:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d3dfd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc112000 con 0x7f20cc1029b0 2024-05-17T09:21:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d3dfd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc112300 con 0x7f20cc1089e0 2024-05-17T09:21:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 0x7f20cc19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 0x7f20cc19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44736/0 (socket says 172.21.15.19:44736) 2024-05-17T09:21:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 -- 172.21.15.19:0/3059457783 learned_addr learned my addr 172.21.15.19:0/3059457783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1b99700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20cc1029b0 0x7f20cc19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d239a700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20cc1089e0 0x7f20cc19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 -- 172.21.15.19:0/3059457783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20cc1089e0 msgr2=0x7f20cc19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20cc1089e0 0x7f20cc19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 -- 172.21.15.19:0/3059457783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20cc1029b0 msgr2=0x7f20cc19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20cc1029b0 0x7f20cc19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 -- 172.21.15.19:0/3059457783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc19fbf0 con 0x7f20cc103300 2024-05-17T09:21:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.965+0000 7f20d1398700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 0x7f20cc19b2c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0059d0 tx=0x7f20bc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.966+0000 7f20c2ffd700 1 -- 172.21.15.19:0/3059457783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc027070 con 0x7f20cc103300 2024-05-17T09:21:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.966+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc19fe80 con 0x7f20cc103300 2024-05-17T09:21:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.966+0000 7f20c2ffd700 1 -- 172.21.15.19:0/3059457783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc022470 con 0x7f20cc103300 2024-05-17T09:21:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.966+0000 7f20c2ffd700 1 -- 172.21.15.19:0/3059457783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc017650 con 0x7f20cc103300 2024-05-17T09:21:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.967+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a0330 con 0x7f20cc103300 2024-05-17T09:21:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.969+0000 7f20c2ffd700 1 -- 172.21.15.19:0/3059457783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f20bc009d10 con 0x7f20cc103300 2024-05-17T09:21:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.970+0000 7f20c2ffd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20b80b2750 0x7f20b80b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.970+0000 7f20d1b99700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20b80b2750 0x7f20b80b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.970+0000 7f20c2ffd700 1 -- 172.21.15.19:0/3059457783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f20bc0e2ea0 con 0x7f20cc103300 2024-05-17T09:21:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.971+0000 7f20d1b99700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20b80b2750 0x7f20b80b4c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20cc103af0 tx=0x7f20c800aa30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.971+0000 7f20e2812b80 1 Processor -- start 2024-05-17T09:21:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.971+0000 7f20e2812b80 1 -- start start 2024-05-17T09:21:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c5878a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c58606e40 0x561c5878a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5878ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20e2812b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c586b3d70 con 0x561c5878e7b0 2024-05-17T09:21:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20e2812b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c586a3910 con 0x561c58606e40 2024-05-17T09:21:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20e2812b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561c586a7950 con 0x561c586f2430 2024-05-17T09:21:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5878ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5878ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44750/0 (socket says 172.21.15.19:44750) 2024-05-17T09:21:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 -- 172.21.15.19:0/1451325984 learned_addr learned my addr 172.21.15.19:0/1451325984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d1b99700 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c5878a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d1398700 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c58606e40 0x561c5878a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 -- 172.21.15.19:0/1451325984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 msgr2=0x561c5878a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c5878a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 -- 172.21.15.19:0/1451325984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c58606e40 msgr2=0x561c5878a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c58606e40 0x561c5878a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 -- 172.21.15.19:0/1451325984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5864cfb0 con 0x561c5878e7b0 2024-05-17T09:21:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.972+0000 7f20d239a700 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5878ebc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b6c0 tx=0x7f20c400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52706ef9dd4b5d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.973+0000 7f20b7fff700 1 -- 172.21.15.19:0/1451325984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004a30 con 0x561c5878e7b0 2024-05-17T09:21:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.973+0000 7f20b7fff700 1 -- 172.21.15.19:0/1451325984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c4015070 con 0x561c5878e7b0 2024-05-17T09:21:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.973+0000 7f20b7fff700 1 -- 172.21.15.19:0/1451325984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40039a0 con 0x561c5878e7b0 2024-05-17T09:21:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 -- 172.21.15.19:0/1451325984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 msgr2=0x561c5878ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5878ebc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b6c0 tx=0x7f20c400b9d0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 -- 172.21.15.19:0/1451325984 shutdown_connections 2024-05-17T09:21:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c5878a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5878ebc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 --2- 172.21.15.19:0/1451325984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c58606e40 0x561c5878a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 -- 172.21.15.19:0/1451325984 >> 172.21.15.19:0/1451325984 conn(0x561c586671c0 msgr2=0x561c587807d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 -- 172.21.15.19:0/1451325984 shutdown_connections 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 -- 172.21.15.19:0/1451325984 wait complete. 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.974+0000 7f20e2812b80 1 -- start start 2024-05-17T09:21:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 Processor -- start 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 0x561c58793ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c58794010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58794550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c585a9fa0 con 0x561c58606e40 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c5860bda0 con 0x561c5878e7b0 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561c5878f560 con 0x561c586f2430 2024-05-17T09:21:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20e2812b80 10 client.0 _ll_register_callbacks cb 0x561c5863e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20d1b99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 0x561c58793ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20d1b99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 0x561c58793ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44752/0 (socket says 172.21.15.19:44752) 2024-05-17T09:21:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.975+0000 7f20d1b99700 1 -- 172.21.15.19:0/2661017857 learned_addr learned my addr 172.21.15.19:0/2661017857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d239a700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58794550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1b99700 1 -- 172.21.15.19:0/2661017857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 msgr2=0x561c58794010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:21:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1398700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c58794010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1b99700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c58794010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1b99700 1 -- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 msgr2=0x561c58794550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1b99700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58794550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1b99700 1 -- 172.21.15.19:0/2661017857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c585f38d0 con 0x561c58606e40 2024-05-17T09:21:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d239a700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58794550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1398700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c58794010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20d1b99700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 0x561c58793ad0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f20c80045b0 tx=0x7f20c8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a7846d03326740 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20b57fa700 1 -- 172.21.15.19:0/2661017857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8028060 con 0x561c58606e40 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.976+0000 7f20b57fa700 1 -- 172.21.15.19:0/2661017857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f20c8016b20 con 0x561c58606e40 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.977+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c4015070 con 0x561c58606e40 2024-05-17T09:21:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.977+0000 7f20b57fa700 1 -- 172.21.15.19:0/2661017857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20c802f050 con 0x561c58606e40 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.977+0000 7f20e2812b80 10 client.16083 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.977+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c5865d760 con 0x561c58606e40 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.977+0000 7f20b57fa700 1 -- 172.21.15.19:0/2661017857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c4015070 con 0x561c58606e40 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.977+0000 7f20b57fa700 1 -- 172.21.15.19:0/2661017857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20c8023cd0 con 0x561c58606e40 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20e2812b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20e2812b80 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20e2812b80 20 client.16083 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c587a38c0 0x561c587a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20e2812b80 4 client.16083 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:21:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c58751ed0 con 0x561c587a38c0 2024-05-17T09:21:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.978+0000 7f20d1398700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c587a38c0 0x561c587a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.979+0000 7f20b57fa700 10 client.16083 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.979+0000 7f20d1398700 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c587a38c0 0x561c587a5d20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61eb2f782747cd5f server_cookie=33c09f82874e9b9b in_seq=0 out_seq=0 2024-05-17T09:21:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.979+0000 7f20b57fa700 1 -- 172.21.15.19:0/2661017857 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561c58751ed0 con 0x561c587a38c0 2024-05-17T09:21:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.979+0000 7f20b57fa700 10 client.16083 handle_command_reply: tid=0 2024-05-17T09:21:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 client.16083 shutdown 2024-05-17T09:21:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 msgr2=0x561c58793ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 0x561c58793ad0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f20c80045b0 tx=0x7f20c8004b40 comp rx=0 tx=0).stop 2024-05-17T09:21:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 shutdown_connections 2024-05-17T09:21:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c587a38c0 0x561c587a5d20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58794550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c58606e40 0x561c58793ad0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2661017857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c586f2430 0x561c58794010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20b57fa700 0 client.16083 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 >> 172.21.15.19:0/2661017857 conn(0x561c586671c0 msgr2=0x561c58780790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 shutdown_connections 2024-05-17T09:21:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 1 -- 172.21.15.19:0/2661017857 wait complete. 2024-05-17T09:21:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:57.980+0000 7f20e2812b80 20 client.16083 trim_cache size 0 max 16384 2024-05-17T09:21:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 Processor -- start 2024-05-17T09:21:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 -- start start 2024-05-17T09:21:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c58792a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c586f2430 0x561c5879f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5877af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1b0610 con 0x561c5878e7b0 2024-05-17T09:21:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1adea0 con 0x561c586f2430 2024-05-17T09:21:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.001+0000 7f20e2812b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1abcf0 con 0x561c58606e40 2024-05-17T09:21:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5877af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5877af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44754/0 (socket says 172.21.15.19:44754) 2024-05-17T09:21:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 -- 172.21.15.19:0/248113181 learned_addr learned my addr 172.21.15.19:0/248113181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d1b99700 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c58792a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d1398700 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c586f2430 0x561c5879f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 -- 172.21.15.19:0/248113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 msgr2=0x561c58792a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c58792a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 -- 172.21.15.19:0/248113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c586f2430 msgr2=0x561c5879f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c586f2430 0x561c5879f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 -- 172.21.15.19:0/248113181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c58781990 con 0x561c5878e7b0 2024-05-17T09:21:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20d239a700 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5877af10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b6c0 tx=0x7f20c400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1017a616c110e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20b77fe700 1 -- 172.21.15.19:0/248113181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4028070 con 0x561c5878e7b0 2024-05-17T09:21:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20b77fe700 1 -- 172.21.15.19:0/248113181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20c4015070 con 0x561c5878e7b0 2024-05-17T09:21:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.002+0000 7f20b77fe700 1 -- 172.21.15.19:0/248113181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004950 con 0x561c5878e7b0 2024-05-17T09:21:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 -- 172.21.15.19:0/248113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 msgr2=0x561c5877af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5877af10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b6c0 tx=0x7f20c400bf90 comp rx=0 tx=0).stop 2024-05-17T09:21:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 -- 172.21.15.19:0/248113181 shutdown_connections 2024-05-17T09:21:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c58792a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c5878e7b0 0x561c5877af10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 --2- 172.21.15.19:0/248113181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c586f2430 0x561c5879f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 -- 172.21.15.19:0/248113181 >> 172.21.15.19:0/248113181 conn(0x561c586671c0 msgr2=0x561c5877a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 -- 172.21.15.19:0/248113181 shutdown_connections 2024-05-17T09:21:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.003+0000 7f20e2812b80 1 -- 172.21.15.19:0/248113181 wait complete. 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.004+0000 7f20e2812b80 1 -- start start 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.004+0000 7f20e2812b80 1 Processor -- start 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c5879bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 0x561c58799350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58799890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ac360 con 0x561c586f2430 2024-05-17T09:21:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ad7d0 con 0x561c5878e7b0 2024-05-17T09:21:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1aca30 con 0x561c58606e40 2024-05-17T09:21:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20e2812b80 10 client.0 _ll_register_callbacks cb 0x561c5863e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:21:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 0x561c58799350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 0x561c58799350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44762/0 (socket says 172.21.15.19:44762) 2024-05-17T09:21:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 -- 172.21.15.19:0/2727071664 learned_addr learned my addr 172.21.15.19:0/2727071664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d239a700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58799890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1b99700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c5879bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 -- 172.21.15.19:0/2727071664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 msgr2=0x561c5879bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c5879bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 -- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 msgr2=0x561c58799890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58799890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 -- 172.21.15.19:0/2727071664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c58799dd0 con 0x561c586f2430 2024-05-17T09:21:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20d1398700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 0x561c58799350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0a6220 tx=0x7f20bc0a9910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7b1973cd9f2c3b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:21:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.005+0000 7f20b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:21:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20b57fa700 1 -- 172.21.15.19:0/2727071664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0aa1a0 con 0x561c586f2430 2024-05-17T09:21:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20b57fa700 1 -- 172.21.15.19:0/2727071664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f20bc0b2ae0 con 0x561c586f2430 2024-05-17T09:21:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c4015070 con 0x561c586f2430 2024-05-17T09:21:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20b57fa700 1 -- 172.21.15.19:0/2727071664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20bc0ae550 con 0x561c586f2430 2024-05-17T09:21:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 10 client.16089 fetch_fsmap learned FSMap version 27 2024-05-17T09:21:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c5880e0e0 con 0x561c586f2430 2024-05-17T09:21:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20b57fa700 1 -- 172.21.15.19:0/2727071664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc0b2f70 con 0x561c586f2430 2024-05-17T09:21:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20b57fa700 1 -- 172.21.15.19:0/2727071664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20bc0e0ab0 con 0x561c586f2430 2024-05-17T09:21:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 20 client.16089 populate_metadata read hostname 'smithi019' 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c5880e810 0x561c58810c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 4 client.16089 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.006+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f20c4015070 con 0x561c5880e810 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20d1b99700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c5880e810 0x561c58810c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:21:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20b57fa700 10 client.16089 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20d1b99700 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c5880e810 0x561c58810c30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6773e4c3b591dd5f server_cookie=cb95efed93214478 in_seq=0 out_seq=0 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20b57fa700 1 -- 172.21.15.19:0/2727071664 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f20c4015070 con 0x561c5880e810 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20b57fa700 10 client.16089 handle_command_reply: tid=0 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20e2812b80 1 client.16089 shutdown 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 msgr2=0x561c58799350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.007+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 0x561c58799350 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0a6220 tx=0x7f20bc0a9910 comp rx=0 tx=0).stop 2024-05-17T09:21:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 shutdown_connections 2024-05-17T09:21:58.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561c58606e40 0x561c5879bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561c5880e810 0x561c58810c30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c5878e7b0 0x561c58799890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 --2- 172.21.15.19:0/2727071664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c586f2430 0x561c58799350 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20b57fa700 0 client.16089 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:21:58.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 >> 172.21.15.19:0/2727071664 conn(0x561c586671c0 msgr2=0x561c5877a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 shutdown_connections 2024-05-17T09:21:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 1 -- 172.21.15.19:0/2727071664 wait complete. 2024-05-17T09:21:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20e2812b80 20 client.16089 trim_cache size 0 max 16384 2024-05-17T09:21:58.023 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:21:58.024 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:21:58.024 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:21:58.024 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:21:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20b80b2750 msgr2=0x7f20b80b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20b80b2750 0x7f20b80b4c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20cc103af0 tx=0x7f20c800aa30 comp rx=0 tx=0).stop 2024-05-17T09:21:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 msgr2=0x7f20cc19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:21:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.008+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 0x7f20cc19b2c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0059d0 tx=0x7f20bc00bfc0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 shutdown_connections 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f20b80b2750 0x7f20b80b4c60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20cc1089e0 0x7f20cc19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20cc1029b0 0x7f20cc19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 --2- 172.21.15.19:0/3059457783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc103300 0x7f20cc19b2c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 >> 172.21.15.19:0/3059457783 conn(0x7f20cc0fe6e0 msgr2=0x7f20cc100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 shutdown_connections 2024-05-17T09:21:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:21:58.009+0000 7f20d3dfd700 1 -- 172.21.15.19:0/3059457783 wait complete. 2024-05-17T09:21:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:58 smithi069 ceph-mon[35081]: pgmap v504: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T09:21:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:21:58 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:21:58.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:58 smithi019 ceph-mon[31620]: pgmap v504: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T09:21:58.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:21:58 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:21:58.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:58 smithi079 ceph-mon[35502]: pgmap v504: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-17T09:21:58.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:21:58 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:21:58.952 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:21:58.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:21:59.214 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:21:59.538 INFO:tasks.workunit.client.0.smithi019.stderr:Updating files: 89% (11013/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '3790c0feaaca36c33bb1b31470733325d7d91e6a'. 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.306 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-05-17T09:22:00.307 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.307 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-05-17T09:22:00.307 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.307 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-17T09:22:00.307 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-17T09:22:00.307 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 3790c0feaac Merge PR #57332 into wip-pdonnell-testing-20240517.012301-debug 2024-05-17T09:22:00.312 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-17T09:22:00.364 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-17T09:22:00.367 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-17T09:22:00.367 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-17T09:22:00.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:00 smithi019 ceph-mon[31620]: pgmap v505: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:22:00.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:00 smithi069 ceph-mon[35081]: pgmap v505: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:22:00.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:00 smithi079 ceph-mon[35502]: pgmap v505: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 -- 172.21.15.19:0/47297912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c108730 msgr2=0x7f899c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 --2- 172.21.15.19:0/47297912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c108730 0x7f899c110090 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f899400b0d0 tx=0x7f899400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 -- 172.21.15.19:0/47297912 shutdown_connections 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 --2- 172.21.15.19:0/47297912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c1034b0 0x7f899c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 --2- 172.21.15.19:0/47297912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c108730 0x7f899c110090 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 --2- 172.21.15.19:0/47297912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c107de0 0x7f899c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 -- 172.21.15.19:0/47297912 >> 172.21.15.19:0/47297912 conn(0x7f899c0fedb0 msgr2=0x7f899c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 -- 172.21.15.19:0/47297912 shutdown_connections 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.120+0000 7f89a4280700 1 -- 172.21.15.19:0/47297912 wait complete. 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 Processor -- start 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 -- start start 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1034b0 0x7f899c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 0x7f899c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c108730 0x7f899c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113210 con 0x7f899c107de0 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113090 con 0x7f899c1034b0 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a4280700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113390 con 0x7f899c108730 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a181b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 0x7f899c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a181b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 0x7f899c1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44788/0 (socket says 172.21.15.19:44788) 2024-05-17T09:22:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a181b700 1 -- 172.21.15.19:0/2749438365 learned_addr learned my addr 172.21.15.19:0/2749438365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.121+0000 7f89a201c700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1034b0 0x7f899c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a281d700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c108730 0x7f899c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a181b700 1 -- 172.21.15.19:0/2749438365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c108730 msgr2=0x7f899c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a181b700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c108730 0x7f899c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a181b700 1 -- 172.21.15.19:0/2749438365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1034b0 msgr2=0x7f899c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a181b700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1034b0 0x7f899c1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a181b700 1 -- 172.21.15.19:0/2749438365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c19b150 con 0x7f899c107de0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a281d700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c108730 0x7f899c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a181b700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 0x7f899c1a0e00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f898c00d980 tx=0x7f898c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c017070 con 0x7f899c107de0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c19b440 con 0x7f899c107de0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f898c009dc0 con 0x7f899c107de0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00f3c0 con 0x7f899c107de0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.122+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a7750 con 0x7f899c107de0 2024-05-17T09:22:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.123+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c0512b0 con 0x7f899c107de0 2024-05-17T09:22:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.125+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f898c009930 con 0x7f899c107de0 2024-05-17T09:22:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.126+0000 7f89937fe700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f89880b2880 0x7f89880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.126+0000 7f89a201c700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f89880b2880 0x7f89880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.126+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f898c0e20e0 con 0x7f899c107de0 2024-05-17T09:22:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.126+0000 7f89a201c700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f89880b2880 0x7f89880b4d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f899c19bc20 tx=0x7f899800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.129+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f898c0a99f0 con 0x7f899c107de0 2024-05-17T09:22:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.393+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f899c19bfd0 con 0x7f899c107de0 2024-05-17T09:22:01.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.394+0000 7f89937fe700 1 -- 172.21.15.19:0/2749438365 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7f898c0a92f0 con 0x7f899c107de0 2024-05-17T09:22:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:22:01.395 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.396+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f89880b2880 msgr2=0x7f89880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.396+0000 7f89a4280700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f89880b2880 0x7f89880b4d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f899c19bc20 tx=0x7f899800a380 comp rx=0 tx=0).stop 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.396+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 msgr2=0x7f899c1a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.396+0000 7f89a4280700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 0x7f899c1a0e00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f898c00d980 tx=0x7f898c00dc90 comp rx=0 tx=0).stop 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 shutdown_connections 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f89880b2880 0x7f89880b4d90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f899c108730 0x7f899c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1034b0 0x7f899c1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 --2- 172.21.15.19:0/2749438365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899c107de0 0x7f899c1a0e00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:01.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 >> 172.21.15.19:0/2749438365 conn(0x7f899c0fedb0 msgr2=0x7f899c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:01.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 shutdown_connections 2024-05-17T09:22:01.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:01.397+0000 7f89a4280700 1 -- 172.21.15.19:0/2749438365 wait complete. 2024-05-17T09:22:01.398 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:22:01.973 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-17T09:22:02.073 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-17T09:22:02.116 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-17T09:22:02.117 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-17T09:22:02.117 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-17T09:22:02.157 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-17T09:22:02.161 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:22:02.161 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-17T09:22:02.178 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-17T09:22:02.179 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-17T09:22:02.179 DEBUG:teuthology.orchestra.run.smithi019:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3790c0feaaca36c33bb1b31470733325d7d91e6a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-17T09:22:02.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:22:02.263 INFO:tasks.workunit.client.0.smithi019.stderr:+ dbench 1 2024-05-17T09:22:02.279 INFO:tasks.workunit.client.0.smithi019.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-17T09:22:02.279 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-17T09:22:02.280 INFO:tasks.workunit.client.0.smithi019.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-17T09:22:02.289 INFO:tasks.workunit.client.0.smithi019.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-17T09:22:02.390 INFO:tasks.workunit.client.0.smithi019.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-17T09:22:02.390 INFO:tasks.workunit.client.0.smithi019.stdout:releasing clients 2024-05-17T09:22:02.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:02 smithi019 ceph-mon[31620]: pgmap v506: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:22:02.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2749438365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:22:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:02.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:02 smithi069 ceph-mon[35081]: pgmap v506: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:22:02.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2749438365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:22:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:02 smithi079 ceph-mon[35502]: pgmap v506: 129 pgs: 129 active+clean; 673 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:22:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2749438365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:22:03.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 873 21.66 MB/sec warmup 1 sec latency 13.010 ms 2024-05-17T09:22:04.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 1497 13.02 MB/sec warmup 2 sec latency 21.555 ms 2024-05-17T09:22:04.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:04 smithi019 ceph-mon[31620]: pgmap v507: 129 pgs: 129 active+clean; 3.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 234 KiB/s wr, 8 op/s 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 -- 172.21.15.19:0/2188695436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda810e8d0 msgr2=0x7efda8110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 --2- 172.21.15.19:0/2188695436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda810e8d0 0x7efda8110db0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efda000b0a0 tx=0x7efda000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 -- 172.21.15.19:0/2188695436 shutdown_connections 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 --2- 172.21.15.19:0/2188695436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda81054f0 0x7efda8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 --2- 172.21.15.19:0/2188695436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda810e8d0 0x7efda8110db0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 --2- 172.21.15.19:0/2188695436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 0x7efda810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 -- 172.21.15.19:0/2188695436 >> 172.21.15.19:0/2188695436 conn(0x7efda80fedb0 msgr2=0x7efda81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 -- 172.21.15.19:0/2188695436 shutdown_connections 2024-05-17T09:22:04.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.403+0000 7efdaed5a700 1 -- 172.21.15.19:0/2188695436 wait complete. 2024-05-17T09:22:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.404+0000 7efdaed5a700 1 Processor -- start 2024-05-17T09:22:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.404+0000 7efdaed5a700 1 -- start start 2024-05-17T09:22:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdaed5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 0x7efda81a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdaed5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 0x7efda81a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdaed5a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda810e8d0 0x7efda819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdaed5a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efda8114330 con 0x7efda81054f0 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdaed5a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efda81141b0 con 0x7efda8105e40 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdaed5a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efda81144b0 con 0x7efda810e8d0 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 0x7efda81a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 0x7efda81a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52764/0 (socket says 172.21.15.19:52764) 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 -- 172.21.15.19:0/297196181 learned_addr learned my addr 172.21.15.19:0/297196181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efda7fff700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 0x7efda81a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdad2f7700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda810e8d0 0x7efda819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 -- 172.21.15.19:0/297196181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda810e8d0 msgr2=0x7efda819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda810e8d0 0x7efda819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 -- 172.21.15.19:0/297196181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 msgr2=0x7efda81a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 0x7efda81a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdacaf6700 1 -- 172.21.15.19:0/297196181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda819b280 con 0x7efda81054f0 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efdad2f7700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda810e8d0 0x7efda819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.405+0000 7efda7fff700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 0x7efda81a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.406+0000 7efdacaf6700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 0x7efda81a09c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e9a0 tx=0x7efd9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.406+0000 7efda5ffb700 1 -- 172.21.15.19:0/297196181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00ca10 con 0x7efda81054f0 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.406+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda819b510 con 0x7efda81054f0 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.406+0000 7efda5ffb700 1 -- 172.21.15.19:0/297196181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c00cbb0 con 0x7efda81054f0 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.406+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda8103110 con 0x7efda81054f0 2024-05-17T09:22:04.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.407+0000 7efda5ffb700 1 -- 172.21.15.19:0/297196181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c018680 con 0x7efda81054f0 2024-05-17T09:22:04.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.411+0000 7efda5ffb700 1 -- 172.21.15.19:0/297196181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efd9c018820 con 0x7efda81054f0 2024-05-17T09:22:04.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.411+0000 7efdbd76fb80 1 Processor -- start 2024-05-17T09:22:04.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.411+0000 7efdbd76fb80 1 -- start start 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efda5ffb700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd980b2880 0x7efd980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e51c33d0 0x55a1e52c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 0x55a1e52ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 0x55a1e52ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 0x55a1e52ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52772/0 (socket says 172.21.15.19:52772) 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e52c43c0 0x55a1e52c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efda5ffb700 1 -- 172.21.15.19:0/297196181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efd9c0e27e0 con 0x7efda81054f0 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1e51f2c90 con 0x55a1e5145e40 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1e51e2830 con 0x55a1e52c43c0 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1097637595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a1e51e6870 con 0x55a1e51c33d0 2024-05-17T09:22:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e51c33d0 0x55a1e52c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efda7fff700 1 -- 172.21.15.19:0/1097637595 learned_addr learned my addr 172.21.15.19:0/1097637595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdad2f7700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd980b2880 0x7efd980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdacaf6700 1 -- 172.21.15.19:0/1097637595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e52c43c0 msgr2=0x55a1e52c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e52c43c0 0x55a1e52c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdacaf6700 1 -- 172.21.15.19:0/1097637595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 msgr2=0x55a1e52ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 0x55a1e52ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efdacaf6700 1 -- 172.21.15.19:0/1097637595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e51328d0 con 0x55a1e51c33d0 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.412+0000 7efda7fff700 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 0x55a1e52ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.413+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e51c33d0 0x55a1e52c9fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0e02a0 tx=0x7efd9c0b7890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5f411fd1e11f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:04.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.413+0000 7efd92ffd700 1 -- 172.21.15.19:0/1097637595 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0a5070 con 0x55a1e51c33d0 2024-05-17T09:22:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.413+0000 7efd92ffd700 1 -- 172.21.15.19:0/1097637595 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c0b2040 con 0x55a1e51c33d0 2024-05-17T09:22:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.413+0000 7efd92ffd700 1 -- 172.21.15.19:0/1097637595 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0ba220 con 0x55a1e51c33d0 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1097637595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e51c33d0 msgr2=0x55a1e52c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e51c33d0 0x55a1e52c9fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0e02a0 tx=0x7efd9c0b7890 comp rx=0 tx=0).stop 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdad2f7700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd980b2880 0x7efd980b4d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efda0009900 tx=0x7efda0009820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1097637595 shutdown_connections 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e51c33d0 0x55a1e52c9fe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e52c43c0 0x55a1e52c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1097637595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e5145e40 0x55a1e52ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.414+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1097637595 >> 172.21.15.19:0/1097637595 conn(0x55a1e5184c80 msgr2=0x55a1e52beee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.415+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1097637595 shutdown_connections 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.415+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1097637595 wait complete. 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.415+0000 7efdbd76fb80 1 -- start start 2024-05-17T09:22:04.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.415+0000 7efdbd76fb80 1 Processor -- start 2024-05-17T09:22:04.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1e50e8fa0 con 0x55a1e51c33d0 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1e51847c0 con 0x55a1e5145e40 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a1e52c5250 con 0x55a1e52c43c0 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43668/0 (socket says 172.21.15.19:43668) 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdacaf6700 1 -- 172.21.15.19:0/96383112 learned_addr learned my addr 172.21.15.19:0/96383112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdbd76fb80 10 client.0 _ll_register_callbacks cb 0x55a1e517d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdad2f7700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdad2f7700 1 -- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 msgr2=0x55a1e52cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdad2f7700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.416+0000 7efdad2f7700 1 -- 172.21.15.19:0/96383112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 msgr2=0x55a1e52ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efdad2f7700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efdad2f7700 1 -- 172.21.15.19:0/96383112 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e5181690 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efda7fff700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efdad2f7700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52ce7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efda0009fd0 tx=0x7efda000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467699bb0ef46fee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efd87fff700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efd87fff700 1 -- 172.21.15.19:0/96383112 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0021070 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c0b2040 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efd87fff700 1 -- 172.21.15.19:0/96383112 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efda0007b10 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efd87fff700 1 -- 172.21.15.19:0/96383112 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efda0036050 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efdbd76fb80 10 client.25795 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.417+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1e52db6f0 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efd87fff700 1 -- 172.21.15.19:0/96383112 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda001f070 con 0x55a1e52c43c0 2024-05-17T09:22:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efd87fff700 1 -- 172.21.15.19:0/96383112 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efda0011070 con 0x55a1e52c43c0 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdbd76fb80 10 client.25795 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdbd76fb80 10 client.25795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdbd76fb80 20 client.25795 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e52ddc80 0x55a1e52e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdbd76fb80 4 client.25795 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd9c0b2040 con 0x55a1e52ddc80 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.418+0000 7efdacaf6700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e52ddc80 0x55a1e52e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.419+0000 7efd87fff700 10 client.25795 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.419+0000 7efdacaf6700 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e52ddc80 0x55a1e52e00a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8db96befd9d7604b server_cookie=258b4d5fa2dce5ea in_seq=0 out_seq=0 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efd87fff700 1 -- 172.21.15.19:0/96383112 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7efd9c0b2040 con 0x55a1e52ddc80 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efd87fff700 10 client.25795 handle_command_reply: tid=0 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 client.25795 shutdown 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 msgr2=0x55a1e52ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52ce7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efda0009fd0 tx=0x7efda000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 shutdown_connections 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e52ddc80 0x55a1e52e00a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52ce7c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/96383112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efd87fff700 0 client.25795 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.420+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 >> 172.21.15.19:0/96383112 conn(0x55a1e5184c80 msgr2=0x55a1e52beea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.421+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 shutdown_connections 2024-05-17T09:22:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.421+0000 7efdbd76fb80 1 -- 172.21.15.19:0/96383112 wait complete. 2024-05-17T09:22:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.421+0000 7efdbd76fb80 20 client.25795 trim_cache size 0 max 16384 2024-05-17T09:22:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.438+0000 7efdbd76fb80 1 Processor -- start 2024-05-17T09:22:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.438+0000 7efdbd76fb80 1 -- start start 2024-05-17T09:22:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b0a30 con 0x55a1e51c33d0 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efda81ae2c0 con 0x55a1e5145e40 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efda81ac110 con 0x55a1e52c43c0 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52796/0 (socket says 172.21.15.19:52796) 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efda7fff700 1 -- 172.21.15.19:0/3542470804 learned_addr learned my addr 172.21.15.19:0/3542470804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdad2f7700 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 -- 172.21.15.19:0/3542470804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 msgr2=0x55a1e52b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 -- 172.21.15.19:0/3542470804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 msgr2=0x55a1e52ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 -- 172.21.15.19:0/3542470804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e52c5130 con 0x55a1e5145e40 2024-05-17T09:22:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efda7fff700 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.439+0000 7efdacaf6700 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cd060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0a9750 tx=0x7efd9c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cc586b8c513684 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.440+0000 7efd927fc700 1 -- 172.21.15.19:0/3542470804 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0a5040 con 0x55a1e5145e40 2024-05-17T09:22:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.440+0000 7efd927fc700 1 -- 172.21.15.19:0/3542470804 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c0b2040 con 0x55a1e5145e40 2024-05-17T09:22:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.440+0000 7efd927fc700 1 -- 172.21.15.19:0/3542470804 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0a9f10 con 0x55a1e5145e40 2024-05-17T09:22:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 -- 172.21.15.19:0/3542470804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 msgr2=0x55a1e52cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cd060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0a9750 tx=0x7efd9c009790 comp rx=0 tx=0).stop 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 -- 172.21.15.19:0/3542470804 shutdown_connections 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52cd060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/3542470804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 -- 172.21.15.19:0/3542470804 >> 172.21.15.19:0/3542470804 conn(0x55a1e5184c80 msgr2=0x55a1e52b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 -- 172.21.15.19:0/3542470804 shutdown_connections 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 -- 172.21.15.19:0/3542470804 wait complete. 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 -- start start 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.441+0000 7efdbd76fb80 1 Processor -- start 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efda81ac780 con 0x55a1e51c33d0 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efda81adbf0 con 0x55a1e5145e40 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efda81ace50 con 0x55a1e52c43c0 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdbd76fb80 10 client.0 _ll_register_callbacks cb 0x55a1e517d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43682/0 (socket says 172.21.15.19:43682) 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 -- 172.21.15.19:0/1383932718 learned_addr learned my addr 172.21.15.19:0/1383932718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efda7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52d65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52810/0 (socket says 172.21.15.19:52810) 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdad2f7700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 -- 172.21.15.19:0/1383932718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 msgr2=0x55a1e52cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 -- 172.21.15.19:0/1383932718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 msgr2=0x55a1e52d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52d65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdacaf6700 1 -- 172.21.15.19:0/1383932718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e52cdd60 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efdad2f7700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.442+0000 7efda7fff700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efdacaf6700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52d60a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e970 tx=0x7efd9c0adf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aaaeebe2d94db93 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efd87fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efd87fff700 1 -- 172.21.15.19:0/1383932718 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0a5030 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efd87fff700 1 -- 172.21.15.19:0/1383932718 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efd9c0a9cb0 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c0b2040 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efd87fff700 1 -- 172.21.15.19:0/1383932718 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd9c0a9eb0 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efdbd76fb80 10 client.25796 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1e5348140 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efd87fff700 1 -- 172.21.15.19:0/1383932718 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c0b2420 con 0x55a1e5145e40 2024-05-17T09:22:04.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efd87fff700 1 -- 172.21.15.19:0/1383932718 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd9c0b4bb0 con 0x55a1e5145e40 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efdbd76fb80 10 client.25796 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.443+0000 7efdbd76fb80 10 client.25796 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efdbd76fb80 20 client.25796 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e5348310 0x55a1e534a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efdbd76fb80 4 client.25796 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efd9c0b2040 con 0x55a1e5348310 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efda7fff700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e5348310 0x55a1e534a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efd87fff700 10 client.25796 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efda7fff700 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e5348310 0x55a1e534a730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=835874f43b6d3e8 server_cookie=876b46ac0a2ec82b in_seq=0 out_seq=0 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efd87fff700 1 -- 172.21.15.19:0/1383932718 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efd9c0b2040 con 0x55a1e5348310 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efd87fff700 10 client.25796 handle_command_reply: tid=0 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.444+0000 7efdbd76fb80 1 client.25796 shutdown 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 msgr2=0x55a1e52d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52d60a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e970 tx=0x7efd9c0adf00 comp rx=0 tx=0).stop 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 shutdown_connections 2024-05-17T09:22:04.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55a1e5348310 0x55a1e534a730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1e52c43c0 0x55a1e52cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1e5145e40 0x55a1e52d60a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efd87fff700 0 client.25796 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 --2- 172.21.15.19:0/1383932718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e51c33d0 0x55a1e52d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 >> 172.21.15.19:0/1383932718 conn(0x55a1e5184c80 msgr2=0x55a1e52ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 shutdown_connections 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 1 -- 172.21.15.19:0/1383932718 wait complete. 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.445+0000 7efdbd76fb80 20 client.25796 trim_cache size 0 max 16384 2024-05-17T09:22:04.450 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:22:04.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd980b2880 msgr2=0x7efd980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd980b2880 0x7efd980b4d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efda0009900 tx=0x7efda0009820 comp rx=0 tx=0).stop 2024-05-17T09:22:04.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 msgr2=0x7efda81a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:04.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 0x7efda81a09c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e9a0 tx=0x7efd9c00ecb0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 shutdown_connections 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd980b2880 0x7efd980b4d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efda810e8d0 0x7efda819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda81054f0 0x7efda81a09c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 --2- 172.21.15.19:0/297196181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda8105e40 0x7efda81a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 >> 172.21.15.19:0/297196181 conn(0x7efda80fedb0 msgr2=0x7efda80ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 shutdown_connections 2024-05-17T09:22:04.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:04.446+0000 7efdaed5a700 1 -- 172.21.15.19:0/297196181 wait complete. 2024-05-17T09:22:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:04 smithi069 ceph-mon[35081]: pgmap v507: 129 pgs: 129 active+clean; 3.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 234 KiB/s wr, 8 op/s 2024-05-17T09:22:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:04 smithi079 ceph-mon[35502]: pgmap v507: 129 pgs: 129 active+clean; 3.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 234 KiB/s wr, 8 op/s 2024-05-17T09:22:05.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:22:05.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 2189 8.92 MB/sec warmup 3 sec latency 15.850 ms 2024-05-17T09:22:05.497 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:06.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 2969 8.38 MB/sec warmup 4 sec latency 19.036 ms 2024-05-17T09:22:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:06 smithi069 ceph-mon[35081]: pgmap v508: 129 pgs: 129 active+clean; 10 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 818 KiB/s wr, 16 op/s 2024-05-17T09:22:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:06 smithi019 ceph-mon[31620]: pgmap v508: 129 pgs: 129 active+clean; 10 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 818 KiB/s wr, 16 op/s 2024-05-17T09:22:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:06 smithi079 ceph-mon[35502]: pgmap v508: 129 pgs: 129 active+clean; 10 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 818 KiB/s wr, 16 op/s 2024-05-17T09:22:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 -- 172.21.15.19:0/1105565732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada01070b0 msgr2=0x7fada01094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 --2- 172.21.15.19:0/1105565732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada01070b0 0x7fada01094d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fad94009900 tx=0x7fad9402a410 comp rx=0 tx=0).stop 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 -- 172.21.15.19:0/1105565732 shutdown_connections 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 --2- 172.21.15.19:0/1105565732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fada0109a10 0x7fada010be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 --2- 172.21.15.19:0/1105565732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada01070b0 0x7fada01094d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 --2- 172.21.15.19:0/1105565732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada0104750 0x7fada0106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.209+0000 7fada4c1e640 1 -- 172.21.15.19:0/1105565732 >> 172.21.15.19:0/1105565732 conn(0x7fada00fdd50 msgr2=0x7fada0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.210+0000 7fada4c1e640 1 -- 172.21.15.19:0/1105565732 shutdown_connections 2024-05-17T09:22:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.210+0000 7fada4c1e640 1 -- 172.21.15.19:0/1105565732 wait complete. 2024-05-17T09:22:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.210+0000 7fada4c1e640 1 Processor -- start 2024-05-17T09:22:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.210+0000 7fada4c1e640 1 -- start start 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fada4c1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 0x7fada019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fada4c1e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fada01070b0 0x7fada019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fada4c1e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada0109a10 0x7fada01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fada4c1e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada010f620 con 0x7fada0104750 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fada4c1e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fada010f4a0 con 0x7fada0109a10 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fada4c1e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fada010f7a0 con 0x7fada01070b0 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 0x7fada019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 0x7fada019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52824/0 (socket says 172.21.15.19:52824) 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 -- 172.21.15.19:0/767985569 learned_addr learned my addr 172.21.15.19:0/767985569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fada01070b0 0x7fada019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada0109a10 0x7fada01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 -- 172.21.15.19:0/767985569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fada01070b0 msgr2=0x7fada019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fada01070b0 0x7fada019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 -- 172.21.15.19:0/767985569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada0109a10 msgr2=0x7fada01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada0109a10 0x7fada01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 -- 172.21.15.19:0/767985569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada01a1d30 con 0x7fada0104750 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.211+0000 7fad9e59c640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 0x7fada019a1f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fad9000e9c0 tx=0x7fad9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.212+0000 7fad8f7fe640 1 -- 172.21.15.19:0/767985569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9000cd80 con 0x7fada0104750 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.212+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01a1fc0 con 0x7fada0104750 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.212+0000 7fad8f7fe640 1 -- 172.21.15.19:0/767985569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad900054b0 con 0x7fada0104750 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.212+0000 7fad8f7fe640 1 -- 172.21.15.19:0/767985569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90010430 con 0x7fada0104750 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.212+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a2480 con 0x7fada0104750 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 Processor -- start 2024-05-17T09:22:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fad8f7fe640 1 -- 172.21.15.19:0/767985569 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fad90004f20 con 0x7fada0104750 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 -- start start 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa88a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa88ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa891c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609aa7f8c30 con 0x5609aa4bc2f0 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609aa7e87d0 con 0x5609aa4c44d0 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fadabba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5609aa7ec810 con 0x5609aa4e51b0 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fad9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa88ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fad9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa891c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fad9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa891c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43712/0 (socket says 172.21.15.19:43712) 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fad9dd9b640 1 -- 172.21.15.19:0/3070200746 learned_addr learned my addr 172.21.15.19:0/3070200746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.215+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa88a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9ed9d640 1 -- 172.21.15.19:0/3070200746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 msgr2=0x5609aa891c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa891c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9ed9d640 1 -- 172.21.15.19:0/3070200746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 msgr2=0x5609aa88ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa88ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9ed9d640 1 -- 172.21.15.19:0/3070200746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609aa4fd210 con 0x5609aa4e51b0 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa88a7b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad98005fd0 tx=0x7fad98003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22386d9444630f2e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad8f7fe640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad880b28a0 0x7fad880b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad8cff9640 1 -- 172.21.15.19:0/3070200746 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98011440 con 0x5609aa4e51b0 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad9e59c640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad880b28a0 0x7fad880b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad8cff9640 1 -- 172.21.15.19:0/3070200746 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad98015030 con 0x5609aa4e51b0 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.216+0000 7fad8cff9640 1 -- 172.21.15.19:0/3070200746 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98010910 con 0x5609aa4e51b0 2024-05-17T09:22:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.217+0000 7fad8f7fe640 1 -- 172.21.15.19:0/767985569 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fad900e2b40 con 0x7fada0104750 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 -- 172.21.15.19:0/3070200746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 msgr2=0x5609aa88a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fad9e59c640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad880b28a0 0x7fad880b4db0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5609aa88bb90 tx=0x7fad900e6620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa88a7b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad98005fd0 tx=0x7fad98003e00 comp rx=0 tx=0).stop 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 -- 172.21.15.19:0/3070200746 shutdown_connections 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa891c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa88ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 --2- 172.21.15.19:0/3070200746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa88a7b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 -- 172.21.15.19:0/3070200746 >> 172.21.15.19:0/3070200746 conn(0x5609aa4d9300 msgr2=0x5609aa886140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 -- 172.21.15.19:0/3070200746 shutdown_connections 2024-05-17T09:22:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 -- 172.21.15.19:0/3070200746 wait complete. 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.218+0000 7fadabba8740 1 -- start start 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 Processor -- start 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa894100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa894640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa89b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609aa4ce790 con 0x5609aa4bc2f0 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609aa7a4890 con 0x5609aa4c44d0 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5609aa892680 con 0x5609aa4e51b0 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fadabba8740 10 client.0 _ll_register_callbacks cb 0x5609aa7a7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa894100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa894100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52838/0 (socket says 172.21.15.19:52838) 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9ed9d640 1 -- 172.21.15.19:0/2901528385 learned_addr learned my addr 172.21.15.19:0/2901528385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa894640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa89b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9dd9b640 1 -- 172.21.15.19:0/2901528385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 msgr2=0x5609aa89b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa89b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9dd9b640 1 -- 172.21.15.19:0/2901528385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 msgr2=0x5609aa894100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa894100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9dd9b640 1 -- 172.21.15.19:0/2901528385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609aa5c9620 con 0x5609aa4c44d0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.219+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa894100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa894640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad94031460 tx=0x7fad94031870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8477e1ba232bc130 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fad85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fad85ffb640 1 -- 172.21.15.19:0/2901528385 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94035040 con 0x5609aa4c44d0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad98015030 con 0x5609aa4c44d0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fad85ffb640 1 -- 172.21.15.19:0/2901528385 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fad94004240 con 0x5609aa4c44d0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fad85ffb640 1 -- 172.21.15.19:0/2901528385 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad9404e050 con 0x5609aa4c44d0 2024-05-17T09:22:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fadabba8740 10 client.25808 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609aa89e590 con 0x5609aa4c44d0 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.220+0000 7fad85ffb640 1 -- 172.21.15.19:0/2901528385 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad98015030 con 0x5609aa4c44d0 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fad85ffb640 1 -- 172.21.15.19:0/2901528385 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad9403c4d0 con 0x5609aa4c44d0 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fadabba8740 10 client.25808 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fadabba8740 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fadabba8740 20 client.25808 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fadabba8740 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4a3d80 0x5609aa8a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fadabba8740 4 client.25808 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609aa7fab80 con 0x5609aa4a3d80 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4a3d80 0x5609aa8a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4a3d80 0x5609aa8a1c90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e62ab7fe649cc79 server_cookie=948b6d5ec9479e60 in_seq=0 out_seq=0 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.221+0000 7fad85ffb640 10 client.25808 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.222+0000 7fad85ffb640 1 -- 172.21.15.19:0/2901528385 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5609aa7fab80 con 0x5609aa4a3d80 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.222+0000 7fad85ffb640 10 client.25808 handle_command_reply: tid=0 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.222+0000 7fadabba8740 1 client.25808 shutdown 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 msgr2=0x5609aa894640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa894640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad94031460 tx=0x7fad94031870 comp rx=0 tx=0).stop 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 shutdown_connections 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4a3d80 0x5609aa8a1c90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4e51b0 0x5609aa89b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa894640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fad85ffb640 0 client.25808 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 --2- 172.21.15.19:0/2901528385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4bc2f0 0x5609aa894100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 >> 172.21.15.19:0/2901528385 conn(0x5609aa4d9300 msgr2=0x5609aa880d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 shutdown_connections 2024-05-17T09:22:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 1 -- 172.21.15.19:0/2901528385 wait complete. 2024-05-17T09:22:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.223+0000 7fadabba8740 20 client.25808 trim_cache size 0 max 16384 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 Processor -- start 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 -- start start 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4a3d80 0x5609aa87ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa89b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 0x5609aa8859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ad130 con 0x5609aa4c44d0 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fada01aaa00 con 0x5609aa4a3d80 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.238+0000 7fadabba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fada01a8850 con 0x5609aa4bc2f0 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 0x5609aa8859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 0x5609aa8859f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52846/0 (socket says 172.21.15.19:52846) 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4067980153 learned_addr learned my addr 172.21.15.19:0/4067980153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4a3d80 0x5609aa87ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa89b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4067980153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 msgr2=0x5609aa89b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa89b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4067980153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4a3d80 msgr2=0x5609aa87ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4a3d80 0x5609aa87ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4067980153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609aa891fe0 con 0x5609aa4c44d0 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 0x5609aa8859f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fad98009700 tx=0x7fad98005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302b5bf61c982bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad87fff640 1 -- 172.21.15.19:0/4067980153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98023040 con 0x5609aa4c44d0 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad87fff640 1 -- 172.21.15.19:0/4067980153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad98015070 con 0x5609aa4c44d0 2024-05-17T09:22:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.239+0000 7fad87fff640 1 -- 172.21.15.19:0/4067980153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980047a0 con 0x5609aa4c44d0 2024-05-17T09:22:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 -- 172.21.15.19:0/4067980153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 msgr2=0x5609aa8859f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 0x5609aa8859f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fad98009700 tx=0x7fad98005ad0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 -- 172.21.15.19:0/4067980153 shutdown_connections 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4c44d0 0x5609aa8859f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa89b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 --2- 172.21.15.19:0/4067980153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4a3d80 0x5609aa87ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 -- 172.21.15.19:0/4067980153 >> 172.21.15.19:0/4067980153 conn(0x5609aa4d9300 msgr2=0x5609aa889970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 -- 172.21.15.19:0/4067980153 shutdown_connections 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.240+0000 7fadabba8740 1 -- 172.21.15.19:0/4067980153 wait complete. 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 -- start start 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 Processor -- start 2024-05-17T09:22:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 0x5609aa898840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa898d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa899cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada01a8ec0 con 0x5609aa4a3d80 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fada01aa330 con 0x5609aa4c44d0 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fada01a9590 con 0x5609aa4bc2f0 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fadabba8740 10 client.0 _ll_register_callbacks cb 0x7fad94031500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fad9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 0x5609aa898840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fad9ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 0x5609aa898840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52858/0 (socket says 172.21.15.19:52858) 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4034922110 learned_addr learned my addr 172.21.15.19:0/4034922110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa899cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.241+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa898d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4034922110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 msgr2=0x5609aa898d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa898d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 msgr2=0x5609aa899cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa899cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad9ed9d640 1 -- 172.21.15.19:0/4034922110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609aa89a270 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad9ed9d640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 0x5609aa898840 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fad98005aa0 tx=0x7fad980021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f769108147d4d9da server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad85ffb640 1 -- 172.21.15.19:0/4034922110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98010cd0 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad85ffb640 1 -- 172.21.15.19:0/4034922110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fad98005050 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad98015070 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad85ffb640 1 -- 172.21.15.19:0/4034922110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad98010e70 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fadabba8740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609aa89b500 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.242+0000 7fad85ffb640 1 -- 172.21.15.19:0/4034922110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad98015070 con 0x5609aa4a3d80 2024-05-17T09:22:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fad85ffb640 1 -- 172.21.15.19:0/4034922110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad98029040 con 0x5609aa4a3d80 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fadabba8740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fadabba8740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fadabba8740 20 client.16131 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fadabba8740 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4e51b0 0x5609aa90c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fadabba8740 4 client.16131 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609aa7f4360 con 0x5609aa4e51b0 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4e51b0 0x5609aa90c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fad85ffb640 10 client.16131 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.243+0000 7fad9dd9b640 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4e51b0 0x5609aa90c590 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ccd36dc96a3403 server_cookie=a955207182383031 in_seq=0 out_seq=0 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fad85ffb640 1 -- 172.21.15.19:0/4034922110 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5609aa7f4360 con 0x5609aa4e51b0 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fad85ffb640 10 client.16131 handle_command_reply: tid=0 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 client.16131 shutdown 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 msgr2=0x5609aa898840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 0x5609aa898840 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fad98005aa0 tx=0x7fad980021f0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 shutdown_connections 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609aa4e51b0 0x5609aa90c590 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609aa4c44d0 0x5609aa899cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609aa4bc2f0 0x5609aa898d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 --2- 172.21.15.19:0/4034922110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609aa4a3d80 0x5609aa898840 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fad85ffb640 0 client.16131 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 >> 172.21.15.19:0/4034922110 conn(0x5609aa4d9300 msgr2=0x5609aa889970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 shutdown_connections 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 1 -- 172.21.15.19:0/4034922110 wait complete. 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.311+0000 7fadabba8740 20 client.16131 trim_cache size 0 max 16384 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:22:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:22:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad880b28a0 msgr2=0x7fad880b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad880b28a0 0x7fad880b4db0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5609aa88bb90 tx=0x7fad900e6620 comp rx=0 tx=0).stop 2024-05-17T09:22:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 msgr2=0x7fada019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 0x7fada019a1f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fad9000e9c0 tx=0x7fad9000ee80 comp rx=0 tx=0).stop 2024-05-17T09:22:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 shutdown_connections 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fada0109a10 0x7fada01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fada01070b0 0x7fada019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad880b28a0 0x7fad880b4db0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 --2- 172.21.15.19:0/767985569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada0104750 0x7fada019a1f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 >> 172.21.15.19:0/767985569 conn(0x7fada00fdd50 msgr2=0x7fada0105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 shutdown_connections 2024-05-17T09:22:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:07.312+0000 7fada4c1e640 1 -- 172.21.15.19:0/767985569 wait complete. 2024-05-17T09:22:07.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 3523 7.67 MB/sec warmup 5 sec latency 53.040 ms 2024-05-17T09:22:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 -- 172.21.15.19:0/2851327470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c103a50 msgr2=0x7fb08c110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 --2- 172.21.15.19:0/2851327470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c103a50 0x7fb08c110770 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb080009990 tx=0x7fb080009ca0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 -- 172.21.15.19:0/2851327470 shutdown_connections 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 --2- 172.21.15.19:0/2851327470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c110cb0 0x7fb08c1130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 --2- 172.21.15.19:0/2851327470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103100 0x7fb08c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 --2- 172.21.15.19:0/2851327470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c103a50 0x7fb08c110770 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 -- 172.21.15.19:0/2851327470 >> 172.21.15.19:0/2851327470 conn(0x7fb08c078df0 msgr2=0x7fb08c079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.417+0000 7fb091d92700 1 -- 172.21.15.19:0/2851327470 shutdown_connections 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.418+0000 7fb091d92700 1 -- 172.21.15.19:0/2851327470 wait complete. 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.418+0000 7fb091d92700 1 Processor -- start 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 -- start start 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c103100 0x7fb08c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 0x7fb08c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 0x7fb08c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c116530 con 0x7fb08c110cb0 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1163b0 con 0x7fb08c103a50 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb091d92700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1166b0 con 0x7fb08c103100 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 0x7fb08c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 0x7fb08c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c103100 0x7fb08c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 0x7fb08c1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52882/0 (socket says 172.21.15.19:52882) 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 0x7fb08c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43764/0 (socket says 172.21.15.19:43764) 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08bfff700 1 -- 172.21.15.19:0/2976590525 learned_addr learned my addr 172.21.15.19:0/2976590525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08affd700 1 -- 172.21.15.19:0/2976590525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c103100 msgr2=0x7fb08c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08affd700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c103100 0x7fb08c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08affd700 1 -- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 msgr2=0x7fb08c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.419+0000 7fb08affd700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 0x7fb08c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb08affd700 1 -- 172.21.15.19:0/2976590525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c1a1df0 con 0x7fb08c103a50 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb08bfff700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 0x7fb08c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb08affd700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 0x7fb08c19d4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb080000c00 tx=0x7fb0800047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb088ff9700 1 -- 172.21.15.19:0/2976590525 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080026070 con 0x7fb08c103a50 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb088ff9700 1 -- 172.21.15.19:0/2976590525 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb080021410 con 0x7fb08c103a50 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb088ff9700 1 -- 172.21.15.19:0/2976590525 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080017650 con 0x7fb08c103a50 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c1a2080 con 0x7fb08c103a50 2024-05-17T09:22:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.420+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a2560 con 0x7fb08c103a50 2024-05-17T09:22:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.423+0000 7fb088ff9700 1 -- 172.21.15.19:0/2976590525 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb080029070 con 0x7fb08c103a50 2024-05-17T09:22:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.424+0000 7fb088ff9700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0740b2880 0x7fb0740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.424+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0740b2880 0x7fb0740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.424+0000 7fb088ff9700 1 -- 172.21.15.19:0/2976590525 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0800e32f0 con 0x7fb08c103a50 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.424+0000 7fb0a07a7b80 1 Processor -- start 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.424+0000 7fb0a07a7b80 1 -- start start 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.424+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0740b2880 0x7fb0740b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009c60 tx=0x7fb07c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f4060cd70 con 0x559f4055fe40 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f405fc910 con 0x559f406ebbb0 2024-05-17T09:22:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f40600950 con 0x559f4064b430 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:51644/0 (socket says 172.21.15.19:51644) 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08b7fe700 1 -- 172.21.15.19:0/4242669419 learned_addr learned my addr 172.21.15.19:0/4242669419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08bfff700 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 -- 172.21.15.19:0/4242669419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 msgr2=0x559f406e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 -- 172.21.15.19:0/4242669419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 msgr2=0x559f406ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 -- 172.21.15.19:0/4242669419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f405a5fb0 con 0x559f4055fe40 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08bfff700 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.425+0000 7fb08affd700 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406e7d20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b2e0 tx=0x7fb0800a9e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134c44d51b453a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.426+0000 7fb071ffb700 1 -- 172.21.15.19:0/4242669419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800aacf0 con 0x559f4055fe40 2024-05-17T09:22:07.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.426+0000 7fb071ffb700 1 -- 172.21.15.19:0/4242669419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0800aa5f0 con 0x559f4055fe40 2024-05-17T09:22:07.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.426+0000 7fb071ffb700 1 -- 172.21.15.19:0/4242669419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800aa850 con 0x559f4055fe40 2024-05-17T09:22:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4242669419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 msgr2=0x559f406e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406e7d20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b2e0 tx=0x7fb0800a9e60 comp rx=0 tx=0).stop 2024-05-17T09:22:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4242669419 shutdown_connections 2024-05-17T09:22:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4242669419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406e7d20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4242669419 >> 172.21.15.19:0/4242669419 conn(0x559f405c01c0 msgr2=0x559f406d97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4242669419 shutdown_connections 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4242669419 wait complete. 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 -- start start 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.427+0000 7fb0a07a7b80 1 Processor -- start 2024-05-17T09:22:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4055fe40 0x559f406f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f406f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 0x559f406f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f40502fa0 con 0x559f406ebbb0 2024-05-17T09:22:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f40564da0 con 0x559f4064b430 2024-05-17T09:22:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f406ec720 con 0x559f4055fe40 2024-05-17T09:22:07.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb0a07a7b80 10 client.0 _ll_register_callbacks cb 0x559f40597950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 0x559f406f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f406f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4055fe40 0x559f406f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 0x559f406f0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52894/0 (socket says 172.21.15.19:52894) 2024-05-17T09:22:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08bfff700 1 -- 172.21.15.19:0/2766148218 learned_addr learned my addr 172.21.15.19:0/2766148218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08affd700 1 -- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4055fe40 msgr2=0x559f406f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08affd700 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4055fe40 0x559f406f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08affd700 1 -- 172.21.15.19:0/2766148218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 msgr2=0x559f406f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08affd700 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 0x559f406f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08affd700 1 -- 172.21.15.19:0/2766148218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f4054c8d0 con 0x559f4064b430 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.428+0000 7fb08bfff700 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 0x559f406f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb08affd700 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f406f6de0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0800e0db0 tx=0x7fb08000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2d13822006eeff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb06affd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb06affd700 1 -- 172.21.15.19:0/2766148218 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800c3020 con 0x559f4064b430 2024-05-17T09:22:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0800aa5f0 con 0x559f4064b430 2024-05-17T09:22:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb06affd700 1 -- 172.21.15.19:0/2766148218 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0800aaa60 con 0x559f4064b430 2024-05-17T09:22:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb06affd700 1 -- 172.21.15.19:0/2766148218 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0800aacf0 con 0x559f4064b430 2024-05-17T09:22:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 10 client.25826 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f405b6760 con 0x559f4064b430 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb06affd700 1 -- 172.21.15.19:0/2766148218 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0800aa5f0 con 0x559f4064b430 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb06affd700 1 -- 172.21.15.19:0/2766148218 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0800a9850 con 0x559f4064b430 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 10 client.25826 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 10 client.25826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 20 client.25826 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f407007c0 0x559f40702be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 4 client.25826 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.429+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f406aaed0 con 0x559f407007c0 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.430+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f407007c0 0x559f40702be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.430+0000 7fb06affd700 10 client.25826 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.430+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f407007c0 0x559f40702be0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41b2b876720efe18 server_cookie=6ae4402732d03629 in_seq=0 out_seq=0 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.431+0000 7fb06affd700 1 -- 172.21.15.19:0/2766148218 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f406aaed0 con 0x559f407007c0 2024-05-17T09:22:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.431+0000 7fb06affd700 10 client.25826 handle_command_reply: tid=0 2024-05-17T09:22:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.431+0000 7fb0a07a7b80 1 client.25826 shutdown 2024-05-17T09:22:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.431+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 msgr2=0x559f406f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.431+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f406f6de0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0800e0db0 tx=0x7fb08000bef0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 shutdown_connections 2024-05-17T09:22:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f407007c0 0x559f40702be0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4055fe40 0x559f406f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f406ebbb0 0x559f406f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/2766148218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f406f6de0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb06affd700 0 client.25826 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 >> 172.21.15.19:0/2766148218 conn(0x559f405c01c0 msgr2=0x559f406d9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 shutdown_connections 2024-05-17T09:22:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/2766148218 wait complete. 2024-05-17T09:22:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.432+0000 7fb0a07a7b80 20 client.25826 trim_cache size 0 max 16384 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 Processor -- start 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 -- start start 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f407081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f406ebbb0 0x559f407086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1b0850 con 0x559f4055fe40 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ae0e0 con 0x559f4064b430 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.450+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1abf30 con 0x559f406ebbb0 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406ebfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52904/0 (socket says 172.21.15.19:52904) 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 -- 172.21.15.19:0/4225120569 learned_addr learned my addr 172.21.15.19:0/4225120569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08affd700 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f407081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08bfff700 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f406ebbb0 0x559f407086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 -- 172.21.15.19:0/4225120569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f406ebbb0 msgr2=0x559f407086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f406ebbb0 0x559f407086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 -- 172.21.15.19:0/4225120569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 msgr2=0x559f407081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f407081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 -- 172.21.15.19:0/4225120569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f406f16d0 con 0x559f4055fe40 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08affd700 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f407081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406ebfc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x559f40703150 tx=0x7fb07c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e8eabfe5773559 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb0717fa700 1 -- 172.21.15.19:0/4225120569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c004560 con 0x559f4055fe40 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.451+0000 7fb0717fa700 1 -- 172.21.15.19:0/4225120569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb07c0338c0 con 0x559f4055fe40 2024-05-17T09:22:07.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.452+0000 7fb0717fa700 1 -- 172.21.15.19:0/4225120569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c01d740 con 0x559f4055fe40 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.452+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4225120569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 msgr2=0x559f406ebfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.452+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406ebfc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x559f40703150 tx=0x7fb07c014000 comp rx=0 tx=0).stop 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4225120569 shutdown_connections 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f406ebbb0 0x559f407086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406ebfc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/4225120569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f4064b430 0x559f407081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4225120569 >> 172.21.15.19:0/4225120569 conn(0x559f405c01c0 msgr2=0x559f406ef590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4225120569 shutdown_connections 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/4225120569 wait complete. 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 -- start start 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.453+0000 7fb0a07a7b80 1 Processor -- start 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ac5a0 con 0x559f4055fe40 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ada10 con 0x559f406ebbb0 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1acc70 con 0x559f4064b430 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb0a07a7b80 10 client.0 _ll_register_callbacks cb 0x559f40597950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52914/0 (socket says 172.21.15.19:52914) 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 -- 172.21.15.19:0/3784772360 learned_addr learned my addr 172.21.15.19:0/3784772360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08bfff700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 -- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 msgr2=0x559f406fa500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406fa500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 -- 172.21.15.19:0/3784772360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 msgr2=0x559f406faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08b7fe700 1 -- 172.21.15.19:0/3784772360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f406fb120 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.454+0000 7fb08bfff700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb08b7fe700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406fd130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005920 tx=0x7fb07c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e117d26ac1930b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb06affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb06affd700 1 -- 172.21.15.19:0/3784772360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c041060 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07c0338c0 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb06affd700 1 -- 172.21.15.19:0/3784772360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb07c033c70 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb06affd700 1 -- 172.21.15.19:0/3784772360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb07c048050 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb0a07a7b80 10 client.16149 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f4076af30 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb06affd700 1 -- 172.21.15.19:0/3784772360 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb07c0338c0 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.455+0000 7fb06affd700 1 -- 172.21.15.19:0/3784772360 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb07c004980 con 0x559f4055fe40 2024-05-17T09:22:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb0a07a7b80 10 client.16149 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb0a07a7b80 10 client.16149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:22:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb0a07a7b80 20 client.16149 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f4076b590 0x559f4076d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb0a07a7b80 4 client.16149 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:22:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f4060ecc0 con 0x559f4076b590 2024-05-17T09:22:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb08affd700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f4076b590 0x559f4076d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb06affd700 10 client.16149 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.456+0000 7fb08affd700 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f4076b590 0x559f4076d9b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8f4349d4d8646e0 server_cookie=b50557f38f8fcdc2 in_seq=0 out_seq=0 2024-05-17T09:22:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb06affd700 1 -- 172.21.15.19:0/3784772360 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f4060ecc0 con 0x559f4076b590 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb06affd700 10 client.16149 handle_command_reply: tid=0 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 client.16149 shutdown 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 msgr2=0x559f406fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406fd130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005920 tx=0x7fb07c003ef0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 shutdown_connections 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x559f4076b590 0x559f4076d9b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f406ebbb0 0x559f406faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f4055fe40 0x559f406fd130 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 --2- 172.21.15.19:0/3784772360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f4064b430 0x559f406fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb06affd700 0 client.16149 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 >> 172.21.15.19:0/3784772360 conn(0x559f405c01c0 msgr2=0x559f406ea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 shutdown_connections 2024-05-17T09:22:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 1 -- 172.21.15.19:0/3784772360 wait complete. 2024-05-17T09:22:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.457+0000 7fb0a07a7b80 20 client.16149 trim_cache size 0 max 16384 2024-05-17T09:22:07.464 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:22:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0740b2880 msgr2=0x7fb0740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0740b2880 0x7fb0740b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009c60 tx=0x7fb07c009260 comp rx=0 tx=0).stop 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 msgr2=0x7fb08c19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 0x7fb08c19d4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb080000c00 tx=0x7fb0800047b0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 shutdown_connections 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0740b2880 0x7fb0740b4d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c103100 0x7fb08c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c110cb0 0x7fb08c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 --2- 172.21.15.19:0/2976590525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb08c103a50 0x7fb08c19d4c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 >> 172.21.15.19:0/2976590525 conn(0x7fb08c078df0 msgr2=0x7fb08c1121d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 shutdown_connections 2024-05-17T09:22:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:07.458+0000 7fb091d92700 1 -- 172.21.15.19:0/2976590525 wait complete. 2024-05-17T09:22:08.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:22:08.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 4020 6.68 MB/sec warmup 6 sec latency 12.543 ms 2024-05-17T09:22:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:08 smithi069 ceph-mon[35081]: pgmap v509: 129 pgs: 129 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 28 op/s 2024-05-17T09:22:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:08 smithi019 ceph-mon[31620]: pgmap v509: 129 pgs: 129 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 28 op/s 2024-05-17T09:22:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:08 smithi079 ceph-mon[35502]: pgmap v509: 129 pgs: 129 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 28 op/s 2024-05-17T09:22:09.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 4552 5.86 MB/sec warmup 7 sec latency 21.011 ms 2024-05-17T09:22:10.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 5227 5.67 MB/sec warmup 8 sec latency 22.166 ms 2024-05-17T09:22:10.515 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:10 smithi019 ceph-mon[31620]: pgmap v510: 129 pgs: 129 active+clean; 24 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.3 MiB/s wr, 59 op/s 2024-05-17T09:22:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 -- 172.21.15.19:0/1343828098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 msgr2=0x7f472c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 --2- 172.21.15.19:0/1343828098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c1081f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f47140099c0 tx=0x7f4714009cd0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 -- 172.21.15.19:0/1343828098 shutdown_connections 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 --2- 172.21.15.19:0/1343828098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c108730 0x7f472c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 --2- 172.21.15.19:0/1343828098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c1034b0 0x7f472c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 --2- 172.21.15.19:0/1343828098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c1081f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 -- 172.21.15.19:0/1343828098 >> 172.21.15.19:0/1343828098 conn(0x7f472c0fedb0 msgr2=0x7f472c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 -- 172.21.15.19:0/1343828098 shutdown_connections 2024-05-17T09:22:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.546+0000 7f4731223700 1 -- 172.21.15.19:0/1343828098 wait complete. 2024-05-17T09:22:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.547+0000 7f4731223700 1 Processor -- start 2024-05-17T09:22:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 -- start start 2024-05-17T09:22:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c1034b0 0x7f472c0ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c0ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c108730 0x7f472c0ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1131f0 con 0x7f472c107de0 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f472c113070 con 0x7f472c108730 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f4731223700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f472c113370 con 0x7f472c1034b0 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f472259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c0ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c1034b0 0x7f472c0ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f472b59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c108730 0x7f472c0ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c1034b0 0x7f472c0ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:51672/0 (socket says 172.21.15.19:51672) 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f472259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c0ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52928/0 (socket says 172.21.15.19:52928) 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.548+0000 7f472ad9d700 1 -- 172.21.15.19:0/1497252917 learned_addr learned my addr 172.21.15.19:0/1497252917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f472b59e700 1 -- 172.21.15.19:0/1497252917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c1034b0 msgr2=0x7f472c0ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f472b59e700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c1034b0 0x7f472c0ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f472b59e700 1 -- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 msgr2=0x7f472c0ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f472b59e700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c0ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f472b59e700 1 -- 172.21.15.19:0/1497252917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c1a9070 con 0x7f472c108730 2024-05-17T09:22:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f472b59e700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c108730 0x7f472c0ffd10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4724007f10 tx=0x7f472400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f4728d99700 1 -- 172.21.15.19:0/1497252917 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724004db0 con 0x7f472c108730 2024-05-17T09:22:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f4728d99700 1 -- 172.21.15.19:0/1497252917 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4724009e20 con 0x7f472c108730 2024-05-17T09:22:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f4728d99700 1 -- 172.21.15.19:0/1497252917 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724015400 con 0x7f472c108730 2024-05-17T09:22:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c1a9360 con 0x7f472c108730 2024-05-17T09:22:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.549+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a9870 con 0x7f472c108730 2024-05-17T09:22:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.552+0000 7f4728d99700 1 -- 172.21.15.19:0/1497252917 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4724005240 con 0x7f472c108730 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.553+0000 7f4728d99700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2900 0x7f47180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.553+0000 7f472ad9d700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2900 0x7f47180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.553+0000 7f4728d99700 1 -- 172.21.15.19:0/1497252917 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f47240e1e80 con 0x7f472c108730 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.553+0000 7f473fc38b80 1 Processor -- start 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.553+0000 7f473fc38b80 1 -- start start 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2900 0x7f47180b4e10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f471c009d20 tx=0x7f471c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d65910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d65eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f473fc38b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560664c8dd70 con 0x560664d5fcd0 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f473fc38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560664c7d910 con 0x560664ccc430 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f473fc38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560664c81950 con 0x560664be0e40 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d65910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d65eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d65910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43828/0 (socket says 172.21.15.19:43828) 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 -- 172.21.15.19:0/872243105 learned_addr learned my addr 172.21.15.19:0/872243105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472b59e700 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 -- 172.21.15.19:0/872243105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 msgr2=0x560664d65eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d65eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 -- 172.21.15.19:0/872243105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 msgr2=0x560664d601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472ad9d700 1 -- 172.21.15.19:0/872243105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560664c26fb0 con 0x560664ccc430 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.554+0000 7f472b59e700 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.555+0000 7f472ad9d700 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d65910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e540 tx=0x7f471c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e48b7e717221724 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.555+0000 7f472159a700 1 -- 172.21.15.19:0/872243105 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c026430 con 0x560664ccc430 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.555+0000 7f472159a700 1 -- 172.21.15.19:0/872243105 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f471c01c8a0 con 0x560664ccc430 2024-05-17T09:22:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.555+0000 7f472159a700 1 -- 172.21.15.19:0/872243105 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01d7e0 con 0x560664ccc430 2024-05-17T09:22:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 -- 172.21.15.19:0/872243105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 msgr2=0x560664d65910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d65910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e540 tx=0x7f471c014000 comp rx=0 tx=0).stop 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 -- 172.21.15.19:0/872243105 shutdown_connections 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d65910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 --2- 172.21.15.19:0/872243105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d65eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 -- 172.21.15.19:0/872243105 >> 172.21.15.19:0/872243105 conn(0x560664c411c0 msgr2=0x560664d5a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 -- 172.21.15.19:0/872243105 shutdown_connections 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 -- 172.21.15.19:0/872243105 wait complete. 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.556+0000 7f473fc38b80 1 -- start start 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 Processor -- start 2024-05-17T09:22:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664be0e40 0x560664d69600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d69b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664d5fcd0 0x560664d6a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560664b83fa0 con 0x560664be0e40 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560664be5da0 con 0x560664ccc430 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560664d6c5a0 con 0x560664d5fcd0 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664be0e40 0x560664d69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f473fc38b80 10 client.0 _ll_register_callbacks cb 0x560664c18950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f472259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d69b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664be0e40 0x560664d69600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52946/0 (socket says 172.21.15.19:52946) 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f472ad9d700 1 -- 172.21.15.19:0/3929611561 learned_addr learned my addr 172.21.15.19:0/3929611561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.557+0000 7f472b59e700 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664d5fcd0 0x560664d6a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f472259c700 1 -- 172.21.15.19:0/3929611561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664d5fcd0 msgr2=0x560664d6a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f472259c700 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664d5fcd0 0x560664d6a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f472259c700 1 -- 172.21.15.19:0/3929611561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664be0e40 msgr2=0x560664d69600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f472259c700 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664be0e40 0x560664d69600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f472259c700 1 -- 172.21.15.19:0/3929611561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560664bcd8d0 con 0x560664ccc430 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f472259c700 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d69b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f471400b580 tx=0x7f4714005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82f563896d133a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f47127fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f47127fc700 1 -- 172.21.15.19:0/3929611561 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140039c0 con 0x560664ccc430 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f47127fc700 1 -- 172.21.15.19:0/3929611561 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4714003b60 con 0x560664ccc430 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f471c01c8a0 con 0x560664ccc430 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.558+0000 7f47127fc700 1 -- 172.21.15.19:0/3929611561 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4714031050 con 0x560664ccc430 2024-05-17T09:22:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 10 client.25844 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560664c37760 con 0x560664ccc430 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f47127fc700 1 -- 172.21.15.19:0/3929611561 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f471c01c8a0 con 0x560664ccc430 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f47127fc700 1 -- 172.21.15.19:0/3929611561 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f471402c030 con 0x560664ccc430 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 10 client.25844 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 20 client.25844 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664d794f0 0x560664d7b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 4 client.25844 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560664d2bed0 con 0x560664d794f0 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.559+0000 7f472ad9d700 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664d794f0 0x560664d7b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.560+0000 7f47127fc700 10 client.25844 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.560+0000 7f472ad9d700 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664d794f0 0x560664d7b930 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70b5589d60f72e7 server_cookie=3ee8ca4eced1abaa in_seq=0 out_seq=0 2024-05-17T09:22:10.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f47127fc700 1 -- 172.21.15.19:0/3929611561 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560664d2bed0 con 0x560664d794f0 2024-05-17T09:22:10.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f47127fc700 10 client.25844 handle_command_reply: tid=0 2024-05-17T09:22:10.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 client.25844 shutdown 2024-05-17T09:22:10.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 msgr2=0x560664d69b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d69b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f471400b580 tx=0x7f4714005790 comp rx=0 tx=0).stop 2024-05-17T09:22:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 shutdown_connections 2024-05-17T09:22:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664d5fcd0 0x560664d6a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664d794f0 0x560664d7b930 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664be0e40 0x560664d69600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 --2- 172.21.15.19:0/3929611561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664ccc430 0x560664d69b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 >> 172.21.15.19:0/3929611561 conn(0x560664c411c0 msgr2=0x560664d5a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f47127fc700 0 client.25844 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 shutdown_connections 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 1 -- 172.21.15.19:0/3929611561 wait complete. 2024-05-17T09:22:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.561+0000 7f473fc38b80 20 client.25844 trim_cache size 0 max 16384 2024-05-17T09:22:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 Processor -- start 2024-05-17T09:22:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 -- start start 2024-05-17T09:22:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d68600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664ccc430 0x560664d695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 0x560664d69b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1b4c00 con 0x560664ccc430 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1b2490 con 0x560664d5fcd0 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f473fc38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1b02e0 con 0x560664be0e40 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f472259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664ccc430 0x560664d695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f472b59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 0x560664d69b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f472b59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 0x560664d69b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43854/0 (socket says 172.21.15.19:43854) 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f472b59e700 1 -- 172.21.15.19:0/2923788716 learned_addr learned my addr 172.21.15.19:0/2923788716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.580+0000 7f472259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664ccc430 0x560664d695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52960/0 (socket says 172.21.15.19:52960) 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472ad9d700 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d68600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472b59e700 1 -- 172.21.15.19:0/2923788716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 msgr2=0x560664d68600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472b59e700 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d68600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472b59e700 1 -- 172.21.15.19:0/2923788716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664ccc430 msgr2=0x560664d695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472b59e700 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664ccc430 0x560664d695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472b59e700 1 -- 172.21.15.19:0/2923788716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560664d5ad00 con 0x560664d5fcd0 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f472b59e700 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 0x560664d69b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4724007ee0 tx=0x7f47240a4f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab96711595992e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f4720d99700 1 -- 172.21.15.19:0/2923788716 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240ad480 con 0x560664d5fcd0 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f4720d99700 1 -- 172.21.15.19:0/2923788716 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47240b1f30 con 0x560664d5fcd0 2024-05-17T09:22:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.581+0000 7f4720d99700 1 -- 172.21.15.19:0/2923788716 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240b1aa0 con 0x560664d5fcd0 2024-05-17T09:22:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 -- 172.21.15.19:0/2923788716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 msgr2=0x560664d69b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 0x560664d69b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4724007ee0 tx=0x7f47240a4f90 comp rx=0 tx=0).stop 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 -- 172.21.15.19:0/2923788716 shutdown_connections 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664be0e40 0x560664d68600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664d5fcd0 0x560664d69b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 --2- 172.21.15.19:0/2923788716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664ccc430 0x560664d695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 -- 172.21.15.19:0/2923788716 >> 172.21.15.19:0/2923788716 conn(0x560664c411c0 msgr2=0x560664d67ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 -- 172.21.15.19:0/2923788716 shutdown_connections 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.582+0000 7f473fc38b80 1 -- 172.21.15.19:0/2923788716 wait complete. 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 -- start start 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 Processor -- start 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 0x560664d71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664ccc430 0x560664d71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1b0950 con 0x560664d5fcd0 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1b1dc0 con 0x560664be0e40 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1b1020 con 0x560664ccc430 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.583+0000 7f473fc38b80 10 client.0 _ll_register_callbacks cb 0x560664c18950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 0x560664d71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 0x560664d71a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:43864/0 (socket says 172.21.15.19:43864) 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 -- 172.21.15.19:0/18650581 learned_addr learned my addr 172.21.15.19:0/18650581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472b59e700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472259c700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664ccc430 0x560664d71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 -- 172.21.15.19:0/18650581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664ccc430 msgr2=0x560664d71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664ccc430 0x560664d71fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 -- 172.21.15.19:0/18650581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 msgr2=0x560664d6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d6aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 -- 172.21.15.19:0/18650581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560664d6b1a0 con 0x560664be0e40 2024-05-17T09:22:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472259c700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664ccc430 0x560664d71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f472ad9d700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 0x560664d71a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f471c009be0 tx=0x7f471c011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11d92d2fcae6f3d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f47127fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f47127fc700 1 -- 172.21.15.19:0/18650581 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c005040 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47240b1f30 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.584+0000 7f47127fc700 1 -- 172.21.15.19:0/18650581 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f471c0051e0 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f47127fc700 1 -- 172.21.15.19:0/18650581 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f471c042050 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 10 client.25850 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560664de3a20 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f47127fc700 1 -- 172.21.15.19:0/18650581 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47240b1f30 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f47127fc700 1 -- 172.21.15.19:0/18650581 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f471c01c690 con 0x560664be0e40 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 10 client.25850 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 20 client.25850 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664de3e20 0x560664de6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 4 client.25850 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560664c8fcc0 con 0x560664de3e20 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.585+0000 7f472259c700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664de3e20 0x560664de6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f47127fc700 10 client.25850 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f472259c700 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664de3e20 0x560664de6240 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee93cc54986098b server_cookie=89fa1b4c5e4662b0 in_seq=0 out_seq=0 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f47127fc700 1 -- 172.21.15.19:0/18650581 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560664c8fcc0 con 0x560664de3e20 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f47127fc700 10 client.25850 handle_command_reply: tid=0 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 client.25850 shutdown 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 msgr2=0x560664d71a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 0x560664d71a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f471c009be0 tx=0x7f471c011bb0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 shutdown_connections 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560664de3e20 0x560664de6240 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560664d5fcd0 0x560664d6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560664be0e40 0x560664d71a60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 --2- 172.21.15.19:0/18650581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560664ccc430 0x560664d71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 >> 172.21.15.19:0/18650581 conn(0x560664c411c0 msgr2=0x560664d54800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f47127fc700 0 client.25850 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 shutdown_connections 2024-05-17T09:22:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 1 -- 172.21.15.19:0/18650581 wait complete. 2024-05-17T09:22:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.586+0000 7f473fc38b80 20 client.25850 trim_cache size 0 max 16384 2024-05-17T09:22:10.592 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:22:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2900 msgr2=0x7f47180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2900 0x7f47180b4e10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f471c009d20 tx=0x7f471c009260 comp rx=0 tx=0).stop 2024-05-17T09:22:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c108730 msgr2=0x7f472c0ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c108730 0x7f472c0ffd10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4724007f10 tx=0x7f472400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 shutdown_connections 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47180b2900 0x7f47180b4e10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f472c1034b0 0x7f472c0ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c108730 0x7f472c0ffd10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 --2- 172.21.15.19:0/1497252917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f472c107de0 0x7f472c0ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 >> 172.21.15.19:0/1497252917 conn(0x7f472c0fedb0 msgr2=0x7f472c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 shutdown_connections 2024-05-17T09:22:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:10.587+0000 7f4731223700 1 -- 172.21.15.19:0/1497252917 wait complete. 2024-05-17T09:22:10.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:10 smithi069 ceph-mon[35081]: pgmap v510: 129 pgs: 129 active+clean; 24 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.3 MiB/s wr, 59 op/s 2024-05-17T09:22:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:10 smithi079 ceph-mon[35502]: pgmap v510: 129 pgs: 129 active+clean; 24 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.3 MiB/s wr, 59 op/s 2024-05-17T09:22:11.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 5921 5.22 MB/sec warmup 9 sec latency 17.188 ms 2024-05-17T09:22:12.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 6759 5.34 MB/sec warmup 10 sec latency 40.744 ms 2024-05-17T09:22:12.397 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:22:12.397 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:22:12.664 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:12 smithi069 ceph-mon[35081]: pgmap v511: 129 pgs: 129 active+clean; 25 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.3 MiB/s wr, 66 op/s 2024-05-17T09:22:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:12 smithi019 ceph-mon[31620]: pgmap v511: 129 pgs: 129 active+clean; 25 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.3 MiB/s wr, 66 op/s 2024-05-17T09:22:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:12 smithi079 ceph-mon[35502]: pgmap v511: 129 pgs: 129 active+clean; 25 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.3 MiB/s wr, 66 op/s 2024-05-17T09:22:13.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 7231 5.26 MB/sec warmup 11 sec latency 19.080 ms 2024-05-17T09:22:14.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 7659 4.93 MB/sec warmup 12 sec latency 14.782 ms 2024-05-17T09:22:14.439 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:14 smithi019 ceph-mon[31620]: pgmap v512: 129 pgs: 129 active+clean; 24 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.6 MiB/s wr, 96 op/s 2024-05-17T09:22:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 -- 172.21.15.19:0/1454673804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc108750 msgr2=0x7f30cc1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/1454673804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc108750 0x7f30cc1100b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f30c40099c0 tx=0x7f30c4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 -- 172.21.15.19:0/1454673804 shutdown_connections 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/1454673804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc108750 0x7f30cc1100b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/1454673804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc1034d0 0x7f30cc1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/1454673804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30cc107e00 0x7f30cc108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 -- 172.21.15.19:0/1454673804 >> 172.21.15.19:0/1454673804 conn(0x7f30cc0fef20 msgr2=0x7f30cc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 -- 172.21.15.19:0/1454673804 shutdown_connections 2024-05-17T09:22:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.572+0000 7f30d2ff4700 1 -- 172.21.15.19:0/1454673804 wait complete. 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 Processor -- start 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 -- start start 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30cc1034d0 0x7f30cc19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc107e00 0x7f30cc19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 0x7f30cc19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc113280 con 0x7f30cc107e00 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc113100 con 0x7f30cc108750 2024-05-17T09:22:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d2ff4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc113400 con 0x7f30cc1034d0 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc107e00 0x7f30cc19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d1591700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 0x7f30cc19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d1591700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 0x7f30cc19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58100/0 (socket says 172.21.15.19:58100) 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30d1591700 1 -- 172.21.15.19:0/932415914 learned_addr learned my addr 172.21.15.19:0/932415914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.573+0000 7f30c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc107e00 0x7f30cc19c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45704/0 (socket says 172.21.15.19:45704) 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d0d90700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30cc1034d0 0x7f30cc19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d1591700 1 -- 172.21.15.19:0/932415914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30cc1034d0 msgr2=0x7f30cc19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d1591700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30cc1034d0 0x7f30cc19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d1591700 1 -- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc107e00 msgr2=0x7f30cc19c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d1591700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc107e00 0x7f30cc19c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d1591700 1 -- 172.21.15.19:0/932415914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc1a6f10 con 0x7f30cc108750 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d1591700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 0x7f30cc19c660 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005b40 tx=0x7f30c40055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30c1ffb700 1 -- 172.21.15.19:0/932415914 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4026070 con 0x7f30cc108750 2024-05-17T09:22:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc1a71a0 con 0x7f30cc108750 2024-05-17T09:22:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30c1ffb700 1 -- 172.21.15.19:0/932415914 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c400f460 con 0x7f30cc108750 2024-05-17T09:22:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.574+0000 7f30c1ffb700 1 -- 172.21.15.19:0/932415914 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4009e60 con 0x7f30cc108750 2024-05-17T09:22:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.575+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a7650 con 0x7f30cc108750 2024-05-17T09:22:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.578+0000 7f30c1ffb700 1 -- 172.21.15.19:0/932415914 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f30c4029070 con 0x7f30cc108750 2024-05-17T09:22:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30c1ffb700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f30b40b2880 0x7f30b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 Processor -- start 2024-05-17T09:22:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 -- start start 2024-05-17T09:22:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30d0d90700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f30b40b2880 0x7f30b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30c1ffb700 1 -- 172.21.15.19:0/932415914 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f30c40e2f00 con 0x7f30cc108750 2024-05-17T09:22:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f07df4b0 0x5609f087cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f087d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 0x5609f0877570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609f07a0df0 con 0x5609f06f3ec0 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609f0790990 con 0x5609f0877160 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5609f07949d0 con 0x5609f07df4b0 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30d0d90700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f30b40b2880 0x7f30b40b4d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009da0 tx=0x7f30c800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30d1591700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 0x5609f0877570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f087d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30d1591700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 0x5609f0877570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58114/0 (socket says 172.21.15.19:58114) 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30d1591700 1 -- 172.21.15.19:0/3033373431 learned_addr learned my addr 172.21.15.19:0/3033373431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.579+0000 7f30d0d90700 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f07df4b0 0x5609f087cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30d1591700 1 -- 172.21.15.19:0/3033373431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f07df4b0 msgr2=0x5609f087cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30d1591700 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f07df4b0 0x5609f087cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30d1591700 1 -- 172.21.15.19:0/3033373431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 msgr2=0x5609f087d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30d1591700 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f087d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30d1591700 1 -- 172.21.15.19:0/3033373431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f073a030 con 0x5609f0877160 2024-05-17T09:22:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30d1591700 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 0x5609f0877570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f30c40a7470 tx=0x7f30c40a9c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf27e2964a7de7fc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30b2ffd700 1 -- 172.21.15.19:0/3033373431 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40aa900 con 0x5609f0877160 2024-05-17T09:22:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30b2ffd700 1 -- 172.21.15.19:0/3033373431 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c40aa200 con 0x5609f0877160 2024-05-17T09:22:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.580+0000 7f30b2ffd700 1 -- 172.21.15.19:0/3033373431 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40aa460 con 0x5609f0877160 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 -- 172.21.15.19:0/3033373431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 msgr2=0x5609f0877570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 0x5609f0877570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f30c40a7470 tx=0x7f30c40a9c80 comp rx=0 tx=0).stop 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 -- 172.21.15.19:0/3033373431 shutdown_connections 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f07df4b0 0x5609f087cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f0877160 0x5609f0877570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/3033373431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f087d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 -- 172.21.15.19:0/3033373431 >> 172.21.15.19:0/3033373431 conn(0x5609f0754240 msgr2=0x5609f086d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 -- 172.21.15.19:0/3033373431 shutdown_connections 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.581+0000 7f30e1a09b80 1 -- 172.21.15.19:0/3033373431 wait complete. 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 -- start start 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 Processor -- start 2024-05-17T09:22:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 0x5609f0886460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f07df4b0 0x5609f08869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0880860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609f0697020 con 0x5609f07df4b0 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5609f06f8e20 con 0x5609f06f3ec0 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5609f0877d10 con 0x5609f0877160 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.582+0000 7f30d0d90700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 0x5609f0886460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f07df4b0 0x5609f08869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 0x5609f0886460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58124/0 (socket says 172.21.15.19:58124) 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 -- 172.21.15.19:0/241710400 learned_addr learned my addr 172.21.15.19:0/241710400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30e1a09b80 10 client.0 _ll_register_callbacks cb 0x5609f072b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d1591700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0880860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 -- 172.21.15.19:0/241710400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 msgr2=0x5609f0880860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0880860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 -- 172.21.15.19:0/241710400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f07df4b0 msgr2=0x5609f08869a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f07df4b0 0x5609f08869a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 -- 172.21.15.19:0/241710400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f06e0950 con 0x5609f06f3ec0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d1591700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0880860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30d0d90700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 0x5609f0886460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30c8003fd0 tx=0x7f30c8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43410bf739ee25fb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30abfff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30abfff700 1 -- 172.21.15.19:0/241710400 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8027020 con 0x5609f06f3ec0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30abfff700 1 -- 172.21.15.19:0/241710400 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f30c8015cf0 con 0x5609f06f3ec0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c40aa200 con 0x5609f06f3ec0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.583+0000 7f30abfff700 1 -- 172.21.15.19:0/241710400 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30c8032050 con 0x5609f06f3ec0 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 10 client.25862 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609f074a7e0 con 0x5609f06f3ec0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30abfff700 1 -- 172.21.15.19:0/241710400 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c801e040 con 0x5609f06f3ec0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30abfff700 1 -- 172.21.15.19:0/241710400 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30c8023420 con 0x5609f06f3ec0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 20 client.25862 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08905d0 0x5609f0892a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 4 client.25862 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f30c40aa200 con 0x5609f08905d0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.584+0000 7f30c3fff700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08905d0 0x5609f0892a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.585+0000 7f30abfff700 10 client.25862 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.585+0000 7f30c3fff700 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08905d0 0x5609f0892a40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4413219f7c078 server_cookie=1e5a84b018110d4f in_seq=0 out_seq=0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30abfff700 1 -- 172.21.15.19:0/241710400 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f30c40aa200 con 0x5609f08905d0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30abfff700 10 client.25862 handle_command_reply: tid=0 2024-05-17T09:22:14.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 client.25862 shutdown 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 msgr2=0x5609f0886460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 0x5609f0886460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30c8003fd0 tx=0x7f30c8004ad0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 shutdown_connections 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0880860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08905d0 0x5609f0892a40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f06f3ec0 0x5609f0886460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/241710400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f07df4b0 0x5609f08869a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30abfff700 0 client.25862 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 >> 172.21.15.19:0/241710400 conn(0x5609f0754240 msgr2=0x5609f086d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 shutdown_connections 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.586+0000 7f30e1a09b80 1 -- 172.21.15.19:0/241710400 wait complete. 2024-05-17T09:22:14.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.587+0000 7f30e1a09b80 20 client.25862 trim_cache size 0 max 16384 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.607+0000 7f30e1a09b80 1 Processor -- start 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.607+0000 7f30e1a09b80 1 -- start start 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f0898200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f0898740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0871d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1b0890 con 0x5609f06f3ec0 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ae120 con 0x5609f07df4b0 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1abf90 con 0x5609f0877160 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f0898200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f0898200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45716/0 (socket says 172.21.15.19:45716) 2024-05-17T09:22:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 -- 172.21.15.19:0/196314054 learned_addr learned my addr 172.21.15.19:0/196314054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30c3fff700 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f0898740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d1591700 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0871d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 -- 172.21.15.19:0/196314054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 msgr2=0x5609f0871d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0871d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 -- 172.21.15.19:0/196314054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 msgr2=0x5609f0898740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f0898740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 -- 172.21.15.19:0/196314054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f087edf0 con 0x5609f06f3ec0 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30c3fff700 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f0898740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.608+0000 7f30d0d90700 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f0898200 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f30c8023a00 tx=0x7f30c8003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5aadb2c31bf03a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.609+0000 7f30b27fc700 1 -- 172.21.15.19:0/196314054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8027070 con 0x5609f06f3ec0 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.609+0000 7f30b27fc700 1 -- 172.21.15.19:0/196314054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c8023420 con 0x5609f06f3ec0 2024-05-17T09:22:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.609+0000 7f30b27fc700 1 -- 172.21.15.19:0/196314054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c801d030 con 0x5609f06f3ec0 2024-05-17T09:22:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 -- 172.21.15.19:0/196314054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 msgr2=0x5609f0898200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f0898200 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f30c8023a00 tx=0x7f30c8003a00 comp rx=0 tx=0).stop 2024-05-17T09:22:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 -- 172.21.15.19:0/196314054 shutdown_connections 2024-05-17T09:22:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f0871d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f0898200 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/196314054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f0898740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 -- 172.21.15.19:0/196314054 >> 172.21.15.19:0/196314054 conn(0x5609f0754240 msgr2=0x5609f087ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 -- 172.21.15.19:0/196314054 shutdown_connections 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 -- 172.21.15.19:0/196314054 wait complete. 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 -- start start 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.610+0000 7f30e1a09b80 1 Processor -- start 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f088bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f088c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f08fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ac5e0 con 0x5609f06f3ec0 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1ada50 con 0x5609f07df4b0 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1accb0 con 0x5609f0877160 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f088bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30e1a09b80 10 client.0 _ll_register_callbacks cb 0x5609f072b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f088bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45722/0 (socket says 172.21.15.19:45722) 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 -- 172.21.15.19:0/956937274 learned_addr learned my addr 172.21.15.19:0/956937274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30c3fff700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f088c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d1591700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f08fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 -- 172.21.15.19:0/956937274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 msgr2=0x5609f08fb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f08fb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 -- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 msgr2=0x5609f088c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f088c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.611+0000 7f30d0d90700 1 -- 172.21.15.19:0/956937274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f08fb630 con 0x5609f06f3ec0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30d0d90700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f088bbe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f30c8023930 tx=0x7f30c8006050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3809cf9a3812cef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30abfff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30abfff700 1 -- 172.21.15.19:0/956937274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8027040 con 0x5609f06f3ec0 2024-05-17T09:22:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30abfff700 1 -- 172.21.15.19:0/956937274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f30c8015cf0 con 0x5609f06f3ec0 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c8023420 con 0x5609f06f3ec0 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30abfff700 1 -- 172.21.15.19:0/956937274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30c801d040 con 0x5609f06f3ec0 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30e1a09b80 10 client.16185 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609f08fc820 con 0x5609f06f3ec0 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.612+0000 7f30abfff700 1 -- 172.21.15.19:0/956937274 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30c8023420 con 0x5609f06f3ec0 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30abfff700 1 -- 172.21.15.19:0/956937274 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30c80085a0 con 0x5609f06f3ec0 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30e1a09b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30e1a09b80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30e1a09b80 20 client.16185 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08fce40 0x5609f08ff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30e1a09b80 4 client.16185 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609f083ef50 con 0x5609f08fce40 2024-05-17T09:22:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30c3fff700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08fce40 0x5609f08ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30abfff700 10 client.16185 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.613+0000 7f30c3fff700 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08fce40 0x5609f08ff260 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab203eb2f79c355 server_cookie=5d57d7f1eee6a84c in_seq=0 out_seq=0 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.615+0000 7f30abfff700 1 -- 172.21.15.19:0/956937274 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609f083ef50 con 0x5609f08fce40 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.615+0000 7f30abfff700 10 client.16185 handle_command_reply: tid=0 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.615+0000 7f30e1a09b80 1 client.16185 shutdown 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.615+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 msgr2=0x5609f088bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.615+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f088bbe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f30c8023930 tx=0x7f30c8006050 comp rx=0 tx=0).stop 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 shutdown_connections 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5609f0877160 0x5609f08fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5609f08fce40 0x5609f08ff260 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609f06f3ec0 0x5609f088bbe0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 --2- 172.21.15.19:0/956937274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609f07df4b0 0x5609f088c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30abfff700 0 client.16185 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 >> 172.21.15.19:0/956937274 conn(0x5609f0754240 msgr2=0x5609f0871d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 shutdown_connections 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 1 -- 172.21.15.19:0/956937274 wait complete. 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.616+0000 7f30e1a09b80 20 client.16185 trim_cache size 0 max 16384 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:22:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "f9ddd3f3-e5cc-4221-9a07-d9c0eb027685", 2024-05-17T09:22:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:22:14.620 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:22:14.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f30b40b2880 msgr2=0x7f30b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f30b40b2880 0x7f30b40b4d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009da0 tx=0x7f30c800b040 comp rx=0 tx=0).stop 2024-05-17T09:22:14.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 msgr2=0x7f30cc19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:14.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 0x7f30cc19c660 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005b40 tx=0x7f30c40055f0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 shutdown_connections 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f30b40b2880 0x7f30b40b4d90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30cc1034d0 0x7f30cc19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30cc108750 0x7f30cc19c660 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 --2- 172.21.15.19:0/932415914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc107e00 0x7f30cc19c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 >> 172.21.15.19:0/932415914 conn(0x7f30cc0fef20 msgr2=0x7f30cc107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 shutdown_connections 2024-05-17T09:22:14.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:14.617+0000 7f30d2ff4700 1 -- 172.21.15.19:0/932415914 wait complete. 2024-05-17T09:22:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:14 smithi069 ceph-mon[35081]: pgmap v512: 129 pgs: 129 active+clean; 24 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.6 MiB/s wr, 96 op/s 2024-05-17T09:22:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:14 smithi079 ceph-mon[35502]: pgmap v512: 129 pgs: 129 active+clean; 24 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.6 MiB/s wr, 96 op/s 2024-05-17T09:22:15.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 8258 4.71 MB/sec warmup 13 sec latency 19.555 ms 2024-05-17T09:22:15.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:22:15.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:15 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:22:15.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:15 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:15.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:15 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:22:15.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:15 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:15.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:15 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:22:15.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:15 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:15 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:22:15.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:15 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:15 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:22:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:15 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:15 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:22:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:15 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:15.835 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:16.391 INFO:tasks.workunit.client.0.smithi019.stdout: 1 8889 4.61 MB/sec warmup 14 sec latency 18.390 ms 2024-05-17T09:22:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:16 smithi069 ceph-mon[35081]: pgmap v513: 129 pgs: 129 active+clean; 25 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 2.7 MiB/s wr, 118 op/s 2024-05-17T09:22:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:22:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:16 smithi019 ceph-mon[31620]: pgmap v513: 129 pgs: 129 active+clean; 25 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 2.7 MiB/s wr, 118 op/s 2024-05-17T09:22:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:22:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:16 smithi079 ceph-mon[35502]: pgmap v513: 129 pgs: 129 active+clean; 25 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 2.7 MiB/s wr, 118 op/s 2024-05-17T09:22:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:22:17.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 9444 4.40 MB/sec warmup 15 sec latency 145.340 ms 2024-05-17T09:22:18.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 10130 4.49 MB/sec warmup 16 sec latency 66.066 ms 2024-05-17T09:22:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:18 smithi069 ceph-mon[35081]: pgmap v514: 129 pgs: 129 active+clean; 25 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.4 MiB/s wr, 123 op/s 2024-05-17T09:22:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:18 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:22:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:18 smithi019 ceph-mon[31620]: pgmap v514: 129 pgs: 129 active+clean; 25 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.4 MiB/s wr, 123 op/s 2024-05-17T09:22:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:18 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:22:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:18 smithi079 ceph-mon[35502]: pgmap v514: 129 pgs: 129 active+clean; 25 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.4 MiB/s wr, 123 op/s 2024-05-17T09:22:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:18 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:22:19.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 10627 4.51 MB/sec warmup 17 sec latency 82.223 ms 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 -- 172.21.15.19:0/2622986632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 msgr2=0x7fbdc0103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 --2- 172.21.15.19:0/2622986632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc0103950 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00099c0 tx=0x7fbdb0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 -- 172.21.15.19:0/2622986632 shutdown_connections 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 --2- 172.21.15.19:0/2622986632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdc0108770 0x7fbdc01100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 --2- 172.21.15.19:0/2622986632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc0103950 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 --2- 172.21.15.19:0/2622986632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc0108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 -- 172.21.15.19:0/2622986632 >> 172.21.15.19:0/2622986632 conn(0x7fbdc0078df0 msgr2=0x7fbdc0079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 -- 172.21.15.19:0/2622986632 shutdown_connections 2024-05-17T09:22:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.249+0000 7fbdc690b700 1 -- 172.21.15.19:0/2622986632 wait complete. 2024-05-17T09:22:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 Processor -- start 2024-05-17T09:22:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 -- start start 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdc0108770 0x7fbdc019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc0113250 con 0x7fbdc0103540 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01130d0 con 0x7fbdc0108770 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdc690b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01133d0 con 0x7fbdc0107e20 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdb7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdb7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc019c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35214/0 (socket says 172.21.15.19:35214) 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.250+0000 7fbdb7fff700 1 -- 172.21.15.19:0/206029372 learned_addr learned my addr 172.21.15.19:0/206029372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdc4ea8700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdc0108770 0x7fbdc019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdb7fff700 1 -- 172.21.15.19:0/206029372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdc0108770 msgr2=0x7fbdc019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdc0108770 0x7fbdc019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdb7fff700 1 -- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 msgr2=0x7fbdc019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdb7fff700 1 -- 172.21.15.19:0/206029372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc01a6f40 con 0x7fbdc0107e20 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdbffff700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc019c110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d950 tx=0x7fbda800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdbdffb700 1 -- 172.21.15.19:0/206029372 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8016070 con 0x7fbdc0107e20 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdbdffb700 1 -- 172.21.15.19:0/206029372 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbda8004830 con 0x7fbdc0107e20 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.251+0000 7fbdbdffb700 1 -- 172.21.15.19:0/206029372 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8005420 con 0x7fbdc0107e20 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.252+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc0199a20 con 0x7fbdc0107e20 2024-05-17T09:22:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.252+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc0199f10 con 0x7fbdc0107e20 2024-05-17T09:22:20.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.254+0000 7fbdbdffb700 1 -- 172.21.15.19:0/206029372 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbda8004220 con 0x7fbdc0107e20 2024-05-17T09:22:20.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.255+0000 7fbdbdffb700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbdac0b2880 0x7fbdac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.255+0000 7fbdbffff700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbdac0b2880 0x7fbdac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.255+0000 7fbdbdffb700 1 -- 172.21.15.19:0/206029372 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbda80e22b0 con 0x7fbdc0107e20 2024-05-17T09:22:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdbffff700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbdac0b2880 0x7fbdac0b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c00 tx=0x7fbdb001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 Processor -- start 2024-05-17T09:22:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 -- start start 2024-05-17T09:22:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588b4fe40 0x561588cd4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588cce3c0 0x561588cce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561588bfcc90 con 0x561588cce3c0 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561588bec830 con 0x561588b4fe40 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.256+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561588bf0870 con 0x561588bcd3d0 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdc4ea8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588cce3c0 0x561588cce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35228/0 (socket says 172.21.15.19:35228) 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdbffff700 1 -- 172.21.15.19:0/189539119 learned_addr learned my addr 172.21.15.19:0/189539119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588b4fe40 0x561588cd4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 -- 172.21.15.19:0/189539119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 msgr2=0x561588cd3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 -- 172.21.15.19:0/189539119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588cce3c0 msgr2=0x561588cce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588cce3c0 0x561588cce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 -- 172.21.15.19:0/189539119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561588b3c8d0 con 0x561588b4fe40 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdbffff700 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588b4fe40 0x561588cd4520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbda80dfd70 tx=0x7fbda80a9100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d990ea660c9dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.257+0000 7fbdb67fc700 1 -- 172.21.15.19:0/189539119 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80ad8d0 con 0x561588b4fe40 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdb67fc700 1 -- 172.21.15.19:0/189539119 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbda80b2380 con 0x561588b4fe40 2024-05-17T09:22:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdb67fc700 1 -- 172.21.15.19:0/189539119 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80b20f0 con 0x561588b4fe40 2024-05-17T09:22:20.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 -- 172.21.15.19:0/189539119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588b4fe40 msgr2=0x561588cd4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588b4fe40 0x561588cd4520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbda80dfd70 tx=0x7fbda80a9100 comp rx=0 tx=0).stop 2024-05-17T09:22:20.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 -- 172.21.15.19:0/189539119 shutdown_connections 2024-05-17T09:22:20.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588cce3c0 0x561588cce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/189539119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588b4fe40 0x561588cd4520 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 -- 172.21.15.19:0/189539119 >> 172.21.15.19:0/189539119 conn(0x561588b8ec80 msgr2=0x561588cc8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 -- 172.21.15.19:0/189539119 shutdown_connections 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.258+0000 7fbdd5320b80 1 -- 172.21.15.19:0/189539119 wait complete. 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.259+0000 7fbdd5320b80 1 -- start start 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.259+0000 7fbdd5320b80 1 Processor -- start 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.259+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.259+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.259+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588cd8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561588af2fa0 con 0x561588b4fe40 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561588b8e7c0 con 0x561588cce3c0 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561588ccf1f0 con 0x561588bcd3d0 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cd7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45748/0 (socket says 172.21.15.19:45748) 2024-05-17T09:22:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 -- 172.21.15.19:0/2463872863 learned_addr learned my addr 172.21.15.19:0/2463872863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdd5320b80 10 client.0 _ll_register_callbacks cb 0x561588b87920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdc4ea8700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588cd8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 -- 172.21.15.19:0/2463872863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 msgr2=0x561588cd81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 -- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 msgr2=0x561588cd8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588cd8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 -- 172.21.15.19:0/2463872863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561588b8b690 con 0x561588b4fe40 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbdbffff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cd7cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009990 tx=0x7fbdb0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edcf5715d7a6565 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.260+0000 7fbd9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/2463872863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0035060 con 0x561588b4fe40 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/2463872863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbdb000f660 con 0x561588b4fe40 2024-05-17T09:22:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda80b2380 con 0x561588b4fe40 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/2463872863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdb0003e60 con 0x561588b4fe40 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 10 client.16197 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561588ce55c0 con 0x561588b4fe40 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/2463872863 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbda80b2380 con 0x561588b4fe40 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/2463872863 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbdb0030440 con 0x561588b4fe40 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 20 client.16197 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588ce7c00 0x561588cea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 4 client.16197 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.261+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561588bfebe0 con 0x561588ce7c00 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.262+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588ce7c00 0x561588cea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.262+0000 7fbd9f7fe700 10 client.16197 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.262+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588ce7c00 0x561588cea040 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab13cecc968ee7b server_cookie=4a81d16bf7b2203e in_seq=0 out_seq=0 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/2463872863 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561588bfebe0 con 0x561588ce7c00 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbd9f7fe700 10 client.16197 handle_command_reply: tid=0 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 client.16197 shutdown 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 msgr2=0x561588cd7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cd7cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009990 tx=0x7fbdb0009310 comp rx=0 tx=0).stop 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 shutdown_connections 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588ce7c00 0x561588cea040 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588cd8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cd7cb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/2463872863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 >> 172.21.15.19:0/2463872863 conn(0x561588b8ec80 msgr2=0x561588cc8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.263+0000 7fbd9f7fe700 0 client.16197 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.264+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 shutdown_connections 2024-05-17T09:22:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.264+0000 7fbdd5320b80 1 -- 172.21.15.19:0/2463872863 wait complete. 2024-05-17T09:22:20.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.264+0000 7fbdd5320b80 20 client.16197 trim_cache size 0 max 16384 2024-05-17T09:22:20.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 Processor -- start 2024-05-17T09:22:20.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 -- start start 2024-05-17T09:22:20.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cc3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588ce3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01b4ee0 con 0x561588b4fe40 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01b2770 con 0x561588cce3c0 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01b05c0 con 0x561588bcd3d0 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cc3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cc3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45762/0 (socket says 172.21.15.19:45762) 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 -- 172.21.15.19:0/4211391590 learned_addr learned my addr 172.21.15.19:0/4211391590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdc4ea8700 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588ce3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 -- 172.21.15.19:0/4211391590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 msgr2=0x561588cd34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 -- 172.21.15.19:0/4211391590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 msgr2=0x561588ce3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588ce3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.287+0000 7fbdbffff700 1 -- 172.21.15.19:0/4211391590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561588cd6c80 con 0x561588b4fe40 2024-05-17T09:22:20.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.288+0000 7fbdbffff700 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cc3990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00098e0 tx=0x7fbdb0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ab619f8e819a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.288+0000 7fbdb5ffb700 1 -- 172.21.15.19:0/4211391590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0031e60 con 0x561588b4fe40 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.288+0000 7fbdb5ffb700 1 -- 172.21.15.19:0/4211391590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdb000f460 con 0x561588b4fe40 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.288+0000 7fbdb5ffb700 1 -- 172.21.15.19:0/4211391590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0035040 con 0x561588b4fe40 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4211391590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 msgr2=0x561588cc3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cc3990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00098e0 tx=0x7fbdb0009570 comp rx=0 tx=0).stop 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4211391590 shutdown_connections 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588cce3c0 0x561588ce3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588cc3990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4211391590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588bcd3d0 0x561588cd34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4211391590 >> 172.21.15.19:0/4211391590 conn(0x561588b8ec80 msgr2=0x561588cd18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4211391590 shutdown_connections 2024-05-17T09:22:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4211391590 wait complete. 2024-05-17T09:22:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.289+0000 7fbdd5320b80 1 -- start start 2024-05-17T09:22:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 Processor -- start 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588ce05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588bcd3d0 0x561588cef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588cce3c0 0x561588cef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01b0c30 con 0x561588b4fe40 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01b20a0 con 0x561588bcd3d0 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc01b1300 con 0x561588cce3c0 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdd5320b80 10 client.0 _ll_register_callbacks cb 0x561588cc9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588ce05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588ce05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45768/0 (socket says 172.21.15.19:45768) 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdbffff700 1 -- 172.21.15.19:0/4261868802 learned_addr learned my addr 172.21.15.19:0/4261868802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588bcd3d0 0x561588cef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.290+0000 7fbdc4ea8700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588cce3c0 0x561588cef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdbffff700 1 -- 172.21.15.19:0/4261868802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588cce3c0 msgr2=0x561588cef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdbffff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588cce3c0 0x561588cef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdbffff700 1 -- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588bcd3d0 msgr2=0x561588cef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdbffff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588bcd3d0 0x561588cef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdbffff700 1 -- 172.21.15.19:0/4261868802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561588ceff80 con 0x561588b4fe40 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdc4ea8700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588cce3c0 0x561588cef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588bcd3d0 0x561588cef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdbffff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588ce05a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0005f50 tx=0x7fbdb0003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1161ea81a14e616a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbd9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/4261868802 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0035040 con 0x561588b4fe40 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/4261868802 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbdb000f810 con 0x561588b4fe40 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb000f460 con 0x561588b4fe40 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/4261868802 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdb0030a00 con 0x561588b4fe40 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdd5320b80 10 client.16203 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.291+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561588d52720 con 0x561588b4fe40 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/4261868802 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdb000f460 con 0x561588b4fe40 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/4261868802 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbdb003e020 con 0x561588b4fe40 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdd5320b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdd5320b80 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdd5320b80 20 client.16203 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588d52b80 0x561588d54fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdd5320b80 4 client.16203 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561588bf83c0 con 0x561588d52b80 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.292+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588d52b80 0x561588d54fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.293+0000 7fbd9f7fe700 10 client.16203 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.293+0000 7fbdb7fff700 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588d52b80 0x561588d54fa0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c41fefe3091b8c5 server_cookie=eaeb586a5649d17 in_seq=0 out_seq=0 2024-05-17T09:22:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbd9f7fe700 1 -- 172.21.15.19:0/4261868802 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561588bf83c0 con 0x561588d52b80 2024-05-17T09:22:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbd9f7fe700 10 client.16203 handle_command_reply: tid=0 2024-05-17T09:22:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 client.16203 shutdown 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 msgr2=0x561588ce05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588ce05a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0005f50 tx=0x7fbdb0003aa0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 shutdown_connections 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561588cce3c0 0x561588cef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561588d52b80 0x561588d54fa0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561588b4fe40 0x561588ce05a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbd9f7fe700 0 client.16203 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 --2- 172.21.15.19:0/4261868802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561588bcd3d0 0x561588cef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 >> 172.21.15.19:0/4261868802 conn(0x561588b8ec80 msgr2=0x561588cc36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.295+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 shutdown_connections 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdd5320b80 1 -- 172.21.15.19:0/4261868802 wait complete. 2024-05-17T09:22:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdd5320b80 20 client.16203 trim_cache size 0 max 16384 2024-05-17T09:22:20.298 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:22:20.298 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:22:20.298 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:22:20.298 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:22:20.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbdac0b2880 msgr2=0x7fbdac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbdac0b2880 0x7fbdac0b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c00 tx=0x7fbdb001a040 comp rx=0 tx=0).stop 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 msgr2=0x7fbdc019c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc019c110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d950 tx=0x7fbda800dc60 comp rx=0 tx=0).stop 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 shutdown_connections 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdc0108770 0x7fbdc019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdc0103540 0x7fbdc019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbdac0b2880 0x7fbdac0b4d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 --2- 172.21.15.19:0/206029372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdc0107e20 0x7fbdc019c110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 >> 172.21.15.19:0/206029372 conn(0x7fbdc0078df0 msgr2=0x7fbdc0100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 shutdown_connections 2024-05-17T09:22:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:20.296+0000 7fbdc690b700 1 -- 172.21.15.19:0/206029372 wait complete. 2024-05-17T09:22:20.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 11077 4.35 MB/sec warmup 18 sec latency 15.702 ms 2024-05-17T09:22:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 -- 172.21.15.19:0/1141088175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028102b40 msgr2=0x7f5028103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 --2- 172.21.15.19:0/1141088175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028102b40 0x7f5028103840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5024009900 tx=0x7f502402a410 comp rx=0 tx=0).stop 2024-05-17T09:22:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 -- 172.21.15.19:0/1141088175 shutdown_connections 2024-05-17T09:22:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 --2- 172.21.15.19:0/1141088175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028108a80 0x7f502810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 --2- 172.21.15.19:0/1141088175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5028103d80 0x7f5028104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 --2- 172.21.15.19:0/1141088175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028102b40 0x7f5028103840 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 -- 172.21.15.19:0/1141088175 >> 172.21.15.19:0/1141088175 conn(0x7f50280fdd30 msgr2=0x7f50281001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 -- 172.21.15.19:0/1141088175 shutdown_connections 2024-05-17T09:22:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.423+0000 7f50307d5640 1 -- 172.21.15.19:0/1141088175 wait complete. 2024-05-17T09:22:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 Processor -- start 2024-05-17T09:22:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 -- start start 2024-05-17T09:22:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028102b40 0x7f5028199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5028103d80 0x7f5028199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 0x7f50281a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f502810e4f0 con 0x7f5028108a80 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f502810e370 con 0x7f5028102b40 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f50307d5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f502810e670 con 0x7f5028103d80 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f502ed72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 0x7f50281a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f502e571640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028102b40 0x7f5028199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f502ed72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 0x7f50281a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45786/0 (socket says 172.21.15.19:45786) 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f502ed72640 1 -- 172.21.15.19:0/226938909 learned_addr learned my addr 172.21.15.19:0/226938909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.424+0000 7f502dd70640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5028103d80 0x7f5028199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502ed72640 1 -- 172.21.15.19:0/226938909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5028103d80 msgr2=0x7f5028199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502ed72640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5028103d80 0x7f5028199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502ed72640 1 -- 172.21.15.19:0/226938909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028102b40 msgr2=0x7f5028199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502ed72640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028102b40 0x7f5028199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502ed72640 1 -- 172.21.15.19:0/226938909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50281a0c70 con 0x7f5028108a80 2024-05-17T09:22:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502e571640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028102b40 0x7f5028199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f502ed72640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 0x7f50281a0600 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5020010d20 tx=0x7f502000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f501f7fe640 1 -- 172.21.15.19:0/226938909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020004d20 con 0x7f5028108a80 2024-05-17T09:22:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f501f7fe640 1 -- 172.21.15.19:0/226938909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50200054b0 con 0x7f5028108a80 2024-05-17T09:22:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f501f7fe640 1 -- 172.21.15.19:0/226938909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020018850 con 0x7f5028108a80 2024-05-17T09:22:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502806a6b0 con 0x7f5028108a80 2024-05-17T09:22:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.425+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502806ac00 con 0x7f5028108a80 2024-05-17T09:22:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.428+0000 7f5037765740 1 Processor -- start 2024-05-17T09:22:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.428+0000 7f501f7fe640 1 -- 172.21.15.19:0/226938909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5020005020 con 0x7f5028108a80 2024-05-17T09:22:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.428+0000 7f5037765740 1 -- start start 2024-05-17T09:22:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f5037765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b6f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f5037765740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b6f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f5037765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b76820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f5037765740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9ae1c30 con 0x55a1f97a52f0 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f5037765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9ad17d0 con 0x55a1f97ad4d0 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f5037765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9ad5810 con 0x55a1f97ce1b0 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b6f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502dd70640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b6f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45796/0 (socket says 172.21.15.19:45796) 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 -- 172.21.15.19:0/798591356 learned_addr learned my addr 172.21.15.19:0/798591356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502dd70640 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b6f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f501f7fe640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50140b2920 0x7f50140b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 -- 172.21.15.19:0/798591356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 msgr2=0x55a1f9b6f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b6f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 -- 172.21.15.19:0/798591356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 msgr2=0x55a1f9b76820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b76820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502e571640 1 -- 172.21.15.19:0/798591356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f97e6210 con 0x55a1f97a52f0 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502dd70640 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b76820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.429+0000 7f502ed72640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50140b2920 0x7f50140b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.430+0000 7f502e571640 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b6f920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5024005ab0 tx=0x7f5024005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c805c6e0c68b956 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.430+0000 7f501f7fe640 1 -- 172.21.15.19:0/226938909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f50200e4580 con 0x7f5028108a80 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.430+0000 7f501cff9640 1 -- 172.21.15.19:0/798591356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024035030 con 0x55a1f97a52f0 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.430+0000 7f501cff9640 1 -- 172.21.15.19:0/798591356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5024040070 con 0x55a1f97a52f0 2024-05-17T09:22:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.430+0000 7f501cff9640 1 -- 172.21.15.19:0/798591356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50240039e0 con 0x55a1f97a52f0 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 -- 172.21.15.19:0/798591356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 msgr2=0x55a1f9b6f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f502ed72640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50140b2920 0x7f50140b4e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f50200e2040 tx=0x7f5020009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b6f920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5024005ab0 tx=0x7f5024005d50 comp rx=0 tx=0).stop 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 -- 172.21.15.19:0/798591356 shutdown_connections 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b76820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b6f920 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 --2- 172.21.15.19:0/798591356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b6f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 -- 172.21.15.19:0/798591356 >> 172.21.15.19:0/798591356 conn(0x55a1f97c2300 msgr2=0x55a1f9b6ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 -- 172.21.15.19:0/798591356 shutdown_connections 2024-05-17T09:22:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 -- 172.21.15.19:0/798591356 wait complete. 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.431+0000 7f5037765740 1 -- start start 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 Processor -- start 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b78c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b80130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1f97b7790 con 0x55a1f97ce1b0 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9a8d890 con 0x55a1f97a52f0 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9b77210 con 0x55a1f97ad4d0 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f5037765740 10 client.0 _ll_register_callbacks cb 0x55a1f9a90d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b80130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b78c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b78c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58258/0 (socket says 172.21.15.19:58258) 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 -- 172.21.15.19:0/2736053024 learned_addr learned my addr 172.21.15.19:0/2736053024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502e571640 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 -- 172.21.15.19:0/2736053024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 msgr2=0x55a1f9b791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 -- 172.21.15.19:0/2736053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 msgr2=0x55a1f9b80130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b80130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.432+0000 7f502dd70640 1 -- 172.21.15.19:0/2736053024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f98b2620 con 0x55a1f97a52f0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f502dd70640 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b78c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5018011d00 tx=0x7f5018005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42409fd038b26b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5011ffb640 1 -- 172.21.15.19:0/2736053024 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501801a040 con 0x55a1f97a52f0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5024040070 con 0x55a1f97a52f0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5011ffb640 1 -- 172.21.15.19:0/2736053024 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5018011070 con 0x55a1f97a52f0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5011ffb640 1 -- 172.21.15.19:0/2736053024 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f501800fe50 con 0x55a1f97a52f0 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5037765740 10 client.25898 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1f9b83180 con 0x55a1f97a52f0 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5011ffb640 1 -- 172.21.15.19:0/2736053024 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5024040070 con 0x55a1f97a52f0 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5011ffb640 1 -- 172.21.15.19:0/2736053024 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5018003880 con 0x55a1f97a52f0 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5037765740 10 client.25898 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5037765740 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.433+0000 7f5037765740 20 client.25898 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.434+0000 7f5037765740 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f978cd80 0x55a1f9b867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.434+0000 7f5037765740 4 client.25898 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.434+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1f9ae3b80 con 0x55a1f978cd80 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.434+0000 7f502e571640 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f978cd80 0x55a1f9b867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.434+0000 7f5011ffb640 10 client.25898 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.434+0000 7f502e571640 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f978cd80 0x55a1f9b867e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e364e88e5e2f9e6e server_cookie=42a7ad2d121760fa in_seq=0 out_seq=0 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5011ffb640 1 -- 172.21.15.19:0/2736053024 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a1f9ae3b80 con 0x55a1f978cd80 2024-05-17T09:22:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5011ffb640 10 client.25898 handle_command_reply: tid=0 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 client.25898 shutdown 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 msgr2=0x55a1f9b78c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b78c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5018011d00 tx=0x7f5018005850 comp rx=0 tx=0).stop 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 shutdown_connections 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f978cd80 0x55a1f9b867e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b80130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 --2- 172.21.15.19:0/2736053024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b78c90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 >> 172.21.15.19:0/2736053024 conn(0x55a1f97c2300 msgr2=0x55a1f9b6dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5011ffb640 0 client.25898 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 shutdown_connections 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 1 -- 172.21.15.19:0/2736053024 wait complete. 2024-05-17T09:22:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.435+0000 7f5037765740 20 client.25898 trim_cache size 0 max 16384 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 Processor -- start 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 -- start start 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b8c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b8c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50281ad0e0 con 0x55a1f97a52f0 2024-05-17T09:22:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50281aa9b0 con 0x55a1f97ad4d0 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f5037765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a8800 con 0x55a1f97ce1b0 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502dd70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b8c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502dd70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b8c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45802/0 (socket says 172.21.15.19:45802) 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502dd70640 1 -- 172.21.15.19:0/972762261 learned_addr learned my addr 172.21.15.19:0/972762261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502e571640 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b8c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502dd70640 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502e571640 1 -- 172.21.15.19:0/972762261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 msgr2=0x55a1f9b800c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.452+0000 7f502e571640 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b800c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f502e571640 1 -- 172.21.15.19:0/972762261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 msgr2=0x55a1f9b8c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f502e571640 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b8c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f502e571640 1 -- 172.21.15.19:0/972762261 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f9b79890 con 0x55a1f97ad4d0 2024-05-17T09:22:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f502dd70640 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b8c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f502e571640 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b8c570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f502402a930 tx=0x7f502402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c228e81d0ea1617 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f5013fff640 1 -- 172.21.15.19:0/972762261 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024057020 con 0x55a1f97ad4d0 2024-05-17T09:22:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f5013fff640 1 -- 172.21.15.19:0/972762261 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5024040050 con 0x55a1f97ad4d0 2024-05-17T09:22:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.453+0000 7f5013fff640 1 -- 172.21.15.19:0/972762261 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024035040 con 0x55a1f97ad4d0 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 -- 172.21.15.19:0/972762261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 msgr2=0x55a1f9b8c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b8c570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f502402a930 tx=0x7f502402a410 comp rx=0 tx=0).stop 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 -- 172.21.15.19:0/972762261 shutdown_connections 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b8c570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 --2- 172.21.15.19:0/972762261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97a52f0 0x55a1f9b8c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 -- 172.21.15.19:0/972762261 >> 172.21.15.19:0/972762261 conn(0x55a1f97c2300 msgr2=0x55a1f9b728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 -- 172.21.15.19:0/972762261 shutdown_connections 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 -- 172.21.15.19:0/972762261 wait complete. 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 -- start start 2024-05-17T09:22:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.454+0000 7f5037765740 1 Processor -- start 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b7f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a8e70 con 0x55a1f97ce1b0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50281aa2e0 con 0x55a1f97a52f0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a9540 con 0x55a1f97ad4d0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5037765740 10 client.0 _ll_register_callbacks cb 0x7f5018011b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502dd70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502dd70640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b7f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b7f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35328/0 (socket says 172.21.15.19:35328) 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 -- 172.21.15.19:0/757711164 learned_addr learned my addr 172.21.15.19:0/757711164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 -- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 msgr2=0x55a1f9b7f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b7f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 -- 172.21.15.19:0/757711164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 msgr2=0x55a1f9b7fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b7fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 -- 172.21.15.19:0/757711164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f9becf70 con 0x55a1f97ad4d0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f502e571640 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b7f730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50240057c0 tx=0x7f5024005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6220bbfb1daabcc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.455+0000 7f5011ffb640 1 -- 172.21.15.19:0/757711164 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024057030 con 0x55a1f97ad4d0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5024040050 con 0x55a1f97ad4d0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5011ffb640 1 -- 172.21.15.19:0/757711164 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f50240036b0 con 0x55a1f97ad4d0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5011ffb640 1 -- 172.21.15.19:0/757711164 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f502406d050 con 0x55a1f97ad4d0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 10 client.25903 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1f9bee200 con 0x55a1f97ad4d0 2024-05-17T09:22:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5011ffb640 1 -- 172.21.15.19:0/757711164 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5024040050 con 0x55a1f97ad4d0 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5011ffb640 1 -- 172.21.15.19:0/757711164 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f502403c700 con 0x55a1f97ad4d0 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 10 client.25903 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 20 client.25903 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f97621c0 0x55a1f9bf07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 4 client.25903 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1f9add360 con 0x55a1f97621c0 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.456+0000 7f502dd70640 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f97621c0 0x55a1f9bf07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.457+0000 7f5011ffb640 10 client.25903 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.457+0000 7f502dd70640 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f97621c0 0x55a1f9bf07f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5573f7890d388c server_cookie=3b74829d4e851a50 in_seq=0 out_seq=0 2024-05-17T09:22:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5011ffb640 1 -- 172.21.15.19:0/757711164 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a1f9add360 con 0x55a1f97621c0 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5011ffb640 10 client.25903 handle_command_reply: tid=0 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 client.25903 shutdown 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 msgr2=0x55a1f9b7f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b7f730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50240057c0 tx=0x7f5024005d50 comp rx=0 tx=0).stop 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 shutdown_connections 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a1f97621c0 0x55a1f9bf07f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f97ce1b0 0x55a1f9b7fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5011ffb640 0 client.25903 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1f97ad4d0 0x55a1f9b7f730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 --2- 172.21.15.19:0/757711164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1f97a52f0 0x55a1f9b7f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 >> 172.21.15.19:0/757711164 conn(0x55a1f97c2300 msgr2=0x55a1f9b78ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 shutdown_connections 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 1 -- 172.21.15.19:0/757711164 wait complete. 2024-05-17T09:22:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.510+0000 7f5037765740 20 client.25903 trim_cache size 0 max 16384 2024-05-17T09:22:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:22:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:22:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:22:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50140b2920 msgr2=0x7f50140b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50140b2920 0x7f50140b4e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f50200e2040 tx=0x7f5020009200 comp rx=0 tx=0).stop 2024-05-17T09:22:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 msgr2=0x7f50281a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 0x7f50281a0600 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5020010d20 tx=0x7f502000c6a0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 shutdown_connections 2024-05-17T09:22:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50140b2920 0x7f50140b4e30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5028108a80 0x7f50281a0600 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5028103d80 0x7f5028199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 --2- 172.21.15.19:0/226938909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028102b40 0x7f5028199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 >> 172.21.15.19:0/226938909 conn(0x7f50280fdd30 msgr2=0x7f5028104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 shutdown_connections 2024-05-17T09:22:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:20.511+0000 7f50307d5640 1 -- 172.21.15.19:0/226938909 wait complete. 2024-05-17T09:22:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:20 smithi069 ceph-mon[35081]: pgmap v515: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-05-17T09:22:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:22:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:22:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:22:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:20 smithi019 ceph-mon[31620]: pgmap v515: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-05-17T09:22:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:22:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:22:20.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:22:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:20 smithi079 ceph-mon[35502]: pgmap v515: 129 pgs: 129 active+clean; 25 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-05-17T09:22:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:22:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:22:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:22:21.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:22:21.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:22:21.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 11563 4.16 MB/sec warmup 19 sec latency 18.974 ms 2024-05-17T09:22:21.394 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:22.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 12244 4.18 MB/sec warmup 20 sec latency 22.651 ms 2024-05-17T09:22:22.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:22 smithi019 ceph-mon[31620]: pgmap v516: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-17T09:22:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:22 smithi069 ceph-mon[35081]: pgmap v516: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-17T09:22:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:22 smithi079 ceph-mon[35502]: pgmap v516: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-17T09:22:23.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/2968807452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 msgr2=0x7ff4e4110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:23.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/2968807452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e4110090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00b0d0 tx=0x7ff4dc00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/2968807452 shutdown_connections 2024-05-17T09:22:23.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/2968807452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e41034b0 0x7ff4e41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/2968807452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e4110090 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/2968807452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e4107de0 0x7ff4e41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/2968807452 >> 172.21.15.19:0/2968807452 conn(0x7ff4e40fedb0 msgr2=0x7ff4e41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:23.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.260+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/2968807452 shutdown_connections 2024-05-17T09:22:23.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.261+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/2968807452 wait complete. 2024-05-17T09:22:23.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.261+0000 7ff4eb5b4700 1 Processor -- start 2024-05-17T09:22:23.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 -- start start 2024-05-17T09:22:23.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e41034b0 0x7ff4e4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:23.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e4107de0 0x7ff4e4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:23.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:23.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113210 con 0x7ff4e4108730 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113090 con 0x7ff4e41034b0 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4eb5b4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113390 con 0x7ff4e4107de0 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4e9b51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4e9350700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e41034b0 0x7ff4e4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4e9b51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45832/0 (socket says 172.21.15.19:45832) 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4e9b51700 1 -- 172.21.15.19:0/1329706023 learned_addr learned my addr 172.21.15.19:0/1329706023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.262+0000 7ff4e8b4f700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e4107de0 0x7ff4e4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e8b4f700 1 -- 172.21.15.19:0/1329706023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e41034b0 msgr2=0x7ff4e4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e8b4f700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e41034b0 0x7ff4e4199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e8b4f700 1 -- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 msgr2=0x7ff4e419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:23.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e8b4f700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e8b4f700 1 -- 172.21.15.19:0/1329706023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41a9090 con 0x7ff4e4107de0 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e9350700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e41034b0 0x7ff4e4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4e8b4f700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e4107de0 0x7ff4e4199ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9c0 tx=0x7ff4d400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400c960 con 0x7ff4e4107de0 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d4010430 con 0x7ff4e4107de0 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4018680 con 0x7ff4e4107de0 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41a9380 con 0x7ff4e4107de0 2024-05-17T09:22:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.263+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a98c0 con 0x7ff4e4107de0 2024-05-17T09:22:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.265+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4104960 con 0x7ff4e4107de0 2024-05-17T09:22:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.266+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff4d4010870 con 0x7ff4e4107de0 2024-05-17T09:22:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.267+0000 7ff4da7fc700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4d00b2880 0x7ff4d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.267+0000 7ff4e9350700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4d00b2880 0x7ff4d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.267+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff4d40e2650 con 0x7ff4e4107de0 2024-05-17T09:22:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.267+0000 7ff4e9350700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4d00b2880 0x7ff4d00b4d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0005950 tx=0x7ff4e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.271+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4d40a9f60 con 0x7ff4e4107de0 2024-05-17T09:22:23.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 12957 4.05 MB/sec warmup 21 sec latency 22.738 ms 2024-05-17T09:22:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.590+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4e402e5e0 con 0x7ff4e4107de0 2024-05-17T09:22:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.591+0000 7ff4da7fc700 1 -- 172.21.15.19:0/1329706023 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11852 (secure 0 0 0) 0x7ff4d40a9860 con 0x7ff4e4107de0 2024-05-17T09:22:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:22:23.591 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:19:29.696379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.593+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4d00b2880 msgr2=0x7ff4d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.593+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4d00b2880 0x7ff4d00b4d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0005950 tx=0x7ff4e000b410 comp rx=0 tx=0).stop 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.593+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e4107de0 msgr2=0x7ff4e4199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.593+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e4107de0 0x7ff4e4199ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9c0 tx=0x7ff4d400ecd0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 shutdown_connections 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e4108730 0x7ff4e419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e41034b0 0x7ff4e4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4d00b2880 0x7ff4d00b4d90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 --2- 172.21.15.19:0/1329706023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4e4107de0 0x7ff4e4199ea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 >> 172.21.15.19:0/1329706023 conn(0x7ff4e40fedb0 msgr2=0x7ff4e41075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 shutdown_connections 2024-05-17T09:22:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:23.594+0000 7ff4eb5b4700 1 -- 172.21.15.19:0/1329706023 wait complete. 2024-05-17T09:22:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-17T09:22:24.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 13662 4.13 MB/sec warmup 22 sec latency 14.434 ms 2024-05-17T09:22:24.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:22:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:24 smithi069 ceph-mon[35081]: pgmap v517: 129 pgs: 129 active+clean; 28 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-05-17T09:22:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:24 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1329706023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:22:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:24 smithi019 ceph-mon[31620]: pgmap v517: 129 pgs: 129 active+clean; 28 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-05-17T09:22:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:24 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1329706023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:22:24.706 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:24 smithi079 ceph-mon[35502]: pgmap v517: 129 pgs: 129 active+clean; 28 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-05-17T09:22:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:24 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1329706023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:22:25.390 INFO:tasks.workunit.client.0.smithi019.stdout: 1 14268 4.17 MB/sec warmup 23 sec latency 16.856 ms 2024-05-17T09:22:26.401 INFO:tasks.workunit.client.0.smithi019.stdout: 1 14674 4.06 MB/sec warmup 24 sec latency 53.216 ms 2024-05-17T09:22:26.466 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:26 smithi019 ceph-mon[31620]: pgmap v518: 129 pgs: 129 active+clean; 24 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-05-17T09:22:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 -- 172.21.15.19:0/2088576294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00109670 msgr2=0x7f4b00110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 --2- 172.21.15.19:0/2088576294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00109670 0x7f4b00110fd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b0d0 tx=0x7f4af800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 -- 172.21.15.19:0/2088576294 shutdown_connections 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 --2- 172.21.15.19:0/2088576294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b001043f0 0x7f4b00104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 --2- 172.21.15.19:0/2088576294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00109670 0x7f4b00110fd0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 --2- 172.21.15.19:0/2088576294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b00108d20 0x7f4b00109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 -- 172.21.15.19:0/2088576294 >> 172.21.15.19:0/2088576294 conn(0x7f4b00079d40 msgr2=0x7f4b0007a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 -- 172.21.15.19:0/2088576294 shutdown_connections 2024-05-17T09:22:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.600+0000 7f4b0519f700 1 -- 172.21.15.19:0/2088576294 wait complete. 2024-05-17T09:22:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.601+0000 7f4b0519f700 1 Processor -- start 2024-05-17T09:22:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.601+0000 7f4b0519f700 1 -- start start 2024-05-17T09:22:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0519f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b001043f0 0x7f4b001a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0519f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 0x7f4b001a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0519f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b00109670 0x7f4b0019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0519f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001141a0 con 0x7f4b00108d20 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0519f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00114020 con 0x7f4b001043f0 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0519f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00114320 con 0x7f4b00109670 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 0x7f4b001a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 0x7f4b001a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38240/0 (socket says 172.21.15.19:38240) 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 -- 172.21.15.19:0/1557405 learned_addr learned my addr 172.21.15.19:0/1557405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4afffff700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b001043f0 0x7f4b001a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4b0499e700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b00109670 0x7f4b0019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 -- 172.21.15.19:0/1557405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b00109670 msgr2=0x7f4b0019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b00109670 0x7f4b0019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 -- 172.21.15.19:0/1557405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b001043f0 msgr2=0x7f4b001a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b001043f0 0x7f4b001a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 -- 172.21.15.19:0/1557405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0019b070 con 0x7f4b00108d20 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.602+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 0x7f4b001a0ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4af400e970 tx=0x7f4af400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.603+0000 7f4afd7fa700 1 -- 172.21.15.19:0/1557405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004a30 con 0x7f4b00108d20 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.603+0000 7f4afd7fa700 1 -- 172.21.15.19:0/1557405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af400cc70 con 0x7f4b00108d20 2024-05-17T09:22:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.603+0000 7f4afd7fa700 1 -- 172.21.15.19:0/1557405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4005520 con 0x7f4b00108d20 2024-05-17T09:22:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.603+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0019b360 con 0x7f4b00108d20 2024-05-17T09:22:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.603+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b001a7630 con 0x7f4b00108d20 2024-05-17T09:22:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.607+0000 7f4afd7fa700 1 -- 172.21.15.19:0/1557405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4af400c7e0 con 0x7f4b00108d20 2024-05-17T09:22:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.607+0000 7f4afd7fa700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ae80b2880 0x7f4ae80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4afffff700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ae80b2880 0x7f4ae80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4afd7fa700 1 -- 172.21.15.19:0/1557405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4af40e26e0 con 0x7f4b00108d20 2024-05-17T09:22:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4b14e16b80 1 Processor -- start 2024-05-17T09:22:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4afffff700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ae80b2880 0x7f4ae80b4d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b0019bb00 tx=0x7f4af0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4b14e16b80 1 -- start start 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a204ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a205080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.608+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a1ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae5a12cd70 con 0x55ae5a07fe40 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ae5a11c910 con 0x55ae5a1fef20 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ae5a120950 con 0x55ae5a16b430 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a205080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a205080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38248/0 (socket says 172.21.15.19:38248) 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4aff7fe700 1 -- 172.21.15.19:0/1753600406 learned_addr learned my addr 172.21.15.19:0/1753600406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a1ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4afffff700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a204ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 -- 172.21.15.19:0/1753600406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 msgr2=0x55ae5a204ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a204ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 -- 172.21.15.19:0/1753600406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 msgr2=0x55ae5a205080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a205080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 -- 172.21.15.19:0/1753600406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5a0c5fb0 con 0x55ae5a1fef20 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4afffff700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a204ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a205080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4b0499e700 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a1ff3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b770 tx=0x7f4af80098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea39c50c448bdbdc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4ae67fc700 1 -- 172.21.15.19:0/1753600406 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800baa0 con 0x55ae5a1fef20 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4ae67fc700 1 -- 172.21.15.19:0/1753600406 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af8020070 con 0x55ae5a1fef20 2024-05-17T09:22:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.609+0000 7f4ae67fc700 1 -- 172.21.15.19:0/1753600406 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80052b0 con 0x55ae5a1fef20 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 -- 172.21.15.19:0/1753600406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 msgr2=0x55ae5a1ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a1ff3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b770 tx=0x7f4af80098c0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 -- 172.21.15.19:0/1753600406 shutdown_connections 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a204ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a1ff3f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/1753600406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a205080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 -- 172.21.15.19:0/1753600406 >> 172.21.15.19:0/1753600406 conn(0x55ae5a0e01c0 msgr2=0x55ae5a1f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 -- 172.21.15.19:0/1753600406 shutdown_connections 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 -- 172.21.15.19:0/1753600406 wait complete. 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.611+0000 7f4b14e16b80 1 -- start start 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 Processor -- start 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a2087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a208ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a209220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae5a022fa0 con 0x55ae5a07fe40 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ae5a084da0 con 0x55ae5a1fef20 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ae5a20b770 con 0x55ae5a16b430 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b14e16b80 10 client.0 _ll_register_callbacks cb 0x55ae5a0b7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a2087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a2087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38258/0 (socket says 172.21.15.19:38258) 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 -- 172.21.15.19:0/2621901147 learned_addr learned my addr 172.21.15.19:0/2621901147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b0499e700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a209220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a208ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 -- 172.21.15.19:0/2621901147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 msgr2=0x55ae5a208ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a208ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 -- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 msgr2=0x55ae5a209220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a209220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4afffff700 1 -- 172.21.15.19:0/2621901147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5a06c8d0 con 0x55ae5a07fe40 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.612+0000 7f4b0499e700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a209220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4afffff700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a2087a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4af00044f0 tx=0x7f4af00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f889075bfa4329 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4adf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4adf7fe700 1 -- 172.21.15.19:0/2621901147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0027640 con 0x55ae5a07fe40 2024-05-17T09:22:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4adf7fe700 1 -- 172.21.15.19:0/2621901147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4af00277e0 con 0x55ae5a07fe40 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af8020070 con 0x55ae5a07fe40 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4adf7fe700 1 -- 172.21.15.19:0/2621901147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4af0004520 con 0x55ae5a07fe40 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4b14e16b80 10 client.16239 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.613+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae5a0d6760 con 0x55ae5a07fe40 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4adf7fe700 1 -- 172.21.15.19:0/2621901147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af8020070 con 0x55ae5a07fe40 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4adf7fe700 1 -- 172.21.15.19:0/2621901147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4af0025020 con 0x55ae5a07fe40 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4b14e16b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4b14e16b80 10 client.16239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4b14e16b80 20 client.16239 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a21aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4b14e16b80 4 client.16239 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae5a1caed0 con 0x55ae5a218650 2024-05-17T09:22:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a21aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.614+0000 7f4adf7fe700 10 client.16239 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.615+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a21aa70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f37087ba73ff7741 server_cookie=db341bfe076682c2 in_seq=0 out_seq=0 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4adf7fe700 1 -- 172.21.15.19:0/2621901147 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ae5a1caed0 con 0x55ae5a218650 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4adf7fe700 10 client.16239 handle_command_reply: tid=0 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 client.16239 shutdown 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 msgr2=0x55ae5a2087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a2087a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4af00044f0 tx=0x7f4af00047c0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 shutdown_connections 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a21aa70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4adf7fe700 0 client.16239 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a1fef20 0x55ae5a209220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a07fe40 0x55ae5a2087a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/2621901147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a16b430 0x55ae5a208ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 >> 172.21.15.19:0/2621901147 conn(0x55ae5a0e01c0 msgr2=0x55ae5a1f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 shutdown_connections 2024-05-17T09:22:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 1 -- 172.21.15.19:0/2621901147 wait complete. 2024-05-17T09:22:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.616+0000 7f4b14e16b80 20 client.16239 trim_cache size 0 max 16384 2024-05-17T09:22:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.633+0000 7f4b14e16b80 1 Processor -- start 2024-05-17T09:22:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.633+0000 7f4b14e16b80 1 -- start start 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.633+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a07fe40 0x55ae5a1fe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.633+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 0x55ae5a20ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.633+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a1fef20 0x55ae5a2146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b2a20 con 0x55ae5a16b430 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b02b0 con 0x55ae5a07fe40 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001ae100 con 0x55ae5a1fef20 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 0x55ae5a20ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 0x55ae5a20ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38268/0 (socket says 172.21.15.19:38268) 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 -- 172.21.15.19:0/3978808358 learned_addr learned my addr 172.21.15.19:0/3978808358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4afffff700 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a07fe40 0x55ae5a1fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4b0499e700 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a1fef20 0x55ae5a2146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 -- 172.21.15.19:0/3978808358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a1fef20 msgr2=0x55ae5a2146b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a1fef20 0x55ae5a2146b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 -- 172.21.15.19:0/3978808358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a07fe40 msgr2=0x55ae5a1fe940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a07fe40 0x55ae5a1fe940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 -- 172.21.15.19:0/3978808358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5a20b5a0 con 0x55ae5a16b430 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 0x55ae5a20ab00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4af40dfe80 tx=0x7f4af40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6751350d6ba0f0a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.634+0000 7f4ae5ffb700 1 -- 172.21.15.19:0/3978808358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af40a9460 con 0x55ae5a16b430 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4ae5ffb700 1 -- 172.21.15.19:0/3978808358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af400ce20 con 0x55ae5a16b430 2024-05-17T09:22:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4ae5ffb700 1 -- 172.21.15.19:0/3978808358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af40056c0 con 0x55ae5a16b430 2024-05-17T09:22:26.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 -- 172.21.15.19:0/3978808358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 msgr2=0x55ae5a20ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 0x55ae5a20ab00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4af40dfe80 tx=0x7f4af40cc040 comp rx=0 tx=0).stop 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 -- 172.21.15.19:0/3978808358 shutdown_connections 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a1fef20 0x55ae5a2146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a07fe40 0x55ae5a1fe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/3978808358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a16b430 0x55ae5a20ab00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.635+0000 7f4b14e16b80 1 -- 172.21.15.19:0/3978808358 >> 172.21.15.19:0/3978808358 conn(0x55ae5a0e01c0 msgr2=0x55ae5a2023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.636+0000 7f4b14e16b80 1 -- 172.21.15.19:0/3978808358 shutdown_connections 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.636+0000 7f4b14e16b80 1 -- 172.21.15.19:0/3978808358 wait complete. 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.636+0000 7f4b14e16b80 1 -- start start 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.636+0000 7f4b14e16b80 1 Processor -- start 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a07fe40 0x55ae5a21fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a16b430 0x55ae5a2203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a1fef20 0x55ae5a220900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001ae770 con 0x55ae5a1fef20 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001afbe0 con 0x55ae5a16b430 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001aee40 con 0x55ae5a07fe40 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b14e16b80 10 client.0 _ll_register_callbacks cb 0x55ae5a0b7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4afffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a07fe40 0x55ae5a21fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b0499e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a1fef20 0x55ae5a220900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4afffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a07fe40 0x55ae5a21fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:43490/0 (socket says 172.21.15.19:43490) 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4afffff700 1 -- 172.21.15.19:0/593692951 learned_addr learned my addr 172.21.15.19:0/593692951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a16b430 0x55ae5a2203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b0499e700 1 -- 172.21.15.19:0/593692951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a07fe40 msgr2=0x55ae5a21fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b0499e700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a07fe40 0x55ae5a21fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b0499e700 1 -- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a16b430 msgr2=0x55ae5a2203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:26.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b0499e700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a16b430 0x55ae5a2203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4b0499e700 1 -- 172.21.15.19:0/593692951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5a209590 con 0x55ae5a1fef20 2024-05-17T09:22:26.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.637+0000 7f4aff7fe700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a16b430 0x55ae5a2203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:26.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4b0499e700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a1fef20 0x55ae5a220900 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b420 tx=0x7f4af800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4223dcfb7873dda4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:26.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4adf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4adf7fe700 1 -- 172.21.15.19:0/593692951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8007db0 con 0x55ae5a1fef20 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4adf7fe700 1 -- 172.21.15.19:0/593692951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4af8003a00 con 0x55ae5a1fef20 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af400ce20 con 0x55ae5a1fef20 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4adf7fe700 1 -- 172.21.15.19:0/593692951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4af803a050 con 0x55ae5a1fef20 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4b14e16b80 10 client.16245 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae5a2831a0 con 0x55ae5a1fef20 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.638+0000 7f4adf7fe700 1 -- 172.21.15.19:0/593692951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af400ce20 con 0x55ae5a1fef20 2024-05-17T09:22:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4adf7fe700 1 -- 172.21.15.19:0/593692951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4af8011070 con 0x55ae5a1fef20 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4b14e16b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4b14e16b80 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4b14e16b80 20 client.16245 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a285430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4b14e16b80 4 client.16245 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae5a12ecc0 con 0x55ae5a218650 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4afffff700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a285430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4adf7fe700 10 client.16245 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.639+0000 7f4afffff700 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a285430 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4fe5adb5a070abc server_cookie=9dc4640b19f93b33 in_seq=0 out_seq=0 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4adf7fe700 1 -- 172.21.15.19:0/593692951 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae5a12ecc0 con 0x55ae5a218650 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4adf7fe700 10 client.16245 handle_command_reply: tid=0 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 client.16245 shutdown 2024-05-17T09:22:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a1fef20 msgr2=0x55ae5a220900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a1fef20 0x55ae5a220900 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b420 tx=0x7f4af800bf90 comp rx=0 tx=0).stop 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 shutdown_connections 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x55ae5a218650 0x55ae5a285430 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae5a07fe40 0x55ae5a21fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae5a1fef20 0x55ae5a220900 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4adf7fe700 0 client.16245 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 --2- 172.21.15.19:0/593692951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5a16b430 0x55ae5a2203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 >> 172.21.15.19:0/593692951 conn(0x55ae5a0e01c0 msgr2=0x55ae5a1fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 shutdown_connections 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 1 -- 172.21.15.19:0/593692951 wait complete. 2024-05-17T09:22:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.640+0000 7f4b14e16b80 20 client.16245 trim_cache size 0 max 16384 2024-05-17T09:22:26.644 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:22:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ae80b2880 msgr2=0x7f4ae80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ae80b2880 0x7f4ae80b4d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b0019bb00 tx=0x7f4af0009260 comp rx=0 tx=0).stop 2024-05-17T09:22:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 msgr2=0x7f4b001a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 0x7f4b001a0ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4af400e970 tx=0x7f4af400ec80 comp rx=0 tx=0).stop 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 shutdown_connections 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ae80b2880 0x7f4ae80b4d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b00109670 0x7f4b0019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b001043f0 0x7f4b001a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 --2- 172.21.15.19:0/1557405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b00108d20 0x7f4b001a0ce0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 >> 172.21.15.19:0/1557405 conn(0x7f4b00079d40 msgr2=0x7f4b001017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 shutdown_connections 2024-05-17T09:22:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:26.641+0000 7f4b0519f700 1 -- 172.21.15.19:0/1557405 wait complete. 2024-05-17T09:22:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:26 smithi069 ceph-mon[35081]: pgmap v518: 129 pgs: 129 active+clean; 24 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-05-17T09:22:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:26 smithi079 ceph-mon[35502]: pgmap v518: 129 pgs: 129 active+clean; 24 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-05-17T09:22:27.401 INFO:tasks.workunit.client.0.smithi019.stdout: 1 15247 3.97 MB/sec warmup 25 sec latency 62.592 ms 2024-05-17T09:22:27.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:22:27.742 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:28.401 INFO:tasks.workunit.client.0.smithi019.stdout: 1 15856 3.95 MB/sec warmup 26 sec latency 28.800 ms 2024-05-17T09:22:28.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:28 smithi069 ceph-mon[35081]: pgmap v519: 129 pgs: 129 active+clean; 25 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-17T09:22:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:28 smithi019 ceph-mon[31620]: pgmap v519: 129 pgs: 129 active+clean; 25 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-17T09:22:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:28 smithi079 ceph-mon[35502]: pgmap v519: 129 pgs: 129 active+clean; 25 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-17T09:22:29.416 INFO:tasks.workunit.client.0.smithi019.stdout: 1 16541 3.86 MB/sec warmup 27 sec latency 15.988 ms 2024-05-17T09:22:30.416 INFO:tasks.workunit.client.0.smithi019.stdout: 1 17361 3.95 MB/sec warmup 28 sec latency 31.103 ms 2024-05-17T09:22:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:30 smithi069 ceph-mon[35081]: pgmap v520: 129 pgs: 129 active+clean; 25 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-17T09:22:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:30 smithi019 ceph-mon[31620]: pgmap v520: 129 pgs: 129 active+clean; 25 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-17T09:22:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:30 smithi079 ceph-mon[35502]: pgmap v520: 129 pgs: 129 active+clean; 25 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-17T09:22:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 -- 172.21.15.19:0/68334168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0102900 msgr2=0x7fa7a0102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 --2- 172.21.15.19:0/68334168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0102900 0x7fa7a0102d10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa794009990 tx=0x7fa794009ca0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 -- 172.21.15.19:0/68334168 shutdown_connections 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 --2- 172.21.15.19:0/68334168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0103380 0x7fa7a0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 --2- 172.21.15.19:0/68334168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0105210 0x7fa7a0105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 --2- 172.21.15.19:0/68334168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0102900 0x7fa7a0102d10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 -- 172.21.15.19:0/68334168 >> 172.21.15.19:0/68334168 conn(0x7fa7a00fe6e0 msgr2=0x7fa7a0100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 -- 172.21.15.19:0/68334168 shutdown_connections 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.035+0000 7fa7a6010700 1 -- 172.21.15.19:0/68334168 wait complete. 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.036+0000 7fa7a6010700 1 Processor -- start 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.036+0000 7fa7a6010700 1 -- start start 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa7a6010700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102900 0x7fa7a019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa7a6010700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0103380 0x7fa7a019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa7a6010700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 0x7fa7a01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa7a6010700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0116540 con 0x7fa7a0105210 2024-05-17T09:22:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa7a6010700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01163c0 con 0x7fa7a0103380 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa7a6010700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01166c0 con 0x7fa7a0102900 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 0x7fa7a01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 0x7fa7a01a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38302/0 (socket says 172.21.15.19:38302) 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0103380 0x7fa7a019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 -- 172.21.15.19:0/2685520469 learned_addr learned my addr 172.21.15.19:0/2685520469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102900 0x7fa7a019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 -- 172.21.15.19:0/2685520469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102900 msgr2=0x7fa7a019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102900 0x7fa7a019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 -- 172.21.15.19:0/2685520469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0103380 msgr2=0x7fa7a019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0103380 0x7fa7a019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 -- 172.21.15.19:0/2685520469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a01a1e40 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79effd700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0103380 0x7fa7a019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.037+0000 7fa79ffff700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 0x7fa7a01a17a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa79800ef30 tx=0x7fa79800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.038+0000 7fa79cff9700 1 -- 172.21.15.19:0/2685520469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980092e0 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.038+0000 7fa79cff9700 1 -- 172.21.15.19:0/2685520469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa798004bb0 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.038+0000 7fa79cff9700 1 -- 172.21.15.19:0/2685520469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798004020 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.038+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a01a20d0 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.038+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a01a2590 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.041+0000 7fa79cff9700 1 -- 172.21.15.19:0/2685520469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa79801b030 con 0x7fa7a0105210 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.042+0000 7fa7b4a25b80 1 Processor -- start 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.042+0000 7fa79cff9700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7880b2880 0x7fa7880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.042+0000 7fa7b4a25b80 1 -- start start 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.042+0000 7fa79effd700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7880b2880 0x7fa7880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.042+0000 7fa79cff9700 1 -- 172.21.15.19:0/2685520469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa7980e2080 con 0x7fa7a0105210 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x560533107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560532f83e40 0x560533108520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x5605331027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560533030c90 con 0x5605330013d0 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560533020830 con 0x5605331023c0 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560533024870 con 0x560532f83e40 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x560533107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x560533107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38312/0 (socket says 172.21.15.19:38312) 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79f7fe700 1 -- 172.21.15.19:0/2579352584 learned_addr learned my addr 172.21.15.19:0/2579352584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560532f83e40 0x560533108520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79ffff700 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x5605331027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7880b2880 0x7fa7880b4d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa79400b650 tx=0x7fa794005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 -- 172.21.15.19:0/2579352584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 msgr2=0x5605331027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x5605331027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 -- 172.21.15.19:0/2579352584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 msgr2=0x560533107fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x560533107fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 -- 172.21.15.19:0/2579352584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560532f708d0 con 0x560532f83e40 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x560533107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.043+0000 7fa79effd700 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560532f83e40 0x560533108520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa794026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f945205e6d8f88a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.044+0000 7fa785ffb700 1 -- 172.21.15.19:0/2579352584 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794028040 con 0x560532f83e40 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.044+0000 7fa785ffb700 1 -- 172.21.15.19:0/2579352584 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa794004ca0 con 0x560532f83e40 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.044+0000 7fa785ffb700 1 -- 172.21.15.19:0/2579352584 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794024480 con 0x560532f83e40 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.044+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/2579352584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560532f83e40 msgr2=0x560533108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.044+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560532f83e40 0x560533108520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa794026000 comp rx=0 tx=0).stop 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/2579352584 shutdown_connections 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x5605331027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x560533107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/2579352584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560532f83e40 0x560533108520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/2579352584 >> 172.21.15.19:0/2579352584 conn(0x560532fc2c80 msgr2=0x5605330fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/2579352584 shutdown_connections 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/2579352584 wait complete. 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 -- start start 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.045+0000 7fa7b4a25b80 1 Processor -- start 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x56053310bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x56053310c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560532f26fa0 con 0x560532f83e40 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560532fc27c0 con 0x5605331023c0 2024-05-17T09:22:30.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560533103190 con 0x5605330013d0 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa7b4a25b80 10 client.0 _ll_register_callbacks cb 0x560532fbb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x56053310bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x56053310bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38326/0 (socket says 172.21.15.19:38326) 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79f7fe700 1 -- 172.21.15.19:0/3719137352 learned_addr learned my addr 172.21.15.19:0/3719137352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79ffff700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x56053310c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 -- 172.21.15.19:0/3719137352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 msgr2=0x56053310c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x56053310c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 -- 172.21.15.19:0/3719137352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 msgr2=0x56053310bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x56053310bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 -- 172.21.15.19:0/3719137352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560532fbf690 con 0x5605330013d0 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79ffff700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x56053310c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:30.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa79effd700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x560533107aa0 tx=0x7fa794004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482ee82cde5ff80 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.046+0000 7fa77effd700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa77effd700 1 -- 172.21.15.19:0/3719137352 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794028070 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa77effd700 1 -- 172.21.15.19:0/3719137352 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa794024ad0 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa794004ca0 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa77effd700 1 -- 172.21.15.19:0/3719137352 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa794044050 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 10 client.25939 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560533119540 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa77effd700 1 -- 172.21.15.19:0/3719137352 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa794004ca0 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa77effd700 1 -- 172.21.15.19:0/3719137352 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa794036460 con 0x5605330013d0 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 10 client.25939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 20 client.25939 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x56053311ba10 0x56053311de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 4 client.25939 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.047+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560533032be0 con 0x56053311ba10 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.048+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x56053311ba10 0x56053311de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.048+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x56053311ba10 0x56053311de30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e903ec8ddd32cc33 server_cookie=33693e7ab765b897 in_seq=0 out_seq=0 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.048+0000 7fa77effd700 10 client.25939 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa77effd700 1 -- 172.21.15.19:0/3719137352 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560533032be0 con 0x56053311ba10 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa77effd700 10 client.25939 handle_command_reply: tid=0 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 client.25939 shutdown 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 msgr2=0x56053310c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x560533107aa0 tx=0x7fa794004810 comp rx=0 tx=0).stop 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 shutdown_connections 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x56053311ba10 0x56053311de30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x56053310c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x56053310bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa77effd700 0 client.25939 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/3719137352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c1a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 >> 172.21.15.19:0/3719137352 conn(0x560532fc2c80 msgr2=0x5605330fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 shutdown_connections 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/3719137352 wait complete. 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.049+0000 7fa7b4a25b80 20 client.25939 trim_cache size 0 max 16384 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 Processor -- start 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 -- start start 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x560533117d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x560533120e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b4c40 con 0x560532f83e40 2024-05-17T09:22:30.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b24d0 con 0x5605331023c0 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.067+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b0320 con 0x5605330013d0 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x560533117d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x560533117d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38332/0 (socket says 172.21.15.19:38332) 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 -- 172.21.15.19:0/1838901958 learned_addr learned my addr 172.21.15.19:0/1838901958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79effd700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79ffff700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x560533120e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 -- 172.21.15.19:0/1838901958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 msgr2=0x56053310c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 -- 172.21.15.19:0/1838901958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 msgr2=0x560533120e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x560533120e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 -- 172.21.15.19:0/1838901958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605330feea0 con 0x560532f83e40 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79effd700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79ffff700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x560533120e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:30.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x560533117d80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa790009d90 tx=0x7fa790004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4300c47042bf63b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.068+0000 7fa7857fa700 1 -- 172.21.15.19:0/1838901958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790003d70 con 0x560532f83e40 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.069+0000 7fa7857fa700 1 -- 172.21.15.19:0/1838901958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa790003f10 con 0x560532f83e40 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.069+0000 7fa7857fa700 1 -- 172.21.15.19:0/1838901958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790010430 con 0x560532f83e40 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/1838901958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 msgr2=0x560533117d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x560533117d80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa790009d90 tx=0x7fa790004660 comp rx=0 tx=0).stop 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/1838901958 shutdown_connections 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605331023c0 0x560533120e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560532f83e40 0x560533117d80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/1838901958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605330013d0 0x56053310c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/1838901958 >> 172.21.15.19:0/1838901958 conn(0x560532fc2c80 msgr2=0x560533107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/1838901958 shutdown_connections 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/1838901958 wait complete. 2024-05-17T09:22:30.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 -- start start 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.070+0000 7fa7b4a25b80 1 Processor -- start 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560532f83e40 0x56053311e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x56053311e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605331023c0 0x560533184270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b0990 con 0x5605330013d0 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b1e00 con 0x560532f83e40 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b1060 con 0x5605331023c0 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa7b4a25b80 10 client.0 _ll_register_callbacks cb 0x560532fbb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x56053311e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x56053311e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38340/0 (socket says 172.21.15.19:38340) 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79effd700 1 -- 172.21.15.19:0/4087980452 learned_addr learned my addr 172.21.15.19:0/4087980452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560532f83e40 0x56053311e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79ffff700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605331023c0 0x560533184270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 -- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605331023c0 msgr2=0x560533184270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605331023c0 0x560533184270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 -- 172.21.15.19:0/4087980452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 msgr2=0x56053311e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x56053311e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 -- 172.21.15.19:0/4087980452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605330fe000 con 0x560532f83e40 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79effd700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x56053311e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa79f7fe700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560532f83e40 0x56053311e1f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa790005e90 tx=0x7fa790003d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=613a4e957112a20b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.071+0000 7fa77effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa77effd700 1 -- 172.21.15.19:0/4087980452 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790000e60 con 0x560532f83e40 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa790003f10 con 0x560532f83e40 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa77effd700 1 -- 172.21.15.19:0/4087980452 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa790004750 con 0x560532f83e40 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa77effd700 1 -- 172.21.15.19:0/4087980452 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa790037050 con 0x560532f83e40 2024-05-17T09:22:30.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 10 client.25946 fetch_fsmap learned FSMap version 27 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605331856c0 con 0x560532f83e40 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa77effd700 1 -- 172.21.15.19:0/4087980452 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa790003f10 con 0x560532f83e40 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa77effd700 1 -- 172.21.15.19:0/4087980452 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa79002f020 con 0x560532f83e40 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 10 client.25946 fetch_fsmap finished waiting for FSMap version 27 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 10 client.25946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 20 client.25946 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560533185dc0 0x5605331881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 4 client.25946 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.072+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56053302c3c0 con 0x560533185dc0 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.073+0000 7fa79effd700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560533185dc0 0x5605331881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.073+0000 7fa77effd700 10 client.25946 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.073+0000 7fa79effd700 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560533185dc0 0x5605331881e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f754002b7a621ee8 server_cookie=1badfbdc882d0148 in_seq=0 out_seq=0 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.073+0000 7fa77effd700 1 -- 172.21.15.19:0/4087980452 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56053302c3c0 con 0x560533185dc0 2024-05-17T09:22:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.073+0000 7fa77effd700 10 client.25946 handle_command_reply: tid=0 2024-05-17T09:22:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 client.25946 shutdown 2024-05-17T09:22:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560532f83e40 msgr2=0x56053311e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560532f83e40 0x56053311e1f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa790005e90 tx=0x7fa790003d30 comp rx=0 tx=0).stop 2024-05-17T09:22:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 shutdown_connections 2024-05-17T09:22:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x560533185dc0 0x5605331881e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.828 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5605331023c0 0x560533184270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560532f83e40 0x56053311e1f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa77effd700 0 client.25946 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 --2- 172.21.15.19:0/4087980452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5605330013d0 0x56053311e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 >> 172.21.15.19:0/4087980452 conn(0x560532fc2c80 msgr2=0x560533107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 shutdown_connections 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 1 -- 172.21.15.19:0/4087980452 wait complete. 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7b4a25b80 20 client.25946 trim_cache size 0 max 16384 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7880b2880 msgr2=0x7fa7880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7a6010700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7880b2880 0x7fa7880b4d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa79400b650 tx=0x7fa794005fb0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 msgr2=0x7fa7a01a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.074+0000 7fa7a6010700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 0x7fa7a01a17a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa79800ef30 tx=0x7fa79800c3a0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 shutdown_connections 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa7880b2880 0x7fa7880b4d90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102900 0x7fa7a019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0105210 0x7fa7a01a17a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 --2- 172.21.15.19:0/2685520469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0103380 0x7fa7a019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 >> 172.21.15.19:0/2685520469 conn(0x7fa7a00fe6e0 msgr2=0x7fa7a0100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 shutdown_connections 2024-05-17T09:22:30.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:30.075+0000 7fa7a6010700 1 -- 172.21.15.19:0/2685520469 wait complete. 2024-05-17T09:22:31.416 INFO:tasks.workunit.client.0.smithi019.stdout: 1 17856 3.97 MB/sec warmup 29 sec latency 16.260 ms 2024-05-17T09:22:32.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 18289 3.88 MB/sec warmup 30 sec latency 15.604 ms 2024-05-17T09:22:32.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:32 smithi019 ceph-mon[31620]: pgmap v521: 129 pgs: 129 active+clean; 25 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-17T09:22:32.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:22:32.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:32 smithi069 ceph-mon[35081]: pgmap v521: 129 pgs: 129 active+clean; 25 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-17T09:22:32.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:32 smithi079 ceph-mon[35502]: pgmap v521: 129 pgs: 129 active+clean; 25 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-17T09:22:32.914 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:33.416 INFO:tasks.workunit.client.0.smithi019.stdout: 1 18800 3.82 MB/sec warmup 31 sec latency 21.394 ms 2024-05-17T09:22:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 -- 172.21.15.19:0/1449590404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 msgr2=0x7f931010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 --2- 172.21.15.19:0/1449590404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f931010aea0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9308009ca0 tx=0x7f930802a8c0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 -- 172.21.15.19:0/1449590404 shutdown_connections 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 --2- 172.21.15.19:0/1449590404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f931010aea0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 --2- 172.21.15.19:0/1449590404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9310103d80 0x7f9310104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 --2- 172.21.15.19:0/1449590404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310102b40 0x7f9310103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 -- 172.21.15.19:0/1449590404 >> 172.21.15.19:0/1449590404 conn(0x7f93100fdd30 msgr2=0x7f93101001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 -- 172.21.15.19:0/1449590404 shutdown_connections 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.619+0000 7f9317fb4640 1 -- 172.21.15.19:0/1449590404 wait complete. 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 Processor -- start 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 -- start start 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9310102b40 0x7f931019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310103d80 0x7f931019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f93101a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e570 con 0x7f9310108a80 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e3f0 con 0x7f9310103d80 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9317fb4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e6f0 con 0x7f9310102b40 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f93101a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f93101a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38346/0 (socket says 172.21.15.19:38346) 2024-05-17T09:22:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 -- 172.21.15.19:0/2095967941 learned_addr learned my addr 172.21.15.19:0/2095967941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f931554f640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310103d80 0x7f931019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9315d50640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9310102b40 0x7f931019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 -- 172.21.15.19:0/2095967941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9310102b40 msgr2=0x7f931019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9310102b40 0x7f931019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 -- 172.21.15.19:0/2095967941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310103d80 msgr2=0x7f931019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.620+0000 7f9316551640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310103d80 0x7f931019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9316551640 1 -- 172.21.15.19:0/2095967941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93101a1cd0 con 0x7f9310108a80 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f931554f640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310103d80 0x7f931019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9316551640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f93101a1660 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9308031040 tx=0x7f9308003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9306ffd640 1 -- 172.21.15.19:0/2095967941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308003d60 con 0x7f9310108a80 2024-05-17T09:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93101a1f60 con 0x7f9310108a80 2024-05-17T09:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9306ffd640 1 -- 172.21.15.19:0/2095967941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93080044d0 con 0x7f9310108a80 2024-05-17T09:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9306ffd640 1 -- 172.21.15.19:0/2095967941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308030c50 con 0x7f9310108a80 2024-05-17T09:22:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.621+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a23c0 con 0x7f9310108a80 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.623+0000 7f931ee61740 1 Processor -- start 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f9306ffd640 1 -- 172.21.15.19:0/2095967941 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9308004040 con 0x7f9310108a80 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 -- start start 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbde3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbde920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d9db50c30 con 0x557d9d8142f0 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557d9db407d0 con 0x557d9d83d1b0 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.624+0000 7f931ee61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557d9db44810 con 0x557d9d81c4d0 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f9315d50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbde920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbde3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f9315d50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbde920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38350/0 (socket says 172.21.15.19:38350) 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f9315d50640 1 -- 172.21.15.19:0/1230398966 learned_addr learned my addr 172.21.15.19:0/1230398966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 -- 172.21.15.19:0/1230398966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 msgr2=0x557d9dbe5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 -- 172.21.15.19:0/1230398966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 msgr2=0x557d9dbde920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbde920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 -- 172.21.15.19:0/1230398966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9d855210 con 0x557d9d83d1b0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f9306ffd640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f92fc0b28a0 0x7f92fc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f931554f640 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbde3a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f931019b230 tx=0x7f9300005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a201697f8d6ac63 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f9315d50640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f92fc0b28a0 0x7f92fc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f92fbfff640 1 -- 172.21.15.19:0/1230398966 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300004030 con 0x557d9d83d1b0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f92fbfff640 1 -- 172.21.15.19:0/1230398966 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f930000f940 con 0x557d9d83d1b0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f9306ffd640 1 -- 172.21.15.19:0/2095967941 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9308103bc0 con 0x7f9310108a80 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.625+0000 7f92fbfff640 1 -- 172.21.15.19:0/1230398966 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300003b40 con 0x557d9d83d1b0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 -- 172.21.15.19:0/1230398966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 msgr2=0x557d9dbde3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f9315d50640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f92fc0b28a0 0x7f92fc0b4db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x557d9dbdf780 tx=0x7f930c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbde3a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f931019b230 tx=0x7f9300005850 comp rx=0 tx=0).stop 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 -- 172.21.15.19:0/1230398966 shutdown_connections 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 --2- 172.21.15.19:0/1230398966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbde3a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.626+0000 7f931ee61740 1 -- 172.21.15.19:0/1230398966 >> 172.21.15.19:0/1230398966 conn(0x557d9d831300 msgr2=0x557d9dbd9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.627+0000 7f931ee61740 1 -- 172.21.15.19:0/1230398966 shutdown_connections 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.627+0000 7f931ee61740 1 -- 172.21.15.19:0/1230398966 wait complete. 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.627+0000 7f931ee61740 1 -- start start 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.627+0000 7f931ee61740 1 Processor -- start 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d8142f0 0x557d9dbe7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d83d1b0 0x557d9dbef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d9d826790 con 0x557d9d83d1b0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557d9dafc890 con 0x557d9d8142f0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557d9dbe6270 con 0x557d9d81c4d0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931ee61740 10 client.0 _ll_register_callbacks cb 0x557d9daffd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f9315d50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d8142f0 0x557d9dbe7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f9315d50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d8142f0 0x557d9dbe7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55894/0 (socket says 172.21.15.19:55894) 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f9315d50640 1 -- 172.21.15.19:0/506171332 learned_addr learned my addr 172.21.15.19:0/506171332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f9316551640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d83d1b0 0x557d9dbef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 -- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d8142f0 msgr2=0x557d9dbe7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d8142f0 0x557d9dbe7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 -- 172.21.15.19:0/506171332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d83d1b0 msgr2=0x557d9dbef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d83d1b0 0x557d9dbef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 -- 172.21.15.19:0/506171332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d9d921620 con 0x557d9d81c4d0 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f9316551640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d83d1b0 0x557d9dbef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f931554f640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe8310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f931019b230 tx=0x7f930000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779aef3a5b35d957 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.628+0000 7f92f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f92f97fa640 1 -- 172.21.15.19:0/506171332 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300013270 con 0x557d9d81c4d0 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f92f97fa640 1 -- 172.21.15.19:0/506171332 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9300005240 con 0x557d9d81c4d0 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930000f940 con 0x557d9d81c4d0 2024-05-17T09:22:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f92f97fa640 1 -- 172.21.15.19:0/506171332 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9300034050 con 0x557d9d81c4d0 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 10 client.25957 fetch_fsmap learned FSMap version 28 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d9dbf2180 con 0x557d9d81c4d0 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f92f97fa640 1 -- 172.21.15.19:0/506171332 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f930000f940 con 0x557d9d81c4d0 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f92f97fa640 1 -- 172.21.15.19:0/506171332 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f930001e070 con 0x557d9d81c4d0 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 10 client.25957 fetch_fsmap finished waiting for FSMap version 28 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 20 client.25957 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7fbd80 0x557d9dbf5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 4 client.25957 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d9db52b80 con 0x557d9d7fbd80 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.629+0000 7f9315d50640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7fbd80 0x557d9dbf5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.630+0000 7f92f97fa640 10 client.25957 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.630+0000 7f9315d50640 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7fbd80 0x557d9dbf5750 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0bcff31cd57d59 server_cookie=748d1f34e7c4e41e in_seq=0 out_seq=0 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.630+0000 7f92f97fa640 1 -- 172.21.15.19:0/506171332 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557d9db52b80 con 0x557d9d7fbd80 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.630+0000 7f92f97fa640 10 client.25957 handle_command_reply: tid=0 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 client.25957 shutdown 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 msgr2=0x557d9dbe8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe8310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f931019b230 tx=0x7f930000ff90 comp rx=0 tx=0).stop 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 shutdown_connections 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7fbd80 0x557d9dbf5750 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d83d1b0 0x557d9dbef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbe8310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f92f97fa640 0 client.25957 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 --2- 172.21.15.19:0/506171332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d8142f0 0x557d9dbe7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 >> 172.21.15.19:0/506171332 conn(0x557d9d831300 msgr2=0x557d9dbd8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 shutdown_connections 2024-05-17T09:22:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 1 -- 172.21.15.19:0/506171332 wait complete. 2024-05-17T09:22:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.631+0000 7f931ee61740 20 client.25957 trim_cache size 0 max 16384 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 Processor -- start 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 -- start start 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbd8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b13d0 con 0x557d9d8142f0 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93101aeca0 con 0x557d9d83d1b0 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f931ee61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93101acaf0 con 0x557d9d81c4d0 2024-05-17T09:22:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f9315d50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbd8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f9315d50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbd8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38378/0 (socket says 172.21.15.19:38378) 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f9315d50640 1 -- 172.21.15.19:0/3390618874 learned_addr learned my addr 172.21.15.19:0/3390618874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.648+0000 7f9316551640 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f931554f640 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f9315d50640 1 -- 172.21.15.19:0/3390618874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 msgr2=0x557d9dbef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f9315d50640 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbef120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f9315d50640 1 -- 172.21.15.19:0/3390618874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 msgr2=0x557d9dbef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f9315d50640 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f9315d50640 1 -- 172.21.15.19:0/3390618874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9dbe56f0 con 0x557d9d8142f0 2024-05-17T09:22:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f9315d50640 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbd8330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f930c004620 tx=0x7f930c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53072089a5b72002 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f92fb7fe640 1 -- 172.21.15.19:0/3390618874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c005020 con 0x557d9d8142f0 2024-05-17T09:22:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f92fb7fe640 1 -- 172.21.15.19:0/3390618874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f930c0047a0 con 0x557d9d8142f0 2024-05-17T09:22:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.649+0000 7f92fb7fe640 1 -- 172.21.15.19:0/3390618874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c005650 con 0x557d9d8142f0 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 -- 172.21.15.19:0/3390618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 msgr2=0x557d9dbd8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbd8330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f930c004620 tx=0x7f930c01a000 comp rx=0 tx=0).stop 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 -- 172.21.15.19:0/3390618874 shutdown_connections 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dbef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d81c4d0 0x557d9dbef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 --2- 172.21.15.19:0/3390618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d8142f0 0x557d9dbd8330 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 -- 172.21.15.19:0/3390618874 >> 172.21.15.19:0/3390618874 conn(0x557d9d831300 msgr2=0x557d9dbe17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 -- 172.21.15.19:0/3390618874 shutdown_connections 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 -- 172.21.15.19:0/3390618874 wait complete. 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.650+0000 7f931ee61740 1 -- start start 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 Processor -- start 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d8142f0 0x557d9dbecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 0x557d9dbed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dc5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93101ad160 con 0x557d9d81c4d0 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93101ae5d0 con 0x557d9d83d1b0 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93101ad830 con 0x557d9d8142f0 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931ee61740 10 client.0 _ll_register_callbacks cb 0x7f9300005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931554f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 0x557d9dbed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931554f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 0x557d9dbed040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38380/0 (socket says 172.21.15.19:38380) 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f931554f640 1 -- 172.21.15.19:0/343164793 learned_addr learned my addr 172.21.15.19:0/343164793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f9316551640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dc5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.651+0000 7f9315d50640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d8142f0 0x557d9dbecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931554f640 1 -- 172.21.15.19:0/343164793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d8142f0 msgr2=0x557d9dbecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931554f640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d8142f0 0x557d9dbecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931554f640 1 -- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 msgr2=0x557d9dc5bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931554f640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dc5bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931554f640 1 -- 172.21.15.19:0/343164793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d9dc5c390 con 0x557d9d81c4d0 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f9316551640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dc5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931554f640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 0x557d9dbed040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f930002e690 tx=0x7f9300009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33636468b7488e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f92f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f92f97fa640 1 -- 172.21.15.19:0/343164793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300031030 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f92f97fa640 1 -- 172.21.15.19:0/343164793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f930002e0b0 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c0047a0 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f92f97fa640 1 -- 172.21.15.19:0/343164793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f930003f050 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931ee61740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d9dc5d620 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.652+0000 7f92f97fa640 1 -- 172.21.15.19:0/343164793 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f930c0047a0 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f92f97fa640 1 -- 172.21.15.19:0/343164793 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f9300037020 con 0x557d9d81c4d0 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f931ee61740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f931ee61740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f931ee61740 20 client.16281 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f931ee61740 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7d11c0 0x557d9dc60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f931ee61740 4 client.16281 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:22:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d9db4c360 con 0x557d9d7d11c0 2024-05-17T09:22:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f9315d50640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7d11c0 0x557d9dc60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f9315d50640 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7d11c0 0x557d9dc60060 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151ffc22f9452270 server_cookie=55565698d946b15c in_seq=0 out_seq=0 2024-05-17T09:22:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.653+0000 7f92f97fa640 10 client.16281 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f92f97fa640 1 -- 172.21.15.19:0/343164793 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557d9db4c360 con 0x557d9d7d11c0 2024-05-17T09:22:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f92f97fa640 10 client.16281 handle_command_reply: tid=0 2024-05-17T09:22:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 client.16281 shutdown 2024-05-17T09:22:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 msgr2=0x557d9dbed040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 0x557d9dbed040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f930002e690 tx=0x7f9300009b30 comp rx=0 tx=0).stop 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 shutdown_connections 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557d9d7d11c0 0x557d9dc60060 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d9d83d1b0 0x557d9dc5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d9d81c4d0 0x557d9dbed040 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f92f97fa640 0 client.16281 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 --2- 172.21.15.19:0/343164793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d9d8142f0 0x557d9dbecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 >> 172.21.15.19:0/343164793 conn(0x557d9d831300 msgr2=0x557d9dbe21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 shutdown_connections 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 1 -- 172.21.15.19:0/343164793 wait complete. 2024-05-17T09:22:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.687+0000 7f931ee61740 20 client.16281 trim_cache size 0 max 16384 2024-05-17T09:22:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:22:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:22:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f92fc0b28a0 msgr2=0x7f92fc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f92fc0b28a0 0x7f92fc0b4db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x557d9dbdf780 tx=0x7f930c005f90 comp rx=0 tx=0).stop 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 msgr2=0x7f93101a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f93101a1660 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9308031040 tx=0x7f9308003970 comp rx=0 tx=0).stop 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 shutdown_connections 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f92fc0b28a0 0x7f92fc0b4db0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9310108a80 0x7f93101a1660 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9310103d80 0x7f931019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 --2- 172.21.15.19:0/2095967941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9310102b40 0x7f931019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 >> 172.21.15.19:0/2095967941 conn(0x7f93100fdd30 msgr2=0x7f93101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 shutdown_connections 2024-05-17T09:22:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:33.688+0000 7f9317fb4640 1 -- 172.21.15.19:0/2095967941 wait complete. 2024-05-17T09:22:34.416 INFO:tasks.workunit.client.0.smithi019.stdout: 1 19382 3.80 MB/sec warmup 32 sec latency 24.270 ms 2024-05-17T09:22:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:34 smithi069 ceph-mon[35081]: pgmap v522: 129 pgs: 129 active+clean; 26 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-17T09:22:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:34 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:34 smithi019 ceph-mon[31620]: pgmap v522: 129 pgs: 129 active+clean; 26 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-17T09:22:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:34 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:34 smithi079 ceph-mon[35502]: pgmap v522: 129 pgs: 129 active+clean; 26 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-17T09:22:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:34 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:34.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 -- 172.21.15.19:0/3529561974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8628107de0 msgr2=0x7f86281081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 --2- 172.21.15.19:0/3529561974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8628107de0 0x7f86281081f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f86180099c0 tx=0x7f8618009cd0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 -- 172.21.15.19:0/3529561974 shutdown_connections 2024-05-17T09:22:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 --2- 172.21.15.19:0/3529561974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86281034b0 0x7f86281038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 --2- 172.21.15.19:0/3529561974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 0x7f8628110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 --2- 172.21.15.19:0/3529561974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8628107de0 0x7f86281081f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 -- 172.21.15.19:0/3529561974 >> 172.21.15.19:0/3529561974 conn(0x7f86280fedb0 msgr2=0x7f86281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 -- 172.21.15.19:0/3529561974 shutdown_connections 2024-05-17T09:22:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.778+0000 7f862f329700 1 -- 172.21.15.19:0/3529561974 wait complete. 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 Processor -- start 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 -- start start 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 0x7f862819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8628107de0 0x7f862819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 0x7f86281a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86281131f0 con 0x7f86281034b0 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8628113070 con 0x7f8628108730 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862f329700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8628113370 con 0x7f8628107de0 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 0x7f862819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 0x7f862819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35922/0 (socket says 172.21.15.19:35922) 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862d0c5700 1 -- 172.21.15.19:0/2802018462 learned_addr learned my addr 172.21.15.19:0/2802018462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862c8c4700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8628107de0 0x7f862819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.779+0000 7f862d8c6700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 0x7f86281a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d0c5700 1 -- 172.21.15.19:0/2802018462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8628107de0 msgr2=0x7f862819c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d0c5700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8628107de0 0x7f862819c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d0c5700 1 -- 172.21.15.19:0/2802018462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 msgr2=0x7f86281a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d0c5700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 0x7f86281a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d0c5700 1 -- 172.21.15.19:0/2802018462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862819c8e0 con 0x7f86281034b0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862c8c4700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8628107de0 0x7f862819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d8c6700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 0x7f86281a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862d0c5700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 0x7f862819baf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f862400e9a0 tx=0x7f862400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f861e7fc700 1 -- 172.21.15.19:0/2802018462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400ca10 con 0x7f86281034b0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f861e7fc700 1 -- 172.21.15.19:0/2802018462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f862400cbb0 con 0x7f86281034b0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86281a71d0 con 0x7f86281034b0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f861e7fc700 1 -- 172.21.15.19:0/2802018462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624018680 con 0x7f86281034b0 2024-05-17T09:22:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.780+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86281a7690 con 0x7f86281034b0 2024-05-17T09:22:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.784+0000 7f861e7fc700 1 -- 172.21.15.19:0/2802018462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8624018820 con 0x7f86281034b0 2024-05-17T09:22:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.784+0000 7f863dd3eb80 1 Processor -- start 2024-05-17T09:22:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f861e7fc700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86140b2880 0x7f86140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 -- start start 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f862d8c6700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86140b2880 0x7f86140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f861e7fc700 1 -- 172.21.15.19:0/2802018462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86240e27d0 con 0x7f86281034b0 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826318e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x55826318e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263188be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5582630b2c90 con 0x5582630833d0 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5582630a2830 con 0x558263005e40 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5582630a6870 con 0x5582631887d0 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826318e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826318e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35926/0 (socket says 172.21.15.19:35926) 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f862d0c5700 1 -- 172.21.15.19:0/1598960101 learned_addr learned my addr 172.21.15.19:0/1598960101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f862d8c6700 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263188be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.785+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x55826318e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862c8c4700 1 -- 172.21.15.19:0/1598960101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 msgr2=0x558263188be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263188be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862c8c4700 1 -- 172.21.15.19:0/1598960101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 msgr2=0x55826318e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862d8c6700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86140b2880 0x7f86140b4d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8620003ee0 tx=0x7f8620004320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826318e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862c8c4700 1 -- 172.21.15.19:0/1598960101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558262ff28d0 con 0x558263005e40 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862d0c5700 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826318e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x55826318e8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f861800b580 tx=0x7f8618005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd75ae611c436e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.786+0000 7f86137fe700 1 -- 172.21.15.19:0/1598960101 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618027070 con 0x558263005e40 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.787+0000 7f86137fe700 1 -- 172.21.15.19:0/1598960101 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f861800bcf0 con 0x558263005e40 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.787+0000 7f86137fe700 1 -- 172.21.15.19:0/1598960101 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618003bb0 con 0x558263005e40 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1598960101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 msgr2=0x55826318e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x55826318e8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f861800b580 tx=0x7f8618005fb0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1598960101 shutdown_connections 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263188be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826318e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1598960101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x55826318e8c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1598960101 >> 172.21.15.19:0/1598960101 conn(0x558263044c80 msgr2=0x55826317eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:34.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1598960101 shutdown_connections 2024-05-17T09:22:34.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1598960101 wait complete. 2024-05-17T09:22:34.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.788+0000 7f863dd3eb80 1 -- start start 2024-05-17T09:22:34.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 Processor -- start 2024-05-17T09:22:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x5582631920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x5582631925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263192b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558262fa8fa0 con 0x5582630833d0 2024-05-17T09:22:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5582630447c0 con 0x558263005e40 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558263189600 con 0x5582631887d0 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f863dd3eb80 10 client.0 _ll_register_callbacks cb 0x55826303d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f862c8c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x5582631925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x5582631920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f862c8c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x5582631925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35932/0 (socket says 172.21.15.19:35932) 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f862c8c4700 1 -- 172.21.15.19:0/889875795 learned_addr learned my addr 172.21.15.19:0/889875795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.789+0000 7f862d8c6700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263192b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862d0c5700 1 -- 172.21.15.19:0/889875795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 msgr2=0x558263192b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862d0c5700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263192b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862d0c5700 1 -- 172.21.15.19:0/889875795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 msgr2=0x5582631925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862d0c5700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x5582631925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862d0c5700 1 -- 172.21.15.19:0/889875795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558263041690 con 0x558263005e40 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862c8c4700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x5582631925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f862d0c5700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x5582631920b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f86240a6cb0 tx=0x7f86240a5300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b259058b1b947f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f8610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f8610ff9700 1 -- 172.21.15.19:0/889875795 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240addf0 con 0x558263005e40 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f8610ff9700 1 -- 172.21.15.19:0/889875795 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86240b2410 con 0x558263005e40 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f861800bcf0 con 0x558263005e40 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f8610ff9700 1 -- 172.21.15.19:0/889875795 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86240b28a0 con 0x558263005e40 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f863dd3eb80 10 client.25976 fetch_fsmap learned FSMap version 28 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.790+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55826319fa40 con 0x558263005e40 2024-05-17T09:22:34.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f8610ff9700 1 -- 172.21.15.19:0/889875795 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f861800bcf0 con 0x558263005e40 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f8610ff9700 1 -- 172.21.15.19:0/889875795 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f8624018b10 con 0x558263005e40 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f863dd3eb80 10 client.25976 fetch_fsmap finished waiting for FSMap version 28 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f863dd3eb80 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f863dd3eb80 20 client.25976 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5582631a2070 0x5582631a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f863dd3eb80 4 client.25976 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582630b4be0 con 0x5582631a2070 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.791+0000 7f862c8c4700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5582631a2070 0x5582631a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.792+0000 7f8610ff9700 10 client.25976 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.792+0000 7f862c8c4700 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5582631a2070 0x5582631a44d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401672f30350db94 server_cookie=21a7c30313498db0 in_seq=0 out_seq=0 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.792+0000 7f8610ff9700 1 -- 172.21.15.19:0/889875795 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5582630b4be0 con 0x5582631a2070 2024-05-17T09:22:34.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.792+0000 7f8610ff9700 10 client.25976 handle_command_reply: tid=0 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.792+0000 7f863dd3eb80 1 client.25976 shutdown 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 msgr2=0x5582631920b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x5582631920b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f86240a6cb0 tx=0x7f86240a5300 comp rx=0 tx=0).stop 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 shutdown_connections 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582631887d0 0x558263192b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5582631a2070 0x5582631a44d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558263005e40 0x5582631920b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/889875795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x5582631925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 >> 172.21.15.19:0/889875795 conn(0x558263044c80 msgr2=0x55826317eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f8610ff9700 0 client.25976 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 shutdown_connections 2024-05-17T09:22:34.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 1 -- 172.21.15.19:0/889875795 wait complete. 2024-05-17T09:22:34.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.793+0000 7f863dd3eb80 20 client.25976 trim_cache size 0 max 16384 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 Processor -- start 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 -- start start 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 0x558263194550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582630833d0 0x558263192970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86281b08a0 con 0x558263005e40 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86281ae130 con 0x5582631887d0 2024-05-17T09:22:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86281abf80 con 0x5582630833d0 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 0x558263194550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.811+0000 7f862d0c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 0x558263194550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35948/0 (socket says 172.21.15.19:35948) 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 -- 172.21.15.19:0/988822821 learned_addr learned my addr 172.21.15.19:0/988822821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862c8c4700 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582630833d0 0x558263192970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d8c6700 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 -- 172.21.15.19:0/988822821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582630833d0 msgr2=0x558263192970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582630833d0 0x558263192970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 -- 172.21.15.19:0/988822821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 msgr2=0x55826319dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 -- 172.21.15.19:0/988822821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558263178c80 con 0x558263005e40 2024-05-17T09:22:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f862d0c5700 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 0x558263194550 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f86240dff70 tx=0x7f8624009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91837bca585cc8d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f8612ffd700 1 -- 172.21.15.19:0/988822821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240addf0 con 0x558263005e40 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f8612ffd700 1 -- 172.21.15.19:0/988822821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86240a9a40 con 0x558263005e40 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.812+0000 7f8612ffd700 1 -- 172.21.15.19:0/988822821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240a9810 con 0x558263005e40 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 -- 172.21.15.19:0/988822821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 msgr2=0x558263194550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 0x558263194550 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f86240dff70 tx=0x7f8624009790 comp rx=0 tx=0).stop 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 -- 172.21.15.19:0/988822821 shutdown_connections 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558263005e40 0x558263194550 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/988822821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5582630833d0 0x558263192970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 -- 172.21.15.19:0/988822821 >> 172.21.15.19:0/988822821 conn(0x558263044c80 msgr2=0x5582631909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 -- 172.21.15.19:0/988822821 shutdown_connections 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.813+0000 7f863dd3eb80 1 -- 172.21.15.19:0/988822821 wait complete. 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 -- start start 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 Processor -- start 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558263005e40 0x55826318fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826319a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86281ac5f0 con 0x5582630833d0 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86281ada60 con 0x5582631887d0 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.814+0000 7f863dd3eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86281accc0 con 0x558263005e40 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f863dd3eb80 10 client.0 _ll_register_callbacks cb 0x55826303d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826319a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826319a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35958/0 (socket says 172.21.15.19:35958) 2024-05-17T09:22:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 -- 172.21.15.19:0/1550214199 learned_addr learned my addr 172.21.15.19:0/1550214199 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862d8c6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862d0c5700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558263005e40 0x55826318fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 -- 172.21.15.19:0/1550214199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558263005e40 msgr2=0x55826318fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558263005e40 0x55826318fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 -- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 msgr2=0x55826319a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 -- 172.21.15.19:0/1550214199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826318e360 con 0x5582630833d0 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862d8c6700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f862c8c4700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826319a1a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8618005910 tx=0x7f8618005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd12e89c54832877 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f8610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f8610ff9700 1 -- 172.21.15.19:0/1550214199 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618027040 con 0x5582630833d0 2024-05-17T09:22:34.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.815+0000 7f8610ff9700 1 -- 172.21.15.19:0/1550214199 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86180039c0 con 0x5582630833d0 2024-05-17T09:22:34.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86240a9a40 con 0x5582630833d0 2024-05-17T09:22:34.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f8610ff9700 1 -- 172.21.15.19:0/1550214199 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f861800bcf0 con 0x5582630833d0 2024-05-17T09:22:34.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 10 client.16299 fetch_fsmap learned FSMap version 28 2024-05-17T09:22:34.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55826320bc90 con 0x5582630833d0 2024-05-17T09:22:34.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f8610ff9700 1 -- 172.21.15.19:0/1550214199 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86240a9a40 con 0x5582630833d0 2024-05-17T09:22:34.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f8610ff9700 1 -- 172.21.15.19:0/1550214199 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f8618021070 con 0x5582630833d0 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 20 client.16299 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55826320beb0 0x55826320e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 4 client.16299 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582630ae3c0 con 0x55826320beb0 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.816+0000 7f862d0c5700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55826320beb0 0x55826320e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.817+0000 7f8610ff9700 10 client.16299 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.817+0000 7f862d0c5700 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55826320beb0 0x55826320e2d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c762767505311eb server_cookie=dc621699e5eec910 in_seq=0 out_seq=0 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f8610ff9700 1 -- 172.21.15.19:0/1550214199 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582630ae3c0 con 0x55826320beb0 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f8610ff9700 10 client.16299 handle_command_reply: tid=0 2024-05-17T09:22:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 client.16299 shutdown 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 msgr2=0x55826319a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826319a1a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8618005910 tx=0x7f8618005940 comp rx=0 tx=0).stop 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 shutdown_connections 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558263005e40 0x55826318fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55826320beb0 0x55826320e2d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582631887d0 0x55826319a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 --2- 172.21.15.19:0/1550214199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582630833d0 0x55826319a1a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 >> 172.21.15.19:0/1550214199 conn(0x558263044c80 msgr2=0x558263178f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f8610ff9700 0 client.16299 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 shutdown_connections 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 1 -- 172.21.15.19:0/1550214199 wait complete. 2024-05-17T09:22:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.818+0000 7f863dd3eb80 20 client.16299 trim_cache size 0 max 16384 2024-05-17T09:22:34.822 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86140b2880 msgr2=0x7f86140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86140b2880 0x7f86140b4d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8620003ee0 tx=0x7f8620004320 comp rx=0 tx=0).stop 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 msgr2=0x7f862819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 0x7f862819baf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f862400e9a0 tx=0x7f862400ecb0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 shutdown_connections 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8628108730 0x7f86281a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86281034b0 0x7f862819baf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86140b2880 0x7f86140b4d90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 --2- 172.21.15.19:0/2802018462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8628107de0 0x7f862819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 >> 172.21.15.19:0/2802018462 conn(0x7f86280fedb0 msgr2=0x7f86281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 shutdown_connections 2024-05-17T09:22:34.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:34.819+0000 7f862f329700 1 -- 172.21.15.19:0/2802018462 wait complete. 2024-05-17T09:22:35.416 INFO:tasks.workunit.client.0.smithi019.stdout: 1 20077 3.74 MB/sec warmup 33 sec latency 29.342 ms 2024-05-17T09:22:36.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 20542 3.75 MB/sec warmup 34 sec latency 16.089 ms 2024-05-17T09:22:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:36 smithi069 ceph-mon[35081]: pgmap v523: 129 pgs: 129 active+clean; 26 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-17T09:22:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:36 smithi019 ceph-mon[31620]: pgmap v523: 129 pgs: 129 active+clean; 26 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-17T09:22:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:36 smithi079 ceph-mon[35502]: pgmap v523: 129 pgs: 129 active+clean; 26 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-05-17T09:22:36.803 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-17T09:22:36.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-17T09:22:37.088 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:37.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 21110 3.74 MB/sec warmup 35 sec latency 15.829 ms 2024-05-17T09:22:38.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 21595 3.76 MB/sec warmup 36 sec latency 16.096 ms 2024-05-17T09:22:38.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:38 smithi019 ceph-mon[31620]: pgmap v524: 129 pgs: 129 active+clean; 28 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-17T09:22:38.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:38 smithi069 ceph-mon[35081]: pgmap v524: 129 pgs: 129 active+clean; 28 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-17T09:22:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:38 smithi079 ceph-mon[35502]: pgmap v524: 129 pgs: 129 active+clean; 28 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-17T09:22:38.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.977+0000 7fdab5a26700 1 -- 172.21.15.19:0/2551345956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0108730 msgr2=0x7fdab0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 --2- 172.21.15.19:0/2551345956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0108730 0x7fdab0110090 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b0d0 tx=0x7fdaa800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 -- 172.21.15.19:0/2551345956 shutdown_connections 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 --2- 172.21.15.19:0/2551345956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0108730 0x7fdab0110090 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 --2- 172.21.15.19:0/2551345956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab01034b0 0x7fdab01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 --2- 172.21.15.19:0/2551345956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0107de0 0x7fdab01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 -- 172.21.15.19:0/2551345956 >> 172.21.15.19:0/2551345956 conn(0x7fdab00fedb0 msgr2=0x7fdab01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 -- 172.21.15.19:0/2551345956 shutdown_connections 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 -- 172.21.15.19:0/2551345956 wait complete. 2024-05-17T09:22:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.978+0000 7fdab5a26700 1 Processor -- start 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 -- start start 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01034b0 0x7fdab019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0107de0 0x7fdab019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab0108730 0x7fdab019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0113260 con 0x7fdab0107de0 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01130e0 con 0x7fdab0108730 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdab5a26700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01133e0 con 0x7fdab01034b0 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdaaeffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01034b0 0x7fdab019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0107de0 0x7fdab019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdaaeffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01034b0 0x7fdab019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:44616/0 (socket says 172.21.15.19:44616) 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdaaeffd700 1 -- 172.21.15.19:0/1198987035 learned_addr learned my addr 172.21.15.19:0/1198987035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.979+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab0108730 0x7fdab019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdaaf7fe700 1 -- 172.21.15.19:0/1198987035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01034b0 msgr2=0x7fdab019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:38.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01034b0 0x7fdab019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdaaf7fe700 1 -- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0107de0 msgr2=0x7fdab019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0107de0 0x7fdab019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdaaf7fe700 1 -- 172.21.15.19:0/1198987035 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab019fc30 con 0x7fdab0108730 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab0108730 0x7fdab019f590 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b770 tx=0x7fdaa8007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdab4a24700 1 -- 172.21.15.19:0/1198987035 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa801f070 con 0x7fdab0108730 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdab4a24700 1 -- 172.21.15.19:0/1198987035 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa8007dc0 con 0x7fdab0108730 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab019fec0 con 0x7fdab0108730 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdab4a24700 1 -- 172.21.15.19:0/1198987035 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa801b430 con 0x7fdab0108730 2024-05-17T09:22:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.980+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a03a0 con 0x7fdab0108730 2024-05-17T09:22:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.983+0000 7fdab4a24700 1 -- 172.21.15.19:0/1198987035 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdaa8022070 con 0x7fdab0108730 2024-05-17T09:22:38.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.984+0000 7fdab4a24700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda980b2880 0x7fda980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.984+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda980b2880 0x7fda980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.984+0000 7fdab4a24700 1 -- 172.21.15.19:0/1198987035 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdaa800f020 con 0x7fdab0108730 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.984+0000 7fdac443bb80 1 Processor -- start 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 -- start start 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda980b2880 0x7fda980b4d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009d20 tx=0x7fdaa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba21099990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba21099f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba21094220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba20fc1df0 con 0x55ba21093d50 2024-05-17T09:22:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba20fb1990 con 0x55ba20f14ec0 2024-05-17T09:22:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdac443bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba20fb59d0 con 0x55ba210004b0 2024-05-17T09:22:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba21099f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaaf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba21094220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba21099f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58302/0 (socket says 172.21.15.19:58302) 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 -- 172.21.15.19:0/296357143 learned_addr learned my addr 172.21.15.19:0/296357143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaaf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba21094220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35994/0 (socket says 172.21.15.19:35994) 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba21099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 -- 172.21.15.19:0/296357143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 msgr2=0x55ba21099990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba21099990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 -- 172.21.15.19:0/296357143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 msgr2=0x55ba21094220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba21094220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.985+0000 7fdaae7fc700 1 -- 172.21.15.19:0/296357143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba20f5b030 con 0x55ba20f14ec0 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.986+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba21099f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009cb0 tx=0x7fdaa4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a44767e0083561 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.986+0000 7fda95ffb700 1 -- 172.21.15.19:0/296357143 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4011d20 con 0x55ba20f14ec0 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.986+0000 7fda95ffb700 1 -- 172.21.15.19:0/296357143 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa40044e0 con 0x55ba20f14ec0 2024-05-17T09:22:38.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.986+0000 7fda95ffb700 1 -- 172.21.15.19:0/296357143 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4005220 con 0x55ba20f14ec0 2024-05-17T09:22:38.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 -- 172.21.15.19:0/296357143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 msgr2=0x55ba21099f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba21099f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009cb0 tx=0x7fdaa4011440 comp rx=0 tx=0).stop 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 -- 172.21.15.19:0/296357143 shutdown_connections 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba21099990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba21094220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 --2- 172.21.15.19:0/296357143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba21099f30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 -- 172.21.15.19:0/296357143 >> 172.21.15.19:0/296357143 conn(0x55ba20f75240 msgr2=0x55ba2108e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 -- 172.21.15.19:0/296357143 shutdown_connections 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 -- 172.21.15.19:0/296357143 wait complete. 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.987+0000 7fdac443bb80 1 -- start start 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 Processor -- start 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba2109d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba2109db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba2109e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba20eb8020 con 0x55ba21093d50 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba20f19e20 con 0x55ba20f14ec0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba210a0620 con 0x55ba210004b0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdaaeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba2109d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdac443bb80 10 client.0 _ll_register_callbacks cb 0x55ba20f4c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdaaf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba2109e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdaaeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba2109d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58306/0 (socket says 172.21.15.19:58306) 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdaaeffd700 1 -- 172.21.15.19:0/2121032245 learned_addr learned my addr 172.21.15.19:0/2121032245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.988+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba2109db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaaeffd700 1 -- 172.21.15.19:0/2121032245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 msgr2=0x55ba2109db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba2109db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaaeffd700 1 -- 172.21.15.19:0/2121032245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 msgr2=0x55ba2109e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba2109e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaaeffd700 1 -- 172.21.15.19:0/2121032245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba20f01950 con 0x55ba20f14ec0 2024-05-17T09:22:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba2109db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:38.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba2109d620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaa001ecd0 tx=0x7fdaa00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d092f4d3fdc6a7b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:38.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fda8effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:38.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fda8effd700 1 -- 172.21.15.19:0/2121032245 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa002b060 con 0x55ba20f14ec0 2024-05-17T09:22:38.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.989+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa40044e0 con 0x55ba20f14ec0 2024-05-17T09:22:38.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fda8effd700 1 -- 172.21.15.19:0/2121032245 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdaa000ea40 con 0x55ba20f14ec0 2024-05-17T09:22:38.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fda8effd700 1 -- 172.21.15.19:0/2121032245 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdaa0032050 con 0x55ba20f14ec0 2024-05-17T09:22:38.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fdac443bb80 10 client.25994 fetch_fsmap learned FSMap version 29 2024-05-17T09:22:38.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba20f6b7e0 con 0x55ba20f14ec0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fda8effd700 1 -- 172.21.15.19:0/2121032245 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa40044e0 con 0x55ba20f14ec0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fda8effd700 1 -- 172.21.15.19:0/2121032245 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa0025030 con 0x55ba20f14ec0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fdac443bb80 10 client.25994 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.990+0000 7fdac443bb80 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fdac443bb80 20 client.25994 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fdac443bb80 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba210ad5b0 0x55ba210af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fdac443bb80 4 client.25994 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba2105ff50 con 0x55ba210ad5b0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba210ad5b0 0x55ba210af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fda8effd700 10 client.25994 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.991+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba210ad5b0 0x55ba210af9f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4c4ff2b0fb428e server_cookie=e886cf8e333169b2 in_seq=0 out_seq=0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.992+0000 7fda8effd700 1 -- 172.21.15.19:0/2121032245 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ba2105ff50 con 0x55ba210ad5b0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.992+0000 7fda8effd700 10 client.25994 handle_command_reply: tid=0 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.992+0000 7fdac443bb80 1 client.25994 shutdown 2024-05-17T09:22:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.992+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 msgr2=0x55ba2109d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.992+0000 7fdac443bb80 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba2109d620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaa001ecd0 tx=0x7fdaa00047c0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.992+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 shutdown_connections 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba210ad5b0 0x55ba210af9f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba21093d50 0x55ba2109e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba20f14ec0 0x55ba2109d620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 --2- 172.21.15.19:0/2121032245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba210004b0 0x55ba2109db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 >> 172.21.15.19:0/2121032245 conn(0x55ba20f75240 msgr2=0x55ba2108e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fda8effd700 0 client.25994 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 shutdown_connections 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 1 -- 172.21.15.19:0/2121032245 wait complete. 2024-05-17T09:22:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:38.993+0000 7fdac443bb80 20 client.25994 trim_cache size 0 max 16384 2024-05-17T09:22:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 Processor -- start 2024-05-17T09:22:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 -- start start 2024-05-17T09:22:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba2109fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba21098cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01b08a0 con 0x55ba210004b0 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ae130 con 0x55ba21093d50 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdac443bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01abf80 con 0x55ba20f14ec0 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba21098cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba21098cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36012/0 (socket says 172.21.15.19:36012) 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 -- 172.21.15.19:0/1245631572 learned_addr learned my addr 172.21.15.19:0/1245631572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba2109fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 -- 172.21.15.19:0/1245631572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 msgr2=0x55ba2109fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba2109fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 -- 172.21.15.19:0/1245631572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 msgr2=0x55ba210a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.016+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.017+0000 7fdaae7fc700 1 -- 172.21.15.19:0/1245631572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2109bb90 con 0x55ba210004b0 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.017+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.017+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba21098cd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011790 tx=0x7fdaa4009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c180dbaf5fad648f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.017+0000 7fda957fa700 1 -- 172.21.15.19:0/1245631572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4011d20 con 0x55ba210004b0 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.017+0000 7fda957fa700 1 -- 172.21.15.19:0/1245631572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa4003b10 con 0x55ba210004b0 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.017+0000 7fda957fa700 1 -- 172.21.15.19:0/1245631572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40207d0 con 0x55ba210004b0 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 -- 172.21.15.19:0/1245631572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 msgr2=0x55ba21098cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba21098cd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011790 tx=0x7fdaa4009710 comp rx=0 tx=0).stop 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 -- 172.21.15.19:0/1245631572 shutdown_connections 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba2109fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1245631572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba21098cd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 -- 172.21.15.19:0/1245631572 >> 172.21.15.19:0/1245631572 conn(0x55ba20f75240 msgr2=0x55ba21092d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:39.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 -- 172.21.15.19:0/1245631572 shutdown_connections 2024-05-17T09:22:39.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 -- 172.21.15.19:0/1245631572 wait complete. 2024-05-17T09:22:39.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.018+0000 7fdac443bb80 1 -- start start 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 Processor -- start 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba210a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba210a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ac5f0 con 0x55ba210004b0 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ada60 con 0x55ba21093d50 2024-05-17T09:22:39.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01accc0 con 0x55ba20f14ec0 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdac443bb80 10 client.0 _ll_register_callbacks cb 0x55ba20f4c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba210a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdaae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba210a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36018/0 (socket says 172.21.15.19:36018) 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdaae7fc700 1 -- 172.21.15.19:0/1542852971 learned_addr learned my addr 172.21.15.19:0/1542852971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.019+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba210a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaaf7fe700 1 -- 172.21.15.19:0/1542852971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 msgr2=0x55ba210a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba210a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaaf7fe700 1 -- 172.21.15.19:0/1542852971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 msgr2=0x55ba210a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba210a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaaf7fe700 1 -- 172.21.15.19:0/1542852971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba210a2d80 con 0x55ba21093d50 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaae7fc700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba210a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdaaf7fe700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a2840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80a4fd0 tx=0x7fdaa80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb6886c72944d3a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fda8effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fda8effd700 1 -- 172.21.15.19:0/1542852971 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80aafb0 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa4003b10 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fda8effd700 1 -- 172.21.15.19:0/1542852971 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdaa80b10b0 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fda8effd700 1 -- 172.21.15.19:0/1542852971 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdaa80aa8b0 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdac443bb80 10 client.26000 fetch_fsmap learned FSMap version 29 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.020+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba21118800 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fda8effd700 1 -- 172.21.15.19:0/1542852971 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaa4003b10 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fda8effd700 1 -- 172.21.15.19:0/1542852971 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa8011070 con 0x55ba21093d50 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdac443bb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 29 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdac443bb80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdac443bb80 20 client.26000 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba21118f40 0x55ba2111b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdac443bb80 4 client.26000 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-17T09:22:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba20fc3d40 con 0x55ba21118f40 2024-05-17T09:22:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba21118f40 0x55ba2111b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.021+0000 7fda8effd700 10 client.26000 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.022+0000 7fdaaeffd700 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba21118f40 0x55ba2111b360 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c454c7a622b8c08 server_cookie=4fe05470e20f9432 in_seq=0 out_seq=0 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fda8effd700 1 -- 172.21.15.19:0/1542852971 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba20fc3d40 con 0x55ba21118f40 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fda8effd700 10 client.26000 handle_command_reply: tid=0 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 client.26000 shutdown 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 msgr2=0x55ba210a2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a2840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80a4fd0 tx=0x7fdaa80099e0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 shutdown_connections 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba20f14ec0 0x55ba210a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ba21118f40 0x55ba2111b360 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba21093d50 0x55ba210a2840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 --2- 172.21.15.19:0/1542852971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba210004b0 0x55ba210a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 >> 172.21.15.19:0/1542852971 conn(0x55ba20f75240 msgr2=0x55ba21097a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fda8effd700 0 client.26000 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 shutdown_connections 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 1 -- 172.21.15.19:0/1542852971 wait complete. 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.023+0000 7fdac443bb80 20 client.26000 trim_cache size 0 max 16384 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "573b75b0-98c4-4dcc-8c59-96a9b986cdb2", 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-17T09:22:39.026 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda980b2880 msgr2=0x7fda980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda980b2880 0x7fda980b4d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009d20 tx=0x7fdaa0009260 comp rx=0 tx=0).stop 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab0108730 msgr2=0x7fdab019f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab0108730 0x7fdab019f590 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b770 tx=0x7fdaa8007980 comp rx=0 tx=0).stop 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 shutdown_connections 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda980b2880 0x7fda980b4d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01034b0 0x7fdab019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdab0108730 0x7fdab019f590 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 --2- 172.21.15.19:0/1198987035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0107de0 0x7fdab019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 >> 172.21.15.19:0/1198987035 conn(0x7fdab00fedb0 msgr2=0x7fdab006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 shutdown_connections 2024-05-17T09:22:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:39.024+0000 7fdab5a26700 1 -- 172.21.15.19:0/1198987035 wait complete. 2024-05-17T09:22:39.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 21840 3.67 MB/sec warmup 37 sec latency 22.766 ms 2024-05-17T09:22:39.430 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:22:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h[64444]: 2024-05-17T09:22:39.030+0000 7f9afd89d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004bf [2,head] /volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC auth v514 ap=1 s=15360 n(v0 rc2024-05-17T09:22:33.598569+0000 b15360 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15228=0-4194304@1} caps={15228=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15228 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5587bdd6c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000004bf:[<0x10000000208/HANDOUT.DOC v514>,<0x10000000009/WORD v5394>,<0x10000000008/~dmtmp v116>,<0x10000000007/client0 v98>,<0x10000000006/clients v78>,<0x10000000005/tmp v78>,<0x10000000003/client.0 v76>,<0x10000000002/b1c9b9fa-b203-4512-bd57-a87511b595df v90>,<0x10000000001/sv_0 v107>,<0x10000000000/qa v92>,<0x1/volumes v90>]//[3]","memoryvalue":"(4)0x100000004bf:[<0x10000000208/HANDOUT.DOC v514>,<0x10000000009/WORD v5447>,<0x10000000008/~dmtmp v119>,<0x10000000007/client0 v101>,<0x10000000006/clients v81>,<0x10000000005/tmp v81>,<0x10000000003/client.0 v78>,<0x10000000002/b1c9b9fa-b203-4512-bd57-a87511b595df v93>,<0x10000000001/sv_0 v109>,<0x10000000000/qa v82>,<0x1/volumes v78>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T09:22:39.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:39 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:39.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:39 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:22:39.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:39 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:39.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:39 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:22:39.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:39 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:39.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:39 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:39 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:22:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:39 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:39 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:22:39.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:39 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:39.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:22:39.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:39 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:39.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:39 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:22:39.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:39 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:39.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:39 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:22:39.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:39 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.191 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:22:40.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22076 3.58 MB/sec warmup 38 sec latency 398.588 ms 2024-05-17T09:22:40.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: pgmap v525: 129 pgs: 129 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-17T09:22:40.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub queued for path: / 2024-05-17T09:22:40.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:22:40.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:22:40.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: Scrub error on inode 0x100000004bf (/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC) see mds.h log and `damage ls` output for details 2024-05-17T09:22:40.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: active paths [/] 2024-05-17T09:22:40.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:40 smithi019 ceph-mon[31620]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: pgmap v525: 129 pgs: 129 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub queued for path: / 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: Scrub error on inode 0x100000004bf (/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC) see mds.h log and `damage ls` output for details 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: active paths [/] 2024-05-17T09:22:40.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:40 smithi079 ceph-mon[35502]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: pgmap v525: 129 pgs: 129 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub queued for path: / 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: Scrub error on inode 0x100000004bf (/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC) see mds.h log and `damage ls` output for details 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: active paths [/] 2024-05-17T09:22:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:40.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:40 smithi069 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-05-17T09:22:41.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22076 3.49 MB/sec warmup 39 sec latency 1398.642 ms 2024-05-17T09:22:41.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:41 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:22:41.436+0000 7f49787cc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T09:22:41.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:41 smithi019 ceph-mon[31620]: pgmap v526: 129 pgs: 129 active+clean; 28 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-05-17T09:22:41.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:41 smithi019 ceph-mon[31620]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T09:22:41.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:41 smithi019 ceph-mon[31620]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:active 2024-05-17T09:22:41.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:41 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:41.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:41 smithi079 ceph-mon[35502]: pgmap v526: 129 pgs: 129 active+clean; 28 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-05-17T09:22:41.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:41 smithi079 ceph-mon[35502]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T09:22:41.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:41 smithi079 ceph-mon[35502]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:active 2024-05-17T09:22:41.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:41 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:41.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:41 smithi069 ceph-mon[35081]: pgmap v526: 129 pgs: 129 active+clean; 28 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-05-17T09:22:41.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:41 smithi069 ceph-mon[35081]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T09:22:41.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:41 smithi069 ceph-mon[35081]: mds.? [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] up:active 2024-05-17T09:22:41.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:41 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 -- 172.21.15.19:0/1870175448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc1034d0 msgr2=0x7f52fc1038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 --2- 172.21.15.19:0/1870175448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc1034d0 0x7f52fc1038e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0099c0 tx=0x7f52ec009cd0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 -- 172.21.15.19:0/1870175448 shutdown_connections 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 --2- 172.21.15.19:0/1870175448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc108750 0x7f52fc1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 --2- 172.21.15.19:0/1870175448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc1034d0 0x7f52fc1038e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 --2- 172.21.15.19:0/1870175448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc107e00 0x7f52fc108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 -- 172.21.15.19:0/1870175448 >> 172.21.15.19:0/1870175448 conn(0x7f52fc0fef20 msgr2=0x7f52fc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 -- 172.21.15.19:0/1870175448 shutdown_connections 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.989+0000 7f5301e12700 1 -- 172.21.15.19:0/1870175448 wait complete. 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.990+0000 7f5301e12700 1 Processor -- start 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.990+0000 7f5301e12700 1 -- start start 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f5301e12700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc1034d0 0x7f52fc19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f5301e12700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc107e00 0x7f52fc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f5301e12700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 0x7f52fc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f5301e12700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc113280 con 0x7f52fc108750 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f5301e12700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc113100 con 0x7f52fc1034d0 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f5301e12700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc113400 con 0x7f52fc107e00 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 0x7f52fc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52faffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc107e00 0x7f52fc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 0x7f52fc199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36046/0 (socket says 172.21.15.19:36046) 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52fbfff700 1 -- 172.21.15.19:0/3035787417 learned_addr learned my addr 172.21.15.19:0/3035787417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc1034d0 0x7f52fc19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52faffd700 1 -- 172.21.15.19:0/3035787417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc1034d0 msgr2=0x7f52fc19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52faffd700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc1034d0 0x7f52fc19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52faffd700 1 -- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 msgr2=0x7f52fc199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52faffd700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 0x7f52fc199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52faffd700 1 -- 172.21.15.19:0/3035787417 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc19a1e0 con 0x7f52fc107e00 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.991+0000 7f52fbfff700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 0x7f52fc199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.992+0000 7f52faffd700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc107e00 0x7f52fc19fe20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52fc076aa0 tx=0x7f52f000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.992+0000 7f52f8ff9700 1 -- 172.21.15.19:0/3035787417 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0016070 con 0x7f52fc107e00 2024-05-17T09:22:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.992+0000 7f52f8ff9700 1 -- 172.21.15.19:0/3035787417 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52f000fd20 con 0x7f52fc107e00 2024-05-17T09:22:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.992+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc19a4d0 con 0x7f52fc107e00 2024-05-17T09:22:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.992+0000 7f52f8ff9700 1 -- 172.21.15.19:0/3035787417 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f000e3b0 con 0x7f52fc107e00 2024-05-17T09:22:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.992+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc06c030 con 0x7f52fc107e00 2024-05-17T09:22:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.996+0000 7f52f8ff9700 1 -- 172.21.15.19:0/3035787417 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f52f0019070 con 0x7f52fc107e00 2024-05-17T09:22:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 Processor -- start 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 -- start start 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52f8ff9700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f52e40b2900 0x7f52e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60acd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60acd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60ac77d0 0x563c60ac7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c609f1c90 con 0x563c60ac77d0 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563c609e1830 con 0x563c60944e40 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f5310827b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563c609e5870 con 0x563c609c23d0 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52fbfff700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f52e40b2900 0x7f52e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52f8ff9700 1 -- 172.21.15.19:0/3035787417 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f52f00e1fc0 con 0x7f52fc107e00 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60acd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60ac77d0 0x563c60ac7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60acd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58342/0 (socket says 172.21.15.19:58342) 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52faffd700 1 -- 172.21.15.19:0/1901810786 learned_addr learned my addr 172.21.15.19:0/1901810786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.997+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60acd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52faffd700 1 -- 172.21.15.19:0/1901810786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 msgr2=0x563c60acd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52faffd700 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60acd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52faffd700 1 -- 172.21.15.19:0/1901810786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60ac77d0 msgr2=0x563c60ac7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52faffd700 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60ac77d0 0x563c60ac7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52faffd700 1 -- 172.21.15.19:0/1901810786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c609318d0 con 0x563c60944e40 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52fbfff700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f52e40b2900 0x7f52e40b4e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009550 tx=0x7f52f4007ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60acd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52faffd700 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60acd8c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52f00105a0 tx=0x7f52f00a89c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d88f2b15dfc0e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52e1ffb700 1 -- 172.21.15.19:0/1901810786 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0020040 con 0x563c60944e40 2024-05-17T09:22:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52e1ffb700 1 -- 172.21.15.19:0/1901810786 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52f00a8ce0 con 0x563c60944e40 2024-05-17T09:22:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.998+0000 7f52e1ffb700 1 -- 172.21.15.19:0/1901810786 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f00b2030 con 0x563c60944e40 2024-05-17T09:22:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 -- 172.21.15.19:0/1901810786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 msgr2=0x563c60acd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60acd8c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52f00105a0 tx=0x7f52f00a89c0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 -- 172.21.15.19:0/1901810786 shutdown_connections 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60acd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60ac77d0 0x563c60ac7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 --2- 172.21.15.19:0/1901810786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60acd8c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 -- 172.21.15.19:0/1901810786 >> 172.21.15.19:0/1901810786 conn(0x563c60983c80 msgr2=0x563c60abdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 -- 172.21.15.19:0/1901810786 shutdown_connections 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 -- 172.21.15.19:0/1901810786 wait complete. 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:41.999+0000 7f5310827b80 1 -- start start 2024-05-17T09:22:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 Processor -- start 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ad10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60ad1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c608e7fa0 con 0x563c60944e40 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563c609837c0 con 0x563c60ac77d0 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563c60ac8610 con 0x563c609c23d0 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f5310827b80 10 client.0 _ll_register_callbacks cb 0x563c6097c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f52fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ad10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f52fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ad10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36068/0 (socket says 172.21.15.19:36068) 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3050032660 learned_addr learned my addr 172.21.15.19:0/3050032660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f52faffd700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.000+0000 7f52fbfff700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60ad1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3050032660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 msgr2=0x563c60ad1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 msgr2=0x563c60ad1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60ad1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3050032660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c60980690 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ad10c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0099c0 tx=0x7f52ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8a89ae70816b00 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52daffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52daffd700 1 -- 172.21.15.19:0/3050032660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00b990 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f00a8ce0 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52daffd700 1 -- 172.21.15.19:0/3050032660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f52ec00bb30 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52daffd700 1 -- 172.21.15.19:0/3050032660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52ec030050 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f5310827b80 10 client.16329 fetch_fsmap learned FSMap version 30 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c60adea20 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.001+0000 7f52daffd700 1 -- 172.21.15.19:0/3050032660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52f00a8ce0 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f52daffd700 1 -- 172.21.15.19:0/3050032660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f52ec00f4f0 con 0x563c60944e40 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f5310827b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f5310827b80 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f5310827b80 20 client.16329 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f5310827b80 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60ae3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f5310827b80 4 client.16329 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c609f3be0 con 0x563c60ae0fb0 2024-05-17T09:22:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f52faffd700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60ae3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f52faffd700 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60ae3460 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83eb3aba010e7aa0 server_cookie=6874b763eb96e24c in_seq=0 out_seq=0 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.002+0000 7f52daffd700 10 client.16329 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f52daffd700 1 -- 172.21.15.19:0/3050032660 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563c609f3be0 con 0x563c60ae0fb0 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f52daffd700 10 client.16329 handle_command_reply: tid=0 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f5310827b80 1 client.16329 shutdown 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 msgr2=0x563c60ad10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f5310827b80 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ad10c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0099c0 tx=0x7f52ec005800 comp rx=0 tx=0).stop 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 shutdown_connections 2024-05-17T09:22:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.003+0000 7f5310827b80 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60ae3460 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60ad1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ad10c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f52daffd700 0 client.16329 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 1 --2- 172.21.15.19:0/3050032660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 >> 172.21.15.19:0/3050032660 conn(0x563c60983c80 msgr2=0x563c60abdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 shutdown_connections 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 1 -- 172.21.15.19:0/3050032660 wait complete. 2024-05-17T09:22:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.004+0000 7f5310827b80 20 client.16329 trim_cache size 0 max 16384 2024-05-17T09:22:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 Processor -- start 2024-05-17T09:22:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 -- start start 2024-05-17T09:22:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ab8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60adcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1b0b00 con 0x563c60944e40 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1ae390 con 0x563c60ac77d0 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f5310827b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1ac1e0 con 0x563c609c23d0 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ab8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ab8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36080/0 (socket says 172.21.15.19:36080) 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3693310362 learned_addr learned my addr 172.21.15.19:0/3693310362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fbfff700 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60adcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52faffd700 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3693310362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 msgr2=0x563c60ad1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3693310362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 msgr2=0x563c60adcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60adcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52fb7fe700 1 -- 172.21.15.19:0/3693310362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c60abfb60 con 0x563c60944e40 2024-05-17T09:22:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.025+0000 7f52faffd700 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:22:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.026+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ab8990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00f910 tx=0x7f52ec00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc8c7ca50482847 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.026+0000 7f52e17fa700 1 -- 172.21.15.19:0/3693310362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00f660 con 0x563c60944e40 2024-05-17T09:22:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.026+0000 7f52e17fa700 1 -- 172.21.15.19:0/3693310362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52ec027070 con 0x563c60944e40 2024-05-17T09:22:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.026+0000 7f52e17fa700 1 -- 172.21.15.19:0/3693310362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec003a60 con 0x563c60944e40 2024-05-17T09:22:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 -- 172.21.15.19:0/3693310362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 msgr2=0x563c60ab8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:42.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ab8990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00f910 tx=0x7f52ec00b540 comp rx=0 tx=0).stop 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 -- 172.21.15.19:0/3693310362 shutdown_connections 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60ac77d0 0x563c60adcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c60944e40 0x563c60ab8990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 --2- 172.21.15.19:0/3693310362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c609c23d0 0x563c60ad1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 -- 172.21.15.19:0/3693310362 >> 172.21.15.19:0/3693310362 conn(0x563c60983c80 msgr2=0x563c60acfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 -- 172.21.15.19:0/3693310362 shutdown_connections 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 -- 172.21.15.19:0/3693310362 wait complete. 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 -- start start 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.027+0000 7f5310827b80 1 Processor -- start 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60ad91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 0x563c60ad9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c60ac77d0 0x563c60ad6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1ac850 con 0x563c609c23d0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1adcc0 con 0x563c60944e40 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1acf20 con 0x563c60ac77d0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f5310827b80 10 client.0 _ll_register_callbacks cb 0x563c6097c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 0x563c60ad9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 0x563c60ad9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36088/0 (socket says 172.21.15.19:36088) 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 -- 172.21.15.19:0/3734705974 learned_addr learned my addr 172.21.15.19:0/3734705974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60ad91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52fbfff700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c60ac77d0 0x563c60ad6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 -- 172.21.15.19:0/3734705974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c60ac77d0 msgr2=0x563c60ad6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c60ac77d0 0x563c60ad6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 -- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 msgr2=0x563c60ad91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60ad91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 -- 172.21.15.19:0/3734705974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c60ad9d40 con 0x563c609c23d0 2024-05-17T09:22:42.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52faffd700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 0x563c60ad9700 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f52f00a5250 tx=0x7f52f0056c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6868a1b1b928538 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.028+0000 7f52daffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f52daffd700 1 -- 172.21.15.19:0/3734705974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0020040 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f52daffd700 1 -- 172.21.15.19:0/3734705974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f52f00a9430 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ec027070 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f52daffd700 1 -- 172.21.15.19:0/3734705974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52f00a8ce0 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 10 client.16335 fetch_fsmap learned FSMap version 30 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c60b4c000 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f52daffd700 1 -- 172.21.15.19:0/3734705974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52ec027070 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f52daffd700 1 -- 172.21.15.19:0/3734705974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f52f00b1ba0 con 0x563c609c23d0 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 10 client.16335 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 20 client.16335 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60b4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 4 client.16335 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:22:42.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c609ed3c0 con 0x563c60ae0fb0 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.029+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60b4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f52daffd700 10 client.16335 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f52fb7fe700 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60b4e2f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d21448c11b32ca7 server_cookie=4f5b4159a2d3085e in_seq=0 out_seq=0 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f52daffd700 1 -- 172.21.15.19:0/3734705974 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563c609ed3c0 con 0x563c60ae0fb0 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f52daffd700 10 client.16335 handle_command_reply: tid=0 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f5310827b80 1 client.16335 shutdown 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 msgr2=0x563c60ad9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.030+0000 7f5310827b80 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 0x563c60ad9700 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f52f00a5250 tx=0x7f52f0056c90 comp rx=0 tx=0).stop 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 shutdown_connections 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563c60ac77d0 0x563c60ad6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563c60ae0fb0 0x563c60b4e2f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c60944e40 0x563c60ad91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 --2- 172.21.15.19:0/3734705974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c609c23d0 0x563c60ad9700 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 >> 172.21.15.19:0/3734705974 conn(0x563c60983c80 msgr2=0x563c60ac23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f52daffd700 0 client.16335 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 shutdown_connections 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 1 -- 172.21.15.19:0/3734705974 wait complete. 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5310827b80 20 client.16335 trim_cache size 0 max 16384 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stdout: "573b75b0-98c4-4dcc-8c59-96a9b986cdb2": { 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-17T09:22:42.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "573b75b0-98c4-4dcc-8c59-96a9b986cdb2", 2024-05-17T09:22:42.036 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-17T09:22:42.036 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:22:42.036 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:22:42.036 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:22:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f52e40b2900 msgr2=0x7f52e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f52e40b2900 0x7f52e40b4e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009550 tx=0x7f52f4007ed0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc107e00 msgr2=0x7f52fc19fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc107e00 0x7f52fc19fe20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52fc076aa0 tx=0x7f52f000dc30 comp rx=0 tx=0).stop 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 shutdown_connections 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52fc108750 0x7f52fc199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52fc1034d0 0x7f52fc19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f52e40b2900 0x7f52e40b4e10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 --2- 172.21.15.19:0/3035787417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52fc107e00 0x7f52fc19fe20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.031+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 >> 172.21.15.19:0/3035787417 conn(0x7f52fc0fef20 msgr2=0x7f52fc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.032+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 shutdown_connections 2024-05-17T09:22:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:22:42.032+0000 7f5301e12700 1 -- 172.21.15.19:0/3035787417 wait complete. 2024-05-17T09:22:42.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22153 3.40 MB/sec warmup 40 sec latency 2188.009 ms 2024-05-17T09:22:42.870 INFO:tasks.cephfs.filesystem:scrub status for tag:573b75b0-98c4-4dcc-8c59-96a9b986cdb2 - {'path': '/', 'tag': '573b75b0-98c4-4dcc-8c59-96a9b986cdb2', 'options': 'recursive,force'} 2024-05-17T09:22:43.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22480 3.36 MB/sec warmup 41 sec latency 335.828 ms 2024-05-17T09:22:44.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22480 3.28 MB/sec warmup 42 sec latency 1335.923 ms 2024-05-17T09:22:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:44 smithi079 ceph-mon[35502]: pgmap v527: 129 pgs: 129 active+clean; 28 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-17T09:22:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:44 smithi069 ceph-mon[35081]: pgmap v527: 129 pgs: 129 active+clean; 28 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-17T09:22:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:44 smithi019 ceph-mon[31620]: pgmap v527: 129 pgs: 129 active+clean; 28 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-17T09:22:45.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22576 3.20 MB/sec warmup 43 sec latency 1365.140 ms 2024-05-17T09:22:46.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22576 3.13 MB/sec warmup 44 sec latency 1723.633 ms 2024-05-17T09:22:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:46 smithi079 ceph-mon[35502]: pgmap v528: 129 pgs: 129 active+clean; 27 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-17T09:22:46.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:46 smithi069 ceph-mon[35081]: pgmap v528: 129 pgs: 129 active+clean; 27 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-17T09:22:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:46 smithi019 ceph-mon[31620]: pgmap v528: 129 pgs: 129 active+clean; 27 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 -- 172.21.15.19:0/3396855016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 msgr2=0x7f77ac103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 --2- 172.21.15.19:0/3396855016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac103840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f779c009900 tx=0x7f779c02a410 comp rx=0 tx=0).stop 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 -- 172.21.15.19:0/3396855016 shutdown_connections 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 --2- 172.21.15.19:0/3396855016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 0x7f77ac10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 --2- 172.21.15.19:0/3396855016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77ac103d80 0x7f77ac104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 --2- 172.21.15.19:0/3396855016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac103840 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 -- 172.21.15.19:0/3396855016 >> 172.21.15.19:0/3396855016 conn(0x7f77ac0fdd30 msgr2=0x7f77ac1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 -- 172.21.15.19:0/3396855016 shutdown_connections 2024-05-17T09:22:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.802+0000 7f77b0e34640 1 -- 172.21.15.19:0/3396855016 wait complete. 2024-05-17T09:22:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.803+0000 7f77b0e34640 1 Processor -- start 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.803+0000 7f77b0e34640 1 -- start start 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.803+0000 7f77b0e34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.803+0000 7f77b0e34640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77ac103d80 0x7f77ac199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77b0e34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 0x7f77ac1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77b0e34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac10e4f0 con 0x7f77ac102b40 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77b0e34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac10e370 con 0x7f77ac108a80 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77b0e34640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac10e670 con 0x7f77ac103d80 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49444/0 (socket says 172.21.15.19:49444) 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 -- 172.21.15.19:0/2798781307 learned_addr learned my addr 172.21.15.19:0/2798781307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77a3fff640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77ac103d80 0x7f77ac199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aad9d640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 0x7f77ac1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 -- 172.21.15.19:0/2798781307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77ac103d80 msgr2=0x7f77ac199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77ac103d80 0x7f77ac199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 -- 172.21.15.19:0/2798781307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 msgr2=0x7f77ac1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 0x7f77ac1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 -- 172.21.15.19:0/2798781307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac1a0c90 con 0x7f77ac102b40 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aad9d640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 0x7f77ac1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77aa59c640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac199150 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f779c005760 tx=0x7f779c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77a37fe640 1 -- 172.21.15.19:0/2798781307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c03f030 con 0x7f77ac102b40 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.804+0000 7f77a37fe640 1 -- 172.21.15.19:0/2798781307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f779c004440 con 0x7f77ac102b40 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.805+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac1a0f20 con 0x7f77ac102b40 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.805+0000 7f77a37fe640 1 -- 172.21.15.19:0/2798781307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c035070 con 0x7f77ac102b40 2024-05-17T09:22:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.805+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a1380 con 0x7f77ac102b40 2024-05-17T09:22:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.807+0000 7f77b7dc5740 1 Processor -- start 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.807+0000 7f77a37fe640 1 -- 172.21.15.19:0/2798781307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f779c042030 con 0x7f77ac102b40 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 -- start start 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e2205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e220b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e227a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f92e190c30 con 0x55f92de5c4d0 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f92e1807d0 con 0x55f92de542f0 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f92e184810 con 0x55f92de7d1b0 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e227a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e227a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49452/0 (socket says 172.21.15.19:49452) 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77a3fff640 1 -- 172.21.15.19:0/334630113 learned_addr learned my addr 172.21.15.19:0/334630113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77aa59c640 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e220b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.808+0000 7f77aad9d640 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e2205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a3fff640 1 -- 172.21.15.19:0/334630113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 msgr2=0x55f92e2205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a3fff640 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e2205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a3fff640 1 -- 172.21.15.19:0/334630113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 msgr2=0x55f92e220b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a3fff640 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e220b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a3fff640 1 -- 172.21.15.19:0/334630113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92de95210 con 0x55f92de5c4d0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a37fe640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77980b2920 0x7f77980b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77aad9d640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77980b2920 0x7f77980b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a3fff640 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e227a30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7794009ca0 tx=0x7f7794011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cb724b884c5664 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a37fe640 1 -- 172.21.15.19:0/2798781307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f779c102950 con 0x7f77ac102b40 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a0ff9640 1 -- 172.21.15.19:0/334630113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005220 con 0x55f92de5c4d0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a0ff9640 1 -- 172.21.15.19:0/334630113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7794004c30 con 0x55f92de5c4d0 2024-05-17T09:22:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.809+0000 7f77a0ff9640 1 -- 172.21.15.19:0/334630113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794019690 con 0x55f92de5c4d0 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 -- 172.21.15.19:0/334630113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 msgr2=0x55f92e227a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77aad9d640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77980b2920 0x7f77980b4e30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55f92e221780 tx=0x7f77a4009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e227a30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7794009ca0 tx=0x7f7794011810 comp rx=0 tx=0).stop 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 -- 172.21.15.19:0/334630113 shutdown_connections 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e227a30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e220b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/334630113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e2205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 -- 172.21.15.19:0/334630113 >> 172.21.15.19:0/334630113 conn(0x55f92de71300 msgr2=0x55f92e21bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 -- 172.21.15.19:0/334630113 shutdown_connections 2024-05-17T09:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.810+0000 7f77b7dc5740 1 -- 172.21.15.19:0/334630113 wait complete. 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 -- start start 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 Processor -- start 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e229f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e22a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f92de66790 con 0x55f92de5c4d0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f92e13c890 con 0x55f92de542f0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f92e228480 con 0x55f92de7d1b0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77b7dc5740 10 client.0 _ll_register_callbacks cb 0x55f92e13fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e22a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e22a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49454/0 (socket says 172.21.15.19:49454) 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77a3fff640 1 -- 172.21.15.19:0/1130985559 learned_addr learned my addr 172.21.15.19:0/1130985559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77aad9d640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.811+0000 7f77aa59c640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e229f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77a3fff640 1 -- 172.21.15.19:0/1130985559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 msgr2=0x55f92e231360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77a3fff640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77a3fff640 1 -- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 msgr2=0x55f92e229f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77a3fff640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e229f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77a3fff640 1 -- 172.21.15.19:0/1130985559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92df61620 con 0x55f92de5c4d0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77aad9d640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77a3fff640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e22a470 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7794009710 tx=0x7f7794005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da57f7e0a1493a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f7791ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f7791ffb640 1 -- 172.21.15.19:0/1130985559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794004230 con 0x55f92de5c4d0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794004c30 con 0x55f92de5c4d0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f7791ffb640 1 -- 172.21.15.19:0/1130985559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f77940043d0 con 0x55f92de5c4d0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f7791ffb640 1 -- 172.21.15.19:0/1130985559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7794034050 con 0x55f92de5c4d0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77b7dc5740 10 client.16347 fetch_fsmap learned FSMap version 30 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f92e234290 con 0x55f92de5c4d0 2024-05-17T09:22:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.812+0000 7f7791ffb640 1 -- 172.21.15.19:0/1130985559 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7794004c30 con 0x55f92de5c4d0 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f7791ffb640 1 -- 172.21.15.19:0/1130985559 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7794023430 con 0x55f92de5c4d0 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77b7dc5740 10 client.16347 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77b7dc5740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77b7dc5740 20 client.16347 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de3bd80 0x55f92e237870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77b7dc5740 4 client.16347 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f92e192b80 con 0x55f92de3bd80 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.813+0000 7f77aa59c640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de3bd80 0x55f92e237870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.814+0000 7f7791ffb640 10 client.16347 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.814+0000 7f77aa59c640 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de3bd80 0x55f92e237870 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ccb4bd5da93d3a4 server_cookie=18a2b1345fb57199 in_seq=0 out_seq=0 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.814+0000 7f7791ffb640 1 -- 172.21.15.19:0/1130985559 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f92e192b80 con 0x55f92de3bd80 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.814+0000 7f7791ffb640 10 client.16347 handle_command_reply: tid=0 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.814+0000 7f77b7dc5740 1 client.16347 shutdown 2024-05-17T09:22:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 msgr2=0x55f92e22a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e22a470 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7794009710 tx=0x7f7794005090 comp rx=0 tx=0).stop 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 shutdown_connections 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de3bd80 0x55f92e237870 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e22a470 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f7791ffb640 0 client.16347 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1130985559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e229f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 >> 172.21.15.19:0/1130985559 conn(0x55f92de71300 msgr2=0x55f92e218d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 shutdown_connections 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1130985559 wait complete. 2024-05-17T09:22:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.815+0000 7f77b7dc5740 20 client.16347 trim_cache size 0 max 16384 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 Processor -- start 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 -- start start 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e21f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e21f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1ad160 con 0x55f92de5c4d0 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1aaa30 con 0x55f92de542f0 2024-05-17T09:22:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1a8880 con 0x55f92de7d1b0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e21f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e21f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49470/0 (socket says 172.21.15.19:49470) 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 -- 172.21.15.19:0/197820306 learned_addr learned my addr 172.21.15.19:0/197820306 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77aa59c640 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e21f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77aad9d640 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 -- 172.21.15.19:0/197820306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 msgr2=0x55f92e231280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 -- 172.21.15.19:0/197820306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 msgr2=0x55f92e21f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e21f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77a3fff640 1 -- 172.21.15.19:0/197820306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92e2200c0 con 0x55f92de5c4d0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.831+0000 7f77aad9d640 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.832+0000 7f77a3fff640 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e21f990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7794012fd0 tx=0x7f7794005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2506773c02e2d49f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.832+0000 7f7793fff640 1 -- 172.21.15.19:0/197820306 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005490 con 0x55f92de5c4d0 2024-05-17T09:22:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.832+0000 7f7793fff640 1 -- 172.21.15.19:0/197820306 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7794005630 con 0x55f92de5c4d0 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.832+0000 7f7793fff640 1 -- 172.21.15.19:0/197820306 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794019760 con 0x55f92de5c4d0 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.832+0000 7f77b7dc5740 1 -- 172.21.15.19:0/197820306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 msgr2=0x55f92e21f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.832+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e21f990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7794012fd0 tx=0x7f7794005970 comp rx=0 tx=0).stop 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- 172.21.15.19:0/197820306 shutdown_connections 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e231280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e21f990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/197820306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e21f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- 172.21.15.19:0/197820306 >> 172.21.15.19:0/197820306 conn(0x55f92de71300 msgr2=0x55f92e21ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- 172.21.15.19:0/197820306 shutdown_connections 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- 172.21.15.19:0/197820306 wait complete. 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- start start 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 Processor -- start 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e231560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e231aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e22e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1a8ef0 con 0x55f92de5c4d0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1aa360 con 0x55f92de542f0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1a95c0 con 0x55f92de7d1b0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77b7dc5740 10 client.0 _ll_register_callbacks cb 0x7f7794004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e231aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.833+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e231aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49484/0 (socket says 172.21.15.19:49484) 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 -- 172.21.15.19:0/1161014324 learned_addr learned my addr 172.21.15.19:0/1161014324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77aa59c640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e231560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77aad9d640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e22e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 -- 172.21.15.19:0/1161014324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 msgr2=0x55f92e22e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e22e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 -- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 msgr2=0x55f92e231560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e231560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 -- 172.21.15.19:0/1161014324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92e22e9a0 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77aad9d640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e22e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77a3fff640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e231aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7794023800 tx=0x7f7794009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e207f52768cb7f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f7791ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f7791ffb640 1 -- 172.21.15.19:0/1161014324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794045060 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f92e18c360 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f7791ffb640 1 -- 172.21.15.19:0/1161014324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7794023a20 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f7791ffb640 1 -- 172.21.15.19:0/1161014324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f779404c050 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77b7dc5740 10 client.16353 fetch_fsmap learned FSMap version 30 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.834+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f92e29ebe0 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f7791ffb640 1 -- 172.21.15.19:0/1161014324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f92e18c360 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f7791ffb640 1 -- 172.21.15.19:0/1161014324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7794019bc0 con 0x55f92de5c4d0 2024-05-17T09:22:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77b7dc5740 10 client.16353 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77b7dc5740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77b7dc5740 20 client.16353 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de111c0 0x55f92e2a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77b7dc5740 4 client.16353 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f92e1065f0 con 0x55f92de111c0 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.835+0000 7f77aa59c640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de111c0 0x55f92e2a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.836+0000 7f7791ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.836+0000 7f77aa59c640 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de111c0 0x55f92e2a1640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18caa1795b3434b4 server_cookie=107830b1c3a97c00 in_seq=0 out_seq=0 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f7791ffb640 1 -- 172.21.15.19:0/1161014324 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f92e1065f0 con 0x55f92de111c0 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f7791ffb640 10 client.16353 handle_command_reply: tid=0 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 client.16353 shutdown 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 msgr2=0x55f92e231aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e231aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7794023800 tx=0x7f7794009f80 comp rx=0 tx=0).stop 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 shutdown_connections 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f92de111c0 0x55f92e2a1640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f92de7d1b0 0x55f92e22e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f92de5c4d0 0x55f92e231aa0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 --2- 172.21.15.19:0/1161014324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f92de542f0 0x55f92e231560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 >> 172.21.15.19:0/1161014324 conn(0x55f92de71300 msgr2=0x55f92e21f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f7791ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 shutdown_connections 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 1 -- 172.21.15.19:0/1161014324 wait complete. 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.854+0000 7f77b7dc5740 20 client.16353 trim_cache size 0 max 16384 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:22:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77980b2920 msgr2=0x7f77980b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77980b2920 0x7f77980b4e30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55f92e221780 tx=0x7f77a4009920 comp rx=0 tx=0).stop 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 msgr2=0x7f77ac199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac199150 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f779c005760 tx=0x7f779c004040 comp rx=0 tx=0).stop 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 shutdown_connections 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77ac108a80 0x7f77ac1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77ac103d80 0x7f77ac199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f77980b2920 0x7f77980b4e30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 --2- 172.21.15.19:0/2798781307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ac102b40 0x7f77ac199150 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 >> 172.21.15.19:0/2798781307 conn(0x7f77ac0fdd30 msgr2=0x7f77ac104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 shutdown_connections 2024-05-17T09:22:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:46.855+0000 7f77b0e34640 1 -- 172.21.15.19:0/2798781307 wait complete. 2024-05-17T09:22:47.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22576 3.06 MB/sec warmup 45 sec latency 2723.708 ms 2024-05-17T09:22:48.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22576 3.00 MB/sec warmup 46 sec latency 3723.813 ms 2024-05-17T09:22:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:48 smithi079 ceph-mon[35502]: pgmap v529: 129 pgs: 129 active+clean; 27 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-17T09:22:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:48 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:22:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:48 smithi069 ceph-mon[35081]: pgmap v529: 129 pgs: 129 active+clean; 27 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-17T09:22:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:48 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:22:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:48 smithi019 ceph-mon[31620]: pgmap v529: 129 pgs: 129 active+clean; 27 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-17T09:22:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:48 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:22:49.417 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22576 2.93 MB/sec warmup 47 sec latency 4723.908 ms 2024-05-17T09:22:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:49 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:22:49.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:49 smithi079 ceph-mon[35502]: scrub summary: idle 2024-05-17T09:22:49.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:49 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:22:49.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:49 smithi069 ceph-mon[35081]: scrub summary: idle 2024-05-17T09:22:49.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:49 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:22:49.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:49 smithi019 ceph-mon[31620]: scrub summary: idle 2024-05-17T09:22:50.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22599 2.87 MB/sec warmup 48 sec latency 4754.589 ms 2024-05-17T09:22:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:50 smithi079 ceph-mon[35502]: pgmap v530: 129 pgs: 129 active+clean; 27 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 640 KiB/s wr, 88 op/s 2024-05-17T09:22:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:50 smithi069 ceph-mon[35081]: pgmap v530: 129 pgs: 129 active+clean; 27 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 640 KiB/s wr, 88 op/s 2024-05-17T09:22:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:50 smithi019 ceph-mon[31620]: pgmap v530: 129 pgs: 129 active+clean; 27 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 640 KiB/s wr, 88 op/s 2024-05-17T09:22:51.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22599 2.81 MB/sec warmup 49 sec latency 1888.082 ms 2024-05-17T09:22:52.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22599 2.76 MB/sec warmup 50 sec latency 2888.168 ms 2024-05-17T09:22:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:52 smithi079 ceph-mon[35502]: pgmap v531: 129 pgs: 129 active+clean; 27 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 403 KiB/s wr, 70 op/s 2024-05-17T09:22:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:52 smithi069 ceph-mon[35081]: pgmap v531: 129 pgs: 129 active+clean; 27 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 403 KiB/s wr, 70 op/s 2024-05-17T09:22:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:52 smithi019 ceph-mon[31620]: pgmap v531: 129 pgs: 129 active+clean; 27 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 403 KiB/s wr, 70 op/s 2024-05-17T09:22:53.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22599 2.70 MB/sec warmup 51 sec latency 3888.274 ms 2024-05-17T09:22:54.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 22599 2.65 MB/sec warmup 52 sec latency 4888.325 ms 2024-05-17T09:22:54.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:54 smithi079 ceph-mon[35502]: pgmap v532: 129 pgs: 129 active+clean; 23 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 339 KiB/s wr, 71 op/s 2024-05-17T09:22:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:54 smithi069 ceph-mon[35081]: pgmap v532: 129 pgs: 129 active+clean; 23 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 339 KiB/s wr, 71 op/s 2024-05-17T09:22:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:54 smithi019 ceph-mon[31620]: pgmap v532: 129 pgs: 129 active+clean; 23 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 339 KiB/s wr, 71 op/s 2024-05-17T09:22:55.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 23185 2.66 MB/sec warmup 53 sec latency 4918.876 ms 2024-05-17T09:22:56.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 23750 2.64 MB/sec warmup 54 sec latency 16.288 ms 2024-05-17T09:22:56.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:56 smithi079 ceph-mon[35502]: pgmap v533: 129 pgs: 129 active+clean; 23 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 203 KiB/s wr, 44 op/s 2024-05-17T09:22:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:56 smithi069 ceph-mon[35081]: pgmap v533: 129 pgs: 129 active+clean; 23 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 203 KiB/s wr, 44 op/s 2024-05-17T09:22:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:56 smithi019 ceph-mon[31620]: pgmap v533: 129 pgs: 129 active+clean; 23 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 203 KiB/s wr, 44 op/s 2024-05-17T09:22:57.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 24496 2.71 MB/sec warmup 55 sec latency 15.390 ms 2024-05-17T09:22:58.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 24890 2.73 MB/sec warmup 56 sec latency 15.615 ms 2024-05-17T09:22:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:22:58 smithi079 ceph-mon[35502]: pgmap v534: 129 pgs: 129 active+clean; 24 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 229 KiB/s wr, 43 op/s 2024-05-17T09:22:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:22:58 smithi069 ceph-mon[35081]: pgmap v534: 129 pgs: 129 active+clean; 24 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 229 KiB/s wr, 43 op/s 2024-05-17T09:22:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:22:58 smithi019 ceph-mon[31620]: pgmap v534: 129 pgs: 129 active+clean; 24 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 229 KiB/s wr, 43 op/s 2024-05-17T09:22:59.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 25247 2.72 MB/sec warmup 57 sec latency 15.795 ms 2024-05-17T09:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.967+0000 7f91c751b640 1 -- 172.21.15.19:0/2888234983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c0106d60 msgr2=0x7f91c0109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 --2- 172.21.15.19:0/2888234983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c0106d60 0x7f91c0109240 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009900 tx=0x7f91b002a410 comp rx=0 tx=0).stop 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 -- 172.21.15.19:0/2888234983 shutdown_connections 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 --2- 172.21.15.19:0/2888234983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c01098d0 0x7f91c010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 --2- 172.21.15.19:0/2888234983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c0106d60 0x7f91c0109240 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 --2- 172.21.15.19:0/2888234983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0104370 0x7f91c0106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 -- 172.21.15.19:0/2888234983 >> 172.21.15.19:0/2888234983 conn(0x7f91c00fdd30 msgr2=0x7f91c01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 -- 172.21.15.19:0/2888234983 shutdown_connections 2024-05-17T09:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.968+0000 7f91c751b640 1 -- 172.21.15.19:0/2888234983 wait complete. 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.969+0000 7f91c751b640 1 Processor -- start 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 -- start start 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c0104370 0x7f91c0103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 0x7f91c0102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c01098d0 0x7f91c01027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91c010f5f0 con 0x7f91c01098d0 2024-05-17T09:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91c010f470 con 0x7f91c0106d60 2024-05-17T09:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c751b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91c010f770 con 0x7f91c0104370 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 0x7f91c0102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 0x7f91c0102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35368/0 (socket says 172.21.15.19:35368) 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c4ab6640 1 -- 172.21.15.19:0/1273506717 learned_addr learned my addr 172.21.15.19:0/1273506717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c01098d0 0x7f91c01027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c52b7640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c0104370 0x7f91c0103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c52b7640 1 -- 172.21.15.19:0/1273506717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 msgr2=0x7f91c0102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.970+0000 7f91c52b7640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 0x7f91c0102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c52b7640 1 -- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c01098d0 msgr2=0x7f91c01027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c52b7640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c01098d0 0x7f91c01027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c52b7640 1 -- 172.21.15.19:0/1273506717 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c0102ef0 con 0x7f91c0104370 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 0x7f91c0102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c52b7640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c0104370 0x7f91c0103be0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00e9c0 tx=0x7f91bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91b67fc640 1 -- 172.21.15.19:0/1273506717 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00cd80 con 0x7f91c0104370 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c01a4230 con 0x7f91c0104370 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91b67fc640 1 -- 172.21.15.19:0/1273506717 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91bc0108c0 con 0x7f91c0104370 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91b67fc640 1 -- 172.21.15.19:0/1273506717 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0187a0 con 0x7f91c0104370 2024-05-17T09:22:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.971+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c01a4720 con 0x7f91c0104370 2024-05-17T09:22:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.974+0000 7f91ce3cd740 1 Processor -- start 2024-05-17T09:22:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 -- start start 2024-05-17T09:22:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91b67fc640 1 -- 172.21.15.19:0/1273506717 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f91bc01f020 con 0x7f91c0104370 2024-05-17T09:22:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 0x55f5076043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507604920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072424d0 0x55f50760b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f507576c30 con 0x55f5072424d0 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f5075667d0 con 0x55f50723a2f0 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f50756a810 con 0x55f5072631b0 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 0x55f5076043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072424d0 0x55f50760b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c52b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507604920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 0x55f5076043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:60984/0 (socket says 172.21.15.19:60984) 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 -- 172.21.15.19:0/939468710 learned_addr learned my addr 172.21.15.19:0/939468710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 -- 172.21.15.19:0/939468710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 msgr2=0x55f507604920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507604920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 -- 172.21.15.19:0/939468710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072424d0 msgr2=0x55f50760b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072424d0 0x55f50760b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c5ab8640 1 -- 172.21.15.19:0/939468710 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50727b210 con 0x55f5072631b0 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.975+0000 7f91c52b7640 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507604920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 0x55f5076043a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91b8010370 tx=0x7f91b8010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d98c2ba61cde4145 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91b67fc640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f91ac0b28a0 0x7f91ac0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91c52b7640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f91ac0b28a0 0x7f91ac0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91ab7fe640 1 -- 172.21.15.19:0/939468710 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8016070 con 0x55f5072631b0 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91ab7fe640 1 -- 172.21.15.19:0/939468710 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b8004d20 con 0x55f5072631b0 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91ab7fe640 1 -- 172.21.15.19:0/939468710 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80052c0 con 0x55f5072631b0 2024-05-17T09:22:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.976+0000 7f91b67fc640 1 -- 172.21.15.19:0/1273506717 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f91bc0e2550 con 0x7f91c0104370 2024-05-17T09:22:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 -- 172.21.15.19:0/939468710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 msgr2=0x55f5076043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 0x55f5076043a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91b8010370 tx=0x7f91b8010830 comp rx=0 tx=0).stop 2024-05-17T09:22:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91c52b7640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f91ac0b28a0 0x7f91ac0b4db0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0e5fd0 tx=0x7f91bc0b2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 -- 172.21.15.19:0/939468710 shutdown_connections 2024-05-17T09:22:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072424d0 0x55f50760b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507604920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/939468710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072631b0 0x55f5076043a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 -- 172.21.15.19:0/939468710 >> 172.21.15.19:0/939468710 conn(0x55f507257300 msgr2=0x55f5075ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 -- 172.21.15.19:0/939468710 shutdown_connections 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.977+0000 7f91ce3cd740 1 -- 172.21.15.19:0/939468710 wait complete. 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.978+0000 7f91ce3cd740 1 -- start start 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.978+0000 7f91ce3cd740 1 Processor -- start 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.978+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f50760dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072424d0 0x55f50760e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072631b0 0x55f507615190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f50724c790 con 0x55f5072631b0 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f507522890 con 0x55f50723a2f0 2024-05-17T09:22:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f50760c270 con 0x55f5072424d0 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 10 client.0 _ll_register_callbacks cb 0x55f507525d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072424d0 0x55f50760e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f50760dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f50760dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35384/0 (socket says 172.21.15.19:35384) 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072424d0 0x55f50760e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:60994/0 (socket says 172.21.15.19:60994) 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 -- 172.21.15.19:0/3770227551 learned_addr learned my addr 172.21.15.19:0/3770227551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072631b0 0x55f507615190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 -- 172.21.15.19:0/3770227551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072424d0 msgr2=0x55f50760e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072424d0 0x55f50760e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 -- 172.21.15.19:0/3770227551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072631b0 msgr2=0x55f507615190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072631b0 0x55f507615190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 -- 172.21.15.19:0/3770227551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f507347620 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f50760dcf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005550 tx=0x7f91b8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1045e8287a02278d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91a8ff9640 1 -- 172.21.15.19:0/3770227551 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8016070 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91a8ff9640 1 -- 172.21.15.19:0/3770227551 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f91b8003b70 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.979+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b8004d20 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91a8ff9640 1 -- 172.21.15.19:0/3770227551 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91b8032050 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 10 client.26045 fetch_fsmap learned FSMap version 30 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5076180c0 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91a8ff9640 1 -- 172.21.15.19:0/3770227551 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b8004d20 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91a8ff9640 1 -- 172.21.15.19:0/3770227551 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f91b800f040 con 0x55f50723a2f0 2024-05-17T09:22:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 10 client.26045 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 20 client.26045 populate_metadata read hostname 'smithi019' 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f507221d80 0x55f50761b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 4 client.26045 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f507578b80 con 0x55f507221d80 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.980+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f507221d80 0x55f50761b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.981+0000 7f91a8ff9640 10 client.26045 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.981+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f507221d80 0x55f50761b5c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62dcc94f8f12050 server_cookie=82a47be36ef34a2a in_seq=0 out_seq=0 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91a8ff9640 1 -- 172.21.15.19:0/3770227551 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f507578b80 con 0x55f507221d80 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91a8ff9640 10 client.26045 handle_command_reply: tid=0 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 client.26045 shutdown 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 msgr2=0x55f50760dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f50760dcf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005550 tx=0x7f91b8009bb0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 shutdown_connections 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f507221d80 0x55f50761b5c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5072631b0 0x55f507615190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91a8ff9640 0 client.26045 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5072424d0 0x55f50760e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/3770227551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f50760dcf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 >> 172.21.15.19:0/3770227551 conn(0x55f507257300 msgr2=0x55f5075fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 shutdown_connections 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 1 -- 172.21.15.19:0/3770227551 wait complete. 2024-05-17T09:22:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:22:59.982+0000 7f91ce3cd740 20 client.26045 trim_cache size 0 max 16384 2024-05-17T09:23:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 Processor -- start 2024-05-17T09:23:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 -- start start 2024-05-17T09:23:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f507257300 0x55f507608730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507615120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f50723a2f0 0x55f5075fe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01b14e0 con 0x55f507221d80 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01aedb0 con 0x55f507257300 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01acc00 con 0x55f50723a2f0 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507615120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507615120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59246/0 (socket says 172.21.15.19:59246) 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91c4ab6640 1 -- 172.21.15.19:0/284990019 learned_addr learned my addr 172.21.15.19:0/284990019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.001+0000 7f91c5ab8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f507257300 0x55f507608730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f50723a2f0 0x55f5075fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c4ab6640 1 -- 172.21.15.19:0/284990019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f50723a2f0 msgr2=0x55f5075fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f50723a2f0 0x55f5075fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c4ab6640 1 -- 172.21.15.19:0/284990019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f507257300 msgr2=0x55f507608730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f507257300 0x55f507608730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c4ab6640 1 -- 172.21.15.19:0/284990019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50760b680 con 0x55f507221d80 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f507257300 0x55f507608730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507615120 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009230 tx=0x7f91b00042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7808c205add2eb1b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91aaffd640 1 -- 172.21.15.19:0/284990019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b004a020 con 0x55f507221d80 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91aaffd640 1 -- 172.21.15.19:0/284990019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b0003a80 con 0x55f507221d80 2024-05-17T09:23:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.002+0000 7f91aaffd640 1 -- 172.21.15.19:0/284990019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0035040 con 0x55f507221d80 2024-05-17T09:23:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 -- 172.21.15.19:0/284990019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 msgr2=0x55f507615120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507615120 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009230 tx=0x7f91b00042d0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 -- 172.21.15.19:0/284990019 shutdown_connections 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f50723a2f0 0x55f5075fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507615120 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/284990019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f507257300 0x55f507608730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 -- 172.21.15.19:0/284990019 >> 172.21.15.19:0/284990019 conn(0x55f50760e400 msgr2=0x55f507603560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 -- 172.21.15.19:0/284990019 shutdown_connections 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 -- 172.21.15.19:0/284990019 wait complete. 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 -- start start 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.003+0000 7f91ce3cd740 1 Processor -- start 2024-05-17T09:23:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507612690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507612bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f507257300 0x55f507613b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01ad270 con 0x55f507221d80 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01ae6e0 con 0x55f50723a2f0 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01ad940 con 0x55f507257300 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91ce3cd740 10 client.0 _ll_register_callbacks cb 0x7f91b8005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507612690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507612bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c4ab6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507612bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35404/0 (socket says 172.21.15.19:35404) 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c4ab6640 1 -- 172.21.15.19:0/2804601737 learned_addr learned my addr 172.21.15.19:0/2804601737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f507257300 0x55f507613b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 -- 172.21.15.19:0/2804601737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f507257300 msgr2=0x55f507613b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f507257300 0x55f507613b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 -- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 msgr2=0x55f507612bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507612bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c5ab8640 1 -- 172.21.15.19:0/2804601737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5076140c0 con 0x55f507221d80 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.004+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507612bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91c5ab8640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507612690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005520 tx=0x7f91b8009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e42c6d0ab1eed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91a8ff9640 1 -- 172.21.15.19:0/2804601737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8016040 con 0x55f507221d80 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91a8ff9640 1 -- 172.21.15.19:0/2804601737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f91b8012d00 con 0x55f507221d80 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b0003a80 con 0x55f507221d80 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91a8ff9640 1 -- 172.21.15.19:0/2804601737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91b8023040 con 0x55f507221d80 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 10 client.16371 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5076153c0 con 0x55f507221d80 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91a8ff9640 1 -- 172.21.15.19:0/2804601737 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91b0003a80 con 0x55f507221d80 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91a8ff9640 1 -- 172.21.15.19:0/2804601737 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f91b80056a0 con 0x55f507221d80 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 10 client.16371 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 20 client.16371 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5072424d0 0x55f507686380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 4 client.16371 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.005+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f507572360 con 0x55f5072424d0 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.006+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5072424d0 0x55f507686380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.006+0000 7f91a8ff9640 10 client.16371 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.006+0000 7f91c4ab6640 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5072424d0 0x55f507686380 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908e2ba509706c84 server_cookie=252838e2d2bd0262 in_seq=0 out_seq=0 2024-05-17T09:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91a8ff9640 1 -- 172.21.15.19:0/2804601737 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f507572360 con 0x55f5072424d0 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91a8ff9640 10 client.16371 handle_command_reply: tid=0 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 client.16371 shutdown 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 msgr2=0x55f507612690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507612690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005520 tx=0x7f91b8009c50 comp rx=0 tx=0).stop 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 shutdown_connections 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5072424d0 0x55f507686380 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f507257300 0x55f507613b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f50723a2f0 0x55f507612bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91a8ff9640 0 client.16371 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 --2- 172.21.15.19:0/2804601737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f507221d80 0x55f507612690 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 >> 172.21.15.19:0/2804601737 conn(0x55f50760e400 msgr2=0x55f5076032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 shutdown_connections 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 1 -- 172.21.15.19:0/2804601737 wait complete. 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.024+0000 7f91ce3cd740 20 client.16371 trim_cache size 0 max 16384 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:23:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f91ac0b28a0 msgr2=0x7f91ac0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f91ac0b28a0 0x7f91ac0b4db0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0e5fd0 tx=0x7f91bc0b2040 comp rx=0 tx=0).stop 2024-05-17T09:23:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c0104370 msgr2=0x7f91c0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c0104370 0x7f91c0103be0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00e9c0 tx=0x7f91bc00ee80 comp rx=0 tx=0).stop 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 shutdown_connections 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f91ac0b28a0 0x7f91ac0b4db0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91c01098d0 0x7f91c01027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c0106d60 0x7f91c0102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 --2- 172.21.15.19:0/1273506717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91c0104370 0x7f91c0103be0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 >> 172.21.15.19:0/1273506717 conn(0x7f91c00fdd30 msgr2=0x7f91c0106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 shutdown_connections 2024-05-17T09:23:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:00.025+0000 7f91c751b640 1 -- 172.21.15.19:0/1273506717 wait complete. 2024-05-17T09:23:00.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 25503 2.67 MB/sec warmup 58 sec latency 15.765 ms 2024-05-17T09:23:00.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:00 smithi069 ceph-mon[35081]: pgmap v535: 129 pgs: 129 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 288 KiB/s wr, 49 op/s 2024-05-17T09:23:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:00 smithi019 ceph-mon[31620]: pgmap v535: 129 pgs: 129 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 288 KiB/s wr, 49 op/s 2024-05-17T09:23:00.712 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.3 2024-05-17T09:23:00.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:23:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:00 smithi079 ceph-mon[35502]: pgmap v535: 129 pgs: 129 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 288 KiB/s wr, 49 op/s 2024-05-17T09:23:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:01.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 26005 2.66 MB/sec warmup 59 sec latency 16.298 ms 2024-05-17T09:23:02.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:02 smithi019 ceph-mon[31620]: pgmap v536: 129 pgs: 129 active+clean; 25 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 353 KiB/s wr, 43 op/s 2024-05-17T09:23:02.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 26525 2.67 MB/sec warmup 60 sec latency 22.017 ms 2024-05-17T09:23:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:02 smithi069 ceph-mon[35081]: pgmap v536: 129 pgs: 129 active+clean; 25 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 353 KiB/s wr, 43 op/s 2024-05-17T09:23:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:02 smithi079 ceph-mon[35502]: pgmap v536: 129 pgs: 129 active+clean; 25 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 353 KiB/s wr, 43 op/s 2024-05-17T09:23:02.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 -- 172.21.15.19:0/563455400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684108730 msgr2=0x7f4684110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:02.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 --2- 172.21.15.19:0/563455400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684108730 0x7f4684110090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b0d0 tx=0x7f467c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 -- 172.21.15.19:0/563455400 shutdown_connections 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 --2- 172.21.15.19:0/563455400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46841034b0 0x7f46841038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 --2- 172.21.15.19:0/563455400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684108730 0x7f4684110090 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 --2- 172.21.15.19:0/563455400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684107de0 0x7f46841081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 -- 172.21.15.19:0/563455400 >> 172.21.15.19:0/563455400 conn(0x7f46840fede0 msgr2=0x7f4684101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 -- 172.21.15.19:0/563455400 shutdown_connections 2024-05-17T09:23:02.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.833+0000 7f468c571700 1 -- 172.21.15.19:0/563455400 wait complete. 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 Processor -- start 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 -- start start 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 0x7f468419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4684107de0 0x7f468419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684108730 0x7f46841a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4684113210 con 0x7f46841034b0 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4684113090 con 0x7f4684108730 2024-05-17T09:23:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.834+0000 7f468c571700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4684113390 con 0x7f4684107de0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 0x7f468419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 0x7f468419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59272/0 (socket says 172.21.15.19:59272) 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 -- 172.21.15.19:0/3228694646 learned_addr learned my addr 172.21.15.19:0/3228694646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f4689b0c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4684107de0 0x7f468419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468ab0e700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684108730 0x7f46841a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 -- 172.21.15.19:0/3228694646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4684107de0 msgr2=0x7f468419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4684107de0 0x7f468419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 -- 172.21.15.19:0/3228694646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684108730 msgr2=0x7f46841a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684108730 0x7f46841a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 -- 172.21.15.19:0/3228694646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46841a1da0 con 0x7f46841034b0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468ab0e700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684108730 0x7f46841a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468a30d700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 0x7f468419ced0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f468000e9a0 tx=0x7f468000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468000ca10 con 0x7f46841034b0 2024-05-17T09:23:02.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f468000cbb0 con 0x7f46841034b0 2024-05-17T09:23:02.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46841a2090 con 0x7f46841034b0 2024-05-17T09:23:02.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.836+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46841a99b0 con 0x7f46841034b0 2024-05-17T09:23:02.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.835+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680018680 con 0x7f46841034b0 2024-05-17T09:23:02.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.837+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46840512b0 con 0x7f46841034b0 2024-05-17T09:23:02.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.838+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4680010430 con 0x7f46841034b0 2024-05-17T09:23:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.839+0000 7f467b7fe700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46700b2880 0x7f46700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.840+0000 7f4689b0c700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46700b2880 0x7f46700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.840+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f46800e2760 con 0x7f46841034b0 2024-05-17T09:23:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.840+0000 7f4689b0c700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46700b2880 0x7f46700b4d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f46840769a0 tx=0x7f467400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:02.845+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46800aa0d0 con 0x7f46841034b0 2024-05-17T09:23:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.138+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f468402e5e0 con 0x7f46841034b0 2024-05-17T09:23:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.139+0000 7f467b7fe700 1 -- 172.21.15.19:0/3228694646 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7f46800bee20 con 0x7f46841034b0 2024-05-17T09:23:03.139 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:23:03.139 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:22:41.259768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:23:03.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46700b2880 msgr2=0x7f46700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:03.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46700b2880 0x7f46700b4d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f46840769a0 tx=0x7f467400b410 comp rx=0 tx=0).stop 2024-05-17T09:23:03.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 msgr2=0x7f468419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:03.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 0x7f468419ced0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f468000e9a0 tx=0x7f468000ecb0 comp rx=0 tx=0).stop 2024-05-17T09:23:03.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 shutdown_connections 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4684108730 0x7f46841a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46841034b0 0x7f468419ced0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f46700b2880 0x7f46700b4d90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 --2- 172.21.15.19:0/3228694646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4684107de0 0x7f468419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 >> 172.21.15.19:0/3228694646 conn(0x7f46840fede0 msgr2=0x7f468410ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 shutdown_connections 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:03.142+0000 7f468c571700 1 -- 172.21.15.19:0/3228694646 wait complete. 2024-05-17T09:23:03.143 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-17T09:23:03.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 27125 2.63 MB/sec warmup 61 sec latency 20.435 ms 2024-05-17T09:23:03.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:03 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3228694646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:03.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:03 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3228694646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:03.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:03 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3228694646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:03.893 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 132.3 --await-for 45.0' 2024-05-17T09:23:03.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 132.3 --await-for 45.0 2024-05-17T09:23:04.175 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:04.418 INFO:tasks.workunit.client.0.smithi019.stdout: 1 27630 2.67 MB/sec warmup 62 sec latency 16.967 ms 2024-05-17T09:23:04.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:04 smithi019 ceph-mon[31620]: pgmap v537: 129 pgs: 129 active+clean; 27 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 662 KiB/s wr, 76 op/s 2024-05-17T09:23:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:04 smithi069 ceph-mon[35081]: pgmap v537: 129 pgs: 129 active+clean; 27 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 662 KiB/s wr, 76 op/s 2024-05-17T09:23:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:04 smithi079 ceph-mon[35502]: pgmap v537: 129 pgs: 129 active+clean; 27 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 662 KiB/s wr, 76 op/s 2024-05-17T09:23:05.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28194 2.69 MB/sec warmup 63 sec latency 15.710 ms 2024-05-17T09:23:06.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/1406871221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f041034b0 msgr2=0x7f8f041038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/1406871221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f041034b0 0x7f8f041038c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80099c0 tx=0x7f8ef8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/1406871221 shutdown_connections 2024-05-17T09:23:06.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/1406871221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f04108730 0x7f8f04110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/1406871221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f041034b0 0x7f8f041038c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/1406871221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04107de0 0x7f8f041081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.178+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/1406871221 >> 172.21.15.19:0/1406871221 conn(0x7f8f040fedb0 msgr2=0x7f8f041011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.179+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/1406871221 shutdown_connections 2024-05-17T09:23:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.179+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/1406871221 wait complete. 2024-05-17T09:23:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.179+0000 7f8f0ae7e700 1 Processor -- start 2024-05-17T09:23:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.179+0000 7f8f0ae7e700 1 -- start start 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0ae7e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f041034b0 0x7f8f040ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0ae7e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04107de0 0x7f8f04100200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0ae7e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 0x7f8f04100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0ae7e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04113210 con 0x7f8f04108730 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0ae7e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04113090 con 0x7f8f041034b0 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0ae7e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04113390 con 0x7f8f04107de0 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 0x7f8f04100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 0x7f8f04100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43856/0 (socket says 172.21.15.19:43856) 2024-05-17T09:23:06.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 -- 172.21.15.19:0/4120948649 learned_addr learned my addr 172.21.15.19:0/4120948649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04107de0 0x7f8f04100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f08c1a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f041034b0 0x7f8f040ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 -- 172.21.15.19:0/4120948649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04107de0 msgr2=0x7f8f04100200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04107de0 0x7f8f04100200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 -- 172.21.15.19:0/4120948649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f041034b0 msgr2=0x7f8f040ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f041034b0 0x7f8f040ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 -- 172.21.15.19:0/4120948649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f041a90b0 con 0x7f8f04108730 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.180+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 0x7f8f04100740 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8efc007ee0 tx=0x7f8efc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.181+0000 7f8f01ffb700 1 -- 172.21.15.19:0/4120948649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc004bb0 con 0x7f8f04108730 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.181+0000 7f8f01ffb700 1 -- 172.21.15.19:0/4120948649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8efc004d50 con 0x7f8f04108730 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.181+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f041a93a0 con 0x7f8f04108730 2024-05-17T09:23:06.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.181+0000 7f8f01ffb700 1 -- 172.21.15.19:0/4120948649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0058b0 con 0x7f8f04108730 2024-05-17T09:23:06.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.181+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f041a98e0 con 0x7f8f04108730 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.184+0000 7f8f01ffb700 1 -- 172.21.15.19:0/4120948649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8efc01a020 con 0x7f8f04108730 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.184+0000 7f8f01ffb700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ef40b2900 0x7f8ef40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.185+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ef40b2900 0x7f8ef40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.185+0000 7f8f01ffb700 1 -- 172.21.15.19:0/4120948649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8efc0e1bd0 con 0x7f8f04108730 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.185+0000 7f8f19893b80 1 Processor -- start 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.185+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ef40b2900 0x7f8ef40b4e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.185+0000 7f8f19893b80 1 -- start start 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0aded5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ac68fd0 0x55fc0adedb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0ade7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f19893b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc0ad15e20 con 0x55fc0ade76b0 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f19893b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fc0ad059c0 con 0x55fc0ac68fd0 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f19893b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fc0ad09a00 con 0x55fc0ace6560 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0ade7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f08c1a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0aded5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0ade7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43862/0 (socket says 172.21.15.19:43862) 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 -- 172.21.15.19:0/4193916451 learned_addr learned my addr 172.21.15.19:0/4193916451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f03fff700 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ac68fd0 0x55fc0adedb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 -- 172.21.15.19:0/4193916451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 msgr2=0x55fc0aded5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0aded5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 -- 172.21.15.19:0/4193916451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ac68fd0 msgr2=0x55fc0adedb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ac68fd0 0x55fc0adedb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 -- 172.21.15.19:0/4193916451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc0ac55a60 con 0x55fc0ade76b0 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0aded5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.186+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0ade7ac0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0a60b0 tx=0x7f8efc0a8d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad0a666457959f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.187+0000 7f8eeeffd700 1 -- 172.21.15.19:0/4193916451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0ab070 con 0x55fc0ade76b0 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.187+0000 7f8eeeffd700 1 -- 172.21.15.19:0/4193916451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8efc0a9540 con 0x55fc0ade76b0 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.187+0000 7f8eeeffd700 1 -- 172.21.15.19:0/4193916451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0a92c0 con 0x55fc0ade76b0 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 -- 172.21.15.19:0/4193916451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 msgr2=0x55fc0ade7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0ade7ac0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0a60b0 tx=0x7f8efc0a8d30 comp rx=0 tx=0).stop 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 -- 172.21.15.19:0/4193916451 shutdown_connections 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0aded5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0ade7ac0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4193916451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ac68fd0 0x55fc0adedb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 -- 172.21.15.19:0/4193916451 >> 172.21.15.19:0/4193916451 conn(0x55fc0aca7e10 msgr2=0x55fc0ade25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 -- 172.21.15.19:0/4193916451 shutdown_connections 2024-05-17T09:23:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 -- 172.21.15.19:0/4193916451 wait complete. 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 -- start start 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.188+0000 7f8f19893b80 1 Processor -- start 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 0x55fc0adf17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc0ac0c130 con 0x55fc0ade76b0 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fc0aca7950 con 0x55fc0ace6560 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fc0adfd460 con 0x55fc0ac68fd0 2024-05-17T09:23:06.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f19893b80 10 client.0 _ll_register_callbacks cb 0x55fc0aca0ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 0x55fc0adf17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 0x55fc0adf17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:46532/0 (socket says 172.21.15.19:46532) 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 -- 172.21.15.19:0/369501851 learned_addr learned my addr 172.21.15.19:0/369501851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f0941b700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 -- 172.21.15.19:0/369501851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ac68fd0 msgr2=0x55fc0adf1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 -- 172.21.15.19:0/369501851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 msgr2=0x55fc0adf1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.189+0000 7f8f03fff700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8f03fff700 1 -- 172.21.15.19:0/369501851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0aca4820 con 0x55fc0ace6560 2024-05-17T09:23:06.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8f03fff700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 0x55fc0adf17b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0012770 tx=0x7f8ef0012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ab1b3ab0677833 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8ee3fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8ee3fff700 1 -- 172.21.15.19:0/369501851 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000f480 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8efc0a9540 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8ee3fff700 1 -- 172.21.15.19:0/369501851 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8ef000f620 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8ee3fff700 1 -- 172.21.15.19:0/369501851 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ef000f820 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8f19893b80 10 client.26069 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc0adfee00 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.190+0000 7f8ee3fff700 1 -- 172.21.15.19:0/369501851 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8efc0a9540 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8ee3fff700 1 -- 172.21.15.19:0/369501851 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef00108f0 con 0x55fc0ace6560 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f19893b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:06.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f19893b80 10 client.26069 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f19893b80 20 client.26069 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f19893b80 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae01510 0x55fc0ae03960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f19893b80 4 client.26069 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc0ad17d70 con 0x55fc0ae01510 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae01510 0x55fc0ae03960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8ee3fff700 10 client.26069 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.191+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae01510 0x55fc0ae03960 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf7b3c6409b07d8 server_cookie=9c050f2bb03c524d in_seq=0 out_seq=0 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8ee3fff700 1 -- 172.21.15.19:0/369501851 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fc0ad17d70 con 0x55fc0ae01510 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8ee3fff700 10 client.26069 handle_command_reply: tid=0 2024-05-17T09:23:06.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 client.26069 shutdown 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 msgr2=0x55fc0adf17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 0x55fc0adf17b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0012770 tx=0x7f8ef0012a80 comp rx=0 tx=0).stop 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 shutdown_connections 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae01510 0x55fc0ae03960 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 --2- 172.21.15.19:0/369501851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ace6560 0x55fc0adf17b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8ee3fff700 0 client.26069 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:06.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.192+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 >> 172.21.15.19:0/369501851 conn(0x55fc0aca7e10 msgr2=0x55fc0addc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:06.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.193+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 shutdown_connections 2024-05-17T09:23:06.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.193+0000 7f8f19893b80 1 -- 172.21.15.19:0/369501851 wait complete. 2024-05-17T09:23:06.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.193+0000 7f8f19893b80 20 client.26069 trim_cache size 0 max 16384 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 Processor -- start 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 -- start start 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041b4db0 con 0x55fc0ac68fd0 2024-05-17T09:23:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041b2640 con 0x55fc0ade76b0 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f19893b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041b0490 con 0x55fc0ace6560 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f08c1a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f0941b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f08c1a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43878/0 (socket says 172.21.15.19:43878) 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f08c1a700 1 -- 172.21.15.19:0/4052900589 learned_addr learned my addr 172.21.15.19:0/4052900589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f0941b700 1 -- 172.21.15.19:0/4052900589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 msgr2=0x55fc0adec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f0941b700 1 -- 172.21.15.19:0/4052900589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 msgr2=0x55fc0adf0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f0941b700 1 -- 172.21.15.19:0/4052900589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc0adfec10 con 0x55fc0ade76b0 2024-05-17T09:23:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f03fff700 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.215+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.216+0000 7f8f0941b700 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55fc0ade8100 tx=0x7f8efc007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7027b8433ec747a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.216+0000 7f8eee7fc700 1 -- 172.21.15.19:0/4052900589 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0ab040 con 0x55fc0ade76b0 2024-05-17T09:23:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.216+0000 7f8eee7fc700 1 -- 172.21.15.19:0/4052900589 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8efc0c7070 con 0x55fc0ade76b0 2024-05-17T09:23:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.216+0000 7f8eee7fc700 1 -- 172.21.15.19:0/4052900589 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0a9300 con 0x55fc0ade76b0 2024-05-17T09:23:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 -- 172.21.15.19:0/4052900589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 msgr2=0x55fc0adf1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55fc0ade8100 tx=0x7f8efc007f90 comp rx=0 tx=0).stop 2024-05-17T09:23:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 -- 172.21.15.19:0/4052900589 shutdown_connections 2024-05-17T09:23:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf1b30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 --2- 172.21.15.19:0/4052900589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 -- 172.21.15.19:0/4052900589 >> 172.21.15.19:0/4052900589 conn(0x55fc0aca7e10 msgr2=0x55fc0adeaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 -- 172.21.15.19:0/4052900589 shutdown_connections 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 -- 172.21.15.19:0/4052900589 wait complete. 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.217+0000 7f8f19893b80 1 -- start start 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 Processor -- start 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adf9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041b0b00 con 0x55fc0ac68fd0 2024-05-17T09:23:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041b1f70 con 0x55fc0ade76b0 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041b11d0 con 0x55fc0ace6560 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f19893b80 10 client.0 _ll_register_callbacks cb 0x55fc0aca0ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f08c1a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adf9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f08c1a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43888/0 (socket says 172.21.15.19:43888) 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f08c1a700 1 -- 172.21.15.19:0/234009092 learned_addr learned my addr 172.21.15.19:0/234009092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.218+0000 7f8f0941b700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f03fff700 1 -- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 msgr2=0x55fc0adf6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f03fff700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f03fff700 1 -- 172.21.15.19:0/234009092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 msgr2=0x55fc0adf93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f03fff700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f03fff700 1 -- 172.21.15.19:0/234009092 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0adf9fa0 con 0x55fc0ace6560 2024-05-17T09:23:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f0941b700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f03fff700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adf9900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000c00 tx=0x7f8ef001be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0e80291dc967a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8ee3fff700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8ee3fff700 1 -- 172.21.15.19:0/234009092 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000fcd0 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8efc0c7070 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8ee3fff700 1 -- 172.21.15.19:0/234009092 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8ef001b3d0 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8ee3fff700 1 -- 172.21.15.19:0/234009092 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ef000fe70 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f19893b80 10 client.26068 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc0ae6c530 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.219+0000 7f8ee3fff700 1 -- 172.21.15.19:0/234009092 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8efc0c7070 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8ee3fff700 1 -- 172.21.15.19:0/234009092 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef0010430 con 0x55fc0ace6560 2024-05-17T09:23:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f19893b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f19893b80 10 client.26068 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f19893b80 20 client.26068 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f19893b80 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae6ccd0 0x55fc0ae6f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f19893b80 4 client.26068 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.3, "await_for": 45.0}] 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.3, "await_for": 45.0}) -- 0x55fc0ad11550 con 0x55fc0ae6ccd0 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.220+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae6ccd0 0x55fc0ae6f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.221+0000 7f8ee3fff700 10 client.26068 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.221+0000 7f8f08c1a700 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae6ccd0 0x55fc0ae6f0f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5840b84ea131a6a server_cookie=755f6929c068706 in_seq=0 out_seq=0 2024-05-17T09:23:06.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.322+0000 7f8ee3fff700 1 -- 172.21.15.19:0/234009092 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fc0ad11550 con 0x55fc0ae6ccd0 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.322+0000 7f8ee3fff700 10 client.26068 handle_command_reply: tid=0 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 client.26068 shutdown 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 msgr2=0x55fc0adf9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adf9900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000c00 tx=0x7f8ef001be30 comp rx=0 tx=0).stop 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 shutdown_connections 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fc0ae6ccd0 0x55fc0ae6f0f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc0ade76b0 0x55fc0adf6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc0ac68fd0 0x55fc0adf93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 --2- 172.21.15.19:0/234009092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fc0ace6560 0x55fc0adf9900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8ee3fff700 0 client.26068 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 >> 172.21.15.19:0/234009092 conn(0x55fc0aca7e10 msgr2=0x55fc0addcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 shutdown_connections 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 1 -- 172.21.15.19:0/234009092 wait complete. 2024-05-17T09:23:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.323+0000 7f8f19893b80 20 client.26068 trim_cache size 0 max 16384 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 10, 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "34bbc80e": { 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 10, 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:23:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 132.3, 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ef40b2900 msgr2=0x7f8ef40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ef40b2900 0x7f8ef40b4e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef801a040 comp rx=0 tx=0).stop 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 msgr2=0x7f8f04100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 0x7f8f04100740 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8efc007ee0 tx=0x7f8efc00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 shutdown_connections 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f04108730 0x7f8f04100740 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f041034b0 0x7f8f040ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8ef40b2900 0x7f8ef40b4e10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 --2- 172.21.15.19:0/4120948649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04107de0 0x7f8f04100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 >> 172.21.15.19:0/4120948649 conn(0x7f8f040fedb0 msgr2=0x7f8f04101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 shutdown_connections 2024-05-17T09:23:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:06.324+0000 7f8f0ae7e700 1 -- 172.21.15.19:0/4120948649 wait complete. 2024-05-17T09:23:06.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.70 MB/sec warmup 64 sec latency 81.716 ms 2024-05-17T09:23:06.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:06 smithi019 ceph-mon[31620]: pgmap v538: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 803 KiB/s wr, 97 op/s 2024-05-17T09:23:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:06 smithi069 ceph-mon[35081]: pgmap v538: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 803 KiB/s wr, 97 op/s 2024-05-17T09:23:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:06 smithi079 ceph-mon[35502]: pgmap v538: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 803 KiB/s wr, 97 op/s 2024-05-17T09:23:07.014 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '34bbc80e', quiesce took 6.3 seconds. Will release after: 6.000000000000001 2024-05-17T09:23:07.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.66 MB/sec warmup 65 sec latency 1081.753 ms 2024-05-17T09:23:08.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.62 MB/sec warmup 66 sec latency 2082.215 ms 2024-05-17T09:23:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:08 smithi069 ceph-mon[35081]: pgmap v539: 129 pgs: 129 active+clean; 27 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:23:08.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:08 smithi019 ceph-mon[31620]: pgmap v539: 129 pgs: 129 active+clean; 27 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:23:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:08 smithi079 ceph-mon[35502]: pgmap v539: 129 pgs: 129 active+clean; 27 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:23:09.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.58 MB/sec warmup 67 sec latency 3082.349 ms 2024-05-17T09:23:10.419 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.54 MB/sec warmup 68 sec latency 4082.460 ms 2024-05-17T09:23:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:10 smithi069 ceph-mon[35081]: pgmap v540: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:23:10.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:10 smithi019 ceph-mon[31620]: pgmap v540: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:23:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:10 smithi079 ceph-mon[35502]: pgmap v540: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:23:11.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.50 MB/sec warmup 69 sec latency 5082.556 ms 2024-05-17T09:23:12.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.47 MB/sec warmup 70 sec latency 6082.678 ms 2024-05-17T09:23:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:12 smithi069 ceph-mon[35081]: pgmap v541: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1019 KiB/s wr, 111 op/s 2024-05-17T09:23:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:12 smithi019 ceph-mon[31620]: pgmap v541: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1019 KiB/s wr, 111 op/s 2024-05-17T09:23:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:12 smithi079 ceph-mon[35502]: pgmap v541: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1019 KiB/s wr, 111 op/s 2024-05-17T09:23:12.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-17T09:23:13.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 -- 172.21.15.19:0/3690418048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 msgr2=0x7f6ccc104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/3690418048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc104190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009900 tx=0x7f6cc002a410 comp rx=0 tx=0).stop 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 -- 172.21.15.19:0/3690418048 shutdown_connections 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/3690418048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ccc108a80 0x7f6ccc10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/3690418048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc104190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/3690418048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc102b40 0x7f6ccc103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 -- 172.21.15.19:0/3690418048 >> 172.21.15.19:0/3690418048 conn(0x7f6ccc0fdd30 msgr2=0x7f6ccc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 -- 172.21.15.19:0/3690418048 shutdown_connections 2024-05-17T09:23:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.135+0000 7f6cd08bf640 1 -- 172.21.15.19:0/3690418048 wait complete. 2024-05-17T09:23:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.136+0000 7f6cd08bf640 1 Processor -- start 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.136+0000 7f6cd08bf640 1 -- start start 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cd08bf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ccc102b40 0x7f6ccc1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cd08bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cd08bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc108a80 0x7f6ccc1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cd08bf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc10e4f0 con 0x7f6ccc103d80 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cd08bf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc10e370 con 0x7f6ccc108a80 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cd08bf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc10e670 con 0x7f6ccc102b40 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43918/0 (socket says 172.21.15.19:43918) 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/2418370955 learned_addr learned my addr 172.21.15.19:0/2418370955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cca59c640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ccc102b40 0x7f6ccc1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc108a80 0x7f6ccc1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/2418370955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ccc102b40 msgr2=0x7f6ccc1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ccc102b40 0x7f6ccc1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/2418370955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc108a80 msgr2=0x7f6ccc1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc108a80 0x7f6ccc1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.137+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/2418370955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc1a0c00 con 0x7f6ccc103d80 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.138+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc199600 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009230 tx=0x7f6cc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.138+0000 7f6cbb7fe640 1 -- 172.21.15.19:0/2418370955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc003f030 con 0x7f6ccc103d80 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.138+0000 7f6cbb7fe640 1 -- 172.21.15.19:0/2418370955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cc0003c90 con 0x7f6ccc103d80 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.138+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc1a0e90 con 0x7f6ccc103d80 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.138+0000 7f6cbb7fe640 1 -- 172.21.15.19:0/2418370955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0035070 con 0x7f6ccc103d80 2024-05-17T09:23:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.138+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1a12f0 con 0x7f6ccc103d80 2024-05-17T09:23:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 Processor -- start 2024-05-17T09:23:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cbb7fe640 1 -- 172.21.15.19:0/2418370955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6cc0042030 con 0x7f6ccc103d80 2024-05-17T09:23:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 -- start start 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a9e61fc30 con 0x558a9e30c1b0 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a9e60f7d0 con 0x558a9e2e32f0 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cd7770740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558a9e613810 con 0x558a9e2eb4d0 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43924/0 (socket says 172.21.15.19:43924) 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6ccad9d640 1 -- 172.21.15.19:0/1196682021 learned_addr learned my addr 172.21.15.19:0/1196682021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cca59c640 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.142+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6ccad9d640 1 -- 172.21.15.19:0/1196682021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 msgr2=0x558a9e6b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6ccad9d640 1 -- 172.21.15.19:0/1196682021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 msgr2=0x558a9e6ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6ccad9d640 1 -- 172.21.15.19:0/1196682021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9e324210 con 0x558a9e30c1b0 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6ad3a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009f10 tx=0x7f6cc4010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6de8133f87261ec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6cbb7fe640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb40b2920 0x7f6cb40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6cb8ff9640 1 -- 172.21.15.19:0/1196682021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc401e070 con 0x558a9e30c1b0 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6cb8ff9640 1 -- 172.21.15.19:0/1196682021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cc400d8f0 con 0x558a9e30c1b0 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6cb8ff9640 1 -- 172.21.15.19:0/1196682021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4019880 con 0x558a9e30c1b0 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6cca59c640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb40b2920 0x7f6cb40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.143+0000 7f6cbb7fe640 1 -- 172.21.15.19:0/2418370955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6cc0102ae0 con 0x7f6ccc103d80 2024-05-17T09:23:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 -- 172.21.15.19:0/1196682021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 msgr2=0x558a9e6ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6ad3a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009f10 tx=0x7f6cc4010930 comp rx=0 tx=0).stop 2024-05-17T09:23:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cca59c640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb40b2920 0x7f6cb40b4e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x558a9e6ae780 tx=0x7f6cbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 -- 172.21.15.19:0/1196682021 shutdown_connections 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 --2- 172.21.15.19:0/1196682021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6ad3a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 -- 172.21.15.19:0/1196682021 >> 172.21.15.19:0/1196682021 conn(0x558a9e300300 msgr2=0x558a9e6a8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.144+0000 7f6cd7770740 1 -- 172.21.15.19:0/1196682021 shutdown_connections 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 -- 172.21.15.19:0/1196682021 wait complete. 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 -- start start 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 Processor -- start 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a9e2f5790 con 0x558a9e30c1b0 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a9e5cb890 con 0x558a9e2e32f0 2024-05-17T09:23:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558a9e6b95c0 con 0x558a9e2eb4d0 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6cd7770740 10 client.0 _ll_register_callbacks cb 0x558a9e5ced60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.145+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43938/0 (socket says 172.21.15.19:43938) 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 -- 172.21.15.19:0/3240717300 learned_addr learned my addr 172.21.15.19:0/3240717300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 -- 172.21.15.19:0/3240717300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 msgr2=0x558a9e6bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 -- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 msgr2=0x558a9e6bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 -- 172.21.15.19:0/3240717300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9e3f0620 con 0x558a9e30c1b0 2024-05-17T09:23:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6c25b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4006010 tx=0x7f6cc400d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932a313621633e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3240717300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc401e070 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3240717300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6cc400dd30 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.146+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc400d8f0 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3240717300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6cc4004a40 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 10 client.16407 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a9e6c54e0 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3240717300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cc400d8f0 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3240717300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6cc40056c0 con 0x558a9e30c1b0 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 10 client.16407 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 20 client.16407 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2cad80 0x558a9e6c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 4 client.16407 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.147+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a9e621b80 con 0x558a9e2cad80 2024-05-17T09:23:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.148+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2cad80 0x558a9e6c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.148+0000 7f6cb1ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.148+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2cad80 0x558a9e6c89b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2876be367943f6 server_cookie=e228c650bd483ee3 in_seq=0 out_seq=0 2024-05-17T09:23:13.152 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3240717300 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558a9e621b80 con 0x558a9e2cad80 2024-05-17T09:23:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cb1ffb640 10 client.16407 handle_command_reply: tid=0 2024-05-17T09:23:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 client.16407 shutdown 2024-05-17T09:23:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 msgr2=0x558a9e6c25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6c25b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4006010 tx=0x7f6cc400d720 comp rx=0 tx=0).stop 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 shutdown_connections 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2cad80 0x558a9e6c89b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e6c25b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3240717300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cb1ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 >> 172.21.15.19:0/3240717300 conn(0x558a9e300300 msgr2=0x558a9e6a7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 shutdown_connections 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 1 -- 172.21.15.19:0/3240717300 wait complete. 2024-05-17T09:23:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.149+0000 7f6cd7770740 20 client.16407 trim_cache size 0 max 16384 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.165+0000 7f6cd7770740 1 Processor -- start 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 -- start start 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2eb4d0 0x558a9e6c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e30c1b0 0x558a9e6a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc1ad0d0 con 0x558a9e2e32f0 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc1aa9a0 con 0x558a9e2eb4d0 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cd7770740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc1a87f0 con 0x558a9e30c1b0 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43942/0 (socket says 172.21.15.19:43942) 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 -- 172.21.15.19:0/632185713 learned_addr learned my addr 172.21.15.19:0/632185713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e30c1b0 0x558a9e6a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2eb4d0 0x558a9e6c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 -- 172.21.15.19:0/632185713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e30c1b0 msgr2=0x558a9e6a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e30c1b0 0x558a9e6a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 -- 172.21.15.19:0/632185713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2eb4d0 msgr2=0x558a9e6c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2eb4d0 0x558a9e6c24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.166+0000 7f6cca59c640 1 -- 172.21.15.19:0/632185713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9e6a2fd0 con 0x558a9e2e32f0 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.167+0000 7f6cca59c640 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb820 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc003e90 tx=0x7f6cbc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a831110539e73428 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.167+0000 7f6cb3fff640 1 -- 172.21.15.19:0/632185713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc01eab0 con 0x558a9e2e32f0 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.167+0000 7f6cb3fff640 1 -- 172.21.15.19:0/632185713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cbc02f070 con 0x558a9e2e32f0 2024-05-17T09:23:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.167+0000 7f6cb3fff640 1 -- 172.21.15.19:0/632185713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc02ac60 con 0x558a9e2e32f0 2024-05-17T09:23:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 -- 172.21.15.19:0/632185713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 msgr2=0x558a9e6bb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb820 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc003e90 tx=0x7f6cbc019000 comp rx=0 tx=0).stop 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 -- 172.21.15.19:0/632185713 shutdown_connections 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e30c1b0 0x558a9e6a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2eb4d0 0x558a9e6c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 --2- 172.21.15.19:0/632185713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e2e32f0 0x558a9e6bb820 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 -- 172.21.15.19:0/632185713 >> 172.21.15.19:0/632185713 conn(0x558a9e300300 msgr2=0x558a9e6bb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 -- 172.21.15.19:0/632185713 shutdown_connections 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 -- 172.21.15.19:0/632185713 wait complete. 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 -- start start 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.168+0000 7f6cd7770740 1 Processor -- start 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e72efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc1a8e60 con 0x558a9e30c1b0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc1aa2d0 con 0x558a9e2e32f0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc1a9530 con 0x558a9e2eb4d0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cd7770740 10 client.0 _ll_register_callbacks cb 0x7f6cc4009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e72efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:38642/0 (socket says 172.21.15.19:38642) 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e72efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43956/0 (socket says 172.21.15.19:43956) 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/3966941664 learned_addr learned my addr 172.21.15.19:0/3966941664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 msgr2=0x558a9e6bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/3966941664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 msgr2=0x558a9e72efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e72efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6cc9d9b640 1 -- 172.21.15.19:0/3966941664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9e72f510 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.169+0000 7f6ccad9d640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e72efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cc9d9b640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bf9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0005760 tx=0x7f6cc0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1cf72228d77b88 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3966941664 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0049040 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3966941664 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6cc00ca270 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cbc02f070 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3966941664 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6cc00e9050 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cd7770740 10 client.26086 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a9e730810 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.170+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3966941664 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cbc02f070 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3966941664 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6cc00d2720 con 0x558a9e2eb4d0 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cd7770740 10 client.26086 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cd7770740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cd7770740 20 client.26086 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2a01c0 0x558a9e733110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cd7770740 4 client.26086 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a9e61b360 con 0x558a9e2a01c0 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.171+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2a01c0 0x558a9e733110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.172+0000 7f6cca59c640 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2a01c0 0x558a9e733110 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ff660a9b0e4567 server_cookie=cc2f0353144dd0ec in_seq=0 out_seq=0 2024-05-17T09:23:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.172+0000 7f6cb1ffb640 10 client.26086 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cb1ffb640 1 -- 172.21.15.19:0/3966941664 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a9e61b360 con 0x558a9e2a01c0 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cb1ffb640 10 client.26086 handle_command_reply: tid=0 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 client.26086 shutdown 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 msgr2=0x558a9e6bf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bf9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0005760 tx=0x7f6cc0009680 comp rx=0 tx=0).stop 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 shutdown_connections 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558a9e2a01c0 0x558a9e733110 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a9e30c1b0 0x558a9e72efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a9e2eb4d0 0x558a9e6bf9e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cb1ffb640 0 client.26086 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 --2- 172.21.15.19:0/3966941664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a9e2e32f0 0x558a9e6bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 >> 172.21.15.19:0/3966941664 conn(0x558a9e300300 msgr2=0x558a9e6a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 shutdown_connections 2024-05-17T09:23:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 1 -- 172.21.15.19:0/3966941664 wait complete. 2024-05-17T09:23:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.190+0000 7f6cd7770740 20 client.26086 trim_cache size 0 max 16384 2024-05-17T09:23:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:23:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:23:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb40b2920 msgr2=0x7f6cb40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb40b2920 0x7f6cb40b4e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x558a9e6ae780 tx=0x7f6cbc00b410 comp rx=0 tx=0).stop 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 msgr2=0x7f6ccc199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc199600 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009230 tx=0x7f6cc00042f0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 shutdown_connections 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ccc108a80 0x7f6ccc1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6cb40b2920 0x7f6cb40b4e30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ccc103d80 0x7f6ccc199600 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 --2- 172.21.15.19:0/2418370955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ccc102b40 0x7f6ccc1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 >> 172.21.15.19:0/2418370955 conn(0x7f6ccc0fdd30 msgr2=0x7f6ccc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 shutdown_connections 2024-05-17T09:23:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:13.191+0000 7f6cd08bf640 1 -- 172.21.15.19:0/2418370955 wait complete. 2024-05-17T09:23:13.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.43 MB/sec warmup 71 sec latency 7082.748 ms 2024-05-17T09:23:14.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.40 MB/sec warmup 72 sec latency 8082.786 ms 2024-05-17T09:23:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:14.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:14 smithi019 ceph-mon[31620]: pgmap v542: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 946 KiB/s wr, 112 op/s 2024-05-17T09:23:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:14 smithi069 ceph-mon[35081]: pgmap v542: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 946 KiB/s wr, 112 op/s 2024-05-17T09:23:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:14 smithi079 ceph-mon[35502]: pgmap v542: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 946 KiB/s wr, 112 op/s 2024-05-17T09:23:15.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.36 MB/sec warmup 73 sec latency 9082.815 ms 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3004381938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4109850 msgr2=0x7f0fc41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3004381938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4109850 0x7f0fc41111b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b0d0 tx=0x7f0fbc00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3004381938 shutdown_connections 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3004381938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc41045d0 0x7f0fc41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3004381938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4109850 0x7f0fc41111b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3004381938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc4108f00 0x7f0fc4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3004381938 >> 172.21.15.19:0/3004381938 conn(0x7f0fc40ffec0 msgr2=0x7f0fc4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3004381938 shutdown_connections 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.219+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3004381938 wait complete. 2024-05-17T09:23:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.220+0000 7f0fcc6b4700 1 Processor -- start 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.220+0000 7f0fcc6b4700 1 -- start start 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.220+0000 7f0fcc6b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc41045d0 0x7f0fc41a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.220+0000 7f0fcc6b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 0x7f0fc41a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.220+0000 7f0fcc6b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc4109850 0x7f0fc419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fcc6b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4114330 con 0x7f0fc4108f00 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fcc6b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41141b0 con 0x7f0fc41045d0 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fcc6b4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41144b0 con 0x7f0fc4109850 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 0x7f0fc41a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 0x7f0fc41a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54938/0 (socket says 172.21.15.19:54938) 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/3682941478 learned_addr learned my addr 172.21.15.19:0/3682941478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fca450700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc41045d0 0x7f0fc41a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fcac51700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc4109850 0x7f0fc419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/3682941478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc4109850 msgr2=0x7f0fc419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc4109850 0x7f0fc419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/3682941478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc41045d0 msgr2=0x7f0fc41a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc41045d0 0x7f0fc41a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/3682941478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc419b260 con 0x7f0fc4108f00 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 0x7f0fc41a0ee0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400d950 tx=0x7f0fb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fbb7fe700 1 -- 172.21.15.19:0/3682941478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4016070 con 0x7f0fc4108f00 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc419b550 con 0x7f0fc4108f00 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fbb7fe700 1 -- 172.21.15.19:0/3682941478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fb4004830 con 0x7f0fc4108f00 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.221+0000 7f0fbb7fe700 1 -- 172.21.15.19:0/3682941478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4005420 con 0x7f0fc4108f00 2024-05-17T09:23:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.222+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc41a7750 con 0x7f0fc4108f00 2024-05-17T09:23:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.224+0000 7f0fbb7fe700 1 -- 172.21.15.19:0/3682941478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0fb4004220 con 0x7f0fc4108f00 2024-05-17T09:23:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.225+0000 7f0fbb7fe700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0fb00b2880 0x7f0fb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.225+0000 7f0fca450700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0fb00b2880 0x7f0fb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.225+0000 7f0fbb7fe700 1 -- 172.21.15.19:0/3682941478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0fb40e22b0 con 0x7f0fc4108f00 2024-05-17T09:23:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fca450700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0fb00b2880 0x7f0fb00b4d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009c60 tx=0x7f0fc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 Processor -- start 2024-05-17T09:23:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 -- start start 2024-05-17T09:23:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d40607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c2d3f8ec90 con 0x55c2d3ee1e40 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2d3f7e830 con 0x55c2d3f5f3d0 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c2d3f82870 con 0x55c2d40603c0 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fc9c4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fc9c4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4066520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54948/0 (socket says 172.21.15.19:54948) 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/2820003399 learned_addr learned my addr 172.21.15.19:0/2820003399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fcac51700 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d40607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.226+0000 7f0fca450700 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/2820003399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 msgr2=0x55c2d40607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d40607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/2820003399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 msgr2=0x55c2d4065fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4065fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fc9c4f700 1 -- 172.21.15.19:0/2820003399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d3ece8d0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fca450700 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4066520 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40a5540 tx=0x7f0fb4004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1d00ef17e2220 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fb8ff9700 1 -- 172.21.15.19:0/2820003399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40910d0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fb8ff9700 1 -- 172.21.15.19:0/2820003399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fb40a9bc0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.227+0000 7f0fb8ff9700 1 -- 172.21.15.19:0/2820003399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40a9940 con 0x55c2d3ee1e40 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2820003399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 msgr2=0x55c2d4066520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4066520 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40a5540 tx=0x7f0fb4004590 comp rx=0 tx=0).stop 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2820003399 shutdown_connections 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d40607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2820003399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4066520 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2820003399 >> 172.21.15.19:0/2820003399 conn(0x55c2d3f20c80 msgr2=0x55c2d405aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2820003399 shutdown_connections 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.228+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2820003399 wait complete. 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 -- start start 2024-05-17T09:23:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 Processor -- start 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3ee1e40 0x55c2d4069c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 0x55c2d406a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c2d3e84fa0 con 0x55c2d40603c0 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2d3f207c0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c2d40611f0 con 0x55c2d3f5f3d0 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fdb0c9b80 10 client.0 _ll_register_callbacks cb 0x55c2d3f19920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fcac51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 0x55c2d406a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fcac51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 0x55c2d406a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54954/0 (socket says 172.21.15.19:54954) 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fcac51700 1 -- 172.21.15.19:0/1295462193 learned_addr learned my addr 172.21.15.19:0/1295462193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fca450700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3ee1e40 0x55c2d4069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.229+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fcac51700 1 -- 172.21.15.19:0/1295462193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 msgr2=0x55c2d406a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fcac51700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d406a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fcac51700 1 -- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3ee1e40 msgr2=0x55c2d4069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fcac51700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3ee1e40 0x55c2d4069c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fcac51700 1 -- 172.21.15.19:0/1295462193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d3f1d690 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fcac51700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 0x55c2d406a6e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00bad0 tx=0x7f0fbc00bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794ee3d8ef67ac67 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fadffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fadffb700 1 -- 172.21.15.19:0/1295462193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc003a00 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fadffb700 1 -- 172.21.15.19:0/1295462193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0fbc003ba0 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb40a9bc0 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fadffb700 1 -- 172.21.15.19:0/1295462193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0fbc02e050 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fdb0c9b80 10 client.16425 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2d40775c0 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.230+0000 7f0fadffb700 1 -- 172.21.15.19:0/1295462193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fb40a9bc0 con 0x55c2d40603c0 2024-05-17T09:23:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fadffb700 1 -- 172.21.15.19:0/1295462193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0fbc011070 con 0x55c2d40603c0 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fdb0c9b80 10 client.16425 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fdb0c9b80 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fdb0c9b80 20 client.16425 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d4079c00 0x55c2d407c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fdb0c9b80 4 client.16425 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2d3f90be0 con 0x55c2d4079c00 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fca450700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d4079c00 0x55c2d407c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fadffb700 10 client.16425 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.231+0000 7f0fca450700 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d4079c00 0x55c2d407c040 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3661efadd62cee3 server_cookie=2849370a1c54b383 in_seq=0 out_seq=0 2024-05-17T09:23:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fadffb700 1 -- 172.21.15.19:0/1295462193 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c2d3f90be0 con 0x55c2d4079c00 2024-05-17T09:23:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fadffb700 10 client.16425 handle_command_reply: tid=0 2024-05-17T09:23:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 client.16425 shutdown 2024-05-17T09:23:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 msgr2=0x55c2d406a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 0x55c2d406a6e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00bad0 tx=0x7f0fbc00bb00 comp rx=0 tx=0).stop 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 shutdown_connections 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d4079c00 0x55c2d407c040 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d40603c0 0x55c2d406a6e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3ee1e40 0x55c2d4069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fadffb700 0 client.16425 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/1295462193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.232+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 >> 172.21.15.19:0/1295462193 conn(0x55c2d3f20c80 msgr2=0x55c2d405aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.233+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 shutdown_connections 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.233+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/1295462193 wait complete. 2024-05-17T09:23:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.233+0000 7f0fdb0c9b80 20 client.16425 trim_cache size 0 max 16384 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.253+0000 7f0fdb0c9b80 1 Processor -- start 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.253+0000 7f0fdb0c9b80 1 -- start start 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4065160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d405fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d40603c0 0x55c2d4075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41b2c10 con 0x55c2d3ee1e40 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41b04a0 con 0x55c2d40603c0 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ae2f0 con 0x55c2d3f5f3d0 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4065160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4065160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54964/0 (socket says 172.21.15.19:54964) 2024-05-17T09:23:16.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 -- 172.21.15.19:0/2744009638 learned_addr learned my addr 172.21.15.19:0/2744009638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d405fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 -- 172.21.15.19:0/2744009638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 msgr2=0x55c2d405fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d405fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fcac51700 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d40603c0 0x55c2d4075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 -- 172.21.15.19:0/2744009638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d40603c0 msgr2=0x55c2d4075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d40603c0 0x55c2d4075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 -- 172.21.15.19:0/2744009638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d406c290 con 0x55c2d3ee1e40 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d405fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0fca450700 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4065160 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0013010 tx=0x7f0fc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0eca4021c58276e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0faffff700 1 -- 172.21.15.19:0/2744009638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc002f070 con 0x55c2d3ee1e40 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0faffff700 1 -- 172.21.15.19:0/2744009638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fc0029840 con 0x55c2d3ee1e40 2024-05-17T09:23:16.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.254+0000 7f0faffff700 1 -- 172.21.15.19:0/2744009638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00286f0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2744009638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 msgr2=0x55c2d4065160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4065160 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0013010 tx=0x7f0fc0014000 comp rx=0 tx=0).stop 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2744009638 shutdown_connections 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d40603c0 0x55c2d4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d4065160 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/2744009638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d3f5f3d0 0x55c2d405fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2744009638 >> 172.21.15.19:0/2744009638 conn(0x55c2d3f20c80 msgr2=0x55c2d40638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2744009638 shutdown_connections 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.255+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/2744009638 wait complete. 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 -- start start 2024-05-17T09:23:16.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 Processor -- start 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d40722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4070670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d4070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ae960 con 0x55c2d3ee1e40 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41afdd0 con 0x55c2d3f5f3d0 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41af030 con 0x55c2d40603c0 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fdb0c9b80 10 client.0 _ll_register_callbacks cb 0x55c2d3f19920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fca450700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d40722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fca450700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d40722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54966/0 (socket says 172.21.15.19:54966) 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fca450700 1 -- 172.21.15.19:0/774061621 learned_addr learned my addr 172.21.15.19:0/774061621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.256+0000 7f0fcac51700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d4070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fca450700 1 -- 172.21.15.19:0/774061621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 msgr2=0x55c2d4070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fca450700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d4070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fca450700 1 -- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 msgr2=0x55c2d4070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fca450700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fca450700 1 -- 172.21.15.19:0/774061621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d4071170 con 0x55c2d3ee1e40 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fca450700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d40722e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0fc002d040 tx=0x7f0fc0003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd47ab7d637b9a1f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fadffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fadffb700 1 -- 172.21.15.19:0/774061621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc002f040 con 0x55c2d3ee1e40 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fc0029840 con 0x55c2d3ee1e40 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fadffb700 1 -- 172.21.15.19:0/774061621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0fc0029ab0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fadffb700 1 -- 172.21.15.19:0/774061621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0fc0041050 con 0x55c2d3ee1e40 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fdb0c9b80 10 client.16431 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2d40e97d0 con 0x55c2d3ee1e40 2024-05-17T09:23:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.257+0000 7f0fadffb700 1 -- 172.21.15.19:0/774061621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fc0029840 con 0x55c2d3ee1e40 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fadffb700 1 -- 172.21.15.19:0/774061621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0fc0032070 con 0x55c2d3ee1e40 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fdb0c9b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fdb0c9b80 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fdb0c9b80 20 client.16431 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d40e9e80 0x55c2d40ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fdb0c9b80 4 client.16431 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2d3f8a3c0 con 0x55c2d40e9e80 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d40e9e80 0x55c2d40ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fc9c4f700 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d40e9e80 0x55c2d40ec2a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351a89bfee29cb45 server_cookie=16185b9bd9eba769 in_seq=0 out_seq=0 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.258+0000 7f0fadffb700 10 client.16431 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fadffb700 1 -- 172.21.15.19:0/774061621 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c2d3f8a3c0 con 0x55c2d40e9e80 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fadffb700 10 client.16431 handle_command_reply: tid=0 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 client.16431 shutdown 2024-05-17T09:23:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 msgr2=0x55c2d40722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d40722e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0fc002d040 tx=0x7f0fc0003f00 comp rx=0 tx=0).stop 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 shutdown_connections 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2d40603c0 0x55c2d4070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c2d40e9e80 0x55c2d40ec2a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c2d3ee1e40 0x55c2d40722e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 --2- 172.21.15.19:0/774061621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2d3f5f3d0 0x55c2d4070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fadffb700 0 client.16431 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 >> 172.21.15.19:0/774061621 conn(0x55c2d3f20c80 msgr2=0x55c2d405fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 shutdown_connections 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 1 -- 172.21.15.19:0/774061621 wait complete. 2024-05-17T09:23:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fdb0c9b80 20 client.16431 trim_cache size 0 max 16384 2024-05-17T09:23:16.265 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:23:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-17T09:23:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-17T09:23:16.265 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:23:16.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0fb00b2880 msgr2=0x7f0fb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0fb00b2880 0x7f0fb00b4d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009c60 tx=0x7f0fc0009260 comp rx=0 tx=0).stop 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 msgr2=0x7f0fc41a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.259+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 0x7f0fc41a0ee0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400d950 tx=0x7f0fb400dc60 comp rx=0 tx=0).stop 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 shutdown_connections 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f0fb00b2880 0x7f0fb00b4d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc4109850 0x7f0fc419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc41045d0 0x7f0fc41a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 --2- 172.21.15.19:0/3682941478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4108f00 0x7f0fc41a0ee0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 >> 172.21.15.19:0/3682941478 conn(0x7f0fc40ffec0 msgr2=0x7f0fc41086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 shutdown_connections 2024-05-17T09:23:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:16.260+0000 7f0fcc6b4700 1 -- 172.21.15.19:0/3682941478 wait complete. 2024-05-17T09:23:16.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.33 MB/sec warmup 74 sec latency 10082.885 ms 2024-05-17T09:23:16.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:16 smithi019 ceph-mon[31620]: pgmap v543: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 642 KiB/s wr, 81 op/s 2024-05-17T09:23:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:16 smithi069 ceph-mon[35081]: pgmap v543: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 642 KiB/s wr, 81 op/s 2024-05-17T09:23:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:16 smithi079 ceph-mon[35502]: pgmap v543: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 642 KiB/s wr, 81 op/s 2024-05-17T09:23:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.036+0000 7f62749b0700 1 -- 172.21.15.19:0/161224967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 msgr2=0x7f62701111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 --2- 172.21.15.19:0/161224967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f62701111d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f626800b0d0 tx=0x7f626800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 -- 172.21.15.19:0/161224967 shutdown_connections 2024-05-17T09:23:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 --2- 172.21.15.19:0/161224967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6270104640 0x7f6270104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 --2- 172.21.15.19:0/161224967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f62701111d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 --2- 172.21.15.19:0/161224967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6270108f20 0x7f6270109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 -- 172.21.15.19:0/161224967 >> 172.21.15.19:0/161224967 conn(0x7f6270079ef0 msgr2=0x7f627007a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 -- 172.21.15.19:0/161224967 shutdown_connections 2024-05-17T09:23:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.037+0000 7f62749b0700 1 -- 172.21.15.19:0/161224967 wait complete. 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 Processor -- start 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 -- start start 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6270104640 0x7f6270075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6270108f20 0x7f6270075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f6270076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6270114350 con 0x7f6270109870 2024-05-17T09:23:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62701141d0 con 0x7f6270108f20 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f62749b0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62701144d0 con 0x7f6270104640 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f626ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f6270076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f626ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f6270076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54988/0 (socket says 172.21.15.19:54988) 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f626dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6270108f20 0x7f6270075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f626ed9d700 1 -- 172.21.15.19:0/1293400773 learned_addr learned my addr 172.21.15.19:0/1293400773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.038+0000 7f626e59c700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6270104640 0x7f6270075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f626ed9d700 1 -- 172.21.15.19:0/1293400773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6270104640 msgr2=0x7f6270075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f626ed9d700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6270104640 0x7f6270075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f626ed9d700 1 -- 172.21.15.19:0/1293400773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6270108f20 msgr2=0x7f6270075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f626ed9d700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6270108f20 0x7f6270075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f626ed9d700 1 -- 172.21.15.19:0/1293400773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62701a9080 con 0x7f6270109870 2024-05-17T09:23:17.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f626ed9d700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f6270076530 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6268009850 tx=0x7f6268007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:17.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626801f070 con 0x7f6270109870 2024-05-17T09:23:17.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6268005500 con 0x7f6270109870 2024-05-17T09:23:17.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626801b8f0 con 0x7f6270109870 2024-05-17T09:23:17.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62701a9310 con 0x7f6270109870 2024-05-17T09:23:17.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.039+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a97c0 con 0x7f6270109870 2024-05-17T09:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.040+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62700512b0 con 0x7f6270109870 2024-05-17T09:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.042+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6268022070 con 0x7f6270109870 2024-05-17T09:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.043+0000 7f62677fe700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f625c0b2900 0x7f625c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.043+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f626800f020 con 0x7f6270109870 2024-05-17T09:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.043+0000 7f626e59c700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f625c0b2900 0x7f625c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.043+0000 7f626e59c700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f625c0b2900 0x7f625c0b4e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6260005950 tx=0x7f626000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.046+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6268029020 con 0x7f6270109870 2024-05-17T09:23:17.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.314+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f627002e7a0 con 0x7f6270109870 2024-05-17T09:23:17.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.315+0000 7f62677fe700 1 -- 172.21.15.19:0/1293400773 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7f62680ab750 con 0x7f6270109870 2024-05-17T09:23:17.315 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:23:17.316 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:22:41.259768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f625c0b2900 msgr2=0x7f625c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f625c0b2900 0x7f625c0b4e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6260005950 tx=0x7f626000b410 comp rx=0 tx=0).stop 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 msgr2=0x7f6270076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f6270076530 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6268009850 tx=0x7f6268007d90 comp rx=0 tx=0).stop 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 shutdown_connections 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f625c0b2900 0x7f625c0b4e10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6270104640 0x7f6270075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6270109870 0x7f6270076530 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 --2- 172.21.15.19:0/1293400773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6270108f20 0x7f6270075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 >> 172.21.15.19:0/1293400773 conn(0x7f6270079ef0 msgr2=0x7f6270102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 shutdown_connections 2024-05-17T09:23:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:17.318+0000 7f62749b0700 1 -- 172.21.15.19:0/1293400773 wait complete. 2024-05-17T09:23:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-17T09:23:17.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.30 MB/sec warmup 75 sec latency 11082.938 ms 2024-05-17T09:23:17.610 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T09:23:17.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:23:18.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.27 MB/sec warmup 76 sec latency 12083.003 ms 2024-05-17T09:23:18.448 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 34bbc80e --release --await' 2024-05-17T09:23:18.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 34bbc80e --release --await 2024-05-17T09:23:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:18.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:18 smithi019 ceph-mon[31620]: pgmap v544: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 495 KiB/s wr, 58 op/s 2024-05-17T09:23:18.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:18 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1293400773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:18 smithi069 ceph-mon[35081]: pgmap v544: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 495 KiB/s wr, 58 op/s 2024-05-17T09:23:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:18 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1293400773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:18 smithi079 ceph-mon[35502]: pgmap v544: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 495 KiB/s wr, 58 op/s 2024-05-17T09:23:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:18 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1293400773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:19.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.24 MB/sec warmup 77 sec latency 13083.032 ms 2024-05-17T09:23:20.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.21 MB/sec warmup 78 sec latency 14083.079 ms 2024-05-17T09:23:20.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:20 smithi019 ceph-mon[31620]: pgmap v545: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 207 KiB/s wr, 27 op/s 2024-05-17T09:23:20.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:20 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:23:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:20 smithi069 ceph-mon[35081]: pgmap v545: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 207 KiB/s wr, 27 op/s 2024-05-17T09:23:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:20 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:23:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:20 smithi079 ceph-mon[35502]: pgmap v545: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 207 KiB/s wr, 27 op/s 2024-05-17T09:23:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:20 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:23:21.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28580 2.19 MB/sec warmup 79 sec latency 15083.130 ms 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:23:21.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 -- 172.21.15.19:0/2327790329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde841034b0 msgr2=0x7fde841038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2327790329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde841034b0 0x7fde841038c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde800099c0 tx=0x7fde80009cd0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 -- 172.21.15.19:0/2327790329 shutdown_connections 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2327790329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde841034b0 0x7fde841038c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2327790329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde84108730 0x7fde84110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2327790329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde841081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 -- 172.21.15.19:0/2327790329 >> 172.21.15.19:0/2327790329 conn(0x7fde840fedb0 msgr2=0x7fde841011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 -- 172.21.15.19:0/2327790329 shutdown_connections 2024-05-17T09:23:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.675+0000 7fde8c36b700 1 -- 172.21.15.19:0/2327790329 wait complete. 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 Processor -- start 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 -- start start 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841034b0 0x7fde8419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde8419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde84108730 0x7fde8419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde84113260 con 0x7fde84107de0 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde841130e0 con 0x7fde841034b0 2024-05-17T09:23:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.676+0000 7fde8c36b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde841133e0 con 0x7fde84108730 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde8419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde8419c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55004/0 (socket says 172.21.15.19:55004) 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 -- 172.21.15.19:0/2223325804 learned_addr learned my addr 172.21.15.19:0/2223325804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde8a107700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841034b0 0x7fde8419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde8a908700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde84108730 0x7fde8419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 -- 172.21.15.19:0/2223325804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde84108730 msgr2=0x7fde8419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde84108730 0x7fde8419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 -- 172.21.15.19:0/2223325804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841034b0 msgr2=0x7fde8419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841034b0 0x7fde8419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 -- 172.21.15.19:0/2223325804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde841a6ef0 con 0x7fde84107de0 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde89906700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde8419c0e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ea30 tx=0x7fde7400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:21.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7400cbd0 con 0x7fde84107de0 2024-05-17T09:23:21.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde841a71e0 con 0x7fde84107de0 2024-05-17T09:23:21.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde740044b0 con 0x7fde84107de0 2024-05-17T09:23:21.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.677+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74003cf0 con 0x7fde84107de0 2024-05-17T09:23:21.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.678+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde841a7770 con 0x7fde84107de0 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:23:21.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:23:21.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.679+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde840512b0 con 0x7fde84107de0 2024-05-17T09:23:21.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.680+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fde74021440 con 0x7fde84107de0 2024-05-17T09:23:21.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.681+0000 7fde7b7fe700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde700b2880 0x7fde700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:21.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.681+0000 7fde8a107700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde700b2880 0x7fde700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:21.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.681+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde740e2720 con 0x7fde84107de0 2024-05-17T09:23:21.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.682+0000 7fde8a107700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde700b2880 0x7fde700b4d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde80000c00 tx=0x7fde8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.686+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde740aa030 con 0x7fde84107de0 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:23:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:23:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.954+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde8406bbf0 con 0x7fde84107de0 2024-05-17T09:23:21.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.954+0000 7fde7b7fe700 1 -- 172.21.15.19:0/2223325804 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7fde740a9930 con 0x7fde84107de0 2024-05-17T09:23:21.955 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:23:21.955 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:22:41.259768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde700b2880 msgr2=0x7fde700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde700b2880 0x7fde700b4d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde80000c00 tx=0x7fde8001a040 comp rx=0 tx=0).stop 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 msgr2=0x7fde8419c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde8419c0e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ea30 tx=0x7fde7400ed40 comp rx=0 tx=0).stop 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 shutdown_connections 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde700b2880 0x7fde700b4d90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde84108730 0x7fde8419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde841034b0 0x7fde8419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.956+0000 7fde8c36b700 1 --2- 172.21.15.19:0/2223325804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde84107de0 0x7fde8419c0e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.957+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 >> 172.21.15.19:0/2223325804 conn(0x7fde840fedb0 msgr2=0x7fde84101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.957+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 shutdown_connections 2024-05-17T09:23:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:21.957+0000 7fde8c36b700 1 -- 172.21.15.19:0/2223325804 wait complete. 2024-05-17T09:23:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-17T09:23:22.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 -- 172.21.15.19:0/3095135191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 msgr2=0x7fd4fc10e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 --2- 172.21.15.19:0/3095135191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc10e360 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009990 tx=0x7fd4e4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 -- 172.21.15.19:0/3095135191 shutdown_connections 2024-05-17T09:23:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 --2- 172.21.15.19:0/3095135191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4fc10e8f0 0x7fd4fc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 --2- 172.21.15.19:0/3095135191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc1054c0 0x7fd4fc1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 --2- 172.21.15.19:0/3095135191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc10e360 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 -- 172.21.15.19:0/3095135191 >> 172.21.15.19:0/3095135191 conn(0x7fd4fc0fef10 msgr2=0x7fd4fc101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 -- 172.21.15.19:0/3095135191 shutdown_connections 2024-05-17T09:23:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.175+0000 7fd5022b4700 1 -- 172.21.15.19:0/3095135191 wait complete. 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 Processor -- start 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 -- start start 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4fc1054c0 0x7fd4fc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc10e8f0 0x7fd4fc19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc114300 con 0x7fd4fc105e10 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc114180 con 0x7fd4fc10e8f0 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd5022b4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc114480 con 0x7fd4fc1054c0 2024-05-17T09:23:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd4f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd4f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55032/0 (socket says 172.21.15.19:55032) 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1978008888 learned_addr learned my addr 172.21.15.19:0/1978008888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4fc1054c0 0x7fd4fc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.176+0000 7fd500851700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc10e8f0 0x7fd4fc19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1978008888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4fc1054c0 msgr2=0x7fd4fc19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4fc1054c0 0x7fd4fc19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1978008888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc10e8f0 msgr2=0x7fd4fc19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc10e8f0 0x7fd4fc19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1978008888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc19fbe0 con 0x7fd4fc105e10 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd500851700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc10e8f0 0x7fd4fc19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc19b2b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4006010 tx=0x7fd4e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f9ffb700 1 -- 172.21.15.19:0/1978008888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4027070 con 0x7fd4fc105e10 2024-05-17T09:23:22.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc19fe70 con 0x7fd4fc105e10 2024-05-17T09:23:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f9ffb700 1 -- 172.21.15.19:0/1978008888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4e400fb90 con 0x7fd4fc105e10 2024-05-17T09:23:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd4f9ffb700 1 -- 172.21.15.19:0/1978008888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40173f0 con 0x7fd4fc105e10 2024-05-17T09:23:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.177+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc1a77f0 con 0x7fd4fc105e10 2024-05-17T09:23:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.180+0000 7fd4f9ffb700 1 -- 172.21.15.19:0/1978008888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd4e402a070 con 0x7fd4fc105e10 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.180+0000 7fd4f9ffb700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4e80b2880 0x7fd4e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4e80b2880 0x7fd4e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd4f9ffb700 1 -- 172.21.15.19:0/1978008888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd4e40e3570 con 0x7fd4fc105e10 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 Processor -- start 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 -- start start 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4e80b2880 0x7fd4e80b4d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009c60 tx=0x7fd4ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb8ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df3cab6e70 con 0x55df3ca09f40 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df3caa6a10 con 0x55df3caf5530 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df3caaaa50 con 0x55df3cb89020 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.181+0000 7fd4f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb8f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55042/0 (socket says 172.21.15.19:55042) 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1412080774 learned_addr learned my addr 172.21.15.19:0/1412080774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb8ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd500851700 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1412080774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 msgr2=0x55df3cb894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb894f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1412080774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 msgr2=0x55df3cb8ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb8ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1412080774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3ca500b0 con 0x55df3ca09f40 2024-05-17T09:23:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb8f430 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40a7a50 tx=0x7fd4e4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df20e6a3d58c0cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f27fc700 1 -- 172.21.15.19:0/1412080774 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40aaee0 con 0x55df3ca09f40 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f27fc700 1 -- 172.21.15.19:0/1412080774 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4e40e2070 con 0x55df3ca09f40 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.182+0000 7fd4f27fc700 1 -- 172.21.15.19:0/1412080774 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400fdd0 con 0x55df3ca09f40 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1412080774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 msgr2=0x55df3cb8f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb8f430 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40a7a50 tx=0x7fd4e4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1412080774 shutdown_connections 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb8ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1412080774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb8f430 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1412080774 >> 172.21.15.19:0/1412080774 conn(0x55df3ca6a2c0 msgr2=0x55df3cb83cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1412080774 shutdown_connections 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.183+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1412080774 wait complete. 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 -- start start 2024-05-17T09:23:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 Processor -- start 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb98b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb92a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df3c9ad0a0 con 0x55df3ca09f40 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df3ca0eea0 con 0x55df3caf5530 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.184+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df3cb89d40 con 0x55df3cb89020 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd510cc9b80 10 client.0 _ll_register_callbacks cb 0x55df3ca41a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb98610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55056/0 (socket says 172.21.15.19:55056) 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 -- 172.21.15.19:0/4159387926 learned_addr learned my addr 172.21.15.19:0/4159387926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd500851700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb92a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb98b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 -- 172.21.15.19:0/4159387926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 msgr2=0x55df3cb92a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb92a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 -- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 msgr2=0x55df3cb98b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb98b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 -- 172.21.15.19:0/4159387926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3c9f69d0 con 0x55df3ca09f40 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb98b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb98610 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec003900 tx=0x7fd4ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558fa0183a722ef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4db7fe700 1 -- 172.21.15.19:0/4159387926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec01d820 con 0x55df3ca09f40 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd4db7fe700 1 -- 172.21.15.19:0/4159387926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd4ec01d9c0 con 0x55df3ca09f40 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.185+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e40e2070 con 0x55df3ca09f40 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd4db7fe700 1 -- 172.21.15.19:0/4159387926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec01dbc0 con 0x55df3ca09f40 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 10 client.16455 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df3ca60860 con 0x55df3ca09f40 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd4db7fe700 1 -- 172.21.15.19:0/4159387926 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4e40e2070 con 0x55df3ca09f40 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd4db7fe700 1 -- 172.21.15.19:0/4159387926 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd4ec01c410 con 0x55df3ca09f40 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 10 client.16455 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 20 client.16455 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cba27a0 0x55df3cba4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 4 client.16455 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3cb54fd0 con 0x55df3cba27a0 2024-05-17T09:23:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.186+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cba27a0 0x55df3cba4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.187+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cba27a0 0x55df3cba4c50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2944ec9832fed991 server_cookie=d5c447003b153368 in_seq=0 out_seq=0 2024-05-17T09:23:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.187+0000 7fd4db7fe700 10 client.16455 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.187+0000 7fd4db7fe700 1 -- 172.21.15.19:0/4159387926 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55df3cb54fd0 con 0x55df3cba27a0 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.187+0000 7fd4db7fe700 10 client.16455 handle_command_reply: tid=0 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 client.16455 shutdown 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 msgr2=0x55df3cb98610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb98610 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec003900 tx=0x7fd4ec0047c0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 shutdown_connections 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb92a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cba27a0 0x55df3cba4c50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb98610 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/4159387926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3caf5530 0x55df3cb98b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 >> 172.21.15.19:0/4159387926 conn(0x55df3ca6a2c0 msgr2=0x55df3cb88010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd4db7fe700 0 client.16455 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 shutdown_connections 2024-05-17T09:23:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 1 -- 172.21.15.19:0/4159387926 wait complete. 2024-05-17T09:23:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.188+0000 7fd510cc9b80 20 client.16455 trim_cache size 0 max 16384 2024-05-17T09:23:22.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.208+0000 7fd510cc9b80 1 Processor -- start 2024-05-17T09:23:22.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.208+0000 7fd510cc9b80 1 -- start start 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb92f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3caf5530 0x55df3cbaa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3cb89020 0x55df3cbaa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1b2b60 con 0x55df3ca09f40 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1b03f0 con 0x55df3cb89020 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1ae240 con 0x55df3caf5530 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb92f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb92f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55064/0 (socket says 172.21.15.19:55064) 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 -- 172.21.15.19:0/3538414133 learned_addr learned my addr 172.21.15.19:0/3538414133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3caf5530 0x55df3cbaa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 -- 172.21.15.19:0/3538414133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3caf5530 msgr2=0x55df3cbaa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3caf5530 0x55df3cbaa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 -- 172.21.15.19:0/3538414133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3cb89020 msgr2=0x55df3cbaa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3cb89020 0x55df3cbaa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 -- 172.21.15.19:0/3538414133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3cb8ea20 con 0x55df3ca09f40 2024-05-17T09:23:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb92f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009fd0 tx=0x7fd4ec009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ebe1fb12d51c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4f1ffb700 1 -- 172.21.15.19:0/3538414133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec01cac0 con 0x55df3ca09f40 2024-05-17T09:23:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4f1ffb700 1 -- 172.21.15.19:0/3538414133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4ec01cc60 con 0x55df3ca09f40 2024-05-17T09:23:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.209+0000 7fd4f1ffb700 1 -- 172.21.15.19:0/3538414133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0359d0 con 0x55df3ca09f40 2024-05-17T09:23:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 -- 172.21.15.19:0/3538414133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 msgr2=0x55df3cb92f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb92f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009fd0 tx=0x7fd4ec009390 comp rx=0 tx=0).stop 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 -- 172.21.15.19:0/3538414133 shutdown_connections 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3cb89020 0x55df3cbaa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3ca09f40 0x55df3cb92f00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/3538414133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3caf5530 0x55df3cbaa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 -- 172.21.15.19:0/3538414133 >> 172.21.15.19:0/3538414133 conn(0x55df3ca6a2c0 msgr2=0x55df3cb927f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 -- 172.21.15.19:0/3538414133 shutdown_connections 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.210+0000 7fd510cc9b80 1 -- 172.21.15.19:0/3538414133 wait complete. 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 -- start start 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 Processor -- start 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3ca09f40 0x55df3cb9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 0x55df3cb9d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb9dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1ae8b0 con 0x55df3caf5530 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1afd20 con 0x55df3ca09f40 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1aef80 con 0x55df3cb89020 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd510cc9b80 10 client.0 _ll_register_callbacks cb 0x55df3ca41a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd4f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 0x55df3cb9d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.211+0000 7fd4f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 0x55df3cb9d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55072/0 (socket says 172.21.15.19:55072) 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1055042023 learned_addr learned my addr 172.21.15.19:0/1055042023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3ca09f40 0x55df3cb9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd500851700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb9dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1055042023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 msgr2=0x55df3cb9dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb9dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3ca09f40 msgr2=0x55df3cb9f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3ca09f40 0x55df3cb9f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 -- 172.21.15.19:0/1055042023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3cb85870 con 0x55df3caf5530 2024-05-17T09:23:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4f3fff700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 0x55df3cb9d510 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ffd0 tx=0x7fd4e40aa7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595484c812652fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4db7fe700 1 -- 172.21.15.19:0/1055042023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40a9f50 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4db7fe700 1 -- 172.21.15.19:0/1055042023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd4e400fd40 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4ec01cc60 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd4db7fe700 1 -- 172.21.15.19:0/1055042023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4e40b3640 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd510cc9b80 10 client.16461 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.212+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df3cc0e1e0 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd4db7fe700 1 -- 172.21.15.19:0/1055042023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4ec01cc60 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd4db7fe700 1 -- 172.21.15.19:0/1055042023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd4e4005110 con 0x55df3caf5530 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd510cc9b80 10 client.16461 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd510cc9b80 10 client.16461 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd510cc9b80 20 client.16461 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cc0e830 0x55df3cc10c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd510cc9b80 4 client.16461 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "34bbc80e", "release": true, "await": true}] 2024-05-17T09:23:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "34bbc80e", "release": true, "await": true}) -- 0x55df3cab8dc0 con 0x55df3cc0e830 2024-05-17T09:23:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.213+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cc0e830 0x55df3cc10c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.214+0000 7fd4db7fe700 10 client.16461 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.214+0000 7fd4fbfff700 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cc0e830 0x55df3cc10c50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee5440cc019bab2 server_cookie=98c5a3614bc010e in_seq=0 out_seq=0 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd4db7fe700 1 -- 172.21.15.19:0/1055042023 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55df3cab8dc0 con 0x55df3cc0e830 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd4db7fe700 10 client.16461 handle_command_reply: tid=0 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 client.16461 shutdown 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 msgr2=0x55df3cb9d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 0x55df3cb9d510 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ffd0 tx=0x7fd4e40aa7e0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 shutdown_connections 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df3cb89020 0x55df3cb9dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df3cc0e830 0x55df3cc10c50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df3ca09f40 0x55df3cb9f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 --2- 172.21.15.19:0/1055042023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df3caf5530 0x55df3cb9d510 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 >> 172.21.15.19:0/1055042023 conn(0x55df3ca6a2c0 msgr2=0x55df3cb90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd4db7fe700 0 client.16461 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 shutdown_connections 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.217+0000 7fd510cc9b80 1 -- 172.21.15.19:0/1055042023 wait complete. 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd510cc9b80 20 client.16461 trim_cache size 0 max 16384 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:23:22.219 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 12, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "34bbc80e": { 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 12, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 132.3, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:23:22.220 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:23:22.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4e80b2880 msgr2=0x7fd4e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4e80b2880 0x7fd4e80b4d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009c60 tx=0x7fd4ec009260 comp rx=0 tx=0).stop 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 msgr2=0x7fd4fc19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc19b2b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4006010 tx=0x7fd4e40057c0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 shutdown_connections 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd4e80b2880 0x7fd4e80b4d90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4fc1054c0 0x7fd4fc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4fc10e8f0 0x7fd4fc19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 --2- 172.21.15.19:0/1978008888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4fc105e10 0x7fd4fc19b2b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 >> 172.21.15.19:0/1978008888 conn(0x7fd4fc0fef10 msgr2=0x7fd4fc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 shutdown_connections 2024-05-17T09:23:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:22.218+0000 7fd5022b4700 1 -- 172.21.15.19:0/1978008888 wait complete. 2024-05-17T09:23:22.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28653 2.17 MB/sec warmup 80 sec latency 15884.726 ms 2024-05-17T09:23:22.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:22 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:23:22.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:22 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:23:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:22 smithi019 ceph-mon[31620]: pgmap v546: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-05-17T09:23:22.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:22 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2223325804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:22 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:23:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:22 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:23:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:22 smithi069 ceph-mon[35081]: pgmap v546: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-05-17T09:23:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:22 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2223325804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:22 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:23:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:22 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:23:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:22 smithi079 ceph-mon[35502]: pgmap v546: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-05-17T09:23:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:22 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2223325804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:22.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-17T09:23:23.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 28886 2.15 MB/sec warmup 81 sec latency 16.140 ms 2024-05-17T09:23:23.624 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '34bbc80e', total seconds elapsed: 22.9 2024-05-17T09:23:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:24.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 29307 2.13 MB/sec warmup 82 sec latency 13.650 ms 2024-05-17T09:23:24.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:24 smithi019 ceph-mon[31620]: pgmap v547: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:23:24.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:24 smithi069 ceph-mon[35081]: pgmap v547: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:23:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:24 smithi079 ceph-mon[35502]: pgmap v547: 129 pgs: 129 active+clean; 28 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:23:25.420 INFO:tasks.workunit.client.0.smithi019.stdout: 1 29841 2.16 MB/sec warmup 83 sec latency 17.753 ms 2024-05-17T09:23:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 -- 172.21.15.19:0/23237688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd01043d0 msgr2=0x7efcd01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 --2- 172.21.15.19:0/23237688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd01043d0 0x7efcd01067f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009900 tx=0x7efcc402a410 comp rx=0 tx=0).stop 2024-05-17T09:23:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 -- 172.21.15.19:0/23237688 shutdown_connections 2024-05-17T09:23:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 --2- 172.21.15.19:0/23237688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 0x7efcd010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 --2- 172.21.15.19:0/23237688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106dc0 0x7efcd0109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 --2- 172.21.15.19:0/23237688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd01043d0 0x7efcd01067f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 -- 172.21.15.19:0/23237688 >> 172.21.15.19:0/23237688 conn(0x7efcd00fdd30 msgr2=0x7efcd01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 -- 172.21.15.19:0/23237688 shutdown_connections 2024-05-17T09:23:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.302+0000 7efcd7184640 1 -- 172.21.15.19:0/23237688 wait complete. 2024-05-17T09:23:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.303+0000 7efcd7184640 1 Processor -- start 2024-05-17T09:23:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.303+0000 7efcd7184640 1 -- start start 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd7184640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 0x7efcd019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd7184640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 0x7efcd019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd7184640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 0x7efcd01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd7184640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010f5f0 con 0x7efcd01098d0 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd7184640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010f470 con 0x7efcd01043d0 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd7184640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010f770 con 0x7efcd0106dc0 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd5721640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 0x7efcd01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 0x7efcd019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efccffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 0x7efcd019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 0x7efcd019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41792/0 (socket says 172.21.15.19:41792) 2024-05-17T09:23:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd5721640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 0x7efcd01a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52136/0 (socket says 172.21.15.19:52136) 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efccffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 0x7efcd019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42678/0 (socket says 172.21.15.19:42678) 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 -- 172.21.15.19:0/3246852571 learned_addr learned my addr 172.21.15.19:0/3246852571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 -- 172.21.15.19:0/3246852571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 msgr2=0x7efcd019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 0x7efcd019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 -- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 msgr2=0x7efcd01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 0x7efcd01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efcd4f20640 1 -- 172.21.15.19:0/3246852571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd01a1d30 con 0x7efcd01043d0 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.304+0000 7efccffff640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 0x7efcd019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.305+0000 7efcd4f20640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 0x7efcd019a1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcc4031460 tx=0x7efcc4005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.305+0000 7efccdffb640 1 -- 172.21.15.19:0/3246852571 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc403f030 con 0x7efcd01043d0 2024-05-17T09:23:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.305+0000 7efccdffb640 1 -- 172.21.15.19:0/3246852571 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc40043d0 con 0x7efcd01043d0 2024-05-17T09:23:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.305+0000 7efccdffb640 1 -- 172.21.15.19:0/3246852571 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4035070 con 0x7efcd01043d0 2024-05-17T09:23:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.305+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd01a1fc0 con 0x7efcd01043d0 2024-05-17T09:23:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.305+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd01a23c0 con 0x7efcd01043d0 2024-05-17T09:23:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 Processor -- start 2024-05-17T09:23:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 -- start start 2024-05-17T09:23:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efccdffb640 1 -- 172.21.15.19:0/3246852571 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efcc4042030 con 0x7efcd01043d0 2024-05-17T09:23:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 0x55fccea023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6382f0 0x55fccea02920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea09820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fcce974c30 con 0x55fcce6404d0 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcce9647d0 con 0x55fcce6611b0 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcde039740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fcce968810 con 0x55fcce6382f0 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcd5721640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 0x55fccea023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcd4f20640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6382f0 0x55fccea02920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcd5721640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 0x55fccea023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41806/0 (socket says 172.21.15.19:41806) 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efccffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea09820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.308+0000 7efcd5721640 1 -- 172.21.15.19:0/1882567334 learned_addr learned my addr 172.21.15.19:0/1882567334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd5721640 1 -- 172.21.15.19:0/1882567334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6382f0 msgr2=0x55fccea02920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd5721640 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6382f0 0x55fccea02920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd5721640 1 -- 172.21.15.19:0/1882567334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 msgr2=0x55fccea09820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd5721640 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea09820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd5721640 1 -- 172.21.15.19:0/1882567334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce679210 con 0x55fcce6611b0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd5721640 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 0x55fccea023a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcc8011440 tx=0x7efcc8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef302e30cba0422 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efccdffb640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efcc00b28a0 0x7efcc00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcbaffd640 1 -- 172.21.15.19:0/1882567334 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8003c30 con 0x55fcce6611b0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcbaffd640 1 -- 172.21.15.19:0/1882567334 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc8003dd0 con 0x55fcce6611b0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcbaffd640 1 -- 172.21.15.19:0/1882567334 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc800f980 con 0x55fcce6611b0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efcd4f20640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efcc00b28a0 0x7efcc00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.309+0000 7efccdffb640 1 -- 172.21.15.19:0/3246852571 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efcc4102910 con 0x7efcd01043d0 2024-05-17T09:23:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 -- 172.21.15.19:0/1882567334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 msgr2=0x55fccea023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcd4f20640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efcc00b28a0 0x7efcc00b4db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efcc40c6df0 tx=0x7efcc4108550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 0x55fccea023a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcc8011440 tx=0x7efcc8011900 comp rx=0 tx=0).stop 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 -- 172.21.15.19:0/1882567334 shutdown_connections 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea09820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6382f0 0x55fccea02920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 --2- 172.21.15.19:0/1882567334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6611b0 0x55fccea023a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.310+0000 7efcde039740 1 -- 172.21.15.19:0/1882567334 >> 172.21.15.19:0/1882567334 conn(0x55fcce655300 msgr2=0x55fcce9fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.311+0000 7efcde039740 1 -- 172.21.15.19:0/1882567334 shutdown_connections 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.311+0000 7efcde039740 1 -- 172.21.15.19:0/1882567334 wait complete. 2024-05-17T09:23:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.311+0000 7efcde039740 1 -- start start 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.311+0000 7efcde039740 1 Processor -- start 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6611b0 0x55fccea17600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fcce64a790 con 0x55fcce6404d0 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcce920890 con 0x55fcce6382f0 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fccea0e620 con 0x55fcce6611b0 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcd5721640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea10710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52152/0 (socket says 172.21.15.19:52152) 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcd5721640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41822/0 (socket says 172.21.15.19:41822) 2024-05-17T09:23:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 -- 172.21.15.19:0/3113636749 learned_addr learned my addr 172.21.15.19:0/3113636749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcde039740 10 client.0 _ll_register_callbacks cb 0x55fcce923d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcd5721640 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6611b0 0x55fccea17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 -- 172.21.15.19:0/3113636749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6611b0 msgr2=0x55fccea17600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6611b0 0x55fccea17600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 -- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 msgr2=0x55fccea101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 -- 172.21.15.19:0/3113636749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce745620 con 0x55fcce6404d0 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efccffff640 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea10710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efcbc012d20 tx=0x7efcbc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42ce79a0f818685 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.312+0000 7efcaffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.402+0000 7efcaffff640 1 -- 172.21.15.19:0/3113636749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc012110 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.402+0000 7efcaffff640 1 -- 172.21.15.19:0/3113636749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efcbc004050 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.402+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc8003dd0 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.402+0000 7efcaffff640 1 -- 172.21.15.19:0/3113636749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcbc025050 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.402+0000 7efcde039740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.402+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fccea1a5c0 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcaffff640 1 -- 172.21.15.19:0/3113636749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc8003dd0 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcaffff640 1 -- 172.21.15.19:0/3113636749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efcbc010430 con 0x55fcce6404d0 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcde039740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcde039740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcde039740 20 client.16473 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcde039740 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce61fd80 0x55fccea1de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcde039740 4 client.16473 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcce976b80 con 0x55fcce61fd80 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.403+0000 7efcd5721640 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce61fd80 0x55fccea1de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.404+0000 7efcaffff640 10 client.16473 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.404+0000 7efcd5721640 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce61fd80 0x55fccea1de10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca13d4808fbe77 server_cookie=98c18d64334fb4c9 in_seq=0 out_seq=0 2024-05-17T09:23:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcaffff640 1 -- 172.21.15.19:0/3113636749 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fcce976b80 con 0x55fcce61fd80 2024-05-17T09:23:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcaffff640 10 client.16473 handle_command_reply: tid=0 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 client.16473 shutdown 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 msgr2=0x55fccea10710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea10710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efcbc012d20 tx=0x7efcbc00f790 comp rx=0 tx=0).stop 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 shutdown_connections 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce61fd80 0x55fccea1de10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce6611b0 0x55fccea17600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea10710 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcaffff640 0 client.16473 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 --2- 172.21.15.19:0/3113636749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 >> 172.21.15.19:0/3113636749 conn(0x55fcce655300 msgr2=0x55fcce9fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 shutdown_connections 2024-05-17T09:23:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 1 -- 172.21.15.19:0/3113636749 wait complete. 2024-05-17T09:23:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.405+0000 7efcde039740 20 client.16473 trim_cache size 0 max 16384 2024-05-17T09:23:26.421 INFO:tasks.workunit.client.0.smithi019.stdout: 1 30326 2.14 MB/sec warmup 84 sec latency 93.488 ms 2024-05-17T09:23:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.431+0000 7efcde039740 1 Processor -- start 2024-05-17T09:23:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 -- start start 2024-05-17T09:23:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea06a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 0x55fccea17520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6404d0 0x55fccea17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01ad080 con 0x55fcce6382f0 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01aa950 con 0x55fcce6404d0 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcde039740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a87a0 con 0x55fcce61fd80 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 0x55fccea17520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 0x55fccea17520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52164/0 (socket says 172.21.15.19:52164) 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 -- 172.21.15.19:0/2991034271 learned_addr learned my addr 172.21.15.19:0/2991034271 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcd5721640 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea06a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcd5721640 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6404d0 0x55fccea17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 -- 172.21.15.19:0/2991034271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 msgr2=0x55fccea06a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea06a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 -- 172.21.15.19:0/2991034271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6404d0 msgr2=0x55fccea17a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6404d0 0x55fccea17a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efccffff640 1 -- 172.21.15.19:0/2991034271 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccea057c0 con 0x55fcce6382f0 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.432+0000 7efcd5721640 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6404d0 0x55fccea17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.433+0000 7efccffff640 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 0x55fccea17520 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efcbc010850 tx=0x7efcbc0122a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d78f82aebf61d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.433+0000 7efcba7fc640 1 -- 172.21.15.19:0/2991034271 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc02e070 con 0x55fcce6382f0 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.433+0000 7efcba7fc640 1 -- 172.21.15.19:0/2991034271 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcbc005540 con 0x55fcce6382f0 2024-05-17T09:23:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.433+0000 7efcba7fc640 1 -- 172.21.15.19:0/2991034271 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc003850 con 0x55fcce6382f0 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 -- 172.21.15.19:0/2991034271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 msgr2=0x55fccea17520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 0x55fccea17520 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efcbc010850 tx=0x7efcbc0122a0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 -- 172.21.15.19:0/2991034271 shutdown_connections 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6404d0 0x55fccea17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6382f0 0x55fccea17520 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 --2- 172.21.15.19:0/2991034271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea06a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 -- 172.21.15.19:0/2991034271 >> 172.21.15.19:0/2991034271 conn(0x55fcce655300 msgr2=0x55fccea01560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 -- 172.21.15.19:0/2991034271 shutdown_connections 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 -- 172.21.15.19:0/2991034271 wait complete. 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 -- start start 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.434+0000 7efcde039740 1 Processor -- start 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea15d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea16be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a8e10 con 0x55fcce6404d0 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01aa280 con 0x55fcce6382f0 2024-05-17T09:23:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a94e0 con 0x55fcce61fd80 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcde039740 10 client.0 _ll_register_callbacks cb 0x7efcbc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcd5721640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcd5721640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea16be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea15d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea15d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41842/0 (socket says 172.21.15.19:41842) 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efcd5721640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea16be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52168/0 (socket says 172.21.15.19:52168) 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 -- 172.21.15.19:0/2077668485 learned_addr learned my addr 172.21.15.19:0/2077668485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 -- 172.21.15.19:0/2077668485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 msgr2=0x55fccea157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 -- 172.21.15.19:0/2077668485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 msgr2=0x55fccea16be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea16be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.435+0000 7efccffff640 1 -- 172.21.15.19:0/2077668485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccea171a0 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efccffff640 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea15d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00f7d0 tx=0x7efcbc010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634ff74e20ac14a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcaffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcaffff640 1 -- 172.21.15.19:0/2077668485 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc02e040 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcbc005540 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcaffff640 1 -- 172.21.15.19:0/2077668485 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efcbc005040 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcaffff640 1 -- 172.21.15.19:0/2077668485 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcbc03d050 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 10 client.26150 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fccea865c0 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcaffff640 1 -- 172.21.15.19:0/2077668485 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcbc005540 con 0x55fcce6382f0 2024-05-17T09:23:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcaffff640 1 -- 172.21.15.19:0/2077668485 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efcbc003b10 con 0x55fcce6382f0 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 10 client.26150 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 20 client.26150 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce6611b0 0x55fccea89060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 4 client.26150 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.436+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcce970360 con 0x55fcce6611b0 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.437+0000 7efcd5721640 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce6611b0 0x55fccea89060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.437+0000 7efcaffff640 10 client.26150 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.437+0000 7efcd5721640 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce6611b0 0x55fccea89060 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3684454df1b3adc4 server_cookie=cfd71a2fa69745e9 in_seq=0 out_seq=0 2024-05-17T09:23:26.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:26 smithi019 ceph-mon[31620]: pgmap v548: 129 pgs: 129 active+clean; 28 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcaffff640 1 -- 172.21.15.19:0/2077668485 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fcce970360 con 0x55fcce6611b0 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcaffff640 10 client.26150 handle_command_reply: tid=0 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 client.26150 shutdown 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 msgr2=0x55fccea15d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea15d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00f7d0 tx=0x7efcbc010660 comp rx=0 tx=0).stop 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 shutdown_connections 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcce6611b0 0x55fccea89060 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcce6404d0 0x55fccea16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcce6382f0 0x55fccea15d00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcaffff640 0 client.26150 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 --2- 172.21.15.19:0/2077668485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcce61fd80 0x55fccea157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 >> 172.21.15.19:0/2077668485 conn(0x55fcce655300 msgr2=0x55fccea012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 shutdown_connections 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 1 -- 172.21.15.19:0/2077668485 wait complete. 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.454+0000 7efcde039740 20 client.26150 trim_cache size 0 max 16384 2024-05-17T09:23:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:23:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:23:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:23:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efcc00b28a0 msgr2=0x7efcc00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efcc00b28a0 0x7efcc00b4db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efcc40c6df0 tx=0x7efcc4108550 comp rx=0 tx=0).stop 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 msgr2=0x7efcd019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 0x7efcd019a1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcc4031460 tx=0x7efcc4005e70 comp rx=0 tx=0).stop 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 shutdown_connections 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efcc00b28a0 0x7efcc00b4db0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd01098d0 0x7efcd01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efcd0106dc0 0x7efcd019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 --2- 172.21.15.19:0/3246852571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd01043d0 0x7efcd019a1f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 >> 172.21.15.19:0/3246852571 conn(0x7efcd00fdd30 msgr2=0x7efcd0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 shutdown_connections 2024-05-17T09:23:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:26.455+0000 7efcd7184640 1 -- 172.21.15.19:0/3246852571 wait complete. 2024-05-17T09:23:26.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:26 smithi069 ceph-mon[35081]: pgmap v548: 129 pgs: 129 active+clean; 28 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-05-17T09:23:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:26 smithi079 ceph-mon[35502]: pgmap v548: 129 pgs: 129 active+clean; 28 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-05-17T09:23:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 -- 172.21.15.19:0/109827903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c1045e0 msgr2=0x7f573c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 --2- 172.21.15.19:0/109827903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c1045e0 0x7f573c1049f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f57300099c0 tx=0x7f5730009cd0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 -- 172.21.15.19:0/109827903 shutdown_connections 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 --2- 172.21.15.19:0/109827903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c109860 0x7f573c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 --2- 172.21.15.19:0/109827903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c1045e0 0x7f573c1049f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 --2- 172.21.15.19:0/109827903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 0x7f573c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 -- 172.21.15.19:0/109827903 >> 172.21.15.19:0/109827903 conn(0x7f573c100030 msgr2=0x7f573c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 -- 172.21.15.19:0/109827903 shutdown_connections 2024-05-17T09:23:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.967+0000 7f5742cac700 1 -- 172.21.15.19:0/109827903 wait complete. 2024-05-17T09:23:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 Processor -- start 2024-05-17T09:23:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 -- start start 2024-05-17T09:23:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 0x7f573c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 0x7f573c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 0x7f573c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f573c114360 con 0x7f573c109860 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1141e0 con 0x7f573c108f10 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.968+0000 7f5742cac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1144e0 con 0x7f573c1045e0 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 0x7f573c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5740a48700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 0x7f573c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5740a48700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 0x7f573c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42748/0 (socket says 172.21.15.19:42748) 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 0x7f573c19f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52174/0 (socket says 172.21.15.19:52174) 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5740a48700 1 -- 172.21.15.19:0/3266557173 learned_addr learned my addr 172.21.15.19:0/3266557173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f573bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 0x7f573c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 -- 172.21.15.19:0/3266557173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 msgr2=0x7f573c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 0x7f573c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 -- 172.21.15.19:0/3266557173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 msgr2=0x7f573c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 0x7f573c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 -- 172.21.15.19:0/3266557173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c19fc30 con 0x7f573c109860 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5740a48700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 0x7f573c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f573bfff700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 0x7f573c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5741249700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 0x7f573c19f590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f573400efc0 tx=0x7f573400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5739ffb700 1 -- 172.21.15.19:0/3266557173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340092e0 con 0x7f573c109860 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5739ffb700 1 -- 172.21.15.19:0/3266557173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5734004bb0 con 0x7f573c109860 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c19fec0 con 0x7f573c109860 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.969+0000 7f5739ffb700 1 -- 172.21.15.19:0/3266557173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734004020 con 0x7f573c109860 2024-05-17T09:23:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.970+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c1a7840 con 0x7f573c109860 2024-05-17T09:23:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.973+0000 7f5739ffb700 1 -- 172.21.15.19:0/3266557173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f573401b030 con 0x7f573c109860 2024-05-17T09:23:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.973+0000 7f5739ffb700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f572c0b2900 0x7f572c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f5740a48700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f572c0b2900 0x7f572c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f5739ffb700 1 -- 172.21.15.19:0/3266557173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f57340e1f40 con 0x7f573c109860 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 Processor -- start 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 -- start start 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f5740a48700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f572c0b2900 0x7f572c0b4e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5730009990 tx=0x7f573001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 0x557d75e67910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e67eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75e61cd0 0x557d75e621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d75d8fd70 con 0x557d75dce430 2024-05-17T09:23:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557d75d7f910 con 0x557d75e61cd0 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.974+0000 7f57516c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557d75d83950 con 0x557d75ce2e40 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 0x557d75e67910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 0x557d75e67910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52176/0 (socket says 172.21.15.19:52176) 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 -- 172.21.15.19:0/1973966818 learned_addr learned my addr 172.21.15.19:0/1973966818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5741249700 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75e61cd0 0x557d75e621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f573bfff700 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 -- 172.21.15.19:0/1973966818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 msgr2=0x557d75e67eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e67eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 -- 172.21.15.19:0/1973966818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75e61cd0 msgr2=0x557d75e621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75e61cd0 0x557d75e621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 -- 172.21.15.19:0/1973966818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d75d28fb0 con 0x557d75dce430 2024-05-17T09:23:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f573bfff700 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e67eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5740a48700 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 0x557d75e67910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5730005910 tx=0x7f5730022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595b3fe08611baa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5726ffd700 1 -- 172.21.15.19:0/1973966818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300037c0 con 0x557d75dce430 2024-05-17T09:23:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.975+0000 7f5726ffd700 1 -- 172.21.15.19:0/1973966818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f573000b790 con 0x557d75dce430 2024-05-17T09:23:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.976+0000 7f5726ffd700 1 -- 172.21.15.19:0/1973966818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730030c60 con 0x557d75dce430 2024-05-17T09:23:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.976+0000 7f57516c1b80 1 -- 172.21.15.19:0/1973966818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 msgr2=0x557d75e67910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.976+0000 7f57516c1b80 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 0x557d75e67910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5730005910 tx=0x7f5730022000 comp rx=0 tx=0).stop 2024-05-17T09:23:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 -- 172.21.15.19:0/1973966818 shutdown_connections 2024-05-17T09:23:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75e61cd0 0x557d75e621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75dce430 0x557d75e67910 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 --2- 172.21.15.19:0/1973966818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e67eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 -- 172.21.15.19:0/1973966818 >> 172.21.15.19:0/1973966818 conn(0x557d75d431c0 msgr2=0x557d75e5c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 -- 172.21.15.19:0/1973966818 shutdown_connections 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 -- 172.21.15.19:0/1973966818 wait complete. 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 -- start start 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.977+0000 7f57516c1b80 1 Processor -- start 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75ce2e40 0x557d75e6b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e6bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75e61cd0 0x557d75e6c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d75c85fa0 con 0x557d75ce2e40 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557d75ce7da0 con 0x557d75dce430 2024-05-17T09:23:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557d75e6e5a0 con 0x557d75e61cd0 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f57516c1b80 10 client.0 _ll_register_callbacks cb 0x557d75d1a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f5740a48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75ce2e40 0x557d75e6b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f5740a48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75ce2e40 0x557d75e6b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52190/0 (socket says 172.21.15.19:52190) 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f5740a48700 1 -- 172.21.15.19:0/489534488 learned_addr learned my addr 172.21.15.19:0/489534488 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f573bfff700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e6bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f5741249700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75e61cd0 0x557d75e6c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f573bfff700 1 -- 172.21.15.19:0/489534488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75e61cd0 msgr2=0x557d75e6c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f573bfff700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75e61cd0 0x557d75e6c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f573bfff700 1 -- 172.21.15.19:0/489534488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75ce2e40 msgr2=0x557d75e6b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f573bfff700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75ce2e40 0x557d75e6b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.978+0000 7f573bfff700 1 -- 172.21.15.19:0/489534488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d75ccf8d0 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f573bfff700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e6bb70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f573c107a00 tx=0x7f572800ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ded00d4bdef53102 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f571bfff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f571bfff700 1 -- 172.21.15.19:0/489534488 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572800ea80 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573000b790 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f571bfff700 1 -- 172.21.15.19:0/489534488 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f572800ec20 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f571bfff700 1 -- 172.21.15.19:0/489534488 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f572802d050 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f57516c1b80 10 client.26162 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d75d39760 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.979+0000 7f571bfff700 1 -- 172.21.15.19:0/489534488 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f573000b790 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f571bfff700 1 -- 172.21.15.19:0/489534488 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f572801a0d0 con 0x557d75dce430 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f57516c1b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f57516c1b80 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f57516c1b80 20 client.26162 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f57516c1b80 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75e7b560 0x557d75e7d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f57516c1b80 4 client.26162 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d75e2ded0 con 0x557d75e7b560 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f5740a48700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75e7b560 0x557d75e7d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f571bfff700 10 client.26162 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.980+0000 7f5740a48700 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75e7b560 0x557d75e7d980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc9088e8b7da4116 server_cookie=cca1938ed53a36ba in_seq=0 out_seq=0 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f571bfff700 1 -- 172.21.15.19:0/489534488 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557d75e2ded0 con 0x557d75e7b560 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f571bfff700 10 client.26162 handle_command_reply: tid=0 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 client.26162 shutdown 2024-05-17T09:23:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 msgr2=0x557d75e6bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e6bb70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f573c107a00 tx=0x7f572800ff20 comp rx=0 tx=0).stop 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 shutdown_connections 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75e7b560 0x557d75e7d980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75e61cd0 0x557d75e6c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75ce2e40 0x557d75e6b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 --2- 172.21.15.19:0/489534488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e6bb70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 >> 172.21.15.19:0/489534488 conn(0x557d75d431c0 msgr2=0x557d75e5c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.981+0000 7f571bfff700 0 client.26162 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.982+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 shutdown_connections 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.982+0000 7f57516c1b80 1 -- 172.21.15.19:0/489534488 wait complete. 2024-05-17T09:23:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.982+0000 7f57516c1b80 20 client.26162 trim_cache size 0 max 16384 2024-05-17T09:23:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.999+0000 7f57516c1b80 1 Processor -- start 2024-05-17T09:23:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.999+0000 7f57516c1b80 1 -- start start 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:26.999+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75ce2e40 0x557d75e6a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75dce430 0x557d75e65930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f57516c1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1b0970 con 0x557d75e61cd0 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f57516c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1ae200 con 0x557d75ce2e40 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f57516c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1ac050 con 0x557d75dce430 2024-05-17T09:23:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5741249700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5741249700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e77570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52204/0 (socket says 172.21.15.19:52204) 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5741249700 1 -- 172.21.15.19:0/3812245844 learned_addr learned my addr 172.21.15.19:0/3812245844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75ce2e40 0x557d75e6a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f573bfff700 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75dce430 0x557d75e65930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 -- 172.21.15.19:0/3812245844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75dce430 msgr2=0x557d75e65930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75dce430 0x557d75e65930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 -- 172.21.15.19:0/3812245844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 msgr2=0x557d75e77570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e77570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 -- 172.21.15.19:0/3812245844 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d75e78f30 con 0x557d75ce2e40 2024-05-17T09:23:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5741249700 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:27.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.000+0000 7f5740a48700 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75ce2e40 0x557d75e6a590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x557d75e7e0b0 tx=0x7f5730005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72da01ee4203587f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:27.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.001+0000 7f57267fc700 1 -- 172.21.15.19:0/3812245844 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730043070 con 0x557d75ce2e40 2024-05-17T09:23:27.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.001+0000 7f57267fc700 1 -- 172.21.15.19:0/3812245844 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5730005540 con 0x557d75ce2e40 2024-05-17T09:23:27.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.001+0000 7f57267fc700 1 -- 172.21.15.19:0/3812245844 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730003dd0 con 0x557d75ce2e40 2024-05-17T09:23:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.001+0000 7f57516c1b80 1 -- 172.21.15.19:0/3812245844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75ce2e40 msgr2=0x557d75e6a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.001+0000 7f57516c1b80 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75ce2e40 0x557d75e6a590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x557d75e7e0b0 tx=0x7f5730005040 comp rx=0 tx=0).stop 2024-05-17T09:23:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.001+0000 7f57516c1b80 1 -- 172.21.15.19:0/3812245844 shutdown_connections 2024-05-17T09:23:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75ce2e40 0x557d75e6a590 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 --2- 172.21.15.19:0/3812245844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75dce430 0x557d75e65930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 -- 172.21.15.19:0/3812245844 >> 172.21.15.19:0/3812245844 conn(0x557d75d431c0 msgr2=0x557d75e65220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 -- 172.21.15.19:0/3812245844 shutdown_connections 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 -- 172.21.15.19:0/3812245844 wait complete. 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 -- start start 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.002+0000 7f57516c1b80 1 Processor -- start 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e82e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1ac6c0 con 0x557d75e61cd0 2024-05-17T09:23:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1adb30 con 0x557d75dce430 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1acd90 con 0x557d75ce2e40 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f57516c1b80 10 client.0 _ll_register_callbacks cb 0x557d75d1a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f573bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e83910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52210/0 (socket says 172.21.15.19:52210) 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f573bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e833d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41894/0 (socket says 172.21.15.19:41894) 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 -- 172.21.15.19:0/4194162535 learned_addr learned my addr 172.21.15.19:0/4194162535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5740a48700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e82e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 -- 172.21.15.19:0/4194162535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 msgr2=0x557d75e82e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e82e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 -- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 msgr2=0x557d75e833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e833d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 -- 172.21.15.19:0/4194162535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d75e6c420 con 0x557d75e61cd0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f573bfff700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f5741249700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e83910 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f57340dfa00 tx=0x7f57340b1e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c400d3b7162844 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.003+0000 7f571bfff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f571bfff700 1 -- 172.21.15.19:0/4194162535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340a8d50 con 0x557d75e61cd0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f571bfff700 1 -- 172.21.15.19:0/4194162535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f57340ad2d0 con 0x557d75e61cd0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5730005540 con 0x557d75e61cd0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f571bfff700 1 -- 172.21.15.19:0/4194162535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57340a9940 con 0x557d75e61cd0 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f57516c1b80 10 client.16497 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d75e56590 con 0x557d75e61cd0 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f571bfff700 1 -- 172.21.15.19:0/4194162535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5730005540 con 0x557d75e61cd0 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f571bfff700 1 -- 172.21.15.19:0/4194162535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f57340a4bb0 con 0x557d75e61cd0 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f57516c1b80 10 client.16497 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f57516c1b80 10 client.16497 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.004+0000 7f57516c1b80 20 client.16497 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f57516c1b80 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75ee6540 0x557d75ee8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f57516c1b80 4 client.16497 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 --> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d75d91cc0 con 0x557d75ee6540 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f5740a48700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75ee6540 0x557d75ee8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f571bfff700 10 client.16497 ms_handle_connect on v2:172.21.15.69:6836/1860504493 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f5740a48700 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75ee6540 0x557d75ee8960 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20c4953a9130b752 server_cookie=a6e50ec374c70765 in_seq=0 out_seq=0 2024-05-17T09:23:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f571bfff700 1 -- 172.21.15.19:0/4194162535 <== mds.2 v2:172.21.15.69:6836/1860504493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+233 (crc 0 0 0) 0x557d75d91cc0 con 0x557d75ee6540 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f571bfff700 10 client.16497 handle_command_reply: tid=0 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.005+0000 7f57516c1b80 1 client.16497 shutdown 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 msgr2=0x557d75e83910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e83910 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f57340dfa00 tx=0x7f57340b1e60 comp rx=0 tx=0).stop 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 shutdown_connections 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:6836/1860504493,v1:172.21.15.69:6837/1860504493] conn(0x557d75ee6540 0x557d75ee8960 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557d75ce2e40 0x557d75e82e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f571bfff700 0 client.16497 ms_handle_reset on v2:172.21.15.69:6836/1860504493 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d75e61cd0 0x557d75e83910 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 --2- 172.21.15.19:0/4194162535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d75dce430 0x557d75e833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 >> 172.21.15.19:0/4194162535 conn(0x557d75d431c0 msgr2=0x557d75e60cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 shutdown_connections 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 1 -- 172.21.15.19:0/4194162535 wait complete. 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.006+0000 7f57516c1b80 20 client.16497 trim_cache size 0 max 16384 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3487705699, 2024-05-17T09:23:27.013 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511628991, 2024-05-17T09:23:27.014 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC" 2024-05-17T09:23:27.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:27.014 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-05-17T09:23:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f572c0b2900 msgr2=0x7f572c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f572c0b2900 0x7f572c0b4e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5730009990 tx=0x7f573001a040 comp rx=0 tx=0).stop 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 msgr2=0x7f573c19f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 0x7f573c19f590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f573400efc0 tx=0x7f573400c3a0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 shutdown_connections 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f572c0b2900 0x7f572c0b4e10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f573c1045e0 0x7f573c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f573c109860 0x7f573c19f590 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 --2- 172.21.15.19:0/3266557173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f573c108f10 0x7f573c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 >> 172.21.15.19:0/3266557173 conn(0x7f573c100030 msgr2=0x7f573c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 shutdown_connections 2024-05-17T09:23:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:27.007+0000 7f5742cac700 1 -- 172.21.15.19:0/3266557173 wait complete. 2024-05-17T09:23:27.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 30831 2.13 MB/sec warmup 85 sec latency 59.858 ms 2024-05-17T09:23:27.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-17T09:23:28.066 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:28.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 31436 2.17 MB/sec warmup 86 sec latency 12.170 ms 2024-05-17T09:23:28.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:28 smithi019 ceph-mon[31620]: pgmap v549: 129 pgs: 129 active+clean; 27 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 264 KiB/s wr, 47 op/s 2024-05-17T09:23:28.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:28 smithi069 ceph-mon[35081]: pgmap v549: 129 pgs: 129 active+clean; 27 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 264 KiB/s wr, 47 op/s 2024-05-17T09:23:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:28 smithi079 ceph-mon[35502]: pgmap v549: 129 pgs: 129 active+clean; 27 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 264 KiB/s wr, 47 op/s 2024-05-17T09:23:29.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 31971 2.20 MB/sec warmup 87 sec latency 12.029 ms 2024-05-17T09:23:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 -- 172.21.15.19:0/2445634359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f4103990 msgr2=0x7f29f4103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 --2- 172.21.15.19:0/2445634359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f4103990 0x7f29f4103da0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009990 tx=0x7f29e8009ca0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 -- 172.21.15.19:0/2445634359 shutdown_connections 2024-05-17T09:23:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 --2- 172.21.15.19:0/2445634359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41042e0 0x7f29f4110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 --2- 172.21.15.19:0/2445634359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f41081f0 0x7f29f4108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 --2- 172.21.15.19:0/2445634359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f4103990 0x7f29f4103da0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 -- 172.21.15.19:0/2445634359 >> 172.21.15.19:0/2445634359 conn(0x7f29f40fe4d0 msgr2=0x7f29f4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 -- 172.21.15.19:0/2445634359 shutdown_connections 2024-05-17T09:23:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.962+0000 7f29f9033700 1 -- 172.21.15.19:0/2445634359 wait complete. 2024-05-17T09:23:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.963+0000 7f29f9033700 1 Processor -- start 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 -- start start 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 0x7f29f40fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f41042e0 0x7f29f40feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 0x7f29f40ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4113220 con 0x7f29f41042e0 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41130a0 con 0x7f29f4103990 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f9033700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41133a0 con 0x7f29f41081f0 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 0x7f29f40fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 0x7f29f40ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 0x7f29f40fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41912/0 (socket says 172.21.15.19:41912) 2024-05-17T09:23:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 0x7f29f40ff430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42794/0 (socket says 172.21.15.19:42794) 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f2d9d700 1 -- 172.21.15.19:0/1104238906 learned_addr learned my addr 172.21.15.19:0/1104238906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f259c700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f41042e0 0x7f29f40feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f359e700 1 -- 172.21.15.19:0/1104238906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 msgr2=0x7f29f40fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f359e700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 0x7f29f40fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f359e700 1 -- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f41042e0 msgr2=0x7f29f40feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.964+0000 7f29f359e700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f41042e0 0x7f29f40feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29f359e700 1 -- 172.21.15.19:0/1104238906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f41a9080 con 0x7f29f41081f0 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 0x7f29f40fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29f359e700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 0x7f29f40ff430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00efc0 tx=0x7f29ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29dbfff700 1 -- 172.21.15.19:0/1104238906 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0092e0 con 0x7f29f41081f0 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29dbfff700 1 -- 172.21.15.19:0/1104238906 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ec004bb0 con 0x7f29f41081f0 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f41a9310 con 0x7f29f41081f0 2024-05-17T09:23:29.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29dbfff700 1 -- 172.21.15.19:0/1104238906 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec004020 con 0x7f29f41081f0 2024-05-17T09:23:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.965+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f41a9800 con 0x7f29f41081f0 2024-05-17T09:23:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.968+0000 7f29dbfff700 1 -- 172.21.15.19:0/1104238906 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f29ec01b030 con 0x7f29f41081f0 2024-05-17T09:23:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.968+0000 7f29dbfff700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29dc0b2880 0x7f29dc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29dc0b2880 0x7f29dc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f29dbfff700 1 -- 172.21.15.19:0/1104238906 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f29ec0e20d0 con 0x7f29f41081f0 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f2a07a48b80 1 Processor -- start 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f2a07a48b80 1 -- start start 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29dc0b2880 0x7f29dc0b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f29e4011490 tx=0x7f29e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd847fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 0x5636bd848520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.969+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd8423c0 0x5636bd8427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636bd770c90 con 0x5636bd7413d0 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636bd760830 con 0x5636bd8423c0 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5636bd764870 con 0x5636bd6c3e40 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 0x5636bd848520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd847fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 0x5636bd848520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42798/0 (socket says 172.21.15.19:42798) 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 -- 172.21.15.19:0/1168693292 learned_addr learned my addr 172.21.15.19:0/1168693292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f359e700 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd8423c0 0x5636bd8427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 -- 172.21.15.19:0/1168693292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd8423c0 msgr2=0x5636bd8427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd8423c0 0x5636bd8427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 -- 172.21.15.19:0/1168693292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 msgr2=0x5636bd847fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd847fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 -- 172.21.15.19:0/1168693292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bd6b08d0 con 0x5636bd6c3e40 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f359e700 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd8423c0 0x5636bd8427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29f259c700 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 0x5636bd848520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009990 tx=0x7f29e8005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b41af34fdc77ee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29d97fa700 1 -- 172.21.15.19:0/1168693292 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80054b0 con 0x5636bd6c3e40 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29d97fa700 1 -- 172.21.15.19:0/1168693292 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29e8005650 con 0x5636bd6c3e40 2024-05-17T09:23:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.970+0000 7f29d97fa700 1 -- 172.21.15.19:0/1168693292 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e800bb10 con 0x5636bd6c3e40 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 -- 172.21.15.19:0/1168693292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 msgr2=0x5636bd848520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 0x5636bd848520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009990 tx=0x7f29e8005f00 comp rx=0 tx=0).stop 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 -- 172.21.15.19:0/1168693292 shutdown_connections 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd8423c0 0x5636bd8427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd847fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/1168693292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd6c3e40 0x5636bd848520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 -- 172.21.15.19:0/1168693292 >> 172.21.15.19:0/1168693292 conn(0x5636bd702c80 msgr2=0x5636bd83cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 -- 172.21.15.19:0/1168693292 shutdown_connections 2024-05-17T09:23:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.971+0000 7f2a07a48b80 1 -- 172.21.15.19:0/1168693292 wait complete. 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 -- start start 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 Processor -- start 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd84bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd84c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd8423c0 0x5636bd84c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636bd666fa0 con 0x5636bd7413d0 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636bd7027c0 con 0x5636bd6c3e40 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5636bd843190 con 0x5636bd8423c0 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f2a07a48b80 10 client.0 _ll_register_callbacks cb 0x5636bd6fb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f29f2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd84bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f29f259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd84c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f29f2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd84bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41934/0 (socket says 172.21.15.19:41934) 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f29f2d9d700 1 -- 172.21.15.19:0/4016566524 learned_addr learned my addr 172.21.15.19:0/4016566524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.972+0000 7f29f359e700 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd8423c0 0x5636bd84c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29f2d9d700 1 -- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd8423c0 msgr2=0x5636bd84c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd8423c0 0x5636bd84c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29f2d9d700 1 -- 172.21.15.19:0/4016566524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 msgr2=0x5636bd84c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd84c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29f2d9d700 1 -- 172.21.15.19:0/4016566524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bd6ff690 con 0x5636bd6c3e40 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd84bcc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29e400eec0 tx=0x7f29e40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcfcccf13df6457a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29d6ff5700 1 -- 172.21.15.19:0/4016566524 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e401a5b0 con 0x5636bd6c3e40 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29d6ff5700 1 -- 172.21.15.19:0/4016566524 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f29e401a750 con 0x5636bd6c3e40 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e8005650 con 0x5636bd6c3e40 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f29d6ff5700 1 -- 172.21.15.19:0/4016566524 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29e401a990 con 0x5636bd6c3e40 2024-05-17T09:23:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f2a07a48b80 10 client.26180 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.973+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636bd859570 con 0x5636bd6c3e40 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f29d6ff5700 1 -- 172.21.15.19:0/4016566524 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29e8005650 con 0x5636bd6c3e40 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f29d6ff5700 1 -- 172.21.15.19:0/4016566524 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f29e4026070 con 0x5636bd6c3e40 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f2a07a48b80 10 client.26180 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f2a07a48b80 10 client.26180 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f2a07a48b80 20 client.26180 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd85bc00 0x5636bd85e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f2a07a48b80 4 client.26180 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636bd772be0 con 0x5636bd85bc00 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f29f259c700 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd85bc00 0x5636bd85e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f29f259c700 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd85bc00 0x5636bd85e020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48b88870732a5336 server_cookie=5b982f56da3898b3 in_seq=0 out_seq=0 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.974+0000 7f29d6ff5700 10 client.26180 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:23:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f29d6ff5700 1 -- 172.21.15.19:0/4016566524 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636bd772be0 con 0x5636bd85bc00 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f29d6ff5700 10 client.26180 handle_command_reply: tid=0 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f2a07a48b80 1 client.26180 shutdown 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 msgr2=0x5636bd84bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd84bcc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29e400eec0 tx=0x7f29e40047c0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 shutdown_connections 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.975+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd85bc00 0x5636bd85e020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd8423c0 0x5636bd84c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f29d6ff5700 0 client.26180 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd84bcc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/4016566524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd7413d0 0x5636bd84c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 >> 172.21.15.19:0/4016566524 conn(0x5636bd702c80 msgr2=0x5636bd83cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 shutdown_connections 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 1 -- 172.21.15.19:0/4016566524 wait complete. 2024-05-17T09:23:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.976+0000 7f2a07a48b80 20 client.26180 trim_cache size 0 max 16384 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 Processor -- start 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 -- start start 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd857b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd837670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd8637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41b4dd0 con 0x5636bd8423c0 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41b2660 con 0x5636bd6c3e40 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.992+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41b04b0 con 0x5636bd7413d0 2024-05-17T09:23:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd8637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd8637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52266/0 (socket says 172.21.15.19:52266) 2024-05-17T09:23:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 -- 172.21.15.19:0/2615610807 learned_addr learned my addr 172.21.15.19:0/2615610807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f259c700 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd837670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd857b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 -- 172.21.15.19:0/2615610807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 msgr2=0x5636bd837670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd837670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 -- 172.21.15.19:0/2615610807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 msgr2=0x5636bd857b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd857b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 -- 172.21.15.19:0/2615610807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bd83d3f0 con 0x5636bd8423c0 2024-05-17T09:23:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29f359e700 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd8637e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0df820 tx=0x7f29ec091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12275db16488950 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.993+0000 7f29d8ff9700 1 -- 172.21.15.19:0/2615610807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0a8ee0 con 0x5636bd8423c0 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f29d8ff9700 1 -- 172.21.15.19:0/2615610807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ec0c3070 con 0x5636bd8423c0 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f29d8ff9700 1 -- 172.21.15.19:0/2615610807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0b1d10 con 0x5636bd8423c0 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2615610807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 msgr2=0x5636bd8637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd8637e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0df820 tx=0x7f29ec091040 comp rx=0 tx=0).stop 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2615610807 shutdown_connections 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd8637e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd857b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2615610807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd837670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.994+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2615610807 >> 172.21.15.19:0/2615610807 conn(0x5636bd702c80 msgr2=0x5636bd836f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.995+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2615610807 shutdown_connections 2024-05-17T09:23:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.995+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2615610807 wait complete. 2024-05-17T09:23:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.995+0000 7f2a07a48b80 1 -- start start 2024-05-17T09:23:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.995+0000 7f2a07a48b80 1 Processor -- start 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd853d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd8542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd848020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41b0b20 con 0x5636bd8423c0 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41b1f90 con 0x5636bd6c3e40 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41b11f0 con 0x5636bd7413d0 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f2a07a48b80 10 client.0 _ll_register_callbacks cb 0x5636bd6fb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd848020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd848020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52274/0 (socket says 172.21.15.19:52274) 2024-05-17T09:23:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 -- 172.21.15.19:0/2279411882 learned_addr learned my addr 172.21.15.19:0/2279411882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f259c700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd8542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd853d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 -- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 msgr2=0x5636bd8542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd8542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 -- 172.21.15.19:0/2279411882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 msgr2=0x5636bd853d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd853d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.996+0000 7f29f359e700 1 -- 172.21.15.19:0/2279411882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bd8548c0 con 0x5636bd8423c0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29f359e700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd848020 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0a6640 tx=0x7f29ec0a53c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5793d921fa7f1a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29d6ff5700 1 -- 172.21.15.19:0/2279411882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec06d070 con 0x5636bd8423c0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29d6ff5700 1 -- 172.21.15.19:0/2279411882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f29ec0a8840 con 0x5636bd8423c0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ec0c3070 con 0x5636bd8423c0 2024-05-17T09:23:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29d6ff5700 1 -- 172.21.15.19:0/2279411882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29ec0ca050 con 0x5636bd8423c0 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f2a07a48b80 10 client.16515 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636bd8c5bb0 con 0x5636bd8423c0 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29d6ff5700 1 -- 172.21.15.19:0/2279411882 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ec0c3070 con 0x5636bd8423c0 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.997+0000 7f29d6ff5700 1 -- 172.21.15.19:0/2279411882 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f29ec0dfce0 con 0x5636bd8423c0 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f2a07a48b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f2a07a48b80 10 client.16515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f2a07a48b80 20 client.16515 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd8c6120 0x5636bd8c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f2a07a48b80 4 client.16515 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 --> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636bd76c3c0 con 0x5636bd8c6120 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd8c6120 0x5636bd8c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f29f2d9d700 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd8c6120 0x5636bd8c8540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2e4193a6edaff19 server_cookie=a1a48132e7ace317 in_seq=0 out_seq=0 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.998+0000 7f29d6ff5700 10 client.16515 ms_handle_connect on v2:172.21.15.79:6838/2520420584 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f29d6ff5700 1 -- 172.21.15.19:0/2279411882 <== mds.1 v2:172.21.15.79:6838/2520420584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636bd76c3c0 con 0x5636bd8c6120 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f29d6ff5700 10 client.16515 handle_command_reply: tid=0 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 client.16515 shutdown 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 msgr2=0x5636bd848020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd848020 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0a6640 tx=0x7f29ec0a53c0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 shutdown_connections 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:6838/2520420584,v1:172.21.15.79:6839/2520420584] conn(0x5636bd8c6120 0x5636bd8c8540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636bd8423c0 0x5636bd848020 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636bd6c3e40 0x5636bd853d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 --2- 172.21.15.19:0/2279411882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636bd7413d0 0x5636bd8542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f29d6ff5700 0 client.16515 ms_handle_reset on v2:172.21.15.79:6838/2520420584 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 >> 172.21.15.19:0/2279411882 conn(0x5636bd702c80 msgr2=0x5636bd8413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 shutdown_connections 2024-05-17T09:23:30.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 1 -- 172.21.15.19:0/2279411882 wait complete. 2024-05-17T09:23:30.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:29.999+0000 7f2a07a48b80 20 client.16515 trim_cache size 0 max 16384 2024-05-17T09:23:30.004 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29dc0b2880 msgr2=0x7f29dc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29dc0b2880 0x7f29dc0b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f29e4011490 tx=0x7f29e4009260 comp rx=0 tx=0).stop 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 msgr2=0x7f29f40ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 0x7f29f40ff430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00efc0 tx=0x7f29ec00c3a0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 shutdown_connections 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29dc0b2880 0x7f29dc0b4d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29f41081f0 0x7f29f40ff430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f4103990 0x7f29f40fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 --2- 172.21.15.19:0/1104238906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29f41042e0 0x7f29f40feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 >> 172.21.15.19:0/1104238906 conn(0x7f29f40fe4d0 msgr2=0x7f29f4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 shutdown_connections 2024-05-17T09:23:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:30.000+0000 7f29f9033700 1 -- 172.21.15.19:0/1104238906 wait complete. 2024-05-17T09:23:30.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32331 2.19 MB/sec warmup 88 sec latency 13.915 ms 2024-05-17T09:23:30.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:30 smithi079 ceph-mon[35502]: pgmap v550: 129 pgs: 129 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 461 KiB/s wr, 66 op/s 2024-05-17T09:23:30.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:30 smithi069 ceph-mon[35081]: pgmap v550: 129 pgs: 129 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 461 KiB/s wr, 66 op/s 2024-05-17T09:23:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:30 smithi019 ceph-mon[31620]: pgmap v550: 129 pgs: 129 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 461 KiB/s wr, 66 op/s 2024-05-17T09:23:30.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-17T09:23:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:31.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32581 2.17 MB/sec warmup 89 sec latency 15.141 ms 2024-05-17T09:23:32.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 33086 2.17 MB/sec warmup 90 sec latency 18.044 ms 2024-05-17T09:23:32.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:32 smithi019 ceph-mon[31620]: pgmap v551: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 514 KiB/s wr, 67 op/s 2024-05-17T09:23:32.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:32 smithi069 ceph-mon[35081]: pgmap v551: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 514 KiB/s wr, 67 op/s 2024-05-17T09:23:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:32 smithi079 ceph-mon[35502]: pgmap v551: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 514 KiB/s wr, 67 op/s 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 -- 172.21.15.19:0/3763929004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c108730 msgr2=0x7fe75c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 --2- 172.21.15.19:0/3763929004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c108730 0x7fe75c110090 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe75400b0d0 tx=0x7fe75400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 -- 172.21.15.19:0/3763929004 shutdown_connections 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 --2- 172.21.15.19:0/3763929004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c1034b0 0x7fe75c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 --2- 172.21.15.19:0/3763929004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c108730 0x7fe75c110090 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 --2- 172.21.15.19:0/3763929004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75c107de0 0x7fe75c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 -- 172.21.15.19:0/3763929004 >> 172.21.15.19:0/3763929004 conn(0x7fe75c0fede0 msgr2=0x7fe75c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 -- 172.21.15.19:0/3763929004 shutdown_connections 2024-05-17T09:23:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.173+0000 7fe7609d7700 1 -- 172.21.15.19:0/3763929004 wait complete. 2024-05-17T09:23:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 Processor -- start 2024-05-17T09:23:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 -- start start 2024-05-17T09:23:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c1034b0 0x7fe75c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c107de0 0x7fe75c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75c108730 0x7fe75c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1143b0 con 0x7fe75c1034b0 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c114230 con 0x7fe75c108730 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe7609d7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c114530 con 0x7fe75c107de0 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c1034b0 0x7fe75c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.175+0000 7fe759d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c107de0 0x7fe75c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c1034b0 0x7fe75c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52302/0 (socket says 172.21.15.19:52302) 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe75a59c700 1 -- 172.21.15.19:0/1175203926 learned_addr learned my addr 172.21.15.19:0/1175203926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe75ad9d700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75c108730 0x7fe75c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe759d9b700 1 -- 172.21.15.19:0/1175203926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75c108730 msgr2=0x7fe75c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe759d9b700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75c108730 0x7fe75c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe759d9b700 1 -- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c1034b0 msgr2=0x7fe75c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe759d9b700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c1034b0 0x7fe75c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe759d9b700 1 -- 172.21.15.19:0/1175203926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c1a1e30 con 0x7fe75c107de0 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe759d9b700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c107de0 0x7fe75c19d500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe75000e9c0 tx=0x7fe75000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe74b7fe700 1 -- 172.21.15.19:0/1175203926 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000cb70 con 0x7fe75c107de0 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe74b7fe700 1 -- 172.21.15.19:0/1175203926 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7500044b0 con 0x7fe75c107de0 2024-05-17T09:23:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.176+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c1a2120 con 0x7fe75c107de0 2024-05-17T09:23:33.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.177+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a9a40 con 0x7fe75c107de0 2024-05-17T09:23:33.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.177+0000 7fe74b7fe700 1 -- 172.21.15.19:0/1175203926 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750003cc0 con 0x7fe75c107de0 2024-05-17T09:23:33.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.179+0000 7fe74b7fe700 1 -- 172.21.15.19:0/1175203926 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe750018430 con 0x7fe75c107de0 2024-05-17T09:23:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.180+0000 7fe74b7fe700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe7440b2900 0x7fe7440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.180+0000 7fe75a59c700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe7440b2900 0x7fe7440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.180+0000 7fe74b7fe700 1 -- 172.21.15.19:0/1175203926 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe7500e2770 con 0x7fe75c107de0 2024-05-17T09:23:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.181+0000 7fe75a59c700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe7440b2900 0x7fe7440b4e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009cb0 tx=0x7fe74c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.181+0000 7fe76f3ecb80 1 Processor -- start 2024-05-17T09:23:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.181+0000 7fe76f3ecb80 1 -- start start 2024-05-17T09:23:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8937f7430 0x55f893890910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f893890eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8937b8d70 con 0x55f8937f7430 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f8937a8910 con 0x55f89388acd0 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f8937ac950 con 0x55f89370be40 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f893890eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8937f7430 0x55f893890910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f893890eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42854/0 (socket says 172.21.15.19:42854) 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 -- 172.21.15.19:0/939050504 learned_addr learned my addr 172.21.15.19:0/939050504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe75ad9d700 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 -- 172.21.15.19:0/939050504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 msgr2=0x55f89388b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 -- 172.21.15.19:0/939050504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8937f7430 msgr2=0x55f893890910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8937f7430 0x55f893890910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 -- 172.21.15.19:0/939050504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f893751fb0 con 0x55f89370be40 2024-05-17T09:23:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.182+0000 7fe759d9b700 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f893890eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a9970 tx=0x7fe7500a9400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8448d479849a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.183+0000 7fe748ff9700 1 -- 172.21.15.19:0/939050504 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500a94f0 con 0x55f89370be40 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.183+0000 7fe748ff9700 1 -- 172.21.15.19:0/939050504 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7500aa0e0 con 0x55f89370be40 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.183+0000 7fe748ff9700 1 -- 172.21.15.19:0/939050504 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000cda0 con 0x55f89370be40 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/939050504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 msgr2=0x55f893890eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f893890eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a9970 tx=0x7fe7500a9400 comp rx=0 tx=0).stop 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/939050504 shutdown_connections 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8937f7430 0x55f893890910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/939050504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f893890eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/939050504 >> 172.21.15.19:0/939050504 conn(0x55f89376c1c0 msgr2=0x55f8938857d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/939050504 shutdown_connections 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/939050504 wait complete. 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.184+0000 7fe76f3ecb80 1 -- start start 2024-05-17T09:23:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 Processor -- start 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89370be40 0x55f8938945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f893894b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f893895040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8936aefa0 con 0x55f89388acd0 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f893710da0 con 0x55f89370be40 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f8938975a0 con 0x55f8937f7430 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe76f3ecb80 10 client.0 _ll_register_callbacks cb 0x55f893743950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe75ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f893895040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89370be40 0x55f8938945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe759d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f893894b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe75ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f893895040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52316/0 (socket says 172.21.15.19:52316) 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe75ad9d700 1 -- 172.21.15.19:0/2359432566 learned_addr learned my addr 172.21.15.19:0/2359432566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe759d9b700 1 -- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89370be40 msgr2=0x55f8938945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe759d9b700 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89370be40 0x55f8938945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe759d9b700 1 -- 172.21.15.19:0/2359432566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 msgr2=0x55f893895040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe759d9b700 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f893895040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe759d9b700 1 -- 172.21.15.19:0/2359432566 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8936f88d0 con 0x55f8937f7430 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.185+0000 7fe75a59c700 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89370be40 0x55f8938945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe759d9b700 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f893894b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a9970 tx=0x7fe7500036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f35c30188786b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe741ffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe741ffb700 1 -- 172.21.15.19:0/2359432566 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500a94f0 con 0x55f8937f7430 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe741ffb700 1 -- 172.21.15.19:0/2359432566 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe7500b23b0 con 0x55f8937f7430 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7500aa0e0 con 0x55f8937f7430 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe741ffb700 1 -- 172.21.15.19:0/2359432566 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7500add90 con 0x55f8937f7430 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe76f3ecb80 10 client.26194 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f893762760 con 0x55f8937f7430 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe741ffb700 1 -- 172.21.15.19:0/2359432566 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7500aa0e0 con 0x55f8937f7430 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.186+0000 7fe741ffb700 1 -- 172.21.15.19:0/2359432566 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe7500921d0 con 0x55f8937f7430 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe76f3ecb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe76f3ecb80 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe76f3ecb80 20 client.26194 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f8938a42e0 0x55f8938a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe76f3ecb80 4 client.26194 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f893856ed0 con 0x55f8938a42e0 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe75a59c700 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f8938a42e0 0x55f8938a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe741ffb700 10 client.26194 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.187+0000 7fe75a59c700 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f8938a42e0 0x55f8938a6700 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c1fddbfafd58b3 server_cookie=fb838e5285df2f79 in_seq=0 out_seq=0 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.188+0000 7fe741ffb700 1 -- 172.21.15.19:0/2359432566 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f893856ed0 con 0x55f8938a42e0 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.188+0000 7fe741ffb700 10 client.26194 handle_command_reply: tid=0 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.188+0000 7fe76f3ecb80 1 client.26194 shutdown 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 msgr2=0x55f893894b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f893894b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a9970 tx=0x7fe7500036f0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 shutdown_connections 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f8938a42e0 0x55f8938a6700 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f893895040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89370be40 0x55f8938945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/2359432566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f893894b00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 >> 172.21.15.19:0/2359432566 conn(0x55f89376c1c0 msgr2=0x55f893885790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe741ffb700 0 client.26194 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 shutdown_connections 2024-05-17T09:23:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/2359432566 wait complete. 2024-05-17T09:23:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.189+0000 7fe76f3ecb80 20 client.26194 trim_cache size 0 max 16384 2024-05-17T09:23:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.208+0000 7fe76f3ecb80 1 Processor -- start 2024-05-17T09:23:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.208+0000 7fe76f3ecb80 1 -- start start 2024-05-17T09:23:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89370be40 0x55f893896a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f89388f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b4ee0 con 0x55f89370be40 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b2770 con 0x55f89388acd0 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b05c0 con 0x55f8937f7430 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89370be40 0x55f893896a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89370be40 0x55f893896a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52320/0 (socket says 172.21.15.19:52320) 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe75a59c700 1 -- 172.21.15.19:0/3113473152 learned_addr learned my addr 172.21.15.19:0/3113473152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe759d9b700 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f89388f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe75ad9d700 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe759d9b700 1 -- 172.21.15.19:0/3113473152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 msgr2=0x55f89388fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe759d9b700 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe759d9b700 1 -- 172.21.15.19:0/3113473152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89370be40 msgr2=0x55f893896a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe759d9b700 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89370be40 0x55f893896a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.209+0000 7fe759d9b700 1 -- 172.21.15.19:0/3113473152 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f893890c80 con 0x55f8937f7430 2024-05-17T09:23:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.210+0000 7fe759d9b700 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f89388f650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a6c50 tx=0x7fe7500a59d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4d14ea2bfd7eec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.210+0000 7fe743fff700 1 -- 172.21.15.19:0/3113473152 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500a94f0 con 0x55f8937f7430 2024-05-17T09:23:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.210+0000 7fe743fff700 1 -- 172.21.15.19:0/3113473152 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7500aa0e0 con 0x55f8937f7430 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.210+0000 7fe743fff700 1 -- 172.21.15.19:0/3113473152 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750092350 con 0x55f8937f7430 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/3113473152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 msgr2=0x55f89388f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f89388f650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a6c50 tx=0x7fe7500a59d0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/3113473152 shutdown_connections 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89388acd0 0x55f89388fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89370be40 0x55f893896a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/3113473152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8937f7430 0x55f89388f650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/3113473152 >> 172.21.15.19:0/3113473152 conn(0x55f89376c1c0 msgr2=0x55f89388b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/3113473152 shutdown_connections 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/3113473152 wait complete. 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 -- start start 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.211+0000 7fe76f3ecb80 1 Processor -- start 2024-05-17T09:23:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f8938a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f8937f7430 0x55f8938a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f8938a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b0c30 con 0x55f89388acd0 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b20a0 con 0x55f8937f7430 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b1300 con 0x55f89370be40 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe76f3ecb80 10 client.0 _ll_register_callbacks cb 0x55f893743950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f8938a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f8938a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42866/0 (socket says 172.21.15.19:42866) 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 -- 172.21.15.19:0/1827266178 learned_addr learned my addr 172.21.15.19:0/1827266178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f8938a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe759d9b700 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f8937f7430 0x55f8938a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 -- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f8937f7430 msgr2=0x55f8938a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f8937f7430 0x55f8938a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 -- 172.21.15.19:0/1827266178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 msgr2=0x55f8938a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f8938a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.212+0000 7fe75a59c700 1 -- 172.21.15.19:0/1827266178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89390cb40 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe75a59c700 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f8938a6700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe74c012bc0 tx=0x7fe74c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232b76ade84bbe06 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe741ffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe741ffb700 1 -- 172.21.15.19:0/1827266178 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c02b850 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7500aa0e0 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe741ffb700 1 -- 172.21.15.19:0/1827266178 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe74c02b9f0 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe741ffb700 1 -- 172.21.15.19:0/1827266178 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe74c02bbf0 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe76f3ecb80 10 client.26200 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f89390de20 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.213+0000 7fe741ffb700 1 -- 172.21.15.19:0/1827266178 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7500aa0e0 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe741ffb700 1 -- 172.21.15.19:0/1827266178 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe74c020430 con 0x55f89370be40 2024-05-17T09:23:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe76f3ecb80 10 client.26200 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe76f3ecb80 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe76f3ecb80 20 client.26200 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f89390e4e0 0x55f893910900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe76f3ecb80 4 client.26200 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8937bacc0 con 0x55f89390e4e0 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe759d9b700 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f89390e4e0 0x55f893910900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe741ffb700 10 client.26200 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.214+0000 7fe759d9b700 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f89390e4e0 0x55f893910900 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2db4ba14b19f3c7 server_cookie=d38200b7a9f584fc in_seq=0 out_seq=0 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.215+0000 7fe741ffb700 1 -- 172.21.15.19:0/1827266178 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8937bacc0 con 0x55f89390e4e0 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.215+0000 7fe741ffb700 10 client.26200 handle_command_reply: tid=0 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 client.26200 shutdown 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 msgr2=0x55f8938a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f8938a6700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe74c012bc0 tx=0x7fe74c0047c0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 shutdown_connections 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f89370be40 0x55f8938a6700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f89390e4e0 0x55f893910900 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89388acd0 0x55f8938a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 --2- 172.21.15.19:0/1827266178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f8937f7430 0x55f8938a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 >> 172.21.15.19:0/1827266178 conn(0x55f89376c1c0 msgr2=0x55f89388f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:33.219 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe741ffb700 0 client.26200 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 shutdown_connections 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 1 -- 172.21.15.19:0/1827266178 wait complete. 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.216+0000 7fe76f3ecb80 20 client.26200 trim_cache size 0 max 16384 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe7440b2900 msgr2=0x7fe7440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe7440b2900 0x7fe7440b4e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009cb0 tx=0x7fe74c009260 comp rx=0 tx=0).stop 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c107de0 msgr2=0x7fe75c19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c107de0 0x7fe75c19d500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe75000e9c0 tx=0x7fe75000ecd0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 shutdown_connections 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe75c108730 0x7fe75c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75c1034b0 0x7fe75c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe7440b2900 0x7fe7440b4e10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 --2- 172.21.15.19:0/1175203926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe75c107de0 0x7fe75c19d500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 >> 172.21.15.19:0/1175203926 conn(0x7fe75c0fede0 msgr2=0x7fe75c109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 shutdown_connections 2024-05-17T09:23:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:33.217+0000 7fe7609d7700 1 -- 172.21.15.19:0/1175203926 wait complete. 2024-05-17T09:23:33.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 33604 2.18 MB/sec warmup 91 sec latency 19.165 ms 2024-05-17T09:23:34.084 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 3487705699, 'ino': 1099511628991, 'path': '/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC'}] 2024-05-17T09:23:34.084 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-05-17T09:23:34.422 INFO:tasks.workunit.client.0.smithi019.stdout: 1 34224 2.16 MB/sec warmup 92 sec latency 22.861 ms 2024-05-17T09:23:34.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:34 smithi019 ceph-mon[31620]: pgmap v552: 129 pgs: 129 active+clean; 30 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 861 KiB/s wr, 101 op/s 2024-05-17T09:23:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:34 smithi069 ceph-mon[35081]: pgmap v552: 129 pgs: 129 active+clean; 30 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 861 KiB/s wr, 101 op/s 2024-05-17T09:23:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:34 smithi079 ceph-mon[35502]: pgmap v552: 129 pgs: 129 active+clean; 30 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 861 KiB/s wr, 101 op/s 2024-05-17T09:23:35.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 34739 2.19 MB/sec warmup 93 sec latency 15.296 ms 2024-05-17T09:23:36.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35293 2.21 MB/sec warmup 94 sec latency 16.647 ms 2024-05-17T09:23:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:36 smithi069 ceph-mon[35081]: pgmap v553: 129 pgs: 129 active+clean; 29 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-17T09:23:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:36 smithi019 ceph-mon[31620]: pgmap v553: 129 pgs: 129 active+clean; 29 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-17T09:23:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:36 smithi079 ceph-mon[35502]: pgmap v553: 129 pgs: 129 active+clean; 29 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-17T09:23:37.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35756 2.23 MB/sec warmup 95 sec latency 17.037 ms 2024-05-17T09:23:38.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36011 2.21 MB/sec warmup 96 sec latency 13.495 ms 2024-05-17T09:23:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:38 smithi069 ceph-mon[35081]: pgmap v554: 129 pgs: 129 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-17T09:23:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:38 smithi019 ceph-mon[31620]: pgmap v554: 129 pgs: 129 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-17T09:23:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:38 smithi079 ceph-mon[35502]: pgmap v554: 129 pgs: 129 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-17T09:23:39.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36409 2.19 MB/sec warmup 97 sec latency 15.830 ms 2024-05-17T09:23:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 -- 172.21.15.19:0/1532193851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc103d80 msgr2=0x7fb4dc104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/1532193851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc103d80 0x7fb4dc104190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009900 tx=0x7fb4d002a3f0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 -- 172.21.15.19:0/1532193851 shutdown_connections 2024-05-17T09:23:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/1532193851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4dc108a80 0x7fb4dc10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/1532193851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc103d80 0x7fb4dc104190 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/1532193851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 -- 172.21.15.19:0/1532193851 >> 172.21.15.19:0/1532193851 conn(0x7fb4dc0fdd30 msgr2=0x7fb4dc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 -- 172.21.15.19:0/1532193851 shutdown_connections 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 -- 172.21.15.19:0/1532193851 wait complete. 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.566+0000 7fb4e11f3640 1 Processor -- start 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 -- start start 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4dc103d80 0x7fb4dc19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc108a80 0x7fb4dc1a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc10e4f0 con 0x7fb4dc108a80 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc10e370 con 0x7fb4dc102b40 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4e11f3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc10e670 con 0x7fb4dc103d80 2024-05-17T09:23:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc108a80 0x7fb4dc1a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49172/0 (socket says 172.21.15.19:49172) 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc108a80 0x7fb4dc1a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51768/0 (socket says 172.21.15.19:51768) 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 -- 172.21.15.19:0/2105778250 learned_addr learned my addr 172.21.15.19:0/2105778250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4da59c640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4dc103d80 0x7fb4dc19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 -- 172.21.15.19:0/2105778250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4dc103d80 msgr2=0x7fb4dc19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4dc103d80 0x7fb4dc19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 -- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc108a80 msgr2=0x7fb4dc1a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc108a80 0x7fb4dc1a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.567+0000 7fb4dad9d640 1 -- 172.21.15.19:0/2105778250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc1a3e70 con 0x7fb4dc102b40 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.568+0000 7fb4dad9d640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc19c3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e9f0 tx=0x7fb4cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.568+0000 7fb4c3fff640 1 -- 172.21.15.19:0/2105778250 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00ccc0 con 0x7fb4dc102b40 2024-05-17T09:23:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.568+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc1a4160 con 0x7fb4dc102b40 2024-05-17T09:23:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.568+0000 7fb4c3fff640 1 -- 172.21.15.19:0/2105778250 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4cc0044b0 con 0x7fb4dc102b40 2024-05-17T09:23:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.568+0000 7fb4c3fff640 1 -- 172.21.15.19:0/2105778250 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc010430 con 0x7fb4dc102b40 2024-05-17T09:23:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.568+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc1a4700 con 0x7fb4dc102b40 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.570+0000 7fb4e8181740 1 Processor -- start 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 -- start start 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4c3fff640 1 -- 172.21.15.19:0/2105778250 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb4cc0105d0 con 0x7fb4dc102b40 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639772f0 0x55f163d41920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d48820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f163cb3c30 con 0x55f1639a01b0 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f163ca37d0 con 0x55f16397f4d0 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4e8181740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f163ca7810 con 0x55f1639772f0 2024-05-17T09:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51780/0 (socket says 172.21.15.19:51780) 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 -- 172.21.15.19:0/1538635580 learned_addr learned my addr 172.21.15.19:0/1538635580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4da59c640 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d48820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4dad9d640 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639772f0 0x55f163d41920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 -- 172.21.15.19:0/1538635580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639772f0 msgr2=0x55f163d41920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639772f0 0x55f163d41920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 -- 172.21.15.19:0/1538635580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 msgr2=0x55f163d48820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d48820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.571+0000 7fb4db59e640 1 -- 172.21.15.19:0/1538635580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1639b8210 con 0x55f1639a01b0 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4db59e640 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d413a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4007f40 tx=0x7fb4d400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11b7d02a2ed1cab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4c17fa640 1 -- 172.21.15.19:0/1538635580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4010900 con 0x55f1639a01b0 2024-05-17T09:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4c17fa640 1 -- 172.21.15.19:0/1538635580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4d40183f0 con 0x55f1639a01b0 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4c17fa640 1 -- 172.21.15.19:0/1538635580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4018e60 con 0x55f1639a01b0 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4c3fff640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4c40b28a0 0x7fb4c40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4dad9d640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4c40b28a0 0x7fb4c40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.572+0000 7fb4c3fff640 1 -- 172.21.15.19:0/2105778250 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4cc0e2990 con 0x7fb4dc102b40 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 -- 172.21.15.19:0/1538635580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 msgr2=0x55f163d413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d413a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4007f40 tx=0x7fb4d400f850 comp rx=0 tx=0).stop 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4dad9d640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4c40b28a0 0x7fb4c40b4db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55f163d42780 tx=0x7fb4cc003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 -- 172.21.15.19:0/1538635580 shutdown_connections 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d48820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639772f0 0x55f163d41920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 --2- 172.21.15.19:0/1538635580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d413a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 -- 172.21.15.19:0/1538635580 >> 172.21.15.19:0/1538635580 conn(0x55f163994300 msgr2=0x55f163d3cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 -- 172.21.15.19:0/1538635580 shutdown_connections 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 -- 172.21.15.19:0/1538635580 wait complete. 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 -- start start 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.573+0000 7fb4e8181740 1 Processor -- start 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d4acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d4b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d52150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f163989790 con 0x55f1639a01b0 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f163c5f890 con 0x55f1639772f0 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f163d49270 con 0x55f16397f4d0 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4e8181740 10 client.0 _ll_register_callbacks cb 0x55f163c62d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d52150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d52150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51796/0 (socket says 172.21.15.19:51796) 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 -- 172.21.15.19:0/3506862044 learned_addr learned my addr 172.21.15.19:0/3506862044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4da59c640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d4b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d4acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 -- 172.21.15.19:0/3506862044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 msgr2=0x55f163d4b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d4b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 -- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 msgr2=0x55f163d4acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d4acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 -- 172.21.15.19:0/3506862044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f163a84620 con 0x55f1639a01b0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4db59e640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d52150 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40077d0 tx=0x7fb4d4012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75660e7f28ecee60 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.574+0000 7fb4beff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4beff5640 1 -- 172.21.15.19:0/3506862044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d402c060 con 0x55f1639a01b0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4beff5640 1 -- 172.21.15.19:0/3506862044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4d4004c40 con 0x55f1639a01b0 2024-05-17T09:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d40183f0 con 0x55f1639a01b0 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4beff5640 1 -- 172.21.15.19:0/3506862044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4d4033050 con 0x55f1639a01b0 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 10 client.16545 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f163d55080 con 0x55f1639a01b0 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4beff5640 1 -- 172.21.15.19:0/3506862044 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4d40183f0 con 0x55f1639a01b0 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4beff5640 1 -- 172.21.15.19:0/3506862044 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb4d4024040 con 0x55f1639a01b0 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 10 client.16545 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 20 client.16545 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f16395ed80 0x55f163d58550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 4 client.16545 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.575+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f163cb5b80 con 0x55f16395ed80 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.576+0000 7fb4da59c640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f16395ed80 0x55f163d58550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.576+0000 7fb4beff5640 10 client.16545 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.576+0000 7fb4da59c640 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f16395ed80 0x55f163d58550 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6c6c805cdf7b41 server_cookie=7841c7cdec43ef25 in_seq=0 out_seq=0 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4beff5640 1 -- 172.21.15.19:0/3506862044 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f163cb5b80 con 0x55f16395ed80 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4beff5640 10 client.16545 handle_command_reply: tid=0 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4e8181740 1 client.16545 shutdown 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 msgr2=0x55f163d52150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4e8181740 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d52150 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40077d0 tx=0x7fb4d4012040 comp rx=0 tx=0).stop 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 shutdown_connections 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.577+0000 7fb4e8181740 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f16395ed80 0x55f163d58550 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163d52150 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d4b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4beff5640 0 client.16545 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 1 --2- 172.21.15.19:0/3506862044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d4acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 >> 172.21.15.19:0/3506862044 conn(0x55f163994300 msgr2=0x55f163d3fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 shutdown_connections 2024-05-17T09:23:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 1 -- 172.21.15.19:0/3506862044 wait complete. 2024-05-17T09:23:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.578+0000 7fb4e8181740 20 client.16545 trim_cache size 0 max 16384 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.593+0000 7fb4e8181740 1 Processor -- start 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.593+0000 7fb4e8181740 1 -- start start 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.593+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 0x55f163d457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.593+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d52070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.593+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639a01b0 0x55f163d525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.593+0000 7fb4e8181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1b1520 con 0x55f1639772f0 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4e8181740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1aedf0 con 0x55f16397f4d0 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4e8181740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1acc40 con 0x55f1639a01b0 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 0x55f163d457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 0x55f163d457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51804/0 (socket says 172.21.15.19:51804) 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 -- 172.21.15.19:0/726910866 learned_addr learned my addr 172.21.15.19:0/726910866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4da59c640 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d52070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639a01b0 0x55f163d525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 -- 172.21.15.19:0/726910866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639a01b0 msgr2=0x55f163d525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639a01b0 0x55f163d525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 -- 172.21.15.19:0/726910866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 msgr2=0x55f163d52070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d52070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 -- 172.21.15.19:0/726910866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f163d38540 con 0x55f1639772f0 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4da59c640 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d52070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4db59e640 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 0x55f163d457d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4018eb0 tx=0x7fb4d401af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4f157ba6db0889 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4c0ff9640 1 -- 172.21.15.19:0/726910866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d402c040 con 0x55f1639772f0 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4c0ff9640 1 -- 172.21.15.19:0/726910866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4d400fd50 con 0x55f1639772f0 2024-05-17T09:23:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.594+0000 7fb4c0ff9640 1 -- 172.21.15.19:0/726910866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4005250 con 0x55f1639772f0 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 -- 172.21.15.19:0/726910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 msgr2=0x55f163d457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 0x55f163d457d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4018eb0 tx=0x7fb4d401af70 comp rx=0 tx=0).stop 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 -- 172.21.15.19:0/726910866 shutdown_connections 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1639a01b0 0x55f163d525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f16397f4d0 0x55f163d52070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 --2- 172.21.15.19:0/726910866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639772f0 0x55f163d457d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 -- 172.21.15.19:0/726910866 >> 172.21.15.19:0/726910866 conn(0x55f163994300 msgr2=0x55f163d3fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 -- 172.21.15.19:0/726910866 shutdown_connections 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.595+0000 7fb4e8181740 1 -- 172.21.15.19:0/726910866 wait complete. 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 -- start start 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 Processor -- start 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d51940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163dbeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1ad2b0 con 0x55f1639a01b0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1ae720 con 0x55f1639772f0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1ad980 con 0x55f16397f4d0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4e8181740 10 client.0 _ll_register_callbacks cb 0x7fb4d40189f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4da59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d51940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4db59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d51400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49216/0 (socket says 172.21.15.19:49216) 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4db59e640 1 -- 172.21.15.19:0/2814390551 learned_addr learned my addr 172.21.15.19:0/2814390551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.596+0000 7fb4db59e640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163dbeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4da59c640 1 -- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 msgr2=0x55f163d51400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4da59c640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d51400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4da59c640 1 -- 172.21.15.19:0/2814390551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 msgr2=0x55f163dbeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4da59c640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163dbeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4da59c640 1 -- 172.21.15.19:0/2814390551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f163d52130 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4db59e640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4da59c640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d51940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc19d480 tx=0x7fb4d0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2371b783efc7fe2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4beff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4beff5640 1 -- 172.21.15.19:0/2814390551 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0035040 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4beff5640 1 -- 172.21.15.19:0/2814390551 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4d0031c90 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d400fd50 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4beff5640 1 -- 172.21.15.19:0/2814390551 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4d0055050 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4e8181740 10 client.26215 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.597+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f163dc0110 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4beff5640 1 -- 172.21.15.19:0/2814390551 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4d400fd50 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4beff5640 1 -- 172.21.15.19:0/2814390551 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb4d0046320 con 0x55f16397f4d0 2024-05-17T09:23:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4e8181740 10 client.26215 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4e8181740 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4e8181740 20 client.26215 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4e8181740 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1639341c0 0x55f163dc2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4e8181740 4 client.26215 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f163caf360 con 0x55f1639341c0 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4db59e640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1639341c0 0x55f163dc2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4beff5640 10 client.26215 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.598+0000 7fb4db59e640 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1639341c0 0x55f163dc2be0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12a10e9dcd3b69f server_cookie=3837ab1d3ca84bfe in_seq=0 out_seq=0 2024-05-17T09:23:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.619+0000 7fb4beff5640 1 -- 172.21.15.19:0/2814390551 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f163caf360 con 0x55f1639341c0 2024-05-17T09:23:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.619+0000 7fb4beff5640 10 client.26215 handle_command_reply: tid=0 2024-05-17T09:23:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.619+0000 7fb4e8181740 1 client.26215 shutdown 2024-05-17T09:23:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 msgr2=0x55f163d51940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d51940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc19d480 tx=0x7fb4d0004270 comp rx=0 tx=0).stop 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 shutdown_connections 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f1639341c0 0x55f163dc2be0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1639a01b0 0x55f163dbeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4beff5640 0 client.26215 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f16397f4d0 0x55f163d51940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 --2- 172.21.15.19:0/2814390551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1639772f0 0x55f163d51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 >> 172.21.15.19:0/2814390551 conn(0x55f163994300 msgr2=0x55f163d447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 shutdown_connections 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 1 -- 172.21.15.19:0/2814390551 wait complete. 2024-05-17T09:23:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e8181740 20 client.26215 trim_cache size 0 max 16384 2024-05-17T09:23:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:23:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:23:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4c40b28a0 msgr2=0x7fb4c40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4c40b28a0 0x7fb4c40b4db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55f163d42780 tx=0x7fb4cc003f90 comp rx=0 tx=0).stop 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 msgr2=0x7fb4dc19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.620+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc19c3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e9f0 tx=0x7fb4cc00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 shutdown_connections 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4c40b28a0 0x7fb4c40b4db0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc108a80 0x7fb4dc1a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4dc103d80 0x7fb4dc19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 --2- 172.21.15.19:0/2105778250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4dc102b40 0x7fb4dc19c3e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 >> 172.21.15.19:0/2105778250 conn(0x7fb4dc0fdd30 msgr2=0x7fb4dc100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 shutdown_connections 2024-05-17T09:23:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:39.621+0000 7fb4e11f3640 1 -- 172.21.15.19:0/2105778250 wait complete. 2024-05-17T09:23:40.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36912 2.22 MB/sec warmup 98 sec latency 21.187 ms 2024-05-17T09:23:40.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:40 smithi069 ceph-mon[35081]: pgmap v555: 129 pgs: 129 active+clean; 31 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:23:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:40 smithi019 ceph-mon[31620]: pgmap v555: 129 pgs: 129 active+clean; 31 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:23:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:40 smithi079 ceph-mon[35502]: pgmap v555: 129 pgs: 129 active+clean; 31 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:23:41.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 37503 2.20 MB/sec warmup 99 sec latency 20.575 ms 2024-05-17T09:23:42.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 38006 2.20 MB/sec warmup 100 sec latency 11.905 ms 2024-05-17T09:23:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:42 smithi069 ceph-mon[35081]: pgmap v556: 129 pgs: 129 active+clean; 31 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-17T09:23:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:42 smithi019 ceph-mon[31620]: pgmap v556: 129 pgs: 129 active+clean; 31 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-17T09:23:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:42 smithi079 ceph-mon[35502]: pgmap v556: 129 pgs: 129 active+clean; 31 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-17T09:23:43.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 38781 2.24 MB/sec warmup 101 sec latency 15.467 ms 2024-05-17T09:23:44.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39172 2.25 MB/sec warmup 102 sec latency 14.834 ms 2024-05-17T09:23:44.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:44 smithi019 ceph-mon[31620]: pgmap v557: 129 pgs: 129 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-17T09:23:44.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:44 smithi069 ceph-mon[35081]: pgmap v557: 129 pgs: 129 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-17T09:23:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:44 smithi079 ceph-mon[35502]: pgmap v557: 129 pgs: 129 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-17T09:23:45.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39462 2.25 MB/sec warmup 103 sec latency 13.414 ms 2024-05-17T09:23:46.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39788 2.23 MB/sec warmup 104 sec latency 14.264 ms 2024-05-17T09:23:46.468 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.9 2024-05-17T09:23:46.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:23:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:46 smithi069 ceph-mon[35081]: pgmap v558: 129 pgs: 129 active+clean; 30 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-17T09:23:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:46 smithi019 ceph-mon[31620]: pgmap v558: 129 pgs: 129 active+clean; 30 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-17T09:23:46.744 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:46 smithi079 ceph-mon[35502]: pgmap v558: 129 pgs: 129 active+clean; 30 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-17T09:23:47.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 40242 2.22 MB/sec warmup 105 sec latency 15.288 ms 2024-05-17T09:23:48.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 40827 2.23 MB/sec warmup 106 sec latency 16.437 ms 2024-05-17T09:23:48.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:48 smithi019 ceph-mon[31620]: pgmap v559: 129 pgs: 129 active+clean; 33 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:23:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:48 smithi069 ceph-mon[35081]: pgmap v559: 129 pgs: 129 active+clean; 33 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:23:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 -- 172.21.15.19:0/3956140269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c109850 msgr2=0x7f963c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 --2- 172.21.15.19:0/3956140269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c109850 0x7f963c1111b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f963400b0d0 tx=0x7f963400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 -- 172.21.15.19:0/3956140269 shutdown_connections 2024-05-17T09:23:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 --2- 172.21.15.19:0/3956140269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c109850 0x7f963c1111b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 --2- 172.21.15.19:0/3956140269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 0x7f963c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:48.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 --2- 172.21.15.19:0/3956140269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f963c108f00 0x7f963c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:48.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 -- 172.21.15.19:0/3956140269 >> 172.21.15.19:0/3956140269 conn(0x7f963c0ffec0 msgr2=0x7f963c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:48.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 -- 172.21.15.19:0/3956140269 shutdown_connections 2024-05-17T09:23:48.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.745+0000 7f9641182700 1 -- 172.21.15.19:0/3956140269 wait complete. 2024-05-17T09:23:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.746+0000 7f9641182700 1 Processor -- start 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.746+0000 7f9641182700 1 -- start start 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f9641182700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 0x7f963c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f9641182700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 0x7f963c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f9641182700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f963c109850 0x7f963c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f9641182700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f963c114330 con 0x7f963c108f00 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f9641182700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1141b0 con 0x7f963c1045d0 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f9641182700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1144b0 con 0x7f963c109850 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f963a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 0x7f963c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f963ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 0x7f963c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f963a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 0x7f963c19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53768/0 (socket says 172.21.15.19:53768) 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f963a59c700 1 -- 172.21.15.19:0/3353613219 learned_addr learned my addr 172.21.15.19:0/3353613219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.747+0000 7f963b59e700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f963c109850 0x7f963c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963ad9d700 1 -- 172.21.15.19:0/3353613219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f963c109850 msgr2=0x7f963c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963ad9d700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f963c109850 0x7f963c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963ad9d700 1 -- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 msgr2=0x7f963c19d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963ad9d700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 0x7f963c19d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963ad9d700 1 -- 172.21.15.19:0/3353613219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1a1e30 con 0x7f963c1045d0 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963a59c700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 0x7f963c19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f963ad9d700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 0x7f963c19cf60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f962c00d750 tx=0x7f962c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c014030 con 0x7f963c1045d0 2024-05-17T09:23:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f962c0044b0 con 0x7f963c1045d0 2024-05-17T09:23:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1a2120 con 0x7f963c1045d0 2024-05-17T09:23:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c005260 con 0x7f963c1045d0 2024-05-17T09:23:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.748+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a9a40 con 0x7f963c1045d0 2024-05-17T09:23:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.749+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c105a80 con 0x7f963c1045d0 2024-05-17T09:23:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.751+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f962c0049e0 con 0x7f963c1045d0 2024-05-17T09:23:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.752+0000 7f9623fff700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f96240b2880 0x7f96240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.752+0000 7f963a59c700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f96240b2880 0x7f96240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.752+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f962c0e24f0 con 0x7f963c1045d0 2024-05-17T09:23:48.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.752+0000 7f963a59c700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f96240b2880 0x7f96240b4d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f963c1104d0 tx=0x7f9630009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:48.755+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f962c0a9e60 con 0x7f963c1045d0 2024-05-17T09:23:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:48 smithi079 ceph-mon[35502]: pgmap v559: 129 pgs: 129 active+clean; 33 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:23:49.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.056+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963c0512b0 con 0x7f963c1045d0 2024-05-17T09:23:49.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.057+0000 7f9623fff700 1 -- 172.21.15.19:0/3353613219 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7f963c0512b0 con 0x7f963c1045d0 2024-05-17T09:23:49.058 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:23:49.058 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:22:41.259768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:23:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.059+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f96240b2880 msgr2=0x7f96240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.059+0000 7f9641182700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f96240b2880 0x7f96240b4d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f963c1104d0 tx=0x7f9630009260 comp rx=0 tx=0).stop 2024-05-17T09:23:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.059+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 msgr2=0x7f963c19cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.059+0000 7f9641182700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 0x7f963c19cf60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f962c00d750 tx=0x7f962c00da60 comp rx=0 tx=0).stop 2024-05-17T09:23:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 shutdown_connections 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f96240b2880 0x7f96240b4d90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f963c109850 0x7f963c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c1045d0 0x7f963c19cf60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 --2- 172.21.15.19:0/3353613219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f963c108f00 0x7f963c19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 >> 172.21.15.19:0/3353613219 conn(0x7f963c0ffec0 msgr2=0x7f963c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 shutdown_connections 2024-05-17T09:23:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:49.060+0000 7f9641182700 1 -- 172.21.15.19:0/3353613219 wait complete. 2024-05-17T09:23:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-17T09:23:49.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 41392 2.23 MB/sec warmup 107 sec latency 30.596 ms 2024-05-17T09:23:49.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:49 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3353613219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:49.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:49 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3353613219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:49.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:49 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3353613219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:23:49.836 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 132.9 --await-for 45.0' 2024-05-17T09:23:49.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 132.9 --await-for 45.0 2024-05-17T09:23:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:23:50.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 41834 2.24 MB/sec warmup 108 sec latency 30.391 ms 2024-05-17T09:23:50.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:50 smithi019 ceph-mon[31620]: pgmap v560: 129 pgs: 129 active+clean; 34 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:23:50.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:50 smithi069 ceph-mon[35081]: pgmap v560: 129 pgs: 129 active+clean; 34 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:23:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:50 smithi079 ceph-mon[35502]: pgmap v560: 129 pgs: 129 active+clean; 34 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:23:51.423 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42389 2.25 MB/sec warmup 109 sec latency 11.578 ms 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 -- 172.21.15.19:0/1854862473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108730 msgr2=0x7f3d9c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 --2- 172.21.15.19:0/1854862473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108730 0x7f3d9c110090 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400b0d0 tx=0x7f3d9400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 -- 172.21.15.19:0/1854862473 shutdown_connections 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 --2- 172.21.15.19:0/1854862473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108730 0x7f3d9c110090 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 --2- 172.21.15.19:0/1854862473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 --2- 172.21.15.19:0/1854862473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c107de0 0x7f3d9c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.214+0000 7f3da412a700 1 -- 172.21.15.19:0/1854862473 >> 172.21.15.19:0/1854862473 conn(0x7f3d9c0fedb0 msgr2=0x7f3d9c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.215+0000 7f3da412a700 1 -- 172.21.15.19:0/1854862473 shutdown_connections 2024-05-17T09:23:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.215+0000 7f3da412a700 1 -- 172.21.15.19:0/1854862473 wait complete. 2024-05-17T09:23:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 Processor -- start 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 -- start start 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c107de0 0x7f3d9c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c108730 0x7f3d9c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c113210 con 0x7f3d9c1034b0 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c113090 con 0x7f3d9c108730 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da412a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c113390 con 0x7f3d9c107de0 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53800/0 (socket says 172.21.15.19:53800) 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2110091454 learned_addr learned my addr 172.21.15.19:0/2110091454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c108730 0x7f3d9c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.216+0000 7f3da16c5700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c107de0 0x7f3d9c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2110091454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c107de0 msgr2=0x7f3d9c19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c107de0 0x7f3d9c19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2110091454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c108730 msgr2=0x7f3d9c19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c108730 0x7f3d9c19c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2110091454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c1a6e60 con 0x7f3d9c1034b0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c108730 0x7f3d9c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c19baf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800e9a0 tx=0x7f3d9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3d92ffd700 1 -- 172.21.15.19:0/2110091454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800ca10 con 0x7f3d9c1034b0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3d92ffd700 1 -- 172.21.15.19:0/2110091454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d9800cbb0 con 0x7f3d9c1034b0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c199940 con 0x7f3d9c1034b0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3d92ffd700 1 -- 172.21.15.19:0/2110091454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98018680 con 0x7f3d9c1034b0 2024-05-17T09:23:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.217+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c199ed0 con 0x7f3d9c1034b0 2024-05-17T09:23:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.220+0000 7f3d92ffd700 1 -- 172.21.15.19:0/2110091454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3d98018820 con 0x7f3d9c1034b0 2024-05-17T09:23:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.221+0000 7f3db2b3fb80 1 Processor -- start 2024-05-17T09:23:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3d92ffd700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3d880b2900 0x7f3d880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 -- start start 2024-05-17T09:23:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3d880b2900 0x7f3d880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3d92ffd700 1 -- 172.21.15.19:0/2110091454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3d980e2740 con 0x7f3d9c1034b0 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd5953f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd59544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd594e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bd587bf00 con 0x559bd57cefd0 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559bd586baa0 con 0x559bd594e0b0 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559bd586fae0 con 0x559bd58ba5c0 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd5953f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3da16c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd59544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd5953f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42510/0 (socket says 172.21.15.19:42510) 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.222+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2579002222 learned_addr learned my addr 172.21.15.19:0/2579002222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2579002222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 msgr2=0x559bd594e580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd594e580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2579002222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 msgr2=0x559bd59544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd59544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da1ec6700 1 -- 172.21.15.19:0/2579002222 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd5815140 con 0x559bd58ba5c0 2024-05-17T09:23:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3d880b2900 0x7f3d880b4e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009900 tx=0x7f3d9400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd5953f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d980a96b0 tx=0x7f3d980a9510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d5c4905f648f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3d87fff700 1 -- 172.21.15.19:0/2579002222 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980a99b0 con 0x559bd58ba5c0 2024-05-17T09:23:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3d87fff700 1 -- 172.21.15.19:0/2579002222 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980add60 con 0x559bd58ba5c0 2024-05-17T09:23:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.223+0000 7f3d87fff700 1 -- 172.21.15.19:0/2579002222 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980b2580 con 0x559bd58ba5c0 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2579002222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 msgr2=0x559bd5953f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd5953f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d980a96b0 tx=0x7f3d980a9510 comp rx=0 tx=0).stop 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2579002222 shutdown_connections 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd5953f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd594e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2579002222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd59544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2579002222 >> 172.21.15.19:0/2579002222 conn(0x559bd582f350 msgr2=0x559bd5948d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2579002222 shutdown_connections 2024-05-17T09:23:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.224+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2579002222 wait complete. 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 -- start start 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 Processor -- start 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5957bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd58ba5c0 0x559bd5958110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5958650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bd5772130 con 0x559bd57cefd0 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559bd57d3f30 con 0x559bd58ba5c0 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559bd595abb0 con 0x559bd594e0b0 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.225+0000 7f3db2b3fb80 10 client.0 _ll_register_callbacks cb 0x559bd5806ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5957bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd58ba5c0 0x559bd5958110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5957bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53814/0 (socket says 172.21.15.19:53814) 2024-05-17T09:23:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da1ec6700 1 -- 172.21.15.19:0/778215182 learned_addr learned my addr 172.21.15.19:0/778215182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da26c7700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5958650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 -- 172.21.15.19:0/778215182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 msgr2=0x559bd5958650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5958650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 -- 172.21.15.19:0/778215182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 msgr2=0x559bd5957bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5957bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 -- 172.21.15.19:0/778215182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd57bba60 con 0x559bd58ba5c0 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5957bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da26c7700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5958650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3da16c5700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd58ba5c0 0x559bd5958110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c106820 tx=0x7f3d8c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91697ca083468898 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3d857fa700 1 -- 172.21.15.19:0/778215182 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00d480 con 0x559bd58ba5c0 2024-05-17T09:23:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d980add60 con 0x559bd58ba5c0 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.226+0000 7f3d857fa700 1 -- 172.21.15.19:0/778215182 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3d8c00d620 con 0x559bd58ba5c0 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3d857fa700 1 -- 172.21.15.19:0/778215182 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3d8c00d820 con 0x559bd58ba5c0 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 10 client.26237 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bd58258f0 con 0x559bd58ba5c0 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3d857fa700 1 -- 172.21.15.19:0/778215182 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980add60 con 0x559bd58ba5c0 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3d857fa700 1 -- 172.21.15.19:0/778215182 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3d8c019070 con 0x559bd58ba5c0 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 10 client.26237 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 10 client.26237 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 20 client.26237 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd5970320 0x559bd5972740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 4 client.26237 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.227+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd591a060 con 0x559bd5970320 2024-05-17T09:23:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.228+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd5970320 0x559bd5972740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.228+0000 7f3d857fa700 10 client.26237 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.228+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd5970320 0x559bd5972740 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815122f7665ad096 server_cookie=3b7076b0bb3fa26f in_seq=0 out_seq=0 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3d857fa700 1 -- 172.21.15.19:0/778215182 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559bd591a060 con 0x559bd5970320 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3d857fa700 10 client.26237 handle_command_reply: tid=0 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 client.26237 shutdown 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd58ba5c0 msgr2=0x559bd5958110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd58ba5c0 0x559bd5958110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c106820 tx=0x7f3d8c009c60 comp rx=0 tx=0).stop 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 shutdown_connections 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5958650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd5970320 0x559bd5972740 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5957bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/778215182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd58ba5c0 0x559bd5958110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3d857fa700 0 client.26237 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 >> 172.21.15.19:0/778215182 conn(0x559bd582f350 msgr2=0x559bd594d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 shutdown_connections 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/778215182 wait complete. 2024-05-17T09:23:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.229+0000 7f3db2b3fb80 20 client.26237 trim_cache size 0 max 16384 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.253+0000 7f3db2b3fb80 1 Processor -- start 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.253+0000 7f3db2b3fb80 1 -- start start 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5943680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd596c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd5977ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1b0aa0 con 0x559bd57cefd0 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1ae330 con 0x559bd594e0b0 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1ac180 con 0x559bd58ba5c0 2024-05-17T09:23:52.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5943680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da16c5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd596c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5943680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53826/0 (socket says 172.21.15.19:53826) 2024-05-17T09:23:52.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 -- 172.21.15.19:0/3766046192 learned_addr learned my addr 172.21.15.19:0/3766046192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da16c5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd596c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42528/0 (socket says 172.21.15.19:42528) 2024-05-17T09:23:52.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da26c7700 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd5977ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 -- 172.21.15.19:0/3766046192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 msgr2=0x559bd596c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd596c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 -- 172.21.15.19:0/3766046192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 msgr2=0x559bd5977ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd5977ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 -- 172.21.15.19:0/3766046192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd59451c0 con 0x559bd57cefd0 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5943680 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3d980e0200 tx=0x7f3d9801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35c5129963b7e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.254+0000 7f3d877fe700 1 -- 172.21.15.19:0/3766046192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980a99b0 con 0x559bd57cefd0 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3d877fe700 1 -- 172.21.15.19:0/3766046192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980a5fa0 con 0x559bd57cefd0 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3d877fe700 1 -- 172.21.15.19:0/3766046192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98018b10 con 0x559bd57cefd0 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/3766046192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 msgr2=0x559bd5943680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5943680 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3d980e0200 tx=0x7f3d9801f000 comp rx=0 tx=0).stop 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/3766046192 shutdown_connections 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd594e0b0 0x559bd5977ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd57cefd0 0x559bd5943680 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/3766046192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd58ba5c0 0x559bd596c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.255+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/3766046192 >> 172.21.15.19:0/3766046192 conn(0x559bd582f350 msgr2=0x559bd5951ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.256+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/3766046192 shutdown_connections 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.256+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/3766046192 wait complete. 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.256+0000 7f3db2b3fb80 1 -- start start 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.256+0000 7f3db2b3fb80 1 Processor -- start 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd57cefd0 0x559bd5968830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd58ba5c0 0x559bd5968d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5965610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1ac7f0 con 0x559bd58ba5c0 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1adc60 con 0x559bd57cefd0 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c1acec0 con 0x559bd594e0b0 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da16c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd58ba5c0 0x559bd5968d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5965610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5965610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42542/0 (socket says 172.21.15.19:42542) 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 -- 172.21.15.19:0/2090485433 learned_addr learned my addr 172.21.15.19:0/2090485433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da16c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd58ba5c0 0x559bd5968d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53840/0 (socket says 172.21.15.19:53840) 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3db2b3fb80 10 client.0 _ll_register_callbacks cb 0x559bd5806ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd57cefd0 0x559bd5968830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 -- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd57cefd0 msgr2=0x559bd5968830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd57cefd0 0x559bd5968830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 -- 172.21.15.19:0/2090485433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd58ba5c0 msgr2=0x559bd5968d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd58ba5c0 0x559bd5968d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 -- 172.21.15.19:0/2090485433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd5965db0 con 0x559bd594e0b0 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd57cefd0 0x559bd5968830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3da26c7700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5965610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400d010 tx=0x7f3d9400b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f816316091b27801 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.257+0000 7f3d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:23:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3d857fa700 1 -- 172.21.15.19:0/2090485433 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9402aad0 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3d857fa700 1 -- 172.21.15.19:0/2090485433 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3d9402ac70 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d980a5fa0 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3d857fa700 1 -- 172.21.15.19:0/2090485433 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3d9401c410 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 10 client.26239 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bd59db820 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3d857fa700 1 -- 172.21.15.19:0/2090485433 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980a5fa0 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3d857fa700 1 -- 172.21.15.19:0/2090485433 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3d94011070 con 0x559bd594e0b0 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 10 client.26239 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 10 client.26239 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 20 client.26239 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd59dbff0 0x559bd59de410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 4 client.26239 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.9, "await_for": 45.0}] 2024-05-17T09:23:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.9, "await_for": 45.0}) -- 0x559bd587de50 con 0x559bd59dbff0 2024-05-17T09:23:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.258+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd59dbff0 0x559bd59de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.259+0000 7f3d857fa700 10 client.26239 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.259+0000 7f3da1ec6700 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd59dbff0 0x559bd59de410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda4acbe16023a49 server_cookie=783bd8f1e1524f87 in_seq=0 out_seq=0 2024-05-17T09:23:52.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3d857fa700 1 -- 172.21.15.19:0/2090485433 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559bd587de50 con 0x559bd59dbff0 2024-05-17T09:23:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3d857fa700 10 client.26239 handle_command_reply: tid=0 2024-05-17T09:23:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 client.26239 shutdown 2024-05-17T09:23:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 msgr2=0x559bd5965610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5965610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400d010 tx=0x7f3d9400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 shutdown_connections 2024-05-17T09:23:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559bd594e0b0 0x559bd5965610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559bd59dbff0 0x559bd59de410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559bd57cefd0 0x559bd5968830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 --2- 172.21.15.19:0/2090485433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bd58ba5c0 0x559bd5968d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3d857fa700 0 client.26239 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 >> 172.21.15.19:0/2090485433 conn(0x559bd582f350 msgr2=0x559bd5951ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 shutdown_connections 2024-05-17T09:23:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.361+0000 7f3db2b3fb80 1 -- 172.21.15.19:0/2090485433 wait complete. 2024-05-17T09:23:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3db2b3fb80 20 client.26239 trim_cache size 0 max 16384 2024-05-17T09:23:52.365 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:23:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-05-17T09:23:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:23:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 14, 2024-05-17T09:23:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:23:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "65b87bed": { 2024-05-17T09:23:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 14, 2024-05-17T09:23:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 132.9, 2024-05-17T09:23:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:23:52.369 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3d880b2900 msgr2=0x7f3d880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3d880b2900 0x7f3d880b4e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009900 tx=0x7f3d9400b040 comp rx=0 tx=0).stop 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 msgr2=0x7f3d9c19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c19baf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800e9a0 tx=0x7f3d9800ecb0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 shutdown_connections 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d9c108730 0x7f3d9c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c1034b0 0x7f3d9c19baf0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3d880b2900 0x7f3d880b4e10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 --2- 172.21.15.19:0/2110091454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d9c107de0 0x7f3d9c19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 >> 172.21.15.19:0/2110091454 conn(0x7f3d9c0fedb0 msgr2=0x7f3d9c109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 shutdown_connections 2024-05-17T09:23:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:23:52.362+0000 7f3da412a700 1 -- 172.21.15.19:0/2110091454 wait complete. 2024-05-17T09:23:52.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.27 MB/sec warmup 110 sec latency 103.678 ms 2024-05-17T09:23:52.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:52 smithi069 ceph-mon[35081]: pgmap v561: 129 pgs: 129 active+clean; 34 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-17T09:23:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:52 smithi019 ceph-mon[31620]: pgmap v561: 129 pgs: 129 active+clean; 34 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-17T09:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 -- 172.21.15.19:0/2522994422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 msgr2=0x7f7a6c10ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 --2- 172.21.15.19:0/2522994422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c10ae80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009be0 tx=0x7f7a6402a6c0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 -- 172.21.15.19:0/2522994422 shutdown_connections 2024-05-17T09:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 --2- 172.21.15.19:0/2522994422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c10ae80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 --2- 172.21.15.19:0/2522994422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a6c103d60 0x7f7a6c104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 --2- 172.21.15.19:0/2522994422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 -- 172.21.15.19:0/2522994422 >> 172.21.15.19:0/2522994422 conn(0x7f7a6c0fdd10 msgr2=0x7f7a6c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 -- 172.21.15.19:0/2522994422 shutdown_connections 2024-05-17T09:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.724+0000 7f7a70b34640 1 -- 172.21.15.19:0/2522994422 wait complete. 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 Processor -- start 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 -- start start 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a6c103d60 0x7f7a6c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10e4e0 con 0x7f7a6c102b20 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10e360 con 0x7f7a6c108a60 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a70b34640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10e660 con 0x7f7a6c103d60 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53854/0 (socket says 172.21.15.19:53854) 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3383421499 learned_addr learned my addr 172.21.15.19:0/3383421499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a6c103d60 0x7f7a6c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3383421499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a6c103d60 msgr2=0x7f7a6c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a6c103d60 0x7f7a6c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3383421499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 msgr2=0x7f7a6c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3383421499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c1a1c90 con 0x7f7a6c102b20 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.725+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.726+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c19a150 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e980 tx=0x7f7a5c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.726+0000 7f7a637fe640 1 -- 172.21.15.19:0/3383421499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00cd80 con 0x7f7a6c102b20 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.726+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c1a1f80 con 0x7f7a6c102b20 2024-05-17T09:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.726+0000 7f7a637fe640 1 -- 172.21.15.19:0/3383421499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a5c0108c0 con 0x7f7a6c102b20 2024-05-17T09:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.726+0000 7f7a637fe640 1 -- 172.21.15.19:0/3383421499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c018880 con 0x7f7a6c102b20 2024-05-17T09:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.726+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a2440 con 0x7f7a6c102b20 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 Processor -- start 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a637fe640 1 -- 172.21.15.19:0/3383421499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7a5c01f020 con 0x7f7a6c102b20 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 -- start start 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe846b1b0 0x55bfe880c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe880c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe8813820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bfe877ec30 con 0x55bfe84422f0 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfe876e7d0 con 0x55bfe844a4d0 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a779e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bfe8772810 con 0x55bfe846b1b0 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe880c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe880c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53860/0 (socket says 172.21.15.19:53860) 2024-05-17T09:23:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a6a59c640 1 -- 172.21.15.19:0/891236367 learned_addr learned my addr 172.21.15.19:0/891236367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe8813820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.729+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe846b1b0 0x55bfe880c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6a59c640 1 -- 172.21.15.19:0/891236367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe846b1b0 msgr2=0x55bfe880c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe846b1b0 0x55bfe880c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6a59c640 1 -- 172.21.15.19:0/891236367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 msgr2=0x55bfe8813820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe8813820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6a59c640 1 -- 172.21.15.19:0/891236367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe8483210 con 0x55bfe84422f0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a637fe640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a580b28a0 0x7f7a580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a580b28a0 0x7f7a580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe880c920 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0e8c10 tx=0x7f7a5c0e8d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84d888f734db238 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a637fe640 1 -- 172.21.15.19:0/3383421499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7a5c0e2840 con 0x7f7a6c102b20 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a60ff9640 1 -- 172.21.15.19:0/891236367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0a5970 con 0x55bfe84422f0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a60ff9640 1 -- 172.21.15.19:0/891236367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a5c0a95c0 con 0x55bfe84422f0 2024-05-17T09:23:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.730+0000 7f7a60ff9640 1 -- 172.21.15.19:0/891236367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0a9220 con 0x55bfe84422f0 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 -- 172.21.15.19:0/891236367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 msgr2=0x55bfe880c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe880c920 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0e8c10 tx=0x7f7a5c0e8d00 comp rx=0 tx=0).stop 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a580b28a0 0x7f7a580b4db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a64030800 tx=0x7f7a64004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 -- 172.21.15.19:0/891236367 shutdown_connections 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe8813820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe880c920 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 --2- 172.21.15.19:0/891236367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe846b1b0 0x55bfe880c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 -- 172.21.15.19:0/891236367 >> 172.21.15.19:0/891236367 conn(0x55bfe845f300 msgr2=0x55bfe8807d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 -- 172.21.15.19:0/891236367 shutdown_connections 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.731+0000 7f7a779e5740 1 -- 172.21.15.19:0/891236367 wait complete. 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 -- start start 2024-05-17T09:23:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 Processor -- start 2024-05-17T09:23:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe844a4d0 0x55bfe8816230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe846b1b0 0x55bfe881d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bfe8454790 con 0x55bfe84422f0 2024-05-17T09:23:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfe872a890 con 0x55bfe846b1b0 2024-05-17T09:23:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bfe8814270 con 0x55bfe844a4d0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a779e5740 10 client.0 _ll_register_callbacks cb 0x55bfe872dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53870/0 (socket says 172.21.15.19:53870) 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3584379677 learned_addr learned my addr 172.21.15.19:0/3584379677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe846b1b0 0x55bfe881d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe844a4d0 0x55bfe8816230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3584379677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe844a4d0 msgr2=0x55bfe8816230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe844a4d0 0x55bfe8816230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe846b1b0 msgr2=0x55bfe881d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe846b1b0 0x55bfe881d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.732+0000 7f7a6a59c640 1 -- 172.21.15.19:0/3584379677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe854f620 con 0x55bfe84422f0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe846b1b0 0x55bfe881d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815cf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0e2bf0 tx=0x7f7a5c0e8320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccca12b019fcd96d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3584379677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0c3060 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3584379677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7a5c0a60a0 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5c0a95c0 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3584379677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c0a9400 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 10 client.16587 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfe8820050 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3584379677 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a5c0a95c0 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3584379677 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5c0a5d40 con 0x55bfe84422f0 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 10 client.16587 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 20 client.16587 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe8429d80 0x55bfe88233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 4 client.16587 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.733+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe8780b80 con 0x55bfe8429d80 2024-05-17T09:23:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.734+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe8429d80 0x55bfe88233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.734+0000 7f7a51ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.734+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe8429d80 0x55bfe88233b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b13f39fc97112be server_cookie=f55f23369c8aec5 in_seq=0 out_seq=0 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3584379677 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bfe8780b80 con 0x55bfe8429d80 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a51ffb640 10 client.16587 handle_command_reply: tid=0 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 client.16587 shutdown 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 msgr2=0x55bfe8815cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815cf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0e2bf0 tx=0x7f7a5c0e8320 comp rx=0 tx=0).stop 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 shutdown_connections 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe8429d80 0x55bfe88233b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe846b1b0 0x55bfe881d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a51ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe844a4d0 0x55bfe8816230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3584379677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815cf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 >> 172.21.15.19:0/3584379677 conn(0x55bfe845f300 msgr2=0x55bfe880ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 shutdown_connections 2024-05-17T09:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 1 -- 172.21.15.19:0/3584379677 wait complete. 2024-05-17T09:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.735+0000 7f7a779e5740 20 client.16587 trim_cache size 0 max 16384 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.750+0000 7f7a779e5740 1 Processor -- start 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 -- start start 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe8810880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe881d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1af270 con 0x55bfe84422f0 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1acb40 con 0x55bfe844a4d0 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a779e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1aa990 con 0x55bfe8429d80 2024-05-17T09:23:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53884/0 (socket says 172.21.15.19:53884) 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 -- 172.21.15.19:0/3096882238 learned_addr learned my addr 172.21.15.19:0/3096882238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe8810880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 -- 172.21.15.19:0/3096882238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 msgr2=0x55bfe8810880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe8810880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 -- 172.21.15.19:0/3096882238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 msgr2=0x55bfe881d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe881d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 -- 172.21.15.19:0/3096882238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe8816d60 con 0x55bfe84422f0 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815fd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c19b1f0 tx=0x7f7a54012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd0ac89929e4524 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.751+0000 7f7a53fff640 1 -- 172.21.15.19:0/3096882238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54022020 con 0x55bfe84422f0 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a53fff640 1 -- 172.21.15.19:0/3096882238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a5401eab0 con 0x55bfe84422f0 2024-05-17T09:23:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a53fff640 1 -- 172.21.15.19:0/3096882238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011c10 con 0x55bfe84422f0 2024-05-17T09:23:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 -- 172.21.15.19:0/3096882238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 msgr2=0x55bfe8815fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815fd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c19b1f0 tx=0x7f7a54012820 comp rx=0 tx=0).stop 2024-05-17T09:23:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 -- 172.21.15.19:0/3096882238 shutdown_connections 2024-05-17T09:23:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe844a4d0 0x55bfe881d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe84422f0 0x55bfe8815fd0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3096882238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe8810880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 -- 172.21.15.19:0/3096882238 >> 172.21.15.19:0/3096882238 conn(0x55bfe845f300 msgr2=0x55bfe88158c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 -- 172.21.15.19:0/3096882238 shutdown_connections 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.752+0000 7f7a779e5740 1 -- 172.21.15.19:0/3096882238 wait complete. 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 -- start start 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 Processor -- start 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe881d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe84422f0 0x55bfe881bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 0x55bfe881c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1ab000 con 0x55bfe844a4d0 2024-05-17T09:23:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1ac470 con 0x55bfe84422f0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1ab6d0 con 0x55bfe8429d80 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a779e5740 10 client.0 _ll_register_callbacks cb 0x7f7a5c0e8a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 0x55bfe881c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 0x55bfe881c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53892/0 (socket says 172.21.15.19:53892) 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 -- 172.21.15.19:0/3527241116 learned_addr learned my addr 172.21.15.19:0/3527241116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a69d9b640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe84422f0 0x55bfe881bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe881d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 -- 172.21.15.19:0/3527241116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 msgr2=0x55bfe881d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe881d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 -- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe84422f0 msgr2=0x55bfe881bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe84422f0 0x55bfe881bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.753+0000 7f7a6ad9d640 1 -- 172.21.15.19:0/3527241116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe881cce0 con 0x55bfe844a4d0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a6ad9d640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 0x55bfe881c530 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009be0 tx=0x7f7a64030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e843ecdfe5ef918 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:23:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3527241116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64036050 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3527241116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7a64004450 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5401eab0 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3527241116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a6405d050 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 10 client.16593 fetch_fsmap learned FSMap version 30 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfe888a9c0 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3527241116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a5401eab0 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3527241116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7a64031b80 con 0x55bfe844a4d0 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 10 client.16593 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 20 client.16593 populate_metadata read hostname 'smithi019' 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe846b1b0 0x55bfe888d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 4 client.16593 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:23:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.754+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfe877a360 con 0x55bfe846b1b0 2024-05-17T09:23:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.755+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe846b1b0 0x55bfe888d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:23:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.755+0000 7f7a51ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.755+0000 7f7a6a59c640 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe846b1b0 0x55bfe888d440 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3df82a09e99690 server_cookie=ad57b54b7d22c952 in_seq=0 out_seq=0 2024-05-17T09:23:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:52 smithi079 ceph-mon[35502]: pgmap v561: 129 pgs: 129 active+clean; 34 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a51ffb640 1 -- 172.21.15.19:0/3527241116 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bfe877a360 con 0x55bfe846b1b0 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a51ffb640 10 client.16593 handle_command_reply: tid=0 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 client.16593 shutdown 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 msgr2=0x55bfe881c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 0x55bfe881c530 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009be0 tx=0x7f7a64030db0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 shutdown_connections 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bfe846b1b0 0x55bfe888d440 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe844a4d0 0x55bfe881c530 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfe84422f0 0x55bfe881bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 --2- 172.21.15.19:0/3527241116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bfe8429d80 0x55bfe881d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a51ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 >> 172.21.15.19:0/3527241116 conn(0x55bfe845f300 msgr2=0x55bfe880f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 shutdown_connections 2024-05-17T09:23:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 1 -- 172.21.15.19:0/3527241116 wait complete. 2024-05-17T09:23:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.773+0000 7f7a779e5740 20 client.16593 trim_cache size 0 max 16384 2024-05-17T09:23:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:23:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:23:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a580b28a0 msgr2=0x7f7a580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a580b28a0 0x7f7a580b4db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a64030800 tx=0x7f7a64004a40 comp rx=0 tx=0).stop 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 msgr2=0x7f7a6c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c19a150 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e980 tx=0x7f7a5c00ee40 comp rx=0 tx=0).stop 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 shutdown_connections 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a6c108a60 0x7f7a6c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a6c103d60 0x7f7a6c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a580b28a0 0x7f7a580b4db0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 --2- 172.21.15.19:0/3383421499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6c102b20 0x7f7a6c19a150 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:23:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 >> 172.21.15.19:0/3383421499 conn(0x7f7a6c0fdd10 msgr2=0x7f7a6c109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:23:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 shutdown_connections 2024-05-17T09:23:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:23:52.774+0000 7f7a70b34640 1 -- 172.21.15.19:0/3383421499 wait complete. 2024-05-17T09:23:53.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.25 MB/sec warmup 111 sec latency 1054.139 ms 2024-05-17T09:23:54.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.23 MB/sec warmup 112 sec latency 2054.257 ms 2024-05-17T09:23:54.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '65b87bed', quiesce took 8.0 seconds. Will release after: 4.9 2024-05-17T09:23:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:54 smithi069 ceph-mon[35081]: pgmap v562: 129 pgs: 129 active+clean; 34 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:23:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:54 smithi019 ceph-mon[31620]: pgmap v562: 129 pgs: 129 active+clean; 34 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:23:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:54 smithi079 ceph-mon[35502]: pgmap v562: 129 pgs: 129 active+clean; 34 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:23:55.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.21 MB/sec warmup 113 sec latency 3054.361 ms 2024-05-17T09:23:56.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.19 MB/sec warmup 114 sec latency 4054.502 ms 2024-05-17T09:23:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:56 smithi069 ceph-mon[35081]: pgmap v563: 129 pgs: 129 active+clean; 35 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:23:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:56 smithi019 ceph-mon[31620]: pgmap v563: 129 pgs: 129 active+clean; 35 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:23:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:56 smithi079 ceph-mon[35502]: pgmap v563: 129 pgs: 129 active+clean; 35 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:23:57.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.17 MB/sec warmup 115 sec latency 5054.607 ms 2024-05-17T09:23:58.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.16 MB/sec warmup 116 sec latency 6054.688 ms 2024-05-17T09:23:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:23:58 smithi069 ceph-mon[35081]: pgmap v564: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:23:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:23:58 smithi019 ceph-mon[31620]: pgmap v564: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:23:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:23:58 smithi079 ceph-mon[35502]: pgmap v564: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:23:59.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:23:59.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.14 MB/sec warmup 117 sec latency 7054.746 ms 2024-05-17T09:23:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:24:00.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.12 MB/sec warmup 118 sec latency 8054.832 ms 2024-05-17T09:24:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:00 smithi069 ceph-mon[35081]: pgmap v565: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 856 KiB/s wr, 93 op/s 2024-05-17T09:24:00.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:00 smithi019 ceph-mon[31620]: pgmap v565: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 856 KiB/s wr, 93 op/s 2024-05-17T09:24:00.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:00 smithi079 ceph-mon[35502]: pgmap v565: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 856 KiB/s wr, 93 op/s 2024-05-17T09:24:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.417+0000 7fcd74b29700 1 -- 172.21.15.19:0/2349553688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd70107de0 msgr2=0x7fcd701081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.417+0000 7fcd74b29700 1 --2- 172.21.15.19:0/2349553688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd70107de0 0x7fcd701081f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcd580099c0 tx=0x7fcd58009cd0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 -- 172.21.15.19:0/2349553688 shutdown_connections 2024-05-17T09:24:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 --2- 172.21.15.19:0/2349553688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd70108730 0x7fcd70110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 --2- 172.21.15.19:0/2349553688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd701034b0 0x7fcd701038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 --2- 172.21.15.19:0/2349553688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd70107de0 0x7fcd701081f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 -- 172.21.15.19:0/2349553688 >> 172.21.15.19:0/2349553688 conn(0x7fcd700fedb0 msgr2=0x7fcd701011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 -- 172.21.15.19:0/2349553688 shutdown_connections 2024-05-17T09:24:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 -- 172.21.15.19:0/2349553688 wait complete. 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.418+0000 7fcd74b29700 1 Processor -- start 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 -- start start 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 0x7fcd7019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd70107de0 0x7fcd7019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd70108730 0x7fcd70199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701131f0 con 0x7fcd701034b0 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70113070 con 0x7fcd70107de0 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd74b29700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70113370 con 0x7fcd70108730 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 0x7fcd7019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 0x7fcd7019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51688/0 (socket says 172.21.15.19:51688) 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 -- 172.21.15.19:0/1107726206 learned_addr learned my addr 172.21.15.19:0/1107726206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6dd9b700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd70107de0 0x7fcd7019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6ed9d700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd70108730 0x7fcd70199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 -- 172.21.15.19:0/1107726206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd70108730 msgr2=0x7fcd70199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd70108730 0x7fcd70199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 -- 172.21.15.19:0/1107726206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd70107de0 msgr2=0x7fcd7019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd70107de0 0x7fcd7019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.419+0000 7fcd6e59c700 1 -- 172.21.15.19:0/1107726206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019a140 con 0x7fcd701034b0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.420+0000 7fcd6e59c700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 0x7fcd7019f870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcd600060b0 tx=0x7fcd6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.420+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6001a070 con 0x7fcd701034b0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.420+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019a430 con 0x7fcd701034b0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.420+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6000ddd0 con 0x7fcd701034b0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.420+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60014690 con 0x7fcd701034b0 2024-05-17T09:24:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.420+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7006b600 con 0x7fcd701034b0 2024-05-17T09:24:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.421+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd700512b0 con 0x7fcd701034b0 2024-05-17T09:24:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.422+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcd6001d070 con 0x7fcd701034b0 2024-05-17T09:24:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.423+0000 7fcd677fe700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcd5c0b2900 0x7fcd5c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.423+0000 7fcd6dd9b700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcd5c0b2900 0x7fcd5c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.424+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcd600e2260 con 0x7fcd701034b0 2024-05-17T09:24:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.424+0000 7fcd6dd9b700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcd5c0b2900 0x7fcd5c0b4e10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009990 tx=0x7fcd58005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:01.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 2.10 MB/sec warmup 119 sec latency 9054.873 ms 2024-05-17T09:24:01.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.427+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd60010070 con 0x7fcd701034b0 2024-05-17T09:24:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.696+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd7019afc0 con 0x7fcd701034b0 2024-05-17T09:24:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.697+0000 7fcd677fe700 1 -- 172.21.15.19:0/1107726206 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7fcd600a9bd0 con 0x7fcd701034b0 2024-05-17T09:24:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:24:01.697 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:22:41.259768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:24:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcd5c0b2900 msgr2=0x7fcd5c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcd5c0b2900 0x7fcd5c0b4e10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009990 tx=0x7fcd58005fb0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 msgr2=0x7fcd7019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 0x7fcd7019f870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcd600060b0 tx=0x7fcd6000d3b0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 shutdown_connections 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcd5c0b2900 0x7fcd5c0b4e10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd70108730 0x7fcd70199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd701034b0 0x7fcd7019f870 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 --2- 172.21.15.19:0/1107726206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd70107de0 0x7fcd7019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 >> 172.21.15.19:0/1107726206 conn(0x7fcd700fedb0 msgr2=0x7fcd701011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 shutdown_connections 2024-05-17T09:24:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:01.699+0000 7fcd74b29700 1 -- 172.21.15.19:0/1107726206 wait complete. 2024-05-17T09:24:01.701 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-17T09:24:02.495 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 65b87bed --release --await' 2024-05-17T09:24:02.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 65b87bed --release --await 2024-05-17T09:24:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:02 smithi069 ceph-mon[35081]: pgmap v566: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 633 KiB/s wr, 68 op/s 2024-05-17T09:24:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:02 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1107726206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:02 smithi019 ceph-mon[31620]: pgmap v566: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 633 KiB/s wr, 68 op/s 2024-05-17T09:24:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:02 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1107726206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:24:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:02 smithi079 ceph-mon[35502]: pgmap v566: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 633 KiB/s wr, 68 op/s 2024-05-17T09:24:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:02 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1107726206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:03.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 0.00 MB/sec execute 1 sec latency 11054.973 ms 2024-05-17T09:24:04.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42813 0.00 MB/sec execute 2 sec latency 12055.005 ms 2024-05-17T09:24:04.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:04 smithi019 ceph-mon[31620]: pgmap v567: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 621 KiB/s wr, 66 op/s 2024-05-17T09:24:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 -- 172.21.15.19:0/2785613678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 msgr2=0x7fca8c10e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 --2- 172.21.15.19:0/2785613678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c10e310 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fca80009990 tx=0x7fca80009ca0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 -- 172.21.15.19:0/2785613678 shutdown_connections 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 --2- 172.21.15.19:0/2785613678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca8c105470 0x7fca8c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 --2- 172.21.15.19:0/2785613678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 0x7fca8c110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 --2- 172.21.15.19:0/2785613678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c10e310 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 -- 172.21.15.19:0/2785613678 >> 172.21.15.19:0/2785613678 conn(0x7fca8c0fef40 msgr2=0x7fca8c101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 -- 172.21.15.19:0/2785613678 shutdown_connections 2024-05-17T09:24:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.646+0000 7fca925fd700 1 -- 172.21.15.19:0/2785613678 wait complete. 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.647+0000 7fca925fd700 1 Processor -- start 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.647+0000 7fca925fd700 1 -- start start 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca925fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca8c105470 0x7fca8c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca925fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca925fd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 0x7fca8c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca925fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1143b0 con 0x7fca8c105dc0 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca925fd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c114230 con 0x7fca8c10e8e0 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca925fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c114530 con 0x7fca8c105470 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44844/0 (socket says 172.21.15.19:44844) 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1498221811 learned_addr learned my addr 172.21.15.19:0/1498221811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca8c105470 0x7fca8c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 0x7fca8c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1498221811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca8c105470 msgr2=0x7fca8c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca8c105470 0x7fca8c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1498221811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 msgr2=0x7fca8c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 0x7fca8c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1498221811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c19fc90 con 0x7fca8c105dc0 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 0x7fca8c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.648+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c19b360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fca80000c00 tx=0x7fca800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.649+0000 7fca897fa700 1 -- 172.21.15.19:0/1498221811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80027070 con 0x7fca8c105dc0 2024-05-17T09:24:04.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.649+0000 7fca897fa700 1 -- 172.21.15.19:0/1498221811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca8000fd00 con 0x7fca8c105dc0 2024-05-17T09:24:04.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.649+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c19ff20 con 0x7fca8c105dc0 2024-05-17T09:24:04.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.649+0000 7fca897fa700 1 -- 172.21.15.19:0/1498221811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80022e40 con 0x7fca8c105dc0 2024-05-17T09:24:04.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.649+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c1a0400 con 0x7fca8c105dc0 2024-05-17T09:24:04.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.652+0000 7fca897fa700 1 -- 172.21.15.19:0/1498221811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fca8002a070 con 0x7fca8c105dc0 2024-05-17T09:24:04.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.652+0000 7fca897fa700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fca740b2880 0x7fca740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.652+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fca740b2880 0x7fca740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 Processor -- start 2024-05-17T09:24:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca897fa700 1 -- 172.21.15.19:0/1498221811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fca800e33e0 con 0x7fca8c105dc0 2024-05-17T09:24:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 -- start start 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e8d3530 0x55a59e96aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e7e7f40 0x55a59e967020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e967ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fca740b2880 0x7fca740b4d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fca8c06d140 tx=0x7fca7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a59e894e70 con 0x55a59e967600 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a59e884a10 con 0x55a59e7e7f40 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a59e888a50 con 0x55a59e8d3530 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca90b9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e967ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca90b9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e967ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44850/0 (socket says 172.21.15.19:44850) 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca90b9a700 1 -- 172.21.15.19:0/1845909039 learned_addr learned my addr 172.21.15.19:0/1845909039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e8d3530 0x55a59e96aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.653+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e7e7f40 0x55a59e967020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca90b9a700 1 -- 172.21.15.19:0/1845909039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e8d3530 msgr2=0x55a59e96aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e8d3530 0x55a59e96aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca90b9a700 1 -- 172.21.15.19:0/1845909039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e7e7f40 msgr2=0x55a59e967020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e7e7f40 0x55a59e967020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca90b9a700 1 -- 172.21.15.19:0/1845909039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59e82e0b0 con 0x55a59e967600 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e7e7f40 0x55a59e967020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e967ad0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fca840087c0 tx=0x7fca84008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a956c05e579d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca727fc700 1 -- 172.21.15.19:0/1845909039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84003c00 con 0x55a59e967600 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca727fc700 1 -- 172.21.15.19:0/1845909039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca84003da0 con 0x55a59e967600 2024-05-17T09:24:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.654+0000 7fca727fc700 1 -- 172.21.15.19:0/1845909039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84012820 con 0x55a59e967600 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1845909039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 msgr2=0x55a59e967ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e967ad0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fca840087c0 tx=0x7fca84008ad0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1845909039 shutdown_connections 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e8d3530 0x55a59e96aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e967ad0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1845909039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e7e7f40 0x55a59e967020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1845909039 >> 172.21.15.19:0/1845909039 conn(0x55a59e8482c0 msgr2=0x55a59e961cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1845909039 shutdown_connections 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1845909039 wait complete. 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.655+0000 7fcaa1012b80 1 -- start start 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 Processor -- start 2024-05-17T09:24:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e978d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 0x55a59e9792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e967600 0x55a59e979810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a59e78b0a0 con 0x55a59e8d3530 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a59e7ecea0 con 0x55a59e967600 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a59e97bff0 con 0x55a59e7e7f40 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fcaa1012b80 10 client.0 _ll_register_callbacks cb 0x55a59e81fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 0x55a59e9792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 0x55a59e9792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44858/0 (socket says 172.21.15.19:44858) 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1813894864 learned_addr learned my addr 172.21.15.19:0/1813894864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e978d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.656+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e967600 0x55a59e979810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1813894864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 msgr2=0x55a59e978d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e978d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e967600 msgr2=0x55a59e979810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e967600 0x55a59e979810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca8b7fe700 1 -- 172.21.15.19:0/1813894864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59e7d49d0 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e967600 0x55a59e979810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 0x55a59e9792d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fca800e0b80 tx=0x7fca800228e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6d9df1af8497e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca6b7fe700 1 -- 172.21.15.19:0/1813894864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800aa1f0 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca84003da0 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca6b7fe700 1 -- 172.21.15.19:0/1813894864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fca800a6cd0 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca6b7fe700 1 -- 172.21.15.19:0/1813894864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca800aade0 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fcaa1012b80 10 client.16611 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a59e83e860 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca6b7fe700 1 -- 172.21.15.19:0/1813894864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca84003da0 con 0x55a59e8d3530 2024-05-17T09:24:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.657+0000 7fca6b7fe700 1 -- 172.21.15.19:0/1813894864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fca800920f0 con 0x55a59e8d3530 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fcaa1012b80 10 client.16611 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fcaa1012b80 10 client.16611 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fcaa1012b80 20 client.16611 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9890a0 0x55a59e98b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fcaa1012b80 4 client.16611 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a59e932fd0 con 0x55a59e9890a0 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9890a0 0x55a59e98b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9890a0 0x55a59e98b550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c1d3f94781ae87 server_cookie=cbf91fd2714f4c10 in_seq=0 out_seq=0 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.658+0000 7fca6b7fe700 10 client.16611 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fca6b7fe700 1 -- 172.21.15.19:0/1813894864 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a59e932fd0 con 0x55a59e9890a0 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fca6b7fe700 10 client.16611 handle_command_reply: tid=0 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 client.16611 shutdown 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 msgr2=0x55a59e9792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 0x55a59e9792d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fca800e0b80 tx=0x7fca800228e0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 shutdown_connections 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e978d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9890a0 0x55a59e98b550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e967600 0x55a59e979810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fca6b7fe700 0 client.16611 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1813894864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e8d3530 0x55a59e9792d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.659+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 >> 172.21.15.19:0/1813894864 conn(0x55a59e8482c0 msgr2=0x55a59e964000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.660+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 shutdown_connections 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.660+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1813894864 wait complete. 2024-05-17T09:24:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.660+0000 7fcaa1012b80 20 client.16611 trim_cache size 0 max 16384 2024-05-17T09:24:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:04 smithi069 ceph-mon[35081]: pgmap v567: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 621 KiB/s wr, 66 op/s 2024-05-17T09:24:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 Processor -- start 2024-05-17T09:24:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 -- start start 2024-05-17T09:24:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 0x55a59e95c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e967020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e967600 0x55a59e990d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1b0950 con 0x55a59e7e7f40 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ae1e0 con 0x55a59e8d3530 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ac030 con 0x55a59e967600 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 0x55a59e95c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 0x55a59e95c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44866/0 (socket says 172.21.15.19:44866) 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fca8bfff700 1 -- 172.21.15.19:0/1732066140 learned_addr learned my addr 172.21.15.19:0/1732066140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.680+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e967020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca90b9a700 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e967600 0x55a59e990d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca8bfff700 1 -- 172.21.15.19:0/1732066140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e967600 msgr2=0x55a59e990d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e967600 0x55a59e990d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca8bfff700 1 -- 172.21.15.19:0/1732066140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 msgr2=0x55a59e967020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e967020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca8bfff700 1 -- 172.21.15.19:0/1732066140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59e962d30 con 0x55a59e7e7f40 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca8bfff700 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 0x55a59e95c5f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00e4b0 tx=0x7fca7c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468aa50d568aabb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca71ffb700 1 -- 172.21.15.19:0/1732066140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c01b040 con 0x55a59e7e7f40 2024-05-17T09:24:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca71ffb700 1 -- 172.21.15.19:0/1732066140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca7c029810 con 0x55a59e7e7f40 2024-05-17T09:24:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.681+0000 7fca71ffb700 1 -- 172.21.15.19:0/1732066140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c015a80 con 0x55a59e7e7f40 2024-05-17T09:24:04.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1732066140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 msgr2=0x55a59e95c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 0x55a59e95c5f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00e4b0 tx=0x7fca7c013000 comp rx=0 tx=0).stop 2024-05-17T09:24:04.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1732066140 shutdown_connections 2024-05-17T09:24:04.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e967600 0x55a59e990d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e7e7f40 0x55a59e95c5f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/1732066140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e967020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1732066140 >> 172.21.15.19:0/1732066140 conn(0x55a59e8482c0 msgr2=0x55a59e9742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1732066140 shutdown_connections 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 -- 172.21.15.19:0/1732066140 wait complete. 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.682+0000 7fcaa1012b80 1 -- start start 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 Processor -- start 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e97da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e97df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e9ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ac6a0 con 0x55a59e967600 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1adb10 con 0x55a59e8d3530 2024-05-17T09:24:04.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1acd70 con 0x55a59e7e7f40 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fcaa1012b80 10 client.0 _ll_register_callbacks cb 0x55a59e81fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e97da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e97df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e97df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59238/0 (socket says 172.21.15.19:59238) 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e97da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42288/0 (socket says 172.21.15.19:42288) 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fca8b7fe700 1 -- 172.21.15.19:0/2623697401 learned_addr learned my addr 172.21.15.19:0/2623697401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.683+0000 7fca90b9a700 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e9ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca8b7fe700 1 -- 172.21.15.19:0/2623697401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 msgr2=0x55a59e97da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e97da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca8b7fe700 1 -- 172.21.15.19:0/2623697401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 msgr2=0x55a59e9ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e9ecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca8b7fe700 1 -- 172.21.15.19:0/2623697401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59e9ed190 con 0x55a59e8d3530 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca8b7fe700 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e97df50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca800a6760 tx=0x7fca800a9c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c8022efaef579d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca6b7fe700 1 -- 172.21.15.19:0/2623697401 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800aa1f0 con 0x55a59e8d3530 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7c029810 con 0x55a59e8d3530 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca6b7fe700 1 -- 172.21.15.19:0/2623697401 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fca800a6cd0 con 0x55a59e8d3530 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fca6b7fe700 1 -- 172.21.15.19:0/2623697401 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca800aa6e0 con 0x55a59e8d3530 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fcaa1012b80 10 client.26279 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.684+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a59e9f44d0 con 0x55a59e8d3530 2024-05-17T09:24:04.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fca6b7fe700 1 -- 172.21.15.19:0/2623697401 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca7c029810 con 0x55a59e8d3530 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fca6b7fe700 1 -- 172.21.15.19:0/2623697401 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fca800a93b0 con 0x55a59e8d3530 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fcaa1012b80 10 client.26279 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fcaa1012b80 10 client.26279 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fcaa1012b80 20 client.26279 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9f4ad0 0x55a59e9f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fcaa1012b80 4 client.26279 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "65b87bed", "release": true, "await": true}] 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "65b87bed", "release": true, "await": true}) -- 0x55a59e896dc0 con 0x55a59e9f4ad0 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.685+0000 7fca8bfff700 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9f4ad0 0x55a59e9f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.686+0000 7fca8bfff700 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9f4ad0 0x55a59e9f6ef0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feabd4dd8b9cf98a server_cookie=f4bcd122dd9d2a4b in_seq=0 out_seq=0 2024-05-17T09:24:04.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.686+0000 7fca6b7fe700 10 client.26279 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:04.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fca6b7fe700 1 -- 172.21.15.19:0/2623697401 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a59e896dc0 con 0x55a59e9f4ad0 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fca6b7fe700 10 client.26279 handle_command_reply: tid=0 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 client.26279 shutdown 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 msgr2=0x55a59e97df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e97df50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca800a6760 tx=0x7fca800a9c80 comp rx=0 tx=0).stop 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 shutdown_connections 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a59e7e7f40 0x55a59e97da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a59e9f4ad0 0x55a59e9f6ef0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a59e967600 0x55a59e9ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 --2- 172.21.15.19:0/2623697401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a59e8d3530 0x55a59e97df50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fca6b7fe700 0 client.26279 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 >> 172.21.15.19:0/2623697401 conn(0x55a59e8482c0 msgr2=0x55a59e9742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 shutdown_connections 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 1 -- 172.21.15.19:0/2623697401 wait complete. 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.690+0000 7fcaa1012b80 20 client.26279 trim_cache size 0 max 16384 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 16, 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:24:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: "65b87bed": { 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 16, 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 132.9, 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:24:04.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:04.697 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fca740b2880 msgr2=0x7fca740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fca740b2880 0x7fca740b4d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fca8c06d140 tx=0x7fca7c009190 comp rx=0 tx=0).stop 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 msgr2=0x7fca8c19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c19b360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fca80000c00 tx=0x7fca800047b0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 shutdown_connections 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fca740b2880 0x7fca740b4d90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca8c105470 0x7fca8c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca8c10e8e0 0x7fca8c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 --2- 172.21.15.19:0/1498221811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8c105dc0 0x7fca8c19b360 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 >> 172.21.15.19:0/1498221811 conn(0x7fca8c0fef40 msgr2=0x7fca8c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:04.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 shutdown_connections 2024-05-17T09:24:04.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:04.691+0000 7fca925fd700 1 -- 172.21.15.19:0/1498221811 wait complete. 2024-05-17T09:24:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:04 smithi079 ceph-mon[35502]: pgmap v567: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 621 KiB/s wr, 66 op/s 2024-05-17T09:24:05.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42979 0.10 MB/sec execute 3 sec latency 12323.496 ms 2024-05-17T09:24:05.523 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '65b87bed', total seconds elapsed: 19.1 2024-05-17T09:24:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 -- 172.21.15.19:0/2380153219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c102b40 msgr2=0x7fde1c103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 --2- 172.21.15.19:0/2380153219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c102b40 0x7fde1c103840 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009900 tx=0x7fde0c02a410 comp rx=0 tx=0).stop 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 -- 172.21.15.19:0/2380153219 shutdown_connections 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 --2- 172.21.15.19:0/2380153219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde1c108a80 0x7fde1c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 --2- 172.21.15.19:0/2380153219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c103d80 0x7fde1c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 --2- 172.21.15.19:0/2380153219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c102b40 0x7fde1c103840 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 -- 172.21.15.19:0/2380153219 >> 172.21.15.19:0/2380153219 conn(0x7fde1c0fdd30 msgr2=0x7fde1c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 -- 172.21.15.19:0/2380153219 shutdown_connections 2024-05-17T09:24:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.881+0000 7fde208e2640 1 -- 172.21.15.19:0/2380153219 wait complete. 2024-05-17T09:24:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 Processor -- start 2024-05-17T09:24:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 -- start start 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde1c102b40 0x7fde1c0ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 0x7fde1c0fe260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c108a80 0x7fde1c0fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c10e4d0 con 0x7fde1c103d80 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c10e350 con 0x7fde1c108a80 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde208e2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c10e650 con 0x7fde1c102b40 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 0x7fde1c0fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c108a80 0x7fde1c0fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 0x7fde1c0fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44882/0 (socket says 172.21.15.19:44882) 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde19d9b640 1 -- 172.21.15.19:0/1346049689 learned_addr learned my addr 172.21.15.19:0/1346049689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.882+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde1c102b40 0x7fde1c0ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1346049689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde1c102b40 msgr2=0x7fde1c0ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde1c102b40 0x7fde1c0ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 msgr2=0x7fde1c0fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 0x7fde1c0fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1346049689 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c0feed0 con 0x7fde1c108a80 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde19d9b640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 0x7fde1c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c108a80 0x7fde1c0fe7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde1400e940 tx=0x7fde1400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde0b7fe640 1 -- 172.21.15.19:0/1346049689 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14004d20 con 0x7fde1c108a80 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde0b7fe640 1 -- 172.21.15.19:0/1346049689 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde14005020 con 0x7fde1c108a80 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1a4340 con 0x7fde1c108a80 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde0b7fe640 1 -- 172.21.15.19:0/1346049689 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140116d0 con 0x7fde1c108a80 2024-05-17T09:24:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.883+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a4750 con 0x7fde1c108a80 2024-05-17T09:24:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.886+0000 7fde27792740 1 Processor -- start 2024-05-17T09:24:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.886+0000 7fde0b7fe640 1 -- 172.21.15.19:0/1346049689 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fde1400cb20 con 0x7fde1c108a80 2024-05-17T09:24:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.886+0000 7fde27792740 1 -- start start 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde27792740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447491b0 0x55ab44aea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde27792740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447202f0 0x55ab44aea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde27792740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447284d0 0x55ab44af1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde27792740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ab44a5cc30 con 0x55ab447202f0 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde27792740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ab44a4c7d0 con 0x55ab447491b0 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde27792740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ab44a50810 con 0x55ab447284d0 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447202f0 0x55ab44aea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447491b0 0x55ab44aea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447202f0 0x55ab44aea920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44886/0 (socket says 172.21.15.19:44886) 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde1a59c640 1 -- 172.21.15.19:0/3510252519 learned_addr learned my addr 172.21.15.19:0/3510252519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447284d0 0x55ab44af1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 -- 172.21.15.19:0/3510252519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447284d0 msgr2=0x55ab44af1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447284d0 0x55ab44af1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 -- 172.21.15.19:0/3510252519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447202f0 msgr2=0x55ab44aea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447202f0 0x55ab44aea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde19d9b640 1 -- 172.21.15.19:0/3510252519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab44761210 con 0x55ab447491b0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.887+0000 7fde0b7fe640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde040b2920 0x7fde040b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.888+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447491b0 0x55ab44aea3a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ff560 tx=0x7fde10004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42efe144199f3b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.888+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde040b2920 0x7fde040b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.888+0000 7fde08ff9640 1 -- 172.21.15.19:0/3510252519 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000fc50 con 0x55ab447491b0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.888+0000 7fde08ff9640 1 -- 172.21.15.19:0/3510252519 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde100044c0 con 0x55ab447491b0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.888+0000 7fde08ff9640 1 -- 172.21.15.19:0/3510252519 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100047a0 con 0x55ab447491b0 2024-05-17T09:24:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.888+0000 7fde0b7fe640 1 -- 172.21.15.19:0/1346049689 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde140e2650 con 0x7fde1c108a80 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 -- 172.21.15.19:0/3510252519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447491b0 msgr2=0x55ab44aea3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde040b2920 0x7fde040b4e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55ab44aeb780 tx=0x7fde0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447491b0 0x55ab44aea3a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ff560 tx=0x7fde10004730 comp rx=0 tx=0).stop 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 -- 172.21.15.19:0/3510252519 shutdown_connections 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447284d0 0x55ab44af1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447202f0 0x55ab44aea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 --2- 172.21.15.19:0/3510252519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447491b0 0x55ab44aea3a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 -- 172.21.15.19:0/3510252519 >> 172.21.15.19:0/3510252519 conn(0x55ab4473d300 msgr2=0x55ab44ae5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 -- 172.21.15.19:0/3510252519 shutdown_connections 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 -- 172.21.15.19:0/3510252519 wait complete. 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 -- start start 2024-05-17T09:24:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.889+0000 7fde27792740 1 Processor -- start 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44af3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44af4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 0x55ab44afb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ab44732790 con 0x55ab447491b0 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ab44a08890 con 0x55ab447284d0 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ab44af2270 con 0x55ab447202f0 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde27792740 10 client.0 _ll_register_callbacks cb 0x55ab44a0bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44af3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 0x55ab44afb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 0x55ab44afb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44896/0 (socket says 172.21.15.19:44896) 2024-05-17T09:24:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1106462699 learned_addr learned my addr 172.21.15.19:0/1106462699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44af3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42324/0 (socket says 172.21.15.19:42324) 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde19d9b640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44af4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1106462699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 msgr2=0x55ab44af3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44af3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 msgr2=0x55ab44af4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44af4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1106462699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab4482d620 con 0x55ab447491b0 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde19d9b640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44af4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 0x55ab44afb1b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fde140a6b80 tx=0x7fde14005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18e717c97423665 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.890+0000 7fde01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde01ffb640 1 -- 172.21.15.19:0/1106462699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1401f040 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde01ffb640 1 -- 172.21.15.19:0/1106462699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde14005460 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab44a5eb80 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde01ffb640 1 -- 172.21.15.19:0/1106462699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde140a9910 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 10 client.16629 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab44afe150 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde01ffb640 1 -- 172.21.15.19:0/1106462699 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ab44a5eb80 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde01ffb640 1 -- 172.21.15.19:0/1106462699 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fde1400f040 con 0x55ab447491b0 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 10 client.16629 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 20 client.16629 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab44707d80 0x55ab44b019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 4 client.16629 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab44a58360 con 0x55ab44707d80 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.891+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab44707d80 0x55ab44b019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.892+0000 7fde01ffb640 10 client.16629 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.892+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab44707d80 0x55ab44b019d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d881cc7d012c81b4 server_cookie=fd74ce9c8981a4db in_seq=0 out_seq=0 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde01ffb640 1 -- 172.21.15.19:0/1106462699 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ab44a58360 con 0x55ab44707d80 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde01ffb640 10 client.16629 handle_command_reply: tid=0 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 client.16629 shutdown 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 msgr2=0x55ab44afb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 0x55ab44afb1b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fde140a6b80 tx=0x7fde14005b90 comp rx=0 tx=0).stop 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 shutdown_connections 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab44707d80 0x55ab44b019d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447491b0 0x55ab44afb1b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44af4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde01ffb640 0 client.16629 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 --2- 172.21.15.19:0/1106462699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44af3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 >> 172.21.15.19:0/1106462699 conn(0x55ab4473d300 msgr2=0x55ab44ae8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 shutdown_connections 2024-05-17T09:24:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 1 -- 172.21.15.19:0/1106462699 wait complete. 2024-05-17T09:24:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.893+0000 7fde27792740 20 client.16629 trim_cache size 0 max 16384 2024-05-17T09:24:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 Processor -- start 2024-05-17T09:24:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 -- start start 2024-05-17T09:24:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab44707d80 0x55ab44b07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447202f0 0x55ab44b076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 0x55ab44afb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b1540 con 0x55ab447284d0 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1aee10 con 0x55ab447202f0 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde27792740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1acc60 con 0x55ab44707d80 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 0x55ab44afb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 0x55ab44afb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44910/0 (socket says 172.21.15.19:44910) 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1853326770 learned_addr learned my addr 172.21.15.19:0/1853326770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde19d9b640 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447202f0 0x55ab44b076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab44707d80 0x55ab44b07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1853326770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab44707d80 msgr2=0x55ab44b07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab44707d80 0x55ab44b07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1853326770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447202f0 msgr2=0x55ab44b076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447202f0 0x55ab44b076d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1ad9d640 1 -- 172.21.15.19:0/1853326770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab44af30e0 con 0x55ab447284d0 2024-05-17T09:24:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.910+0000 7fde1a59c640 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab44707d80 0x55ab44b07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:24:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.911+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 0x55ab44afb0d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fde140a98a0 tx=0x7fde1400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e52e905b136d516 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.911+0000 7fde03fff640 1 -- 172.21.15.19:0/1853326770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1401f070 con 0x55ab447284d0 2024-05-17T09:24:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.911+0000 7fde03fff640 1 -- 172.21.15.19:0/1853326770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde140adc10 con 0x55ab447284d0 2024-05-17T09:24:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.911+0000 7fde03fff640 1 -- 172.21.15.19:0/1853326770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14005460 con 0x55ab447284d0 2024-05-17T09:24:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 -- 172.21.15.19:0/1853326770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 msgr2=0x55ab44afb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 0x55ab44afb0d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fde140a98a0 tx=0x7fde1400cf90 comp rx=0 tx=0).stop 2024-05-17T09:24:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 -- 172.21.15.19:0/1853326770 shutdown_connections 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab447284d0 0x55ab44afb0d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447202f0 0x55ab44b076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 --2- 172.21.15.19:0/1853326770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab44707d80 0x55ab44b07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 -- 172.21.15.19:0/1853326770 >> 172.21.15.19:0/1853326770 conn(0x55ab4473d300 msgr2=0x55ab44aed8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 -- 172.21.15.19:0/1853326770 shutdown_connections 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 -- 172.21.15.19:0/1853326770 wait complete. 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 -- start start 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.912+0000 7fde27792740 1 Processor -- start 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 0x55ab44afa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44afa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44afadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ad2d0 con 0x55ab44707d80 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ae740 con 0x55ab447284d0 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ad9a0 con 0x55ab447202f0 2024-05-17T09:24:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 10 client.0 _ll_register_callbacks cb 0x7fde140a9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 0x55ab44afa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 0x55ab44afa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44918/0 (socket says 172.21.15.19:44918) 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 -- 172.21.15.19:0/3037570809 learned_addr learned my addr 172.21.15.19:0/3037570809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1ad9d640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44afadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44afa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 -- 172.21.15.19:0/3037570809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 msgr2=0x55ab44afa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44afa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 -- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 msgr2=0x55ab44afadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44afadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 -- 172.21.15.19:0/3037570809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab44b68180 con 0x55ab44707d80 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde1a59c640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 0x55ab44afa330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009900 tx=0x7fde0c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a002766c8352722f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde01ffb640 1 -- 172.21.15.19:0/3037570809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c056020 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde01ffb640 1 -- 172.21.15.19:0/3037570809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde0c0528f0 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.913+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde140adc10 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde01ffb640 1 -- 172.21.15.19:0/3037570809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde0c061050 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 10 client.16635 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab44b693f0 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde01ffb640 1 -- 172.21.15.19:0/3037570809 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde140adc10 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde01ffb640 1 -- 172.21.15.19:0/3037570809 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fde0c044040 con 0x55ab44707d80 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 20 client.16635 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab447491b0 0x55ab44b6be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 4 client.16635 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:24:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab449d25f0 con 0x55ab447491b0 2024-05-17T09:24:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.914+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab447491b0 0x55ab44b6be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.915+0000 7fde19d9b640 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab447491b0 0x55ab44b6be70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a0c27cedffe0a6 server_cookie=48294d8c7ea0a0bb in_seq=0 out_seq=0 2024-05-17T09:24:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.915+0000 7fde01ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.933+0000 7fde01ffb640 1 -- 172.21.15.19:0/3037570809 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ab449d25f0 con 0x55ab447491b0 2024-05-17T09:24:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.933+0000 7fde01ffb640 10 client.16635 handle_command_reply: tid=0 2024-05-17T09:24:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.933+0000 7fde27792740 1 client.16635 shutdown 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 msgr2=0x55ab44afa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 0x55ab44afa330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009900 tx=0x7fde0c02a410 comp rx=0 tx=0).stop 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 shutdown_connections 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ab447491b0 0x55ab44b6be70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab447284d0 0x55ab44afadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ab447202f0 0x55ab44afa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 --2- 172.21.15.19:0/3037570809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab44707d80 0x55ab44afa330 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde01ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 >> 172.21.15.19:0/3037570809 conn(0x55ab4473d300 msgr2=0x55ab44aee2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 shutdown_connections 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 1 -- 172.21.15.19:0/3037570809 wait complete. 2024-05-17T09:24:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde27792740 20 client.16635 trim_cache size 0 max 16384 2024-05-17T09:24:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:24:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:24:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde040b2920 msgr2=0x7fde040b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde208e2640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde040b2920 0x7fde040b4e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55ab44aeb780 tx=0x7fde0c000bc0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c108a80 msgr2=0x7fde1c0fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.934+0000 7fde208e2640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c108a80 0x7fde1c0fe7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde1400e940 tx=0x7fde1400ee00 comp rx=0 tx=0).stop 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 shutdown_connections 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde1c108a80 0x7fde1c0fe7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde040b2920 0x7fde040b4e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde1c103d80 0x7fde1c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 --2- 172.21.15.19:0/1346049689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde1c102b40 0x7fde1c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 >> 172.21.15.19:0/1346049689 conn(0x7fde1c0fdd30 msgr2=0x7fde1c10a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 shutdown_connections 2024-05-17T09:24:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:05.935+0000 7fde208e2640 1 -- 172.21.15.19:0/1346049689 wait complete. 2024-05-17T09:24:06.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 43274 0.13 MB/sec execute 4 sec latency 16.801 ms 2024-05-17T09:24:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:06 smithi069 ceph-mon[35081]: pgmap v568: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 335 KiB/s wr, 40 op/s 2024-05-17T09:24:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:06 smithi019 ceph-mon[31620]: pgmap v568: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 335 KiB/s wr, 40 op/s 2024-05-17T09:24:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:06 smithi079 ceph-mon[35502]: pgmap v568: 129 pgs: 129 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 335 KiB/s wr, 40 op/s 2024-05-17T09:24:07.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 43774 0.47 MB/sec execute 5 sec latency 15.412 ms 2024-05-17T09:24:08.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 44373 0.94 MB/sec execute 6 sec latency 12.465 ms 2024-05-17T09:24:08.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:08 smithi069 ceph-mon[35081]: pgmap v569: 129 pgs: 129 active+clean; 35 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 156 KiB/s wr, 42 op/s 2024-05-17T09:24:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:08 smithi019 ceph-mon[31620]: pgmap v569: 129 pgs: 129 active+clean; 35 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 156 KiB/s wr, 42 op/s 2024-05-17T09:24:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:08 smithi079 ceph-mon[35502]: pgmap v569: 129 pgs: 129 active+clean; 35 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 156 KiB/s wr, 42 op/s 2024-05-17T09:24:09.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 44975 1.05 MB/sec execute 7 sec latency 12.387 ms 2024-05-17T09:24:10.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 45569 1.61 MB/sec execute 8 sec latency 15.942 ms 2024-05-17T09:24:10.687 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:10 smithi069 ceph-mon[35081]: pgmap v570: 129 pgs: 129 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 170 KiB/s wr, 39 op/s 2024-05-17T09:24:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:10 smithi019 ceph-mon[31620]: pgmap v570: 129 pgs: 129 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 170 KiB/s wr, 39 op/s 2024-05-17T09:24:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:10 smithi079 ceph-mon[35502]: pgmap v570: 129 pgs: 129 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 170 KiB/s wr, 39 op/s 2024-05-17T09:24:11.424 INFO:tasks.workunit.client.0.smithi019.stdout: 1 45995 1.65 MB/sec execute 9 sec latency 15.897 ms 2024-05-17T09:24:12.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46458 1.91 MB/sec execute 10 sec latency 12.320 ms 2024-05-17T09:24:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:12 smithi069 ceph-mon[35081]: pgmap v571: 129 pgs: 129 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 219 KiB/s wr, 38 op/s 2024-05-17T09:24:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:12 smithi019 ceph-mon[31620]: pgmap v571: 129 pgs: 129 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 219 KiB/s wr, 38 op/s 2024-05-17T09:24:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:12 smithi079 ceph-mon[35502]: pgmap v571: 129 pgs: 129 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 219 KiB/s wr, 38 op/s 2024-05-17T09:24:13.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46727 1.77 MB/sec execute 11 sec latency 14.787 ms 2024-05-17T09:24:14.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 47197 1.78 MB/sec execute 12 sec latency 15.545 ms 2024-05-17T09:24:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:14 smithi069 ceph-mon[35081]: pgmap v572: 129 pgs: 129 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 598 KiB/s wr, 87 op/s 2024-05-17T09:24:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:14 smithi019 ceph-mon[31620]: pgmap v572: 129 pgs: 129 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 598 KiB/s wr, 87 op/s 2024-05-17T09:24:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:14 smithi079 ceph-mon[35502]: pgmap v572: 129 pgs: 129 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 598 KiB/s wr, 87 op/s 2024-05-17T09:24:15.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 47744 1.89 MB/sec execute 13 sec latency 16.851 ms 2024-05-17T09:24:16.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 48370 1.80 MB/sec execute 14 sec latency 19.156 ms 2024-05-17T09:24:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:16 smithi069 ceph-mon[35081]: pgmap v573: 129 pgs: 129 active+clean; 33 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 772 KiB/s wr, 90 op/s 2024-05-17T09:24:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:16 smithi019 ceph-mon[31620]: pgmap v573: 129 pgs: 129 active+clean; 33 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 772 KiB/s wr, 90 op/s 2024-05-17T09:24:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:16 smithi079 ceph-mon[35502]: pgmap v573: 129 pgs: 129 active+clean; 33 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 772 KiB/s wr, 90 op/s 2024-05-17T09:24:17.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 48920 2.03 MB/sec execute 15 sec latency 15.098 ms 2024-05-17T09:24:18.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 49472 2.12 MB/sec execute 16 sec latency 16.187 ms 2024-05-17T09:24:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:18 smithi069 ceph-mon[35081]: pgmap v574: 129 pgs: 129 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 997 KiB/s wr, 139 op/s 2024-05-17T09:24:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:18 smithi019 ceph-mon[31620]: pgmap v574: 129 pgs: 129 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 997 KiB/s wr, 139 op/s 2024-05-17T09:24:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:18 smithi079 ceph-mon[35502]: pgmap v574: 129 pgs: 129 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 997 KiB/s wr, 139 op/s 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 -- 172.21.15.19:0/1843080782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988104010 msgr2=0x7f99881028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 --2- 172.21.15.19:0/1843080782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988104010 0x7f99881028f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9978009900 tx=0x7f997802a410 comp rx=0 tx=0).stop 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 -- 172.21.15.19:0/1843080782 shutdown_connections 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 --2- 172.21.15.19:0/1843080782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988108b40 0x7f998810af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 --2- 172.21.15.19:0/1843080782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99881044f0 0x7f9988102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 --2- 172.21.15.19:0/1843080782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988104010 0x7f99881028f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 -- 172.21.15.19:0/1843080782 >> 172.21.15.19:0/1843080782 conn(0x7f99880fd470 msgr2=0x7f99880ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 -- 172.21.15.19:0/1843080782 shutdown_connections 2024-05-17T09:24:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.047+0000 7f998cc3b640 1 -- 172.21.15.19:0/1843080782 wait complete. 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 Processor -- start 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 -- start start 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988104010 0x7f9988199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 0x7f998819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998819f270 0x7f99881a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f998810e4d0 con 0x7f998819f270 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f998810e350 con 0x7f9988108b40 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998cc3b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f998810e650 con 0x7f9988104010 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f9986d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998819f270 0x7f99881a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f997ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 0x7f998819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988104010 0x7f9988199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f997ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 0x7f998819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47116/0 (socket says 172.21.15.19:47116) 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f997ffff640 1 -- 172.21.15.19:0/3968051201 learned_addr learned my addr 172.21.15.19:0/3968051201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f998659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988104010 0x7f9988199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:36988/0 (socket says 172.21.15.19:36988) 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.048+0000 7f9986d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998819f270 0x7f99881a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44164/0 (socket says 172.21.15.19:44164) 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997ffff640 1 -- 172.21.15.19:0/3968051201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988104010 msgr2=0x7f9988199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997ffff640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988104010 0x7f9988199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997ffff640 1 -- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998819f270 msgr2=0x7f99881a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997ffff640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998819f270 0x7f99881a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997ffff640 1 -- 172.21.15.19:0/3968051201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99881a1d00 con 0x7f9988108b40 2024-05-17T09:24:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997ffff640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 0x7f998819a2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f997000e9f0 tx=0x7f997000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997f7fe640 1 -- 172.21.15.19:0/3968051201 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997000cd80 con 0x7f9988108b40 2024-05-17T09:24:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997f7fe640 1 -- 172.21.15.19:0/3968051201 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99700108c0 con 0x7f9988108b40 2024-05-17T09:24:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99881a1ff0 con 0x7f9988108b40 2024-05-17T09:24:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f997f7fe640 1 -- 172.21.15.19:0/3968051201 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99700187c0 con 0x7f9988108b40 2024-05-17T09:24:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.049+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881a2460 con 0x7f9988108b40 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.052+0000 7f9993bc7740 1 Processor -- start 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 -- start start 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x5589095e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f997f7fe640 1 -- 172.21.15.19:0/3968051201 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9970010430 con 0x7f9988108b40 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558909556c30 con 0x5589092224d0 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5589095467d0 con 0x55890921a2f0 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9993bc7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55890954a810 con 0x5589092431b0 2024-05-17T09:24:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f997ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f998659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f998659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095e4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47122/0 (socket says 172.21.15.19:47122) 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f998659c640 1 -- 172.21.15.19:0/2650770216 learned_addr learned my addr 172.21.15.19:0/2650770216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f997ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095eb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44180/0 (socket says 172.21.15.19:44180) 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.053+0000 7f9986d9d640 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x5589095e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f998659c640 1 -- 172.21.15.19:0/2650770216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 msgr2=0x5589095e43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f998659c640 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x5589095e43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f998659c640 1 -- 172.21.15.19:0/2650770216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 msgr2=0x5589095eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f998659c640 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f998659c640 1 -- 172.21.15.19:0/2650770216 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890925b210 con 0x55890921a2f0 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f9986d9d640 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x5589095e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f998659c640 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095e4920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9978031460 tx=0x7f9978005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5410aed4564aa3c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f997f7fe640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f99740b28a0 0x7f99740b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f997cff9640 1 -- 172.21.15.19:0/2650770216 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978035030 con 0x55890921a2f0 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f9986d9d640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f99740b28a0 0x7f99740b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f997cff9640 1 -- 172.21.15.19:0/2650770216 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978040070 con 0x55890921a2f0 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f997cff9640 1 -- 172.21.15.19:0/2650770216 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978003850 con 0x55890921a2f0 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.054+0000 7f997f7fe640 1 -- 172.21.15.19:0/3968051201 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f99700e2950 con 0x7f9988108b40 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 -- 172.21.15.19:0/2650770216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 msgr2=0x5589095e4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9986d9d640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f99740b28a0 0x7f99740b4db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5589095e5570 tx=0x7f9980009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095e4920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9978031460 tx=0x7f9978005fd0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 -- 172.21.15.19:0/2650770216 shutdown_connections 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095e4920 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2650770216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x5589095e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 -- 172.21.15.19:0/2650770216 >> 172.21.15.19:0/2650770216 conn(0x558909237300 msgr2=0x5589095dfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 -- 172.21.15.19:0/2650770216 shutdown_connections 2024-05-17T09:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.055+0000 7f9993bc7740 1 -- 172.21.15.19:0/2650770216 wait complete. 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 -- start start 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 Processor -- start 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55890921a2f0 0x5589095edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092431b0 0x5589095f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55890922c790 con 0x5589092431b0 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558909502890 con 0x5589092224d0 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5589095ec210 con 0x55890921a2f0 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9993bc7740 10 client.0 _ll_register_callbacks cb 0x558909505d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9986d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092431b0 0x5589095f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095ee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47130/0 (socket says 172.21.15.19:47130) 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f9986d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092431b0 0x5589095f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44184/0 (socket says 172.21.15.19:44184) 2024-05-17T09:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 -- 172.21.15.19:0/2150212089 learned_addr learned my addr 172.21.15.19:0/2150212089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f998659c640 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55890921a2f0 0x5589095edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 -- 172.21.15.19:0/2150212089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55890921a2f0 msgr2=0x5589095edd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55890921a2f0 0x5589095edd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 -- 172.21.15.19:0/2150212089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092431b0 msgr2=0x5589095f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.056+0000 7f997ffff640 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092431b0 0x5589095f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f997ffff640 1 -- 172.21.15.19:0/2150212089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558909327620 con 0x5589092224d0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f997ffff640 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095ee250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f997000e9c0 tx=0x7f99700a9ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e6842dde461977 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f996dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f996dffb640 1 -- 172.21.15.19:0/2150212089 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970010720 con 0x5589092224d0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9978040070 con 0x5589092224d0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f996dffb640 1 -- 172.21.15.19:0/2150212089 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f99700a6260 con 0x5589092224d0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f996dffb640 1 -- 172.21.15.19:0/2150212089 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99700b2360 con 0x5589092224d0 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f9993bc7740 10 client.26309 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589095f81a0 con 0x5589092224d0 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f996dffb640 1 -- 172.21.15.19:0/2150212089 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978040070 con 0x5589092224d0 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.057+0000 7f996dffb640 1 -- 172.21.15.19:0/2150212089 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f99700b2040 con 0x5589092224d0 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f9993bc7740 10 client.26309 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f9993bc7740 10 client.26309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f9993bc7740 20 client.26309 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558909201d80 0x5589095fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f9993bc7740 4 client.26309 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558909558b80 con 0x558909201d80 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f998659c640 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558909201d80 0x5589095fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f996dffb640 10 client.26309 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.058+0000 7f998659c640 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558909201d80 0x5589095fb990 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca5252ee9c68ade server_cookie=42828272027b55c6 in_seq=0 out_seq=0 2024-05-17T09:24:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f996dffb640 1 -- 172.21.15.19:0/2150212089 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558909558b80 con 0x558909201d80 2024-05-17T09:24:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f996dffb640 10 client.26309 handle_command_reply: tid=0 2024-05-17T09:24:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 client.26309 shutdown 2024-05-17T09:24:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 msgr2=0x5589095ee250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095ee250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f997000e9c0 tx=0x7f99700a9ac0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 shutdown_connections 2024-05-17T09:24:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558909201d80 0x5589095fb990 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092431b0 0x5589095f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095ee250 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f996dffb640 0 client.26309 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 --2- 172.21.15.19:0/2150212089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55890921a2f0 0x5589095edd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.059+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 >> 172.21.15.19:0/2150212089 conn(0x558909237300 msgr2=0x5589095e2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.060+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 shutdown_connections 2024-05-17T09:24:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.060+0000 7f9993bc7740 1 -- 172.21.15.19:0/2150212089 wait complete. 2024-05-17T09:24:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.060+0000 7f9993bc7740 20 client.26309 trim_cache size 0 max 16384 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.076+0000 7f9993bc7740 1 Processor -- start 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.076+0000 7f9993bc7740 1 -- start start 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.076+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 0x5589095e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909600140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f9993bc7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99881af260 con 0x55890921a2f0 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f9993bc7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99881acb30 con 0x5589092224d0 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f9993bc7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f99881aa980 con 0x5589092431b0 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 0x5589095e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 0x5589095e8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44198/0 (socket says 172.21.15.19:44198) 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 -- 172.21.15.19:0/4262415572 learned_addr learned my addr 172.21.15.19:0/4262415572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f997ffff640 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f9986d9d640 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909600140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 -- 172.21.15.19:0/4262415572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 msgr2=0x558909600140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909600140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 -- 172.21.15.19:0/4262415572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 msgr2=0x5589095f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 -- 172.21.15.19:0/4262415572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589095eee50 con 0x55890921a2f0 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f997ffff640 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f998659c640 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 0x5589095e8950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99780057c0 tx=0x7f9978000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb1dccf8568608a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.077+0000 7f996ffff640 1 -- 172.21.15.19:0/4262415572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978035040 con 0x55890921a2f0 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f996ffff640 1 -- 172.21.15.19:0/4262415572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978040040 con 0x55890921a2f0 2024-05-17T09:24:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f996ffff640 1 -- 172.21.15.19:0/4262415572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978050860 con 0x55890921a2f0 2024-05-17T09:24:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 -- 172.21.15.19:0/4262415572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 msgr2=0x5589095e8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 0x5589095e8950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99780057c0 tx=0x7f9978000bc0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 -- 172.21.15.19:0/4262415572 shutdown_connections 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909600140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589092224d0 0x5589095f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 --2- 172.21.15.19:0/4262415572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55890921a2f0 0x5589095e8950 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 -- 172.21.15.19:0/4262415572 >> 172.21.15.19:0/4262415572 conn(0x558909237300 msgr2=0x5589095e3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 -- 172.21.15.19:0/4262415572 shutdown_connections 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.078+0000 7f9993bc7740 1 -- 172.21.15.19:0/4262415572 wait complete. 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 -- start start 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 Processor -- start 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095f4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909662110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99881aaff0 con 0x5589092224d0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99881ac460 con 0x55890921a2f0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f99881ab6c0 con 0x5589092431b0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f9993bc7740 10 client.0 _ll_register_callbacks cb 0x7f99700a5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f997ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f998659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f998659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47146/0 (socket says 172.21.15.19:47146) 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.079+0000 7f998659c640 1 -- 172.21.15.19:0/68101732 learned_addr learned my addr 172.21.15.19:0/68101732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f9986d9d640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909662110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f997ffff640 1 -- 172.21.15.19:0/68101732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 msgr2=0x558909662110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f997ffff640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909662110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f997ffff640 1 -- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 msgr2=0x5589095f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f997ffff640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f997ffff640 1 -- 172.21.15.19:0/68101732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558909662700 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f998659c640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f9986d9d640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909662110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f997ffff640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095f4940 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f99700a6210 tx=0x7f99700a8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db038e4ba0db771d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f996dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f996dffb640 1 -- 172.21.15.19:0/68101732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99700c4020 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f996dffb640 1 -- 172.21.15.19:0/68101732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f99700a5630 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9978040040 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f996dffb640 1 -- 172.21.15.19:0/68101732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9970010720 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f9993bc7740 10 client.16653 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558909663a00 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.080+0000 7f996dffb640 1 -- 172.21.15.19:0/68101732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978040040 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f996dffb640 1 -- 172.21.15.19:0/68101732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f99700b2040 con 0x5589092224d0 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f9993bc7740 10 client.16653 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f9993bc7740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f9993bc7740 20 client.16653 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f9993bc7740 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5589091d71c0 0x5589096664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f9993bc7740 4 client.16653 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558909552360 con 0x5589091d71c0 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f998659c640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5589091d71c0 0x5589096664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f996dffb640 10 client.16653 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.081+0000 7f998659c640 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5589091d71c0 0x5589096664a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f1a84296f6fea2 server_cookie=373d691c648bfc0 in_seq=0 out_seq=0 2024-05-17T09:24:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f996dffb640 1 -- 172.21.15.19:0/68101732 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558909552360 con 0x5589091d71c0 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f996dffb640 10 client.16653 handle_command_reply: tid=0 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 client.16653 shutdown 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 msgr2=0x5589095f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095f4940 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f99700a6210 tx=0x7f99700a8b50 comp rx=0 tx=0).stop 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 shutdown_connections 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5589091d71c0 0x5589096664a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5589092431b0 0x558909662110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589092224d0 0x5589095f4940 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f996dffb640 0 client.16653 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 --2- 172.21.15.19:0/68101732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55890921a2f0 0x5589095f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 >> 172.21.15.19:0/68101732 conn(0x558909237300 msgr2=0x5589095e32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 shutdown_connections 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 1 -- 172.21.15.19:0/68101732 wait complete. 2024-05-17T09:24:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.097+0000 7f9993bc7740 20 client.16653 trim_cache size 0 max 16384 2024-05-17T09:24:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:24:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:24:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f99740b28a0 msgr2=0x7f99740b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f99740b28a0 0x7f99740b4db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5589095e5570 tx=0x7f9980009500 comp rx=0 tx=0).stop 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 msgr2=0x7f998819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 0x7f998819a2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f997000e9f0 tx=0x7f997000eeb0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 shutdown_connections 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f99740b28a0 0x7f99740b4db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998819f270 0x7f99881a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988108b40 0x7f998819a2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 --2- 172.21.15.19:0/3968051201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9988104010 0x7f9988199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 >> 172.21.15.19:0/3968051201 conn(0x7f99880fd470 msgr2=0x7f9988109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 shutdown_connections 2024-05-17T09:24:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:19.098+0000 7f998cc3b640 1 -- 172.21.15.19:0/3968051201 wait complete. 2024-05-17T09:24:19.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 49936 2.25 MB/sec execute 17 sec latency 15.626 ms 2024-05-17T09:24:20.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 50165 2.14 MB/sec execute 18 sec latency 78.482 ms 2024-05-17T09:24:20.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:20 smithi069 ceph-mon[35081]: pgmap v575: 129 pgs: 129 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:24:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:20 smithi019 ceph-mon[31620]: pgmap v575: 129 pgs: 129 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:24:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:20 smithi079 ceph-mon[35502]: pgmap v575: 129 pgs: 129 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:24:21.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 50561 2.06 MB/sec execute 19 sec latency 15.311 ms 2024-05-17T09:24:21.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:21 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:24:21.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:21 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:24:21.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:21 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:24:22.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 51091 2.18 MB/sec execute 20 sec latency 16.020 ms 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: pgmap v576: 129 pgs: 129 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 996 KiB/s wr, 119 op/s 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:24:22.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:24:22.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:24:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: pgmap v576: 129 pgs: 129 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 996 KiB/s wr, 119 op/s 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:24:22.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: pgmap v576: 129 pgs: 129 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 996 KiB/s wr, 119 op/s 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:24:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:24:22.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:24:23.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 51672 2.10 MB/sec execute 21 sec latency 16.299 ms 2024-05-17T09:24:23.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:23 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:24:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:23 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:24:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:23 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:24:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:23 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:24:23.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:23 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:24:23.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:23 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:24:23.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:23 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:24:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:23 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:24:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:23 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:24:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:23 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:24:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:23 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:24:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:23 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:24:24.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 52194 2.11 MB/sec execute 22 sec latency 15.289 ms 2024-05-17T09:24:24.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:24 smithi069 ceph-mon[35081]: pgmap v577: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-05-17T09:24:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:24 smithi019 ceph-mon[31620]: pgmap v577: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-05-17T09:24:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:24 smithi079 ceph-mon[35502]: pgmap v577: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-05-17T09:24:25.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 52977 2.30 MB/sec execute 23 sec latency 19.596 ms 2024-05-17T09:24:26.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 53393 2.35 MB/sec execute 24 sec latency 14.573 ms 2024-05-17T09:24:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:26 smithi069 ceph-mon[35081]: pgmap v578: 129 pgs: 129 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-17T09:24:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:26 smithi019 ceph-mon[31620]: pgmap v578: 129 pgs: 129 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-17T09:24:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:26 smithi079 ceph-mon[35502]: pgmap v578: 129 pgs: 129 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-17T09:24:27.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 53653 2.31 MB/sec execute 25 sec latency 84.205 ms 2024-05-17T09:24:28.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 53995 2.23 MB/sec execute 26 sec latency 14.316 ms 2024-05-17T09:24:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:28 smithi069 ceph-mon[35081]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-05-17T09:24:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:28 smithi019 ceph-mon[31620]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-05-17T09:24:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:28 smithi079 ceph-mon[35502]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-05-17T09:24:29.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54492 2.22 MB/sec execute 27 sec latency 15.344 ms 2024-05-17T09:24:29.482 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:24:29.482 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.65 2024-05-17T09:24:29.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:24:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:24:30.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 55078 2.25 MB/sec execute 28 sec latency 17.595 ms 2024-05-17T09:24:30.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:30 smithi069 ceph-mon[35081]: pgmap v580: 129 pgs: 129 active+clean; 36 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-17T09:24:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:30 smithi019 ceph-mon[31620]: pgmap v580: 129 pgs: 129 active+clean; 36 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-17T09:24:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:30 smithi079 ceph-mon[35502]: pgmap v580: 129 pgs: 129 active+clean; 36 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-17T09:24:31.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 55626 2.23 MB/sec execute 29 sec latency 63.155 ms 2024-05-17T09:24:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 -- 172.21.15.19:0/1359738342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad01034b0 msgr2=0x7ffad01038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/1359738342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad01034b0 0x7ffad01038c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffac00099c0 tx=0x7ffac0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 -- 172.21.15.19:0/1359738342 shutdown_connections 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/1359738342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad0108730 0x7ffad0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/1359738342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad01034b0 0x7ffad01038c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/1359738342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad0107de0 0x7ffad01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 -- 172.21.15.19:0/1359738342 >> 172.21.15.19:0/1359738342 conn(0x7ffad00fedb0 msgr2=0x7ffad01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 -- 172.21.15.19:0/1359738342 shutdown_connections 2024-05-17T09:24:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.663+0000 7ffad4c7a700 1 -- 172.21.15.19:0/1359738342 wait complete. 2024-05-17T09:24:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 Processor -- start 2024-05-17T09:24:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 -- start start 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 0x7ffad00ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad0107de0 0x7ffad00ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0108730 0x7ffad00ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0113210 con 0x7ffad0108730 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0113090 con 0x7ffad0107de0 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffad4c7a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0113390 con 0x7ffad01034b0 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7fface59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 0x7ffad00ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7fface59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 0x7ffad00ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39106/0 (socket says 172.21.15.19:39106) 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7fface59c700 1 -- 172.21.15.19:0/3214009151 learned_addr learned my addr 172.21.15.19:0/3214009151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.664+0000 7ffaced9d700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0108730 0x7ffad00ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7ffacdd9b700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad0107de0 0x7ffad00ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7fface59c700 1 -- 172.21.15.19:0/3214009151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad0107de0 msgr2=0x7ffad00ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7fface59c700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad0107de0 0x7ffad00ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7fface59c700 1 -- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0108730 msgr2=0x7ffad00ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7fface59c700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0108730 0x7ffad00ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7fface59c700 1 -- 172.21.15.19:0/3214009151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad01a90d0 con 0x7ffad01034b0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7fface59c700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 0x7ffad00ff290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006010 tx=0x7ffac0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0026070 con 0x7ffad01034b0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffac0003c50 con 0x7ffad01034b0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac000fa10 con 0x7ffad01034b0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad01a9360 con 0x7ffad01034b0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.665+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01a9760 con 0x7ffad01034b0 2024-05-17T09:24:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.666+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad00512b0 con 0x7ffad01034b0 2024-05-17T09:24:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.668+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffac0029070 con 0x7ffad01034b0 2024-05-17T09:24:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.669+0000 7ffac77fe700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffabc0b2880 0x7ffabc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:31.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.669+0000 7ffacdd9b700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffabc0b2880 0x7ffabc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:31.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.669+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffac00e2fa0 con 0x7ffad01034b0 2024-05-17T09:24:31.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.669+0000 7ffacdd9b700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffabc0b2880 0x7ffabc0b4d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffad0102420 tx=0x7ffab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.672+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffac0013070 con 0x7ffad01034b0 2024-05-17T09:24:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.984+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad01a9ad0 con 0x7ffad01034b0 2024-05-17T09:24:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.985+0000 7ffac77fe700 1 -- 172.21.15.19:0/3214009151 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7ffac00aa940 con 0x7ffad01034b0 2024-05-17T09:24:31.985 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:24:31.985 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:22:41.259768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:24:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffabc0b2880 msgr2=0x7ffabc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffabc0b2880 0x7ffabc0b4d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffad0102420 tx=0x7ffab8009260 comp rx=0 tx=0).stop 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 msgr2=0x7ffad00ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 0x7ffad00ff290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006010 tx=0x7ffac0004270 comp rx=0 tx=0).stop 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 shutdown_connections 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffabc0b2880 0x7ffabc0b4d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffad01034b0 0x7ffad00ff290 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad0108730 0x7ffad00ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 --2- 172.21.15.19:0/3214009151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad0107de0 0x7ffad00ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 >> 172.21.15.19:0/3214009151 conn(0x7ffad00fedb0 msgr2=0x7ffad0109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 shutdown_connections 2024-05-17T09:24:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:31.988+0000 7ffad4c7a700 1 -- 172.21.15.19:0/3214009151 wait complete. 2024-05-17T09:24:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-17T09:24:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.209+0000 7fd80fc3b640 1 -- 172.21.15.19:0/2021874609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808104e80 msgr2=0x7fd808105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.209+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/2021874609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808104e80 0x7fd808105290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009900 tx=0x7fd7f802a410 comp rx=0 tx=0).stop 2024-05-17T09:24:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.209+0000 7fd80fc3b640 1 -- 172.21.15.19:0/2021874609 shutdown_connections 2024-05-17T09:24:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.209+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/2021874609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 0x7fd80810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.209+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/2021874609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808104e80 0x7fd808105290 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.210+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/2021874609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808103c40 0x7fd808104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.210+0000 7fd80fc3b640 1 -- 172.21.15.19:0/2021874609 >> 172.21.15.19:0/2021874609 conn(0x7fd8080fee30 msgr2=0x7fd8081012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.210+0000 7fd80fc3b640 1 -- 172.21.15.19:0/2021874609 shutdown_connections 2024-05-17T09:24:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.210+0000 7fd80fc3b640 1 -- 172.21.15.19:0/2021874609 wait complete. 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.210+0000 7fd80fc3b640 1 Processor -- start 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.210+0000 7fd80fc3b640 1 -- start start 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80fc3b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 0x7fd808077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80fc3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808104e80 0x7fd808076120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80fc3b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 0x7fd808076660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80fc3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810f5f0 con 0x7fd808104e80 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80fc3b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810f470 con 0x7fd808109b80 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80fc3b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810f770 con 0x7fd808103c40 2024-05-17T09:24:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80d9d7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 0x7fd808077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80d1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808104e80 0x7fd808076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80e1d8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 0x7fd808076660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80d9d7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 0x7fd808077a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39124/0 (socket says 172.21.15.19:39124) 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80d9d7640 1 -- 172.21.15.19:0/176447077 learned_addr learned my addr 172.21.15.19:0/176447077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80e1d8640 1 -- 172.21.15.19:0/176447077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 msgr2=0x7fd808077a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 0x7fd808077a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80e1d8640 1 -- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808104e80 msgr2=0x7fd808076120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808104e80 0x7fd808076120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80e1d8640 1 -- 172.21.15.19:0/176447077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd808076d90 con 0x7fd808109b80 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 0x7fd808077a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.211+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808104e80 0x7fd808076120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.212+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 0x7fd808076660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd800009fd0 tx=0x7fd80000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.212+0000 7fd7feffd640 1 -- 172.21.15.19:0/176447077 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000ce20 con 0x7fd808109b80 2024-05-17T09:24:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.212+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081a4250 con 0x7fd808109b80 2024-05-17T09:24:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.212+0000 7fd7feffd640 1 -- 172.21.15.19:0/176447077 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd800005020 con 0x7fd808109b80 2024-05-17T09:24:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.212+0000 7fd7feffd640 1 -- 172.21.15.19:0/176447077 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800010690 con 0x7fd808109b80 2024-05-17T09:24:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.212+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a46c0 con 0x7fd808109b80 2024-05-17T09:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 Processor -- start 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd7feffd640 1 -- 172.21.15.19:0/176447077 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd800004a40 con 0x7fd808109b80 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 -- start start 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667c21b0 0x557166b633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 0x557166b63920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667a14d0 0x557166b6a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557166ad5c30 con 0x5571667c21b0 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557166ac57d0 con 0x5571667992f0 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd816bc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557166ac9810 con 0x5571667a14d0 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd80d1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667c21b0 0x557166b633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd80d9d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 0x557166b63920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd80d9d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 0x557166b63920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52264/0 (socket says 172.21.15.19:52264) 2024-05-17T09:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.216+0000 7fd80d9d7640 1 -- 172.21.15.19:0/2592556158 learned_addr learned my addr 172.21.15.19:0/2592556158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667a14d0 0x557166b6a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d9d7640 1 -- 172.21.15.19:0/2592556158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667a14d0 msgr2=0x557166b6a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667a14d0 0x557166b6a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d9d7640 1 -- 172.21.15.19:0/2592556158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667c21b0 msgr2=0x557166b633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667c21b0 0x557166b633a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d9d7640 1 -- 172.21.15.19:0/2592556158 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571667da210 con 0x5571667992f0 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd7feffd640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd7f40b28a0 0x7fd7f40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd7f40b28a0 0x7fd7f40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 0x557166b63920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd8040115c0 tx=0x7fd804011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d717c8775878862a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd7feffd640 1 -- 172.21.15.19:0/176447077 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd8000e2ca0 con 0x7fd808109b80 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd7f3fff640 1 -- 172.21.15.19:0/2592556158 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804005020 con 0x5571667992f0 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd7f3fff640 1 -- 172.21.15.19:0/2592556158 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8040047a0 con 0x5571667992f0 2024-05-17T09:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.217+0000 7fd7f3fff640 1 -- 172.21.15.19:0/2592556158 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80400f840 con 0x5571667992f0 2024-05-17T09:24:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 -- 172.21.15.19:0/2592556158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 msgr2=0x557166b63920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 0x557166b63920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd8040115c0 tx=0x7fd804011a80 comp rx=0 tx=0).stop 2024-05-17T09:24:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd7f40b28a0 0x7fd7f40b4db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd808077420 tx=0x7fd7f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 -- 172.21.15.19:0/2592556158 shutdown_connections 2024-05-17T09:24:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667a14d0 0x557166b6a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667992f0 0x557166b63920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 --2- 172.21.15.19:0/2592556158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667c21b0 0x557166b633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 -- 172.21.15.19:0/2592556158 >> 172.21.15.19:0/2592556158 conn(0x5571667b6300 msgr2=0x557166b5ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 -- 172.21.15.19:0/2592556158 shutdown_connections 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.218+0000 7fd816bc9740 1 -- 172.21.15.19:0/2592556158 wait complete. 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 -- start start 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 Processor -- start 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b6ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b6d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667c21b0 0x557166b74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571667ab790 con 0x5571667a14d0 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557166a81890 con 0x5571667c21b0 2024-05-17T09:24:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557166b6b270 con 0x5571667992f0 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd816bc9740 10 client.0 _ll_register_callbacks cb 0x557166a84d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.219+0000 7fd80d1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b6d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80d1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b6d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50632/0 (socket says 172.21.15.19:50632) 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80d1d6640 1 -- 172.21.15.19:0/3881558449 learned_addr learned my addr 172.21.15.19:0/3881558449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667c21b0 0x557166b74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b6ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 -- 172.21.15.19:0/3881558449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 msgr2=0x557166b6ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b6ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 -- 172.21.15.19:0/3881558449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 msgr2=0x557166b6d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b6d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 -- 172.21.15.19:0/3881558449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571668a6620 con 0x5571667c21b0 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b6d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b6ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667c21b0 0x557166b74150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8000a7180 tx=0x7fd8000aa750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85c783a9c44814d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd7f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.220+0000 7fd7f17fa640 1 -- 172.21.15.19:0/3881558449 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000b2d70 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd7f17fa640 1 -- 172.21.15.19:0/3881558449 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd8000051d0 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8040047a0 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd7f17fa640 1 -- 172.21.15.19:0/3881558449 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd800004d30 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 10 client.26333 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557166b77150 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd7f17fa640 1 -- 172.21.15.19:0/3881558449 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8040047a0 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd7f17fa640 1 -- 172.21.15.19:0/3881558449 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd8000a5a10 con 0x5571667c21b0 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 10 client.26333 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 10 client.26333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 20 client.26333 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557166780d80 0x557166b7a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 4 client.26333 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557166ad7b80 con 0x557166780d80 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.221+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557166780d80 0x557166b7a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.222+0000 7fd7f17fa640 10 client.26333 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.222+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557166780d80 0x557166b7a9b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1629457386a294a server_cookie=2afe356b82d222f9 in_seq=0 out_seq=0 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd7f17fa640 1 -- 172.21.15.19:0/3881558449 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557166ad7b80 con 0x557166780d80 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd7f17fa640 10 client.26333 handle_command_reply: tid=0 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 client.26333 shutdown 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667c21b0 msgr2=0x557166b74150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667c21b0 0x557166b74150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8000a7180 tx=0x7fd8000aa750 comp rx=0 tx=0).stop 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 shutdown_connections 2024-05-17T09:24:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557166780d80 0x557166b7a9b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667c21b0 0x557166b74150 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd7f17fa640 0 client.26333 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b6d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 --2- 172.21.15.19:0/3881558449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b6ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 >> 172.21.15.19:0/3881558449 conn(0x5571667b6300 msgr2=0x557166b61c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 shutdown_connections 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 1 -- 172.21.15.19:0/3881558449 wait complete. 2024-05-17T09:24:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.223+0000 7fd816bc9740 20 client.26333 trim_cache size 0 max 16384 2024-05-17T09:24:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 Processor -- start 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 -- start start 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557166780d80 0x557166b5ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b74070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b5d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081b14b0 con 0x5571667a14d0 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081aed80 con 0x557166780d80 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd816bc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081acbd0 con 0x5571667992f0 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd80e1d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b5d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd80d9d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557166780d80 0x557166b5ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd80e1d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b5d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50636/0 (socket says 172.21.15.19:50636) 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd80e1d8640 1 -- 172.21.15.19:0/448142220 learned_addr learned my addr 172.21.15.19:0/448142220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.241+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b74070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d9d7640 1 -- 172.21.15.19:0/448142220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 msgr2=0x557166b74070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b74070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d9d7640 1 -- 172.21.15.19:0/448142220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 msgr2=0x557166b5d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b5d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d9d7640 1 -- 172.21.15.19:0/448142220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557166b6a670 con 0x557166780d80 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b74070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b5d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557166780d80 0x557166b5ee70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd804000c00 tx=0x7fd804005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d90926ea6f699d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd7f37fe640 1 -- 172.21.15.19:0/448142220 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80401d040 con 0x557166780d80 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd7f37fe640 1 -- 172.21.15.19:0/448142220 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8040044d0 con 0x557166780d80 2024-05-17T09:24:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.242+0000 7fd7f37fe640 1 -- 172.21.15.19:0/448142220 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804003cc0 con 0x557166780d80 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 -- 172.21.15.19:0/448142220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557166780d80 msgr2=0x557166b5ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557166780d80 0x557166b5ee70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd804000c00 tx=0x7fd804005d30 comp rx=0 tx=0).stop 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 -- 172.21.15.19:0/448142220 shutdown_connections 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667a14d0 0x557166b5d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5571667992f0 0x557166b74070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 --2- 172.21.15.19:0/448142220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557166780d80 0x557166b5ee70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 -- 172.21.15.19:0/448142220 >> 172.21.15.19:0/448142220 conn(0x5571667b6300 msgr2=0x557166b5e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 -- 172.21.15.19:0/448142220 shutdown_connections 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 -- 172.21.15.19:0/448142220 wait complete. 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 -- start start 2024-05-17T09:24:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.243+0000 7fd816bc9740 1 Processor -- start 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557166780d80 0x557166b71400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 0x557166b71940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667a14d0 0x557166b72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081ad240 con 0x5571667992f0 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081ae6b0 con 0x5571667a14d0 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081ad910 con 0x557166780d80 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd816bc9740 10 client.0 _ll_register_callbacks cb 0x7fd8000a9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 0x557166b71940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 0x557166b71940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50638/0 (socket says 172.21.15.19:50638) 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 -- 172.21.15.19:0/45283605 learned_addr learned my addr 172.21.15.19:0/45283605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80e1d8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667a14d0 0x557166b72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557166780d80 0x557166b71400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 -- 172.21.15.19:0/45283605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557166780d80 msgr2=0x557166b71400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557166780d80 0x557166b71400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 -- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667a14d0 msgr2=0x557166b72820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667a14d0 0x557166b72820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d1d6640 1 -- 172.21.15.19:0/45283605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557166b72de0 con 0x5571667992f0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557166780d80 0x557166b71400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.244+0000 7fd80e1d8640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667a14d0 0x557166b72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd80d1d6640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 0x557166b71940 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd7f802a930 tx=0x7fd7f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54332ffc085b2fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd7f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd7f17fa640 1 -- 172.21.15.19:0/45283605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8035040 con 0x5571667992f0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd7f17fa640 1 -- 172.21.15.19:0/45283605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd7f8003dd0 con 0x5571667992f0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8040044d0 con 0x5571667992f0 2024-05-17T09:24:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd7f17fa640 1 -- 172.21.15.19:0/45283605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7f805f050 con 0x5571667992f0 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd816bc9740 10 client.16677 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557166b74110 con 0x5571667992f0 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.245+0000 7fd7f17fa640 1 -- 172.21.15.19:0/45283605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8040044d0 con 0x5571667992f0 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd7f17fa640 1 -- 172.21.15.19:0/45283605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd7f80594f0 con 0x5571667992f0 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd816bc9740 10 client.16677 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd816bc9740 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd816bc9740 20 client.16677 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd816bc9740 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5571667c21b0 0x557166be5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd816bc9740 4 client.16677 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557166ad1360 con 0x5571667c21b0 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5571667c21b0 0x557166be5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd7f17fa640 10 client.16677 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.246+0000 7fd80d9d7640 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5571667c21b0 0x557166be5590 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89772b3837927ebc server_cookie=3e6c0887539890ff in_seq=0 out_seq=0 2024-05-17T09:24:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.263+0000 7fd7f17fa640 1 -- 172.21.15.19:0/45283605 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557166ad1360 con 0x5571667c21b0 2024-05-17T09:24:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.263+0000 7fd7f17fa640 10 client.16677 handle_command_reply: tid=0 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.263+0000 7fd816bc9740 1 client.16677 shutdown 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 msgr2=0x557166b71940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 0x557166b71940 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd7f802a930 tx=0x7fd7f8000bc0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 shutdown_connections 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5571667c21b0 0x557166be5590 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571667a14d0 0x557166b72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571667992f0 0x557166b71940 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd7f17fa640 0 client.16677 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 --2- 172.21.15.19:0/45283605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557166780d80 0x557166b71400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 >> 172.21.15.19:0/45283605 conn(0x5571667b6300 msgr2=0x557166b667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 shutdown_connections 2024-05-17T09:24:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 1 -- 172.21.15.19:0/45283605 wait complete. 2024-05-17T09:24:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.264+0000 7fd816bc9740 20 client.16677 trim_cache size 0 max 16384 2024-05-17T09:24:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:24:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:24:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd7f40b28a0 msgr2=0x7fd7f40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd7f40b28a0 0x7fd7f40b4db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd808077420 tx=0x7fd7f8005fd0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 msgr2=0x7fd808076660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 0x7fd808076660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd800009fd0 tx=0x7fd80000ef10 comp rx=0 tx=0).stop 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 shutdown_connections 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd808109b80 0x7fd808076660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd7f40b28a0 0x7fd7f40b4db0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd808104e80 0x7fd808076120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 --2- 172.21.15.19:0/176447077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd808103c40 0x7fd808077a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 >> 172.21.15.19:0/176447077 conn(0x7fd8080fee30 msgr2=0x7fd808105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 shutdown_connections 2024-05-17T09:24:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:32.265+0000 7fd80fc3b640 1 -- 172.21.15.19:0/176447077 wait complete. 2024-05-17T09:24:32.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 56166 2.34 MB/sec execute 30 sec latency 55.211 ms 2024-05-17T09:24:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:32 smithi069 ceph-mon[35081]: pgmap v581: 129 pgs: 129 active+clean; 37 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-17T09:24:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:32 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3214009151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:32.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:32 smithi019 ceph-mon[31620]: pgmap v581: 129 pgs: 129 active+clean; 37 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-17T09:24:32.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:32 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3214009151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:32 smithi079 ceph-mon[35502]: pgmap v581: 129 pgs: 129 active+clean; 37 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-17T09:24:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:32 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3214009151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:33.027 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 146.65 --await-for 45.0' 2024-05-17T09:24:33.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 146.65 --await-for 45.0 2024-05-17T09:24:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:24:33.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 56611 2.33 MB/sec execute 31 sec latency 59.874 ms 2024-05-17T09:24:34.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57082 2.39 MB/sec execute 32 sec latency 14.896 ms 2024-05-17T09:24:34.660 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:34 smithi019 ceph-mon[31620]: pgmap v582: 129 pgs: 129 active+clean; 36 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-17T09:24:34.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:34 smithi069 ceph-mon[35081]: pgmap v582: 129 pgs: 129 active+clean; 36 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-17T09:24:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:34 smithi079 ceph-mon[35502]: pgmap v582: 129 pgs: 129 active+clean; 36 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-17T09:24:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 -- 172.21.15.19:0/2964591266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50108750 msgr2=0x7f6a501100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 --2- 172.21.15.19:0/2964591266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50108750 0x7f6a501100b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009fc0 tx=0x7f6a4800b330 comp rx=0 tx=0).stop 2024-05-17T09:24:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 -- 172.21.15.19:0/2964591266 shutdown_connections 2024-05-17T09:24:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 --2- 172.21.15.19:0/2964591266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 --2- 172.21.15.19:0/2964591266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50108750 0x7f6a501100b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 --2- 172.21.15.19:0/2964591266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a50107e00 0x7f6a50108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 -- 172.21.15.19:0/2964591266 >> 172.21.15.19:0/2964591266 conn(0x7f6a500fef20 msgr2=0x7f6a50101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 -- 172.21.15.19:0/2964591266 shutdown_connections 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.274+0000 7f6a56158700 1 -- 172.21.15.19:0/2964591266 wait complete. 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.275+0000 7f6a56158700 1 Processor -- start 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.275+0000 7f6a56158700 1 -- start start 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a56158700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a56158700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107e00 0x7f6a501a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a56158700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a50108750 0x7f6a501ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a56158700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501143b0 con 0x7f6a50107e00 2024-05-17T09:24:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a56158700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50114230 con 0x7f6a50108750 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a56158700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50114530 con 0x7f6a501034d0 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107e00 0x7f6a501a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a4ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a50108750 0x7f6a501ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56094/0 (socket says 172.21.15.19:56094) 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107e00 0x7f6a501a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50456/0 (socket says 172.21.15.19:50456) 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.276+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/978598622 learned_addr learned my addr 172.21.15.19:0/978598622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/978598622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a50108750 msgr2=0x7f6a501ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a50108750 0x7f6a501ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107e00 msgr2=0x7f6a501a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107e00 0x7f6a501a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/978598622 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a501ad620 con 0x7f6a501034d0 2024-05-17T09:24:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501a2060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000e9a0 tx=0x7f6a4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4d7fa700 1 -- 172.21.15.19:0/978598622 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000c9c0 con 0x7f6a501034d0 2024-05-17T09:24:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4d7fa700 1 -- 172.21.15.19:0/978598622 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a40010430 con 0x7f6a501034d0 2024-05-17T09:24:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a501ad850 con 0x7f6a501034d0 2024-05-17T09:24:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.277+0000 7f6a4d7fa700 1 -- 172.21.15.19:0/978598622 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40018680 con 0x7f6a501034d0 2024-05-17T09:24:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.278+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501add90 con 0x7f6a501034d0 2024-05-17T09:24:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.281+0000 7f6a4d7fa700 1 -- 172.21.15.19:0/978598622 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a40018820 con 0x7f6a501034d0 2024-05-17T09:24:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a4d7fa700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a3c0b2980 0x7f6a3c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 Processor -- start 2024-05-17T09:24:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a3c0b2980 0x7f6a3c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a4d7fa700 1 -- 172.21.15.19:0/978598622 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a400e27d0 con 0x7f6a501034d0 2024-05-17T09:24:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 -- start start 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d120600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d120b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d11aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f2d048e30 con 0x556f2d019570 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556f2d0389d0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556f2d03ca10 con 0x556f2d11a6c0 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.283+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a3c0b2980 0x7f6a3c0b4e90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6a50076aa0 tx=0x7f6a38009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a4ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d11aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d120600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50476/0 (socket says 172.21.15.19:50476) 2024-05-17T09:24:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/94448011 learned_addr learned my addr 172.21.15.19:0/94448011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a4ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d11aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56098/0 (socket says 172.21.15.19:56098) 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 -- 172.21.15.19:0/94448011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 msgr2=0x556f2d11aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d11aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 -- 172.21.15.19:0/94448011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 msgr2=0x556f2d120600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d120600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 -- 172.21.15.19:0/94448011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2cf88a70 con 0x556f2cf9bfe0 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.284+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d120b40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a38011c60 tx=0x7f6a38014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23495aff3cc57234 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.285+0000 7f6a45ffb700 1 -- 172.21.15.19:0/94448011 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004830 con 0x556f2cf9bfe0 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.285+0000 7f6a45ffb700 1 -- 172.21.15.19:0/94448011 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a38005020 con 0x556f2cf9bfe0 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.285+0000 7f6a45ffb700 1 -- 172.21.15.19:0/94448011 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380038e0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 -- 172.21.15.19:0/94448011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 msgr2=0x556f2d120b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d120b40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a38011c60 tx=0x7f6a38014000 comp rx=0 tx=0).stop 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 -- 172.21.15.19:0/94448011 shutdown_connections 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d11aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/94448011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d120b40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 -- 172.21.15.19:0/94448011 >> 172.21.15.19:0/94448011 conn(0x556f2cfdae20 msgr2=0x556f2d1155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 -- 172.21.15.19:0/94448011 shutdown_connections 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 -- 172.21.15.19:0/94448011 wait complete. 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.286+0000 7f6a64b6db80 1 -- start start 2024-05-17T09:24:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 Processor -- start 2024-05-17T09:24:35.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d124250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d124790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d124cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f2cf3f140 con 0x556f2d019570 2024-05-17T09:24:35.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556f2cfda960 con 0x556f2cf9bfe0 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556f2d1304a0 con 0x556f2d11a6c0 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a64b6db80 10 client.0 _ll_register_callbacks cb 0x556f2cfd3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d124790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d124790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50518/0 (socket says 172.21.15.19:50518) 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a46ffd700 1 -- 172.21.15.19:0/2681569518 learned_addr learned my addr 172.21.15.19:0/2681569518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a4ffff700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d124cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.287+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d124250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a46ffd700 1 -- 172.21.15.19:0/2681569518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 msgr2=0x556f2d124cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d124cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a46ffd700 1 -- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 msgr2=0x556f2d124250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d124250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a46ffd700 1 -- 172.21.15.19:0/2681569518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2cfd7830 con 0x556f2d019570 2024-05-17T09:24:35.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:24:35.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d124790 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6a38004670 tx=0x7f6a380046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728707a533f3626a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:35.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a2effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a2effd700 1 -- 172.21.15.19:0/2681569518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004c00 con 0x556f2d019570 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a38005020 con 0x556f2d019570 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a2effd700 1 -- 172.21.15.19:0/2681569518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a38004da0 con 0x556f2d019570 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.288+0000 7f6a2effd700 1 -- 172.21.15.19:0/2681569518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a38003680 con 0x556f2d019570 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 10 client.16689 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f2d131dc0 con 0x556f2d019570 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a2effd700 1 -- 172.21.15.19:0/2681569518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a38003cb0 con 0x556f2d019570 2024-05-17T09:24:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a2effd700 1 -- 172.21.15.19:0/2681569518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6a38031a00 con 0x556f2d019570 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 10 client.16689 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 10 client.16689 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 20 client.16689 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d136710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 4 client.16689 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a38005020 con 0x556f2d1342f0 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.289+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d136710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.290+0000 7f6a2effd700 10 client.16689 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.290+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d136710 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd08ea6e3496184 server_cookie=f80d0ad2e052ea91 in_seq=0 out_seq=0 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a2effd700 1 -- 172.21.15.19:0/2681569518 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f6a38005020 con 0x556f2d1342f0 2024-05-17T09:24:35.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a2effd700 10 client.16689 handle_command_reply: tid=0 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 client.16689 shutdown 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 msgr2=0x556f2d124790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d124790 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6a38004670 tx=0x7f6a380046a0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 shutdown_connections 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d124cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d136710 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a2effd700 0 client.16689 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2681569518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d124790 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 >> 172.21.15.19:0/2681569518 conn(0x556f2cfdae20 msgr2=0x556f2d10f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 shutdown_connections 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2681569518 wait complete. 2024-05-17T09:24:35.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.291+0000 7f6a64b6db80 20 client.16689 trim_cache size 0 max 16384 2024-05-17T09:24:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 Processor -- start 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 -- start start 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2cf9bfe0 0x556f2d123230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d10fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2d11a6c0 0x556f2d139730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501b9310 con 0x556f2d019570 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501b6ba0 con 0x556f2d11a6c0 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.316+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501b49f0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d10fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d10fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50528/0 (socket says 172.21.15.19:50528) 2024-05-17T09:24:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a46ffd700 1 -- 172.21.15.19:0/2845655088 learned_addr learned my addr 172.21.15.19:0/2845655088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2d11a6c0 0x556f2d139730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2cf9bfe0 0x556f2d123230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 -- 172.21.15.19:0/2845655088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2cf9bfe0 msgr2=0x556f2d123230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2cf9bfe0 0x556f2d123230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 -- 172.21.15.19:0/2845655088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 msgr2=0x556f2d10fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d10fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 -- 172.21.15.19:0/2845655088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2d1227e0 con 0x556f2d11a6c0 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d10fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a4ffff700 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2d11a6c0 0x556f2d139730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b6c0 tx=0x7f6a48009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662f45493a201329 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a457fa700 1 -- 172.21.15.19:0/2845655088 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48003ac0 con 0x556f2d11a6c0 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.317+0000 7f6a457fa700 1 -- 172.21.15.19:0/2845655088 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a48003c60 con 0x556f2d11a6c0 2024-05-17T09:24:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.318+0000 7f6a457fa700 1 -- 172.21.15.19:0/2845655088 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48007af0 con 0x556f2d11a6c0 2024-05-17T09:24:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.318+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2845655088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2d11a6c0 msgr2=0x556f2d139730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.318+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2d11a6c0 0x556f2d139730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b6c0 tx=0x7f6a48009d60 comp rx=0 tx=0).stop 2024-05-17T09:24:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2845655088 shutdown_connections 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2cf9bfe0 0x556f2d123230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2d11a6c0 0x556f2d139730 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2845655088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d10fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2845655088 >> 172.21.15.19:0/2845655088 conn(0x556f2cfdae20 msgr2=0x556f2d10f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2845655088 shutdown_connections 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2845655088 wait complete. 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 -- start start 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.319+0000 7f6a64b6db80 1 Processor -- start 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d139e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501b5060 con 0x556f2d019570 2024-05-17T09:24:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501b64d0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501b5730 con 0x556f2d11a6c0 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a64b6db80 10 client.0 _ll_register_callbacks cb 0x556f2cfd3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d1382d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50558/0 (socket says 172.21.15.19:50558) 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a46ffd700 1 -- 172.21.15.19:0/2866758347 learned_addr learned my addr 172.21.15.19:0/2866758347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a4ffff700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.320+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2866758347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 msgr2=0x556f2d138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2866758347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 msgr2=0x556f2d1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d1382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2866758347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2d138fb0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d139e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a400a6c50 tx=0x7f6a400dfd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c6066e0c898253 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a2effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a2effd700 1 -- 172.21.15.19:0/2866758347 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a400aa0e0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a2effd700 1 -- 172.21.15.19:0/2866758347 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a40018b10 con 0x556f2cf9bfe0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a48003c60 con 0x556f2cf9bfe0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a2effd700 1 -- 172.21.15.19:0/2866758347 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a400a99e0 con 0x556f2cf9bfe0 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a64b6db80 10 client.26354 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f2d19fa80 con 0x556f2cf9bfe0 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.321+0000 7f6a2effd700 1 -- 172.21.15.19:0/2866758347 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a48003c60 con 0x556f2cf9bfe0 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a2effd700 1 -- 172.21.15.19:0/2866758347 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6a4000cb60 con 0x556f2cf9bfe0 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a64b6db80 10 client.26354 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a64b6db80 10 client.26354 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a64b6db80 20 client.26354 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d1a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a64b6db80 4 client.26354 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.65, "await_for": 45.0}] 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.65, "await_for": 45.0}) -- 0x556f2d04ad80 con 0x556f2d1342f0 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d1a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.322+0000 7f6a2effd700 10 client.26354 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.323+0000 7f6a46ffd700 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d1a21c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a89911b11062645 server_cookie=f288e13ca6d6dbe7 in_seq=0 out_seq=0 2024-05-17T09:24:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a2effd700 1 -- 172.21.15.19:0/2866758347 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556f2d04ad80 con 0x556f2d1342f0 2024-05-17T09:24:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a2effd700 10 client.26354 handle_command_reply: tid=0 2024-05-17T09:24:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 client.26354 shutdown 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 msgr2=0x556f2d139e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d139e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a400a6c50 tx=0x7f6a400dfd90 comp rx=0 tx=0).stop 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 shutdown_connections 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556f2d11a6c0 0x556f2d138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556f2d1342f0 0x556f2d1a21c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556f2cf9bfe0 0x556f2d139e00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 --2- 172.21.15.19:0/2866758347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f2d019570 0x556f2d1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a2effd700 0 client.26354 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 >> 172.21.15.19:0/2866758347 conn(0x556f2cfdae20 msgr2=0x556f2d11e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 shutdown_connections 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.423+0000 7f6a64b6db80 1 -- 172.21.15.19:0/2866758347 wait complete. 2024-05-17T09:24:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a64b6db80 20 client.26354 trim_cache size 0 max 16384 2024-05-17T09:24:35.425 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:24:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-05-17T09:24:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:24:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 18, 2024-05-17T09:24:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "eaabf277": { 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 18, 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 146.7, 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:24:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:24:35.427 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:24:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a3c0b2980 msgr2=0x7f6a3c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a56158700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a3c0b2980 0x7f6a3c0b4e90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6a50076aa0 tx=0x7f6a38009310 comp rx=0 tx=0).stop 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 msgr2=0x7f6a501a2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a56158700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501a2060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000e9a0 tx=0x7f6a4000ecb0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 shutdown_connections 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.424+0000 7f6a56158700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a3c0b2980 0x7f6a3c0b4e90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.425+0000 7f6a56158700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a501034d0 0x7f6a501a2060 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.425+0000 7f6a56158700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a50108750 0x7f6a501ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.425+0000 7f6a56158700 1 --2- 172.21.15.19:0/978598622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107e00 0x7f6a501a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.425+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 >> 172.21.15.19:0/978598622 conn(0x7f6a500fef20 msgr2=0x7f6a501096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.425+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 shutdown_connections 2024-05-17T09:24:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:35.425+0000 7f6a56158700 1 -- 172.21.15.19:0/978598622 wait complete. 2024-05-17T09:24:35.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57295 2.33 MB/sec execute 33 sec latency 78.780 ms 2024-05-17T09:24:36.239 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eaabf277', quiesce took 6.8 seconds. Will release after: 19.849999999999998 2024-05-17T09:24:36.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 2.26 MB/sec execute 34 sec latency 962.217 ms 2024-05-17T09:24:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:36 smithi069 ceph-mon[35081]: pgmap v583: 129 pgs: 129 active+clean; 36 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-17T09:24:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:36 smithi019 ceph-mon[31620]: pgmap v583: 129 pgs: 129 active+clean; 36 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-17T09:24:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:36 smithi079 ceph-mon[35502]: pgmap v583: 129 pgs: 129 active+clean; 36 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-17T09:24:37.425 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 2.19 MB/sec execute 35 sec latency 1962.316 ms 2024-05-17T09:24:38.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 2.13 MB/sec execute 36 sec latency 2962.412 ms 2024-05-17T09:24:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:38 smithi069 ceph-mon[35081]: pgmap v584: 129 pgs: 129 active+clean; 38 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-17T09:24:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:38 smithi019 ceph-mon[31620]: pgmap v584: 129 pgs: 129 active+clean; 38 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-17T09:24:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:38 smithi079 ceph-mon[35502]: pgmap v584: 129 pgs: 129 active+clean; 38 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-17T09:24:39.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 2.08 MB/sec execute 37 sec latency 3962.535 ms 2024-05-17T09:24:40.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 2.02 MB/sec execute 38 sec latency 4962.641 ms 2024-05-17T09:24:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:40 smithi069 ceph-mon[35081]: pgmap v585: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-17T09:24:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:40 smithi019 ceph-mon[31620]: pgmap v585: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-17T09:24:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:40 smithi079 ceph-mon[35502]: pgmap v585: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-17T09:24:41.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.97 MB/sec execute 39 sec latency 5962.725 ms 2024-05-17T09:24:42.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.92 MB/sec execute 40 sec latency 6962.830 ms 2024-05-17T09:24:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:42 smithi069 ceph-mon[35081]: pgmap v586: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 885 KiB/s wr, 110 op/s 2024-05-17T09:24:42.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:42 smithi019 ceph-mon[31620]: pgmap v586: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 885 KiB/s wr, 110 op/s 2024-05-17T09:24:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:42 smithi079 ceph-mon[35502]: pgmap v586: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 885 KiB/s wr, 110 op/s 2024-05-17T09:24:43.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.87 MB/sec execute 41 sec latency 7962.918 ms 2024-05-17T09:24:44.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.83 MB/sec execute 42 sec latency 8963.024 ms 2024-05-17T09:24:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:44 smithi069 ceph-mon[35081]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 845 KiB/s wr, 109 op/s 2024-05-17T09:24:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:44 smithi019 ceph-mon[31620]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 845 KiB/s wr, 109 op/s 2024-05-17T09:24:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:44 smithi079 ceph-mon[35502]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 845 KiB/s wr, 109 op/s 2024-05-17T09:24:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 -- 172.21.15.19:0/2357035461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 msgr2=0x7f141c105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 --2- 172.21.15.19:0/2357035461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c105620 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1418009900 tx=0x7f141802a410 comp rx=0 tx=0).stop 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 -- 172.21.15.19:0/2357035461 shutdown_connections 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 --2- 172.21.15.19:0/2357035461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f141c1023c0 0x7f141c10be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 --2- 172.21.15.19:0/2357035461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141c1018c0 0x7f141c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 --2- 172.21.15.19:0/2357035461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c105620 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 -- 172.21.15.19:0/2357035461 >> 172.21.15.19:0/2357035461 conn(0x7f141c0fd500 msgr2=0x7f141c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 -- 172.21.15.19:0/2357035461 shutdown_connections 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.378+0000 7f1423477640 1 -- 172.21.15.19:0/2357035461 wait complete. 2024-05-17T09:24:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 Processor -- start 2024-05-17T09:24:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 -- start start 2024-05-17T09:24:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f141c1018c0 0x7f141c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141c1023c0 0x7f141c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f141c10f610 con 0x7f141c105210 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f141c10f490 con 0x7f141c1023c0 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1423477640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f141c10f790 con 0x7f141c1018c0 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c1a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55816/0 (socket says 172.21.15.19:55816) 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 -- 172.21.15.19:0/3580301303 learned_addr learned my addr 172.21.15.19:0/3580301303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421213640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f141c1018c0 0x7f141c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1420a12640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141c1023c0 0x7f141c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 -- 172.21.15.19:0/3580301303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f141c1018c0 msgr2=0x7f141c19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f141c1018c0 0x7f141c19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 -- 172.21.15.19:0/3580301303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141c1023c0 msgr2=0x7f141c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141c1023c0 0x7f141c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.379+0000 7f1421a14640 1 -- 172.21.15.19:0/3580301303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1a3f60 con 0x7f141c105210 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.380+0000 7f1421a14640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c1a38f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f141400eaa0 tx=0x7f141400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.380+0000 7f14127fc640 1 -- 172.21.15.19:0/3580301303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141400cdb0 con 0x7f141c105210 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.380+0000 7f14127fc640 1 -- 172.21.15.19:0/3580301303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1414005020 con 0x7f141c105210 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.380+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1a4250 con 0x7f141c105210 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.380+0000 7f14127fc640 1 -- 172.21.15.19:0/3580301303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140116d0 con 0x7f141c105210 2024-05-17T09:24:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.380+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1a4750 con 0x7f141c105210 2024-05-17T09:24:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.383+0000 7f142a405740 1 Processor -- start 2024-05-17T09:24:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.383+0000 7f14127fc640 1 -- 172.21.15.19:0/3580301303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f14140051d0 con 0x7f141c105210 2024-05-17T09:24:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 -- start start 2024-05-17T09:24:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc385b1b0 0x55fcc3bfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc38322f0 0x55fcc3bfc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 0x55fcc3c03820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fcc3b6ec30 con 0x55fcc383a4d0 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcc3b5e7d0 con 0x55fcc385b1b0 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f142a405740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fcc3b62810 con 0x55fcc38322f0 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 0x55fcc3c03820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 0x55fcc3c03820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55826/0 (socket says 172.21.15.19:55826) 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 -- 172.21.15.19:0/3343085645 learned_addr learned my addr 172.21.15.19:0/3343085645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1421213640 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc38322f0 0x55fcc3bfc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc385b1b0 0x55fcc3bfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 -- 172.21.15.19:0/3343085645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc38322f0 msgr2=0x55fcc3bfc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc38322f0 0x55fcc3bfc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 -- 172.21.15.19:0/3343085645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc385b1b0 msgr2=0x55fcc3bfc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc385b1b0 0x55fcc3bfc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.384+0000 7f1420a12640 1 -- 172.21.15.19:0/3343085645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc3873210 con 0x55fcc383a4d0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f1420a12640 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 0x55fcc3c03820 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f140c00efd0 tx=0x7f140c00e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d652a3933d9b2f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f14127fc640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f14080b28a0 0x7f14080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f14077fe640 1 -- 172.21.15.19:0/3343085645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c018030 con 0x55fcc383a4d0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f1421213640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f14080b28a0 0x7f14080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f14077fe640 1 -- 172.21.15.19:0/3343085645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f140c010b30 con 0x55fcc383a4d0 2024-05-17T09:24:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f14077fe640 1 -- 172.21.15.19:0/3343085645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c01cac0 con 0x55fcc383a4d0 2024-05-17T09:24:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.385+0000 7f14127fc640 1 -- 172.21.15.19:0/3580301303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f14140e2860 con 0x7f141c105210 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 -- 172.21.15.19:0/3343085645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 msgr2=0x55fcc3c03820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f1421213640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f14080b28a0 0x7f14080b4db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14180057c0 tx=0x7f1418031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 0x55fcc3c03820 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f140c00efd0 tx=0x7f140c00e610 comp rx=0 tx=0).stop 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 -- 172.21.15.19:0/3343085645 shutdown_connections 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc383a4d0 0x55fcc3c03820 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc38322f0 0x55fcc3bfc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 --2- 172.21.15.19:0/3343085645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc385b1b0 0x55fcc3bfc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 -- 172.21.15.19:0/3343085645 >> 172.21.15.19:0/3343085645 conn(0x55fcc384f300 msgr2=0x55fcc3bf7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 -- 172.21.15.19:0/3343085645 shutdown_connections 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 -- 172.21.15.19:0/3343085645 wait complete. 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.386+0000 7f142a405740 1 -- start start 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 Processor -- start 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c05c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc385b1b0 0x55fcc3c0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fcc3844790 con 0x55fcc38322f0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcc3b1a890 con 0x55fcc383a4d0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fcc3c04270 con 0x55fcc385b1b0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f142a405740 10 client.0 _ll_register_callbacks cb 0x55fcc3b1dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f1421213640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c05c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f1421213640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c05c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55836/0 (socket says 172.21.15.19:55836) 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f1421213640 1 -- 172.21.15.19:0/3444900785 learned_addr learned my addr 172.21.15.19:0/3444900785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f1420a12640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.387+0000 7f1421a14640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc385b1b0 0x55fcc3c0d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1421213640 1 -- 172.21.15.19:0/3444900785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc385b1b0 msgr2=0x55fcc3c0d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1421213640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc385b1b0 0x55fcc3c0d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1421213640 1 -- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 msgr2=0x55fcc3c061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1421213640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1421213640 1 -- 172.21.15.19:0/3444900785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc393f620 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1421213640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c05c80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f14180319f0 tx=0x7f1418009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b57f91e2be95e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1404ff9640 1 -- 172.21.15.19:0/3444900785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141804d030 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1404ff9640 1 -- 172.21.15.19:0/3444900785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1418004450 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140c010b30 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1404ff9640 1 -- 172.21.15.19:0/3444900785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f141805a050 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f142a405740 10 client.16707 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fcc3c100c0 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.388+0000 7f1404ff9640 1 -- 172.21.15.19:0/3444900785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f140c010b30 con 0x55fcc38322f0 2024-05-17T09:24:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f1404ff9640 1 -- 172.21.15.19:0/3444900785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f141803d650 con 0x55fcc38322f0 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f142a405740 10 client.16707 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f142a405740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f142a405740 20 client.16707 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f142a405740 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc3819d80 0x55fcc3c13940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f142a405740 4 client.16707 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcc3b70b80 con 0x55fcc3819d80 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f1420a12640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc3819d80 0x55fcc3c13940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f1420a12640 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc3819d80 0x55fcc3c13940 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2603a1aed8fb307 server_cookie=8acff9158c965aa5 in_seq=0 out_seq=0 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.389+0000 7f1404ff9640 10 client.16707 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f1404ff9640 1 -- 172.21.15.19:0/3444900785 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fcc3b70b80 con 0x55fcc3819d80 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f1404ff9640 10 client.16707 handle_command_reply: tid=0 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 client.16707 shutdown 2024-05-17T09:24:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 msgr2=0x55fcc3c05c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c05c80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f14180319f0 tx=0x7f1418009260 comp rx=0 tx=0).stop 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 shutdown_connections 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc3819d80 0x55fcc3c13940 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc385b1b0 0x55fcc3c0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f1404ff9640 0 client.16707 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 --2- 172.21.15.19:0/3444900785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c05c80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.390+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 >> 172.21.15.19:0/3444900785 conn(0x55fcc384f300 msgr2=0x55fcc3bfac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.391+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 shutdown_connections 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.391+0000 7f142a405740 1 -- 172.21.15.19:0/3444900785 wait complete. 2024-05-17T09:24:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.391+0000 7f142a405740 20 client.16707 trim_cache size 0 max 16384 2024-05-17T09:24:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 Processor -- start 2024-05-17T09:24:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 -- start start 2024-05-17T09:24:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc384f300 0x55fcc3c00850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1b1580 con 0x55fcc38322f0 2024-05-17T09:24:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1aee50 con 0x55fcc383a4d0 2024-05-17T09:24:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.406+0000 7f142a405740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1acc80 con 0x55fcc384f300 2024-05-17T09:24:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55850/0 (socket says 172.21.15.19:55850) 2024-05-17T09:24:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 -- 172.21.15.19:0/2675266528 learned_addr learned my addr 172.21.15.19:0/2675266528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1421213640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc384f300 0x55fcc3c00850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1421a14640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 -- 172.21.15.19:0/2675266528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc384f300 msgr2=0x55fcc3c00850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc384f300 0x55fcc3c00850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 -- 172.21.15.19:0/2675266528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 msgr2=0x55fcc3c0d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 -- 172.21.15.19:0/2675266528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc3c03630 con 0x55fcc38322f0 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1421213640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc384f300 0x55fcc3c00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1421a14640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1420a12640 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0d0b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f140c001eb0 tx=0x7f140c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f8b10cfc7f29b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1406ffd640 1 -- 172.21.15.19:0/2675266528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c02a070 con 0x55fcc38322f0 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1406ffd640 1 -- 172.21.15.19:0/2675266528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f140c010b30 con 0x55fcc38322f0 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.407+0000 7f1406ffd640 1 -- 172.21.15.19:0/2675266528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c018040 con 0x55fcc38322f0 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 -- 172.21.15.19:0/2675266528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 msgr2=0x55fcc3c0d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0d0b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f140c001eb0 tx=0x7f140c01c460 comp rx=0 tx=0).stop 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 -- 172.21.15.19:0/2675266528 shutdown_connections 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0d0b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 --2- 172.21.15.19:0/2675266528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc384f300 0x55fcc3c00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 -- 172.21.15.19:0/2675266528 >> 172.21.15.19:0/2675266528 conn(0x55fcc3c06390 msgr2=0x55fcc3bfb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 -- 172.21.15.19:0/2675266528 shutdown_connections 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 -- 172.21.15.19:0/2675266528 wait complete. 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.408+0000 7f142a405740 1 -- start start 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 Processor -- start 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc384f300 0x55fcc3c79fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1ad310 con 0x55fcc38322f0 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1ae780 con 0x55fcc384f300 2024-05-17T09:24:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1ad9e0 con 0x55fcc383a4d0 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f142a405740 10 client.0 _ll_register_callbacks cb 0x7f1418032a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55854/0 (socket says 172.21.15.19:55854) 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 -- 172.21.15.19:0/1245995599 learned_addr learned my addr 172.21.15.19:0/1245995599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421a14640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc384f300 0x55fcc3c79fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1420a12640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 -- 172.21.15.19:0/1245995599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc383a4d0 msgr2=0x55fcc3c0c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 -- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc384f300 msgr2=0x55fcc3c79fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc384f300 0x55fcc3c79fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1421213640 1 -- 172.21.15.19:0/1245995599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc3c0d100 con 0x55fcc38322f0 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.409+0000 7f1420a12640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:24:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1421213640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0c3d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1418049460 tx=0x7f141803dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9adfa285383f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1404ff9640 1 -- 172.21.15.19:0/1245995599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418057070 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1404ff9640 1 -- 172.21.15.19:0/1245995599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f14180035f0 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140c010b30 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1404ff9640 1 -- 172.21.15.19:0/1245995599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f141804d070 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 10 client.16713 fetch_fsmap learned FSMap version 30 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fcc3c7b470 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1404ff9640 1 -- 172.21.15.19:0/1245995599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f140c010b30 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f1404ff9640 1 -- 172.21.15.19:0/1245995599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1418049690 con 0x55fcc38322f0 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 10 client.16713 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 20 client.16713 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc385b1b0 0x55fcc3c7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 4 client.16713 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:24:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.410+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcc3b6a360 con 0x55fcc385b1b0 2024-05-17T09:24:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.411+0000 7f1420a12640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc385b1b0 0x55fcc3c7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.411+0000 7f1404ff9640 10 client.16713 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.411+0000 7f1420a12640 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc385b1b0 0x55fcc3c7df40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecb703d613f41b3 server_cookie=ff691dc59ea2c55c in_seq=0 out_seq=0 2024-05-17T09:24:45.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.79 MB/sec execute 43 sec latency 9963.069 ms 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f1404ff9640 1 -- 172.21.15.19:0/1245995599 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fcc3b6a360 con 0x55fcc385b1b0 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f1404ff9640 10 client.16713 handle_command_reply: tid=0 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 client.16713 shutdown 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 msgr2=0x55fcc3c0c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0c3d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1418049460 tx=0x7f141803dc80 comp rx=0 tx=0).stop 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 shutdown_connections 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcc385b1b0 0x55fcc3c7df40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcc384f300 0x55fcc3c79fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcc383a4d0 0x55fcc3c0c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f1404ff9640 0 client.16713 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 --2- 172.21.15.19:0/1245995599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcc38322f0 0x55fcc3c0c3d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 >> 172.21.15.19:0/1245995599 conn(0x55fcc3c06390 msgr2=0x55fcc3bfb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 shutdown_connections 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 1 -- 172.21.15.19:0/1245995599 wait complete. 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.430+0000 7f142a405740 20 client.16713 trim_cache size 0 max 16384 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:24:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:24:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f14080b28a0 msgr2=0x7f14080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f14080b28a0 0x7f14080b4db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14180057c0 tx=0x7f1418031400 comp rx=0 tx=0).stop 2024-05-17T09:24:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 msgr2=0x7f141c1a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c1a38f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f141400eaa0 tx=0x7f141400ef60 comp rx=0 tx=0).stop 2024-05-17T09:24:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 shutdown_connections 2024-05-17T09:24:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f14080b28a0 0x7f14080b4db0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c105210 0x7f141c1a38f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141c1023c0 0x7f141c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 --2- 172.21.15.19:0/3580301303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f141c1018c0 0x7f141c19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 >> 172.21.15.19:0/3580301303 conn(0x7f141c0fd500 msgr2=0x7f141c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 shutdown_connections 2024-05-17T09:24:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:45.431+0000 7f1423477640 1 -- 172.21.15.19:0/3580301303 wait complete. 2024-05-17T09:24:46.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.75 MB/sec execute 44 sec latency 10963.115 ms 2024-05-17T09:24:46.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:46 smithi019 ceph-mon[31620]: pgmap v588: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 587 KiB/s wr, 60 op/s 2024-05-17T09:24:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:46 smithi069 ceph-mon[35081]: pgmap v588: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 587 KiB/s wr, 60 op/s 2024-05-17T09:24:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:46 smithi079 ceph-mon[35502]: pgmap v588: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 587 KiB/s wr, 60 op/s 2024-05-17T09:24:47.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.71 MB/sec execute 45 sec latency 11963.186 ms 2024-05-17T09:24:48.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.67 MB/sec execute 46 sec latency 12963.234 ms 2024-05-17T09:24:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:48 smithi069 ceph-mon[35081]: pgmap v589: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 349 KiB/s wr, 57 op/s 2024-05-17T09:24:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:48 smithi019 ceph-mon[31620]: pgmap v589: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 349 KiB/s wr, 57 op/s 2024-05-17T09:24:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:48 smithi079 ceph-mon[35502]: pgmap v589: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 349 KiB/s wr, 57 op/s 2024-05-17T09:24:49.426 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.63 MB/sec execute 47 sec latency 13963.297 ms 2024-05-17T09:24:50.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.60 MB/sec execute 48 sec latency 14963.387 ms 2024-05-17T09:24:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:50 smithi069 ceph-mon[35081]: pgmap v590: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-05-17T09:24:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:50 smithi019 ceph-mon[31620]: pgmap v590: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-05-17T09:24:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:50 smithi079 ceph-mon[35502]: pgmap v590: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-05-17T09:24:51.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.57 MB/sec execute 49 sec latency 15963.475 ms 2024-05-17T09:24:52.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.54 MB/sec execute 50 sec latency 16963.598 ms 2024-05-17T09:24:52.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:52 smithi019 ceph-mon[31620]: pgmap v591: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:24:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:52 smithi079 ceph-mon[35502]: pgmap v591: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:24:52.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:52 smithi069 ceph-mon[35081]: pgmap v591: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:24:53.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.51 MB/sec execute 51 sec latency 17963.670 ms 2024-05-17T09:24:54.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.48 MB/sec execute 52 sec latency 18963.790 ms 2024-05-17T09:24:54.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:54 smithi019 ceph-mon[31620]: pgmap v592: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-17T09:24:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:54 smithi079 ceph-mon[35502]: pgmap v592: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-17T09:24:54.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:54 smithi069 ceph-mon[35081]: pgmap v592: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-17T09:24:55.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.45 MB/sec execute 53 sec latency 19963.843 ms 2024-05-17T09:24:56.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:24:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:24:56.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.42 MB/sec execute 54 sec latency 20963.926 ms 2024-05-17T09:24:56.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:56 smithi019 ceph-mon[31620]: pgmap v593: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-05-17T09:24:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:56 smithi079 ceph-mon[35502]: pgmap v593: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-05-17T09:24:56.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:56 smithi069 ceph-mon[35081]: pgmap v593: 129 pgs: 129 active+clean; 38 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-05-17T09:24:57.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.40 MB/sec execute 55 sec latency 21964.016 ms 2024-05-17T09:24:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/3430532650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34107e10 msgr2=0x7f5b34108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/3430532650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34107e10 0x7f5b34108220 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5b240099c0 tx=0x7f5b24009cd0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/3430532650 shutdown_connections 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/3430532650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b34103530 0x7f5b34103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/3430532650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34108760 0x7f5b341100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/3430532650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34107e10 0x7f5b34108220 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/3430532650 >> 172.21.15.19:0/3430532650 conn(0x7f5b34078e40 msgr2=0x7f5b34079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/3430532650 shutdown_connections 2024-05-17T09:24:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.243+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/3430532650 wait complete. 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 Processor -- start 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 -- start start 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34103530 0x7f5b341108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b34107e10 0x7f5b34110de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 0x7f5b34111320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341143b0 con 0x7f5b34108760 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34114230 con 0x7f5b34103530 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3c3c9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34114530 con 0x7f5b34107e10 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 0x7f5b34111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 0x7f5b34111320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45898/0 (socket says 172.21.15.19:45898) 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 -- 172.21.15.19:0/2130674469 learned_addr learned my addr 172.21.15.19:0/2130674469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a165700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34103530 0x7f5b341108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b39964700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b34107e10 0x7f5b34110de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 -- 172.21.15.19:0/2130674469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b34107e10 msgr2=0x7f5b34110de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b34107e10 0x7f5b34110de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 -- 172.21.15.19:0/2130674469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34103530 msgr2=0x7f5b341108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34103530 0x7f5b341108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.244+0000 7f5b3a966700 1 -- 172.21.15.19:0/2130674469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b341a9100 con 0x7f5b34108760 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.245+0000 7f5b3a966700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 0x7f5b34111320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c007ee0 tx=0x7f5b2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.245+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00dbd0 con 0x7f5b34108760 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.245+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b341a93f0 con 0x7f5b34108760 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.245+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b2c00dd70 con 0x7f5b34108760 2024-05-17T09:24:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.245+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0058d0 con 0x7f5b34108760 2024-05-17T09:24:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.245+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b341a98e0 con 0x7f5b34108760 2024-05-17T09:24:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.246+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b340512b0 con 0x7f5b34108760 2024-05-17T09:24:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.248+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5b2c01a020 con 0x7f5b34108760 2024-05-17T09:24:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.248+0000 7f5b2b7fe700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5b200b2880 0x7f5b200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.249+0000 7f5b3a165700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5b200b2880 0x7f5b200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.249+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5b2c0e2370 con 0x7f5b34108760 2024-05-17T09:24:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.249+0000 7f5b3a165700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5b200b2880 0x7f5b200b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009c60 tx=0x7f5b30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.252+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b2c010070 con 0x7f5b34108760 2024-05-17T09:24:58.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.37 MB/sec execute 56 sec latency 22964.063 ms 2024-05-17T09:24:58.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.522+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b3402e7a0 con 0x7f5b34108760 2024-05-17T09:24:58.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.523+0000 7f5b2b7fe700 1 -- 172.21.15.19:0/2130674469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11854 (secure 0 0 0) 0x7f5b2c0a9ce0 con 0x7f5b34108760 2024-05-17T09:24:58.524 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:24:58.524 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:24:57.624115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24487]},"id":1}]} 2024-05-17T09:24:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5b200b2880 msgr2=0x7f5b200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5b200b2880 0x7f5b200b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009c60 tx=0x7f5b30009260 comp rx=0 tx=0).stop 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 msgr2=0x7f5b34111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 0x7f5b34111320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c007ee0 tx=0x7f5b2c00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 shutdown_connections 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b34108760 0x7f5b34111320 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34103530 0x7f5b341108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5b200b2880 0x7f5b200b4d90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 --2- 172.21.15.19:0/2130674469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b34107e10 0x7f5b34110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 >> 172.21.15.19:0/2130674469 conn(0x7f5b34078e40 msgr2=0x7f5b34101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 shutdown_connections 2024-05-17T09:24:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:24:58.526+0000 7f5b3c3c9700 1 -- 172.21.15.19:0/2130674469 wait complete. 2024-05-17T09:24:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 -- 172.21.15.19:0/3687917687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104080 msgr2=0x7fa6c0102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 --2- 172.21.15.19:0/3687917687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104080 0x7fa6c0102970 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009900 tx=0x7fa6b002a410 comp rx=0 tx=0).stop 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 -- 172.21.15.19:0/3687917687 shutdown_connections 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 --2- 172.21.15.19:0/3687917687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6c0108bc0 0x7fa6c010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 --2- 172.21.15.19:0/3687917687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c0104520 0x7fa6c0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 --2- 172.21.15.19:0/3687917687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104080 0x7fa6c0102970 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 -- 172.21.15.19:0/3687917687 >> 172.21.15.19:0/3687917687 conn(0x7fa6c00fd470 msgr2=0x7fa6c00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 -- 172.21.15.19:0/3687917687 shutdown_connections 2024-05-17T09:24:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.534+0000 7fa6c645a640 1 -- 172.21.15.19:0/3687917687 wait complete. 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 Processor -- start 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 -- start start 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6c0104080 0x7fa6c01022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 0x7fa6c019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c0108bc0 0x7fa6c01a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c010e4c0 con 0x7fa6c0104520 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c010e340 con 0x7fa6c0108bc0 2024-05-17T09:24:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.535+0000 7fa6c645a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c010e640 con 0x7fa6c0104080 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 0x7fa6c019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 0x7fa6c019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45920/0 (socket says 172.21.15.19:45920) 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/1820165783 learned_addr learned my addr 172.21.15.19:0/1820165783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c0108bc0 0x7fa6c01a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bffff640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6c0104080 0x7fa6c01022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/1820165783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6c0104080 msgr2=0x7fa6c01022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6c0104080 0x7fa6c01022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/1820165783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c0108bc0 msgr2=0x7fa6c01a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c0108bc0 0x7fa6c01a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/1820165783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01a1ce0 con 0x7fa6c0104520 2024-05-17T09:24:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 0x7fa6c019ef70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea60 tx=0x7fa6b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.536+0000 7fa6bd7fa640 1 -- 172.21.15.19:0/1820165783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400cde0 con 0x7fa6c0104520 2024-05-17T09:24:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.537+0000 7fa6bd7fa640 1 -- 172.21.15.19:0/1820165783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6b40035f0 con 0x7fa6c0104520 2024-05-17T09:24:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.537+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01a1fd0 con 0x7fa6c0104520 2024-05-17T09:24:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.537+0000 7fa6bd7fa640 1 -- 172.21.15.19:0/1820165783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4010690 con 0x7fa6c0104520 2024-05-17T09:24:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.537+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01a2440 con 0x7fa6c0104520 2024-05-17T09:24:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.539+0000 7fa6cd3ea740 1 Processor -- start 2024-05-17T09:24:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 -- start start 2024-05-17T09:24:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6bd7fa640 1 -- 172.21.15.19:0/1820165783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa6b4010830 con 0x7fa6c0104520 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 0x559a1ac3b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8712f0 0x559a1ac3b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac42820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a1abadc30 con 0x559a1a89a1b0 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559a1ab9d7d0 con 0x559a1a8712f0 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a1aba1810 con 0x559a1a8794d0 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 0x559a1ac3b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 0x559a1ac3b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45924/0 (socket says 172.21.15.19:45924) 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 -- 172.21.15.19:0/288068529 learned_addr learned my addr 172.21.15.19:0/288068529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac42820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6bffff640 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8712f0 0x559a1ac3b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 -- 172.21.15.19:0/288068529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 msgr2=0x559a1ac42820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac42820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 -- 172.21.15.19:0/288068529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8712f0 msgr2=0x559a1ac3b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8712f0 0x559a1ac3b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.540+0000 7fa6c49f7640 1 -- 172.21.15.19:0/288068529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1a8b2210 con 0x559a1a89a1b0 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6bd7fa640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a80b28a0 0x7fa6a80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 0x559a1ac3b3a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a7ca31fcfa7095 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6bffff640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a80b28a0 0x7fa6a80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6bd7fa640 1 -- 172.21.15.19:0/1820165783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6b40e2850 con 0x7fa6c0104520 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6a67fc640 1 -- 172.21.15.19:0/288068529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8004d20 con 0x559a1a89a1b0 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6a67fc640 1 -- 172.21.15.19:0/288068529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6b8004050 con 0x559a1a89a1b0 2024-05-17T09:24:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.541+0000 7fa6a67fc640 1 -- 172.21.15.19:0/288068529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8005260 con 0x559a1a89a1b0 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/288068529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 msgr2=0x559a1ac3b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6bffff640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a80b28a0 0x7fa6a80b4db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0005ab0 tx=0x7fa6b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 0x559a1ac3b3a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b80118d0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/288068529 shutdown_connections 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac42820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8712f0 0x559a1ac3b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/288068529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a89a1b0 0x559a1ac3b3a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/288068529 >> 172.21.15.19:0/288068529 conn(0x559a1a88e300 msgr2=0x559a1ac36d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/288068529 shutdown_connections 2024-05-17T09:24:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.542+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/288068529 wait complete. 2024-05-17T09:24:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.543+0000 7fa6cd3ea740 1 -- start start 2024-05-17T09:24:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.543+0000 7fa6cd3ea740 1 Processor -- start 2024-05-17T09:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac44cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a89a1b0 0x559a1ac4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a1a883790 con 0x559a1a8712f0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559a1ab59890 con 0x559a1a89a1b0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a1ac43270 con 0x559a1a8794d0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6cd3ea740 10 client.0 _ll_register_callbacks cb 0x559a1ab5cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac44cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac44cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45930/0 (socket says 172.21.15.19:45930) 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 -- 172.21.15.19:0/2486965656 learned_addr learned my addr 172.21.15.19:0/2486965656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a89a1b0 0x559a1ac4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 -- 172.21.15.19:0/2486965656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 msgr2=0x559a1ac451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 -- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a89a1b0 msgr2=0x559a1ac4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a89a1b0 0x559a1ac4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 -- 172.21.15.19:0/2486965656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1a97e620 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa6bffff640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac44cb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00057c0 tx=0x7fa6b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac4be552a4791fc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.544+0000 7fa69f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa69f7fe640 1 -- 172.21.15.19:0/2486965656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0035070 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa69f7fe640 1 -- 172.21.15.19:0/2486965656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6b0004240 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b8004050 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa69f7fe640 1 -- 172.21.15.19:0/2486965656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa6b0004440 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 10 client.16731 fetch_fsmap learned FSMap version 31 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559a1ac4f0f0 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa69f7fe640 1 -- 172.21.15.19:0/2486965656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6b8004050 con 0x559a1a8712f0 2024-05-17T09:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa69f7fe640 1 -- 172.21.15.19:0/2486965656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fa6b0043070 con 0x559a1a8712f0 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 10 client.16731 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 20 client.16731 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a858d80 0x559a1ac52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 4 client.16731 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.545+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a1abafb80 con 0x559a1a858d80 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.546+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a858d80 0x559a1ac52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.546+0000 7fa69f7fe640 10 client.16731 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.546+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a858d80 0x559a1ac52900 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3a6dc36d11f8e server_cookie=3d94d306f502577d in_seq=0 out_seq=0 2024-05-17T09:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.547+0000 7fa69f7fe640 1 -- 172.21.15.19:0/2486965656 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559a1abafb80 con 0x559a1a858d80 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.547+0000 7fa69f7fe640 10 client.16731 handle_command_reply: tid=0 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.547+0000 7fa6cd3ea740 1 client.16731 shutdown 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.547+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 msgr2=0x559a1ac44cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.547+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac44cb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00057c0 tx=0x7fa6b002a410 comp rx=0 tx=0).stop 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 shutdown_connections 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a858d80 0x559a1ac52900 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a89a1b0 0x559a1ac4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8794d0 0x559a1ac451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2486965656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac44cb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 >> 172.21.15.19:0/2486965656 conn(0x559a1a88e300 msgr2=0x559a1ac35d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa69f7fe640 0 client.16731 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 shutdown_connections 2024-05-17T09:24:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2486965656 wait complete. 2024-05-17T09:24:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.548+0000 7fa6cd3ea740 20 client.16731 trim_cache size 0 max 16384 2024-05-17T09:24:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.564+0000 7fa6cd3ea740 1 Processor -- start 2024-05-17T09:24:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.564+0000 7fa6cd3ea740 1 -- start start 2024-05-17T09:24:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a88e300 0x559a1ac3a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac4c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8794d0 0x559a1ac4c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b1520 con 0x559a1a8712f0 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01aedf0 con 0x559a1a8794d0 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01acc40 con 0x559a1a88e300 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac4c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac4c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45946/0 (socket says 172.21.15.19:45946) 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/2191568690 learned_addr learned my addr 172.21.15.19:0/2191568690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8794d0 0x559a1ac4c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bffff640 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a88e300 0x559a1ac3a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/2191568690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a88e300 msgr2=0x559a1ac3a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a88e300 0x559a1ac3a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/2191568690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8794d0 msgr2=0x559a1ac4c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8794d0 0x559a1ac4c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/2191568690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1ac3fcd0 con 0x559a1a8712f0 2024-05-17T09:24:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.565+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac4c0e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40a98a0 tx=0x7fa6b40dfe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cab2d0781868bc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.566+0000 7fa6a5ffb640 1 -- 172.21.15.19:0/2191568690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40a9310 con 0x559a1a8712f0 2024-05-17T09:24:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.566+0000 7fa6a5ffb640 1 -- 172.21.15.19:0/2191568690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6b40b2260 con 0x559a1a8712f0 2024-05-17T09:24:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.566+0000 7fa6a5ffb640 1 -- 172.21.15.19:0/2191568690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40a5330 con 0x559a1a8712f0 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2191568690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 msgr2=0x559a1ac4c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac4c0e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40a98a0 tx=0x7fa6b40dfe70 comp rx=0 tx=0).stop 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2191568690 shutdown_connections 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a8794d0 0x559a1ac4c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8712f0 0x559a1ac4c0e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/2191568690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a88e300 0x559a1ac3a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2191568690 >> 172.21.15.19:0/2191568690 conn(0x559a1ac453c0 msgr2=0x559a1ac3e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2191568690 shutdown_connections 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/2191568690 wait complete. 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 -- start start 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.567+0000 7fa6cd3ea740 1 Processor -- start 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8712f0 0x559a1ac49e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 0x559a1ac4a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a88e300 0x559a1ac4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01ad2b0 con 0x559a1a8794d0 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01ae720 con 0x559a1a88e300 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01ad980 con 0x559a1a8712f0 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6cd3ea740 10 client.0 _ll_register_callbacks cb 0x7fa6b00318e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 0x559a1ac4a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 0x559a1ac4a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45950/0 (socket says 172.21.15.19:45950) 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/3358543475 learned_addr learned my addr 172.21.15.19:0/3358543475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bffff640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8712f0 0x559a1ac49e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6c49f7640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a88e300 0x559a1ac4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/3358543475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8712f0 msgr2=0x559a1ac49e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8712f0 0x559a1ac49e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a88e300 msgr2=0x559a1ac4b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a88e300 0x559a1ac4b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.568+0000 7fa6bf7fe640 1 -- 172.21.15.19:0/3358543475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1ac4b8b0 con 0x559a1a8794d0 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6bf7fe640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 0x559a1ac4a3c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40e0310 tx=0x7fa6b40e8170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300f41dc331d5230 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa69f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:24:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa69f7fe640 1 -- 172.21.15.19:0/3358543475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40c4070 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa69f7fe640 1 -- 172.21.15.19:0/3358543475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6b4003ac0 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a1aba9360 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa69f7fe640 1 -- 172.21.15.19:0/3358543475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa6b40ade70 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 10 client.16737 fetch_fsmap learned FSMap version 31 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559a1ac4cbb0 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa69f7fe640 1 -- 172.21.15.19:0/3358543475 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559a1aba9360 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa69f7fe640 1 -- 172.21.15.19:0/3358543475 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fa6b40a5630 con 0x559a1a8794d0 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 10 client.16737 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 20 client.16737 populate_metadata read hostname 'smithi019' 2024-05-17T09:24:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a89a1b0 0x559a1acbd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:24:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 4 client.16737 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:24:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.569+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a1ab235f0 con 0x559a1a89a1b0 2024-05-17T09:24:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.570+0000 7fa6bffff640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a89a1b0 0x559a1acbd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:24:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.570+0000 7fa69f7fe640 10 client.16737 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.570+0000 7fa6bffff640 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a89a1b0 0x559a1acbd710 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbfe4c0bf87f98b server_cookie=ba303df701aa1a92 in_seq=0 out_seq=0 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.586+0000 7fa69f7fe640 1 -- 172.21.15.19:0/3358543475 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559a1ab235f0 con 0x559a1a89a1b0 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.586+0000 7fa69f7fe640 10 client.16737 handle_command_reply: tid=0 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.586+0000 7fa6cd3ea740 1 client.16737 shutdown 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 msgr2=0x559a1ac4a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 0x559a1ac4a3c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40e0310 tx=0x7fa6b40e8170 comp rx=0 tx=0).stop 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 shutdown_connections 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559a1a89a1b0 0x559a1acbd710 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a1a88e300 0x559a1ac4b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa69f7fe640 0 client.16737 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a1a8794d0 0x559a1ac4a3c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 --2- 172.21.15.19:0/3358543475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a1a8712f0 0x559a1ac49e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 >> 172.21.15.19:0/3358543475 conn(0x559a1ac453c0 msgr2=0x559a1ac3a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 shutdown_connections 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 1 -- 172.21.15.19:0/3358543475 wait complete. 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6cd3ea740 20 client.16737 trim_cache size 0 max 16384 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:24:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:24:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a80b28a0 msgr2=0x7fa6a80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.587+0000 7fa6c645a640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a80b28a0 0x7fa6a80b4db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0005ab0 tx=0x7fa6b0000bc0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 msgr2=0x7fa6c019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 0x7fa6c019ef70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea60 tx=0x7fa6b400ef20 comp rx=0 tx=0).stop 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 shutdown_connections 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c0108bc0 0x7fa6c01a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a80b28a0 0x7fa6a80b4db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c0104520 0x7fa6c019ef70 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 --2- 172.21.15.19:0/1820165783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6c0104080 0x7fa6c01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 >> 172.21.15.19:0/1820165783 conn(0x7fa6c00fd470 msgr2=0x7fa6c0104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 shutdown_connections 2024-05-17T09:24:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:24:58.588+0000 7fa6c645a640 1 -- 172.21.15.19:0/1820165783 wait complete. 2024-05-17T09:24:58.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:58 smithi079 ceph-mon[35502]: pgmap v594: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-05-17T09:24:58.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:58 smithi019 ceph-mon[31620]: pgmap v594: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-05-17T09:24:58.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:58 smithi069 ceph-mon[35081]: pgmap v594: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-05-17T09:24:59.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.35 MB/sec execute 57 sec latency 23964.170 ms 2024-05-17T09:24:59.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:59 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:24:59.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:24:59 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2130674469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:59.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:59 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:24:59.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:24:59 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2130674469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:24:59.909 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id eaabf277 --release --await' 2024-05-17T09:24:59.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id eaabf277 --release --await 2024-05-17T09:24:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:59 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:24:59.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:24:59 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2130674469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:25:00.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.32 MB/sec execute 58 sec latency 24964.231 ms 2024-05-17T09:25:00.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:00 smithi019 ceph-mon[31620]: pgmap v595: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-05-17T09:25:00.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:00 smithi079 ceph-mon[35502]: pgmap v595: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-05-17T09:25:00.831 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:00 smithi069 ceph-mon[35081]: pgmap v595: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-05-17T09:25:01.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57312 1.30 MB/sec execute 59 sec latency 25964.275 ms 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/972065646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d0108750 msgr2=0x7fc5d01100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/972065646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d0108750 0x7fc5d01100b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b0d0 tx=0x7fc5c800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/972065646 shutdown_connections 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/972065646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d0108750 0x7fc5d01100b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/972065646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d01034d0 0x7fc5d01038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/972065646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d0107e00 0x7fc5d0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/972065646 >> 172.21.15.19:0/972065646 conn(0x7fc5d00fef40 msgr2=0x7fc5d0101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/972065646 shutdown_connections 2024-05-17T09:25:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.980+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/972065646 wait complete. 2024-05-17T09:25:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.981+0000 7fc5d4e2d700 1 Processor -- start 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.981+0000 7fc5d4e2d700 1 -- start start 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.981+0000 7fc5d4e2d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 0x7fc5d019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.981+0000 7fc5d4e2d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d0107e00 0x7fc5d019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.981+0000 7fc5d4e2d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d0108750 0x7fc5d019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.981+0000 7fc5d4e2d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0114380 con 0x7fc5d01034d0 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5d4e2d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0114200 con 0x7fc5d0107e00 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5d4e2d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0114500 con 0x7fc5d0108750 2024-05-17T09:25:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 0x7fc5d019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 0x7fc5d019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45968/0 (socket says 172.21.15.19:45968) 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 -- 172.21.15.19:0/2901928714 learned_addr learned my addr 172.21.15.19:0/2901928714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d0108750 0x7fc5d019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d0107e00 0x7fc5d019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 -- 172.21.15.19:0/2901928714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d0108750 msgr2=0x7fc5d019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d0108750 0x7fc5d019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 -- 172.21.15.19:0/2901928714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d0107e00 msgr2=0x7fc5d019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d0107e00 0x7fc5d019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 -- 172.21.15.19:0/2901928714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d019fc40 con 0x7fc5d01034d0 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d0107e00 0x7fc5d019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 0x7fc5d019ad70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000c920 tx=0x7fc5c000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.982+0000 7fc5bf7fe700 1 -- 172.21.15.19:0/2901928714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0007b00 con 0x7fc5d01034d0 2024-05-17T09:25:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.983+0000 7fc5bf7fe700 1 -- 172.21.15.19:0/2901928714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5c0004cc0 con 0x7fc5d01034d0 2024-05-17T09:25:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.983+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d019ff30 con 0x7fc5d01034d0 2024-05-17T09:25:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.983+0000 7fc5bf7fe700 1 -- 172.21.15.19:0/2901928714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0005440 con 0x7fc5d01034d0 2024-05-17T09:25:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.983+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d01a7850 con 0x7fc5d01034d0 2024-05-17T09:25:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.985+0000 7fc5bf7fe700 1 -- 172.21.15.19:0/2901928714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5c0004750 con 0x7fc5d01034d0 2024-05-17T09:25:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.986+0000 7fc5bf7fe700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5b80b2880 0x7fc5b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.986+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5b80b2880 0x7fc5b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.986+0000 7fc5bf7fe700 1 -- 172.21.15.19:0/2901928714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5c00e21b0 con 0x7fc5d01034d0 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.986+0000 7fc5e3842b80 1 Processor -- start 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 -- start start 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5b80b2880 0x7fc5b80b4d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4005e10 tx=0x7fc5c4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 0x5654945dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5654945d6020 0x5654945d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565494503e70 con 0x565494456f40 2024-05-17T09:25:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5654944f3a10 con 0x565494542530 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5654944f7a50 con 0x5654945d6020 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5cdd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 0x5654945dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5cdd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 0x5654945dc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45970/0 (socket says 172.21.15.19:45970) 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5cdd9b700 1 -- 172.21.15.19:0/2518122448 learned_addr learned my addr 172.21.15.19:0/2518122448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.987+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5654945d6020 0x5654945d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5cdd9b700 1 -- 172.21.15.19:0/2518122448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5654945d6020 msgr2=0x5654945d64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5654945d6020 0x5654945d64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5cdd9b700 1 -- 172.21.15.19:0/2518122448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 msgr2=0x5654945dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945dbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5cdd9b700 1 -- 172.21.15.19:0/2518122448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549449d0b0 con 0x565494456f40 2024-05-17T09:25:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 0x5654945dc430 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400fd40 tx=0x7fc5c4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e221a4729509ceb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5bcff9700 1 -- 172.21.15.19:0/2518122448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4022070 con 0x565494456f40 2024-05-17T09:25:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5bcff9700 1 -- 172.21.15.19:0/2518122448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5c4011bb0 con 0x565494456f40 2024-05-17T09:25:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.988+0000 7fc5bcff9700 1 -- 172.21.15.19:0/2518122448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c401d3b0 con 0x565494456f40 2024-05-17T09:25:01.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2518122448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 msgr2=0x5654945dc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:01.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 0x5654945dc430 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400fd40 tx=0x7fc5c4012000 comp rx=0 tx=0).stop 2024-05-17T09:25:01.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2518122448 shutdown_connections 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5654945d6020 0x5654945d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2518122448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494456f40 0x5654945dc430 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2518122448 >> 172.21.15.19:0/2518122448 conn(0x5654944b72c0 msgr2=0x5654945d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2518122448 shutdown_connections 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.989+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2518122448 wait complete. 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 -- start start 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 Processor -- start 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494542530 0x5654945e0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 0x5654945e0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5654943fa0a0 con 0x565494542530 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56549445bea0 con 0x5654945d6020 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5654945e2b20 con 0x565494456f40 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5cdd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494542530 0x5654945e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5e3842b80 10 client.0 _ll_register_callbacks cb 0x56549448ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5ced9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 0x5654945e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5ced9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 0x5654945e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33414/0 (socket says 172.21.15.19:33414) 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2295899024 learned_addr learned my addr 172.21.15.19:0/2295899024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.990+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2295899024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 msgr2=0x5654945dfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945dfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2295899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494542530 msgr2=0x5654945e0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494542530 0x5654945e0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2295899024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654944439d0 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 0x5654945e0540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8003c60 tx=0x7fc5c8003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8187c115690ddf7a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/2295899024 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80077d0 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c4011bb0 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/2295899024 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5c8007970 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/2295899024 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5c802e050 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5e3842b80 10 client.26399 fetch_fsmap learned FSMap version 31 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.991+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654944ad860 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/2295899024 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5c4011bb0 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/2295899024 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc5c8011070 con 0x5654945d6020 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5e3842b80 10 client.26399 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T09:25:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5e3842b80 10 client.26399 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5e3842b80 20 client.26399 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5654945ef980 0x5654945f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5e3842b80 4 client.26399 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654945a1fd0 con 0x5654945ef980 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.992+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5654945ef980 0x5654945f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.993+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5654945ef980 0x5654945f1e00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb17990c88276665 server_cookie=77a588a1cac627cd in_seq=0 out_seq=0 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.993+0000 7fc5b5ffb700 10 client.26399 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/2295899024 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654945a1fd0 con 0x5654945ef980 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5b5ffb700 10 client.26399 handle_command_reply: tid=0 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 client.26399 shutdown 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 msgr2=0x5654945e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:01.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 0x5654945e0540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8003c60 tx=0x7fc5c8003c90 comp rx=0 tx=0).stop 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 shutdown_connections 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5654945ef980 0x5654945f1e00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654945d6020 0x5654945e0540 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2295899024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565494542530 0x5654945e0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5b5ffb700 0 client.26399 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 >> 172.21.15.19:0/2295899024 conn(0x5654944b72c0 msgr2=0x5654945d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 shutdown_connections 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2295899024 wait complete. 2024-05-17T09:25:02.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:01.994+0000 7fc5e3842b80 20 client.26399 trim_cache size 0 max 16384 2024-05-17T09:25:02.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 Processor -- start 2024-05-17T09:25:02.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 -- start start 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b2bc0 con 0x5654945d6020 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b0450 con 0x565494542530 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.014+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01ae2a0 con 0x565494456f40 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45990/0 (socket says 172.21.15.19:45990) 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2626695476 learned_addr learned my addr 172.21.15.19:0/2626695476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2626695476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 msgr2=0x5654945ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945ebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2626695476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 msgr2=0x5654945d5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945d5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 -- 172.21.15.19:0/2626695476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654945e2360 con 0x5654945d6020 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945f75c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800ba60 tx=0x7fc5c8005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51da626e69b77c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5b7fff700 1 -- 172.21.15.19:0/2626695476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005c60 con 0x5654945d6020 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5b7fff700 1 -- 172.21.15.19:0/2626695476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5c8005e00 con 0x5654945d6020 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.015+0000 7fc5b7fff700 1 -- 172.21.15.19:0/2626695476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8026430 con 0x5654945d6020 2024-05-17T09:25:02.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2626695476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 msgr2=0x5654945f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945f75c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800ba60 tx=0x7fc5c8005ac0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2626695476 shutdown_connections 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494456f40 0x5654945ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945f75c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/2626695476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494542530 0x5654945d5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2626695476 >> 172.21.15.19:0/2626695476 conn(0x5654944b72c0 msgr2=0x5654945d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2626695476 shutdown_connections 2024-05-17T09:25:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.016+0000 7fc5e3842b80 1 -- 172.21.15.19:0/2626695476 wait complete. 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 -- start start 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 Processor -- start 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494456f40 0x5654945e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494542530 0x5654945e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945e5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01ae910 con 0x5654945d6020 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01afd80 con 0x565494456f40 2024-05-17T09:25:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01aefe0 con 0x565494542530 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5e3842b80 10 client.0 _ll_register_callbacks cb 0x56549448ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945e5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45998/0 (socket says 172.21.15.19:45998) 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 -- 172.21.15.19:0/4021682086 learned_addr learned my addr 172.21.15.19:0/4021682086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494456f40 0x5654945e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5cdd9b700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494542530 0x5654945e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 -- 172.21.15.19:0/4021682086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494542530 msgr2=0x5654945e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494542530 0x5654945e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 -- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494456f40 msgr2=0x5654945e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494456f40 0x5654945e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.017+0000 7fc5ced9d700 1 -- 172.21.15.19:0/4021682086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654945e5920 con 0x5654945d6020 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5ced9d700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945e5280 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc5c801f040 tx=0x7fc5c8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25de0bb9122fb89e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:02.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/4021682086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80050c0 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/4021682086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5c8005260 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c8005e00 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/4021682086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5c8042050 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5e3842b80 10 client.16755 fetch_fsmap learned FSMap version 31 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56549465b2f0 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/4021682086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5c8005e00 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/4021682086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc5c8011040 con 0x5654945d6020 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5e3842b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5e3842b80 10 client.16755 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.018+0000 7fc5e3842b80 20 client.16755 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.019+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56549465b910 0x56549465dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.019+0000 7fc5e3842b80 4 client.16755 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eaabf277", "release": true, "await": true}] 2024-05-17T09:25:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.019+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eaabf277", "release": true, "await": true}) -- 0x565494505dc0 con 0x56549465b910 2024-05-17T09:25:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.019+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56549465b910 0x56549465dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.019+0000 7fc5ce59c700 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56549465b910 0x56549465dd30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aeaa78bbba3d61 server_cookie=7cd4fec793f3c61b in_seq=0 out_seq=0 2024-05-17T09:25:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.019+0000 7fc5b5ffb700 10 client.16755 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5b5ffb700 1 -- 172.21.15.19:0/4021682086 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565494505dc0 con 0x56549465b910 2024-05-17T09:25:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5b5ffb700 10 client.16755 handle_command_reply: tid=0 2024-05-17T09:25:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 client.16755 shutdown 2024-05-17T09:25:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 msgr2=0x5654945e5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945e5280 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc5c801f040 tx=0x7fc5c8007a00 comp rx=0 tx=0).stop 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 shutdown_connections 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56549465b910 0x56549465dd30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654945d6020 0x5654945e5280 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565494456f40 0x5654945e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5b5ffb700 0 client.16755 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 --2- 172.21.15.19:0/4021682086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565494542530 0x5654945e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 >> 172.21.15.19:0/4021682086 conn(0x5654944b72c0 msgr2=0x5654945daa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 shutdown_connections 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 1 -- 172.21.15.19:0/4021682086 wait complete. 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.025+0000 7fc5e3842b80 20 client.16755 trim_cache size 0 max 16384 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 31, 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 20, 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:25:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "eaabf277": { 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 20, 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 146.7, 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:25:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:02.030 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:25:02.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5b80b2880 msgr2=0x7fc5b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:02.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5b80b2880 0x7fc5b80b4d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4005e10 tx=0x7fc5c4005d80 comp rx=0 tx=0).stop 2024-05-17T09:25:02.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 msgr2=0x7fc5d019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:02.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 0x7fc5d019ad70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000c920 tx=0x7fc5c000cc30 comp rx=0 tx=0).stop 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 shutdown_connections 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc5b80b2880 0x7fc5b80b4d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d0108750 0x7fc5d019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d01034d0 0x7fc5d019ad70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 --2- 172.21.15.19:0/2901928714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5d0107e00 0x7fc5d019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.026+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 >> 172.21.15.19:0/2901928714 conn(0x7fc5d00fef40 msgr2=0x7fc5d0107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.027+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 shutdown_connections 2024-05-17T09:25:02.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:02.027+0000 7fc5d4e2d700 1 -- 172.21.15.19:0/2901928714 wait complete. 2024-05-17T09:25:02.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57438 1.28 MB/sec execute 60 sec latency 26560.881 ms 2024-05-17T09:25:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:02 smithi079 ceph-mon[35502]: pgmap v596: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-17T09:25:02.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:02 smithi019 ceph-mon[31620]: pgmap v596: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-17T09:25:02.854 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'eaabf277', total seconds elapsed: 33.4 2024-05-17T09:25:02.855 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.65 2024-05-17T09:25:02.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:25:02.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:02 smithi069 ceph-mon[35081]: pgmap v596: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-17T09:25:03.140 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:25:03.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57935 1.29 MB/sec execute 61 sec latency 15.782 ms 2024-05-17T09:25:04.427 INFO:tasks.workunit.client.0.smithi019.stdout: 1 58485 1.32 MB/sec execute 62 sec latency 15.924 ms 2024-05-17T09:25:04.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:04 smithi019 ceph-mon[31620]: pgmap v597: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-17T09:25:04.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:04 smithi069 ceph-mon[35081]: pgmap v597: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-17T09:25:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:04 smithi079 ceph-mon[35502]: pgmap v597: 129 pgs: 129 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-17T09:25:05.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.168+0000 7f61c24c8700 1 -- 172.21.15.19:0/2452652036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc107de0 msgr2=0x7f61bc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:05.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.168+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2452652036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc107de0 0x7f61bc1081f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f61b00099c0 tx=0x7f61b0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 -- 172.21.15.19:0/2452652036 shutdown_connections 2024-05-17T09:25:05.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2452652036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61bc1034b0 0x7f61bc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2452652036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc108730 0x7f61bc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2452652036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc107de0 0x7f61bc1081f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 -- 172.21.15.19:0/2452652036 >> 172.21.15.19:0/2452652036 conn(0x7f61bc0fedb0 msgr2=0x7f61bc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:05.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 -- 172.21.15.19:0/2452652036 shutdown_connections 2024-05-17T09:25:05.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 -- 172.21.15.19:0/2452652036 wait complete. 2024-05-17T09:25:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 Processor -- start 2024-05-17T09:25:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.169+0000 7f61c24c8700 1 -- start start 2024-05-17T09:25:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61c24c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 0x7f61bc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61c24c8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc107de0 0x7f61bc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61c24c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61bc108730 0x7f61bc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61c24c8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1131f0 con 0x7f61bc1034b0 2024-05-17T09:25:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61c24c8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc113070 con 0x7f61bc107de0 2024-05-17T09:25:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61c24c8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc113370 con 0x7f61bc108730 2024-05-17T09:25:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.170+0000 7f61bbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 0x7f61bc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 0x7f61bc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54134/0 (socket says 172.21.15.19:54134) 2024-05-17T09:25:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 -- 172.21.15.19:0/2873399728 learned_addr learned my addr 172.21.15.19:0/2873399728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bb7fe700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc107de0 0x7f61bc19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61c0a65700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61bc108730 0x7f61bc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 -- 172.21.15.19:0/2873399728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61bc108730 msgr2=0x7f61bc19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61bc108730 0x7f61bc19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 -- 172.21.15.19:0/2873399728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc107de0 msgr2=0x7f61bc19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc107de0 0x7f61bc19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bbfff700 1 -- 172.21.15.19:0/2873399728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc19fc00 con 0x7f61bc1034b0 2024-05-17T09:25:05.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.171+0000 7f61bb7fe700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc107de0 0x7f61bc19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:05.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.172+0000 7f61bbfff700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 0x7f61bc19ad30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e9a0 tx=0x7f61ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:05.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.172+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00ca10 con 0x7f61bc1034b0 2024-05-17T09:25:05.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.172+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61ac00cbb0 con 0x7f61bc1034b0 2024-05-17T09:25:05.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.172+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac018680 con 0x7f61bc1034b0 2024-05-17T09:25:05.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.172+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc19fef0 con 0x7f61bc1034b0 2024-05-17T09:25:05.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.172+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a7810 con 0x7f61bc1034b0 2024-05-17T09:25:05.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.173+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61bc104960 con 0x7f61bc1034b0 2024-05-17T09:25:05.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.175+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f61ac010430 con 0x7f61bc1034b0 2024-05-17T09:25:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.176+0000 7f61b97fa700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61a40b2880 0x7f61a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.177+0000 7f61bb7fe700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61a40b2880 0x7f61a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.177+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f61ac0e2b60 con 0x7f61bc1034b0 2024-05-17T09:25:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.177+0000 7f61bb7fe700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61a40b2880 0x7f61a40b4d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000c00 tx=0x7f61b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.184+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61ac0aa4d0 con 0x7f61bc1034b0 2024-05-17T09:25:05.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 59105 1.33 MB/sec execute 63 sec latency 16.195 ms 2024-05-17T09:25:05.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.520+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61bc1a79e0 con 0x7f61bc1034b0 2024-05-17T09:25:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.520+0000 7f61b97fa700 1 -- 172.21.15.19:0/2873399728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11854 (secure 0 0 0) 0x7f61ac0a9dd0 con 0x7f61bc1034b0 2024-05-17T09:25:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:25:05.521 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:24:57.624115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24487]},"id":1}]} 2024-05-17T09:25:05.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61a40b2880 msgr2=0x7f61a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61a40b2880 0x7f61a40b4d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000c00 tx=0x7f61b001a040 comp rx=0 tx=0).stop 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 msgr2=0x7f61bc19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 0x7f61bc19ad30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e9a0 tx=0x7f61ac00ecb0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 shutdown_connections 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61a40b2880 0x7f61a40b4d90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61bc108730 0x7f61bc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61bc1034b0 0x7f61bc19ad30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 --2- 172.21.15.19:0/2873399728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61bc107de0 0x7f61bc19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.523+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 >> 172.21.15.19:0/2873399728 conn(0x7f61bc0fedb0 msgr2=0x7f61bc06b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.524+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 shutdown_connections 2024-05-17T09:25:05.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:05.524+0000 7f61c24c8700 1 -- 172.21.15.19:0/2873399728 wait complete. 2024-05-17T09:25:05.525 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-05-17T09:25:06.359 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 146.65 --await-for 45.0' 2024-05-17T09:25:06.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 146.65 --await-for 45.0 2024-05-17T09:25:06.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 59553 1.37 MB/sec execute 64 sec latency 19.030 ms 2024-05-17T09:25:06.644 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:25:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:06 smithi069 ceph-mon[35081]: pgmap v598: 129 pgs: 129 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 60 KiB/s wr, 11 op/s 2024-05-17T09:25:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:06 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2873399728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:06 smithi019 ceph-mon[31620]: pgmap v598: 129 pgs: 129 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 60 KiB/s wr, 11 op/s 2024-05-17T09:25:06.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2873399728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:06 smithi079 ceph-mon[35502]: pgmap v598: 129 pgs: 129 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 60 KiB/s wr, 11 op/s 2024-05-17T09:25:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:06 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2873399728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:07.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60105 1.40 MB/sec execute 65 sec latency 17.613 ms 2024-05-17T09:25:08.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60556 1.45 MB/sec execute 66 sec latency 56.069 ms 2024-05-17T09:25:08.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:08 smithi019 ceph-mon[31620]: pgmap v599: 129 pgs: 129 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 324 KiB/s wr, 54 op/s 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 -- 172.21.15.19:0/3477249371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24108700 msgr2=0x7feb24108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 --2- 172.21.15.19:0/3477249371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24108700 0x7feb24108bc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009fc0 tx=0x7feb1c00b330 comp rx=0 tx=0).stop 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 -- 172.21.15.19:0/3477249371 shutdown_connections 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 --2- 172.21.15.19:0/3477249371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24108700 0x7feb24108bc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 --2- 172.21.15.19:0/3477249371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103260 0x7feb24103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 --2- 172.21.15.19:0/3477249371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24103c40 0x7feb241080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 -- 172.21.15.19:0/3477249371 >> 172.21.15.19:0/3477249371 conn(0x7feb240fef40 msgr2=0x7feb24101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 -- 172.21.15.19:0/3477249371 shutdown_connections 2024-05-17T09:25:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.582+0000 7feb2a92a700 1 -- 172.21.15.19:0/3477249371 wait complete. 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 Processor -- start 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 -- start start 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 0x7feb241a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 0x7feb241a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24108700 0x7feb241a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb2411a9d0 con 0x7feb24103c40 2024-05-17T09:25:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb2411a850 con 0x7feb24103260 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.583+0000 7feb2a92a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb2411ab50 con 0x7feb24108700 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 0x7feb241a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 0x7feb241a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24108700 0x7feb241a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 0x7feb241a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49912/0 (socket says 172.21.15.19:49912) 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24108700 0x7feb241a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47558/0 (socket says 172.21.15.19:47558) 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 0x7feb241a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54146/0 (socket says 172.21.15.19:54146) 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 -- 172.21.15.19:0/406657242 learned_addr learned my addr 172.21.15.19:0/406657242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 -- 172.21.15.19:0/406657242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24108700 msgr2=0x7feb241a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24108700 0x7feb241a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 -- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 msgr2=0x7feb241a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 0x7feb241a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 -- 172.21.15.19:0/406657242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb241a61f0 con 0x7feb24103260 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb1bfff700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 0x7feb241a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb23fff700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 0x7feb241a1320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feb1400ec10 tx=0x7feb1400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb21ffb700 1 -- 172.21.15.19:0/406657242 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1400cc90 con 0x7feb24103260 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb21ffb700 1 -- 172.21.15.19:0/406657242 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb1400ce30 con 0x7feb24103260 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.584+0000 7feb21ffb700 1 -- 172.21.15.19:0/406657242 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14010430 con 0x7feb24103260 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.585+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb241a64e0 con 0x7feb24103260 2024-05-17T09:25:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.585+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb241ade00 con 0x7feb24103260 2024-05-17T09:25:08.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.588+0000 7feb21ffb700 1 -- 172.21.15.19:0/406657242 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feb14003f30 con 0x7feb24103260 2024-05-17T09:25:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.588+0000 7feb21ffb700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7feb100b2900 0x7feb100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.588+0000 7feb3933fb80 1 Processor -- start 2024-05-17T09:25:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 -- start start 2024-05-17T09:25:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb1bfff700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7feb100b2900 0x7feb100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb21ffb700 1 -- 172.21.15.19:0/406657242 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7feb140e2830 con 0x7feb24103260 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562096a2c0c0 0x562096a2c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562096959f10 con 0x5620969985d0 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562096949ab0 con 0x5620968acfe0 2024-05-17T09:25:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb3933fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56209694daf0 con 0x562096a2c0c0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb23fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb1bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562096a2c0c0 0x562096a2c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb23fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54152/0 (socket says 172.21.15.19:54152) 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb23fff700 1 -- 172.21.15.19:0/1322046181 learned_addr learned my addr 172.21.15.19:0/1322046181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb23fff700 1 -- 172.21.15.19:0/1322046181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562096a2c0c0 msgr2=0x562096a2c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.589+0000 7feb23fff700 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562096a2c0c0 0x562096a2c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb23fff700 1 -- 172.21.15.19:0/1322046181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 msgr2=0x562096a324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb23fff700 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb23fff700 1 -- 172.21.15.19:0/1322046181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620968f3150 con 0x5620969985d0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb23fff700 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31f30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7feb140a5b20 tx=0x7feb140a93f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1505b641ffeacc4b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb1bfff700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7feb100b2900 0x7feb100b4e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009fd0 tx=0x7feb0c009ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb1a7fc700 1 -- 172.21.15.19:0/1322046181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140c3070 con 0x5620969985d0 2024-05-17T09:25:08.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb1a7fc700 1 -- 172.21.15.19:0/1322046181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb140b2250 con 0x5620969985d0 2024-05-17T09:25:08.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.590+0000 7feb1a7fc700 1 -- 172.21.15.19:0/1322046181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140b2900 con 0x5620969985d0 2024-05-17T09:25:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 -- 172.21.15.19:0/1322046181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 msgr2=0x562096a31f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31f30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7feb140a5b20 tx=0x7feb140a93f0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 -- 172.21.15.19:0/1322046181 shutdown_connections 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562096a2c0c0 0x562096a2c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31f30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 --2- 172.21.15.19:0/1322046181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 -- 172.21.15.19:0/1322046181 >> 172.21.15.19:0/1322046181 conn(0x56209690d360 msgr2=0x562096a26d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 -- 172.21.15.19:0/1322046181 shutdown_connections 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.591+0000 7feb3933fb80 1 -- 172.21.15.19:0/1322046181 wait complete. 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 -- start start 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 Processor -- start 2024-05-17T09:25:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 0x562096a35c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 0x562096a366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562096850140 con 0x5620969985d0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5620968b1f40 con 0x562096a2c0c0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562096a38bc0 con 0x5620968acfe0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb3933fb80 10 client.0 _ll_register_callbacks cb 0x5620968e4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb23fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 0x562096a35c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 0x562096a366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb23fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 0x562096a35c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47578/0 (socket says 172.21.15.19:47578) 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 0x562096a366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49932/0 (socket says 172.21.15.19:49932) 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb23fff700 1 -- 172.21.15.19:0/3711601275 learned_addr learned my addr 172.21.15.19:0/3711601275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.592+0000 7feb1bfff700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb28ec7700 1 -- 172.21.15.19:0/3711601275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 msgr2=0x562096a35c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb28ec7700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 0x562096a35c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb28ec7700 1 -- 172.21.15.19:0/3711601275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 msgr2=0x562096a36160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb28ec7700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a36160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb28ec7700 1 -- 172.21.15.19:0/3711601275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562096899a70 con 0x562096a2c0c0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb23fff700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 0x562096a35c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb28ec7700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 0x562096a366a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b6c0 tx=0x7feb1c009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d15e7957dd8cdfa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb037fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb037fe700 1 -- 172.21.15.19:0/3711601275 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c00be00 con 0x562096a2c0c0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb037fe700 1 -- 172.21.15.19:0/3711601275 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7feb1c007500 con 0x562096a2c0c0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.593+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb140b2250 con 0x562096a2c0c0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb037fe700 1 -- 172.21.15.19:0/3711601275 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb1c007700 con 0x562096a2c0c0 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 10 client.26423 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562096903900 con 0x562096a2c0c0 2024-05-17T09:25:08.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb037fe700 1 -- 172.21.15.19:0/3711601275 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb140b2250 con 0x562096a2c0c0 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb037fe700 1 -- 172.21.15.19:0/3711601275 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7feb1c024070 con 0x562096a2c0c0 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 10 client.26423 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 10 client.26423 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 20 client.26423 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a45af0 0x562096a47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 4 client.26423 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620969f8070 con 0x562096a45af0 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.594+0000 7feb23fff700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a45af0 0x562096a47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.595+0000 7feb037fe700 10 client.26423 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.595+0000 7feb23fff700 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a45af0 0x562096a47f70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d46500a7171ea0 server_cookie=b7df3568dedcf526 in_seq=0 out_seq=0 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.595+0000 7feb037fe700 1 -- 172.21.15.19:0/3711601275 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5620969f8070 con 0x562096a45af0 2024-05-17T09:25:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb037fe700 10 client.26423 handle_command_reply: tid=0 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 client.26423 shutdown 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 msgr2=0x562096a366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 0x562096a366a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b6c0 tx=0x7feb1c009d50 comp rx=0 tx=0).stop 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 shutdown_connections 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620968acfe0 0x562096a35c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a45af0 0x562096a47f70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562096a2c0c0 0x562096a366a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 --2- 172.21.15.19:0/3711601275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 >> 172.21.15.19:0/3711601275 conn(0x56209690d360 msgr2=0x562096a2b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb037fe700 0 client.26423 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 shutdown_connections 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 1 -- 172.21.15.19:0/3711601275 wait complete. 2024-05-17T09:25:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.596+0000 7feb3933fb80 20 client.26423 trim_cache size 0 max 16384 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 Processor -- start 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 -- start start 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56209690d360 0x562096a34f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb241b91c0 con 0x5620969985d0 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb241b6a50 con 0x5620968acfe0 2024-05-17T09:25:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb3933fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb241b48a0 con 0x56209690d360 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb23fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56209690d360 0x562096a34f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb28ec7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54176/0 (socket says 172.21.15.19:54176) 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.620+0000 7feb28ec7700 1 -- 172.21.15.19:0/2648813366 learned_addr learned my addr 172.21.15.19:0/2648813366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb1bfff700 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb23fff700 1 -- 172.21.15.19:0/2648813366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 msgr2=0x562096a380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb23fff700 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb23fff700 1 -- 172.21.15.19:0/2648813366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 msgr2=0x562096a31310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb23fff700 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb23fff700 1 -- 172.21.15.19:0/2648813366 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562096a41c20 con 0x56209690d360 2024-05-17T09:25:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb23fff700 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56209690d360 0x562096a34f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x562096a354a0 tx=0x7feb1400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a1078f1d6c1c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb19ffb700 1 -- 172.21.15.19:0/2648813366 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140c3040 con 0x56209690d360 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb19ffb700 1 -- 172.21.15.19:0/2648813366 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb140b2250 con 0x56209690d360 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.621+0000 7feb19ffb700 1 -- 172.21.15.19:0/2648813366 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14003680 con 0x56209690d360 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 -- 172.21.15.19:0/2648813366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56209690d360 msgr2=0x562096a34f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56209690d360 0x562096a34f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x562096a354a0 tx=0x7feb1400ef90 comp rx=0 tx=0).stop 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 -- 172.21.15.19:0/2648813366 shutdown_connections 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56209690d360 0x562096a34f40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620969985d0 0x562096a31310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 --2- 172.21.15.19:0/2648813366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620968acfe0 0x562096a380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 -- 172.21.15.19:0/2648813366 >> 172.21.15.19:0/2648813366 conn(0x562096a482c0 msgr2=0x562096a2f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 -- 172.21.15.19:0/2648813366 shutdown_connections 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.622+0000 7feb3933fb80 1 -- 172.21.15.19:0/2648813366 wait complete. 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 -- start start 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 Processor -- start 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620968acfe0 0x562096a3ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 0x562096a3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620969985d0 0x562096a3b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb241b4f10 con 0x5620968acfe0 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb241b6380 con 0x56209690d360 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb241b55e0 con 0x5620969985d0 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.623+0000 7feb3933fb80 10 client.0 _ll_register_callbacks cb 0x5620968e4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 0x562096a3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 0x562096a3b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49952/0 (socket says 172.21.15.19:49952) 2024-05-17T09:25:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 -- 172.21.15.19:0/585116201 learned_addr learned my addr 172.21.15.19:0/585116201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb28ec7700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620969985d0 0x562096a3b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb23fff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620968acfe0 0x562096a3ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 -- 172.21.15.19:0/585116201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620969985d0 msgr2=0x562096a3b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620969985d0 0x562096a3b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 -- 172.21.15.19:0/585116201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620968acfe0 msgr2=0x562096a3ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620968acfe0 0x562096a3ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 -- 172.21.15.19:0/585116201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562096ab0610 con 0x56209690d360 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb28ec7700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620969985d0 0x562096a3b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb23fff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620968acfe0 0x562096a3ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb1bfff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 0x562096a3b320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feb24101800 tx=0x7feb0c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b674aaf6f13898be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.624+0000 7feb037fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb037fe700 1 -- 172.21.15.19:0/585116201 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c013e60 con 0x56209690d360 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb140b2250 con 0x56209690d360 2024-05-17T09:25:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb037fe700 1 -- 172.21.15.19:0/585116201 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7feb0c01c040 con 0x56209690d360 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb037fe700 1 -- 172.21.15.19:0/585116201 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb0c035050 con 0x56209690d360 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 10 client.26429 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562096ab1800 con 0x56209690d360 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb037fe700 1 -- 172.21.15.19:0/585116201 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb140b2250 con 0x56209690d360 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb037fe700 1 -- 172.21.15.19:0/585116201 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7feb0c020050 con 0x56209690d360 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 10 client.26429 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 10 client.26429 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 20 client.26429 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a2c0c0 0x562096ab3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 4 client.26429 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.65, "await_for": 45.0}] 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.65, "await_for": 45.0}) -- 0x56209695be60 con 0x562096a2c0c0 2024-05-17T09:25:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.625+0000 7feb23fff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a2c0c0 0x562096ab3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.626+0000 7feb037fe700 10 client.26429 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.626+0000 7feb23fff700 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a2c0c0 0x562096ab3fb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3249316001444d server_cookie=fe5d0bc323ee4810 in_seq=0 out_seq=0 2024-05-17T09:25:08.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:08 smithi069 ceph-mon[35081]: pgmap v599: 129 pgs: 129 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 324 KiB/s wr, 54 op/s 2024-05-17T09:25:08.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.713+0000 7feb037fe700 1 -- 172.21.15.19:0/585116201 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56209695be60 con 0x562096a2c0c0 2024-05-17T09:25:08.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.713+0000 7feb037fe700 10 client.26429 handle_command_reply: tid=0 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.713+0000 7feb3933fb80 1 client.26429 shutdown 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 msgr2=0x562096a3b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 0x562096a3b320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feb24101800 tx=0x7feb0c01d000 comp rx=0 tx=0).stop 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 shutdown_connections 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620969985d0 0x562096a3b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562096a2c0c0 0x562096ab3fb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5620968acfe0 0x562096a3ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 --2- 172.21.15.19:0/585116201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56209690d360 0x562096a3b320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb037fe700 0 client.26429 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 >> 172.21.15.19:0/585116201 conn(0x562096a482c0 msgr2=0x562096a34ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 shutdown_connections 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 1 -- 172.21.15.19:0/585116201 wait complete. 2024-05-17T09:25:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.714+0000 7feb3933fb80 20 client.26429 trim_cache size 0 max 16384 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 22, 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "863fb380": { 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 22, 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:25:08.716 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 146.7, 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:08.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:08.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:08.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:08.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:08.718 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:25:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7feb100b2900 msgr2=0x7feb100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7feb100b2900 0x7feb100b4e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009fd0 tx=0x7feb0c009ec0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 msgr2=0x7feb241a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 0x7feb241a1320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feb1400ec10 tx=0x7feb1400ef20 comp rx=0 tx=0).stop 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 shutdown_connections 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7feb100b2900 0x7feb100b4e10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb24108700 0x7feb241a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb24103260 0x7feb241a1320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 --2- 172.21.15.19:0/406657242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb24103c40 0x7feb241a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 >> 172.21.15.19:0/406657242 conn(0x7feb240fef40 msgr2=0x7feb24106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 shutdown_connections 2024-05-17T09:25:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:08.715+0000 7feb2a92a700 1 -- 172.21.15.19:0/406657242 wait complete. 2024-05-17T09:25:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:08 smithi079 ceph-mon[35502]: pgmap v599: 129 pgs: 129 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 324 KiB/s wr, 54 op/s 2024-05-17T09:25:09.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.43 MB/sec execute 67 sec latency 703.050 ms 2024-05-17T09:25:09.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:09 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:25:09.586 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '863fb380', quiesce took 6.7 seconds. Will release after: 19.95 2024-05-17T09:25:09.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:09 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:25:09.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:09 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:25:10.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.41 MB/sec execute 68 sec latency 1703.097 ms 2024-05-17T09:25:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:10 smithi069 ceph-mon[35081]: pgmap v600: 129 pgs: 129 active+clean; 38 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 484 KiB/s wr, 60 op/s 2024-05-17T09:25:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:10 smithi019 ceph-mon[31620]: pgmap v600: 129 pgs: 129 active+clean; 38 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 484 KiB/s wr, 60 op/s 2024-05-17T09:25:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:10 smithi079 ceph-mon[35502]: pgmap v600: 129 pgs: 129 active+clean; 38 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 484 KiB/s wr, 60 op/s 2024-05-17T09:25:11.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.39 MB/sec execute 69 sec latency 2703.195 ms 2024-05-17T09:25:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 -- 172.21.15.19:0/1600943540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530102b20 msgr2=0x7fe530103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 --2- 172.21.15.19:0/1600943540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530102b20 0x7fe530103820 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe520009900 tx=0x7fe52002a410 comp rx=0 tx=0).stop 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 -- 172.21.15.19:0/1600943540 shutdown_connections 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 --2- 172.21.15.19:0/1600943540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe530108a60 0x7fe53010ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 --2- 172.21.15.19:0/1600943540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe530103d60 0x7fe530104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 --2- 172.21.15.19:0/1600943540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530102b20 0x7fe530103820 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 -- 172.21.15.19:0/1600943540 >> 172.21.15.19:0/1600943540 conn(0x7fe5300fdd10 msgr2=0x7fe5301001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 -- 172.21.15.19:0/1600943540 shutdown_connections 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 -- 172.21.15.19:0/1600943540 wait complete. 2024-05-17T09:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.703+0000 7fe535284640 1 Processor -- start 2024-05-17T09:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 -- start start 2024-05-17T09:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe530102b20 0x7fe530075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe530103d60 0x7fe53019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 0x7fe5301a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010e4d0 con 0x7fe530108a60 2024-05-17T09:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010e350 con 0x7fe530102b20 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe535284640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010e650 con 0x7fe530103d60 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 0x7fe5301a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 0x7fe5301a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54204/0 (socket says 172.21.15.19:54204) 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1288312610 learned_addr learned my addr 172.21.15.19:0/1288312610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe5267fc640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe530103d60 0x7fe53019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52effd640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe530102b20 0x7fe530075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1288312610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe530103d60 msgr2=0x7fe53019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe530103d60 0x7fe53019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1288312610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe530102b20 msgr2=0x7fe530075960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe530102b20 0x7fe530075960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.704+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1288312610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5301a1cf0 con 0x7fe530108a60 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.705+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 0x7fe5301a14d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe52800eb10 tx=0x7fe52800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.705+0000 7fe52cff9640 1 -- 172.21.15.19:0/1288312610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528004b20 con 0x7fe530108a60 2024-05-17T09:25:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.705+0000 7fe52cff9640 1 -- 172.21.15.19:0/1288312610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe528004cc0 con 0x7fe530108a60 2024-05-17T09:25:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.705+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5301a1fe0 con 0x7fe530108a60 2024-05-17T09:25:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.705+0000 7fe52cff9640 1 -- 172.21.15.19:0/1288312610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528010690 con 0x7fe530108a60 2024-05-17T09:25:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.705+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5301a2440 con 0x7fe530108a60 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 Processor -- start 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe52cff9640 1 -- 172.21.15.19:0/1288312610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe528005020 con 0x7fe530108a60 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 -- start start 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc34920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56035fba6c30 con 0x56035f8931b0 2024-05-17T09:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56035fb967d0 con 0x56035f8724d0 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.708+0000 7fe53c20f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56035fb9a810 con 0x56035f86a2f0 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc343a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54220/0 (socket says 172.21.15.19:54220) 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 -- 172.21.15.19:0/4192541743 learned_addr learned my addr 172.21.15.19:0/4192541743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe52effd640 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc34920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 -- 172.21.15.19:0/4192541743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 msgr2=0x56035fc34920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc34920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 -- 172.21.15.19:0/4192541743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 msgr2=0x56035fc3b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc3b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 -- 172.21.15.19:0/4192541743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035f8ab210 con 0x56035f8931b0 2024-05-17T09:25:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe52cff9640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe51c0b28a0 0x7fe51c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5267fc640 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc343a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe518012630 tx=0x7fe51800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6789d61b7918eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe52effd640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe51c0b28a0 0x7fe51c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe52cff9640 1 -- 172.21.15.19:0/1288312610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe5280e2a90 con 0x7fe530108a60 2024-05-17T09:25:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.709+0000 7fe5257fa640 1 -- 172.21.15.19:0/4192541743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518005020 con 0x56035f8931b0 2024-05-17T09:25:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.710+0000 7fe5257fa640 1 -- 172.21.15.19:0/4192541743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe518005650 con 0x56035f8931b0 2024-05-17T09:25:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.710+0000 7fe5257fa640 1 -- 172.21.15.19:0/4192541743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800e710 con 0x56035f8931b0 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.710+0000 7fe53c20f740 1 -- 172.21.15.19:0/4192541743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 msgr2=0x56035fc343a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.710+0000 7fe52effd640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe51c0b28a0 0x7fe51c0b4db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe520005760 tx=0x7fe520000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.710+0000 7fe53c20f740 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc343a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe518012630 tx=0x7fe51800b8c0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- 172.21.15.19:0/4192541743 shutdown_connections 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc3b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc34920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 --2- 172.21.15.19:0/4192541743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc343a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- 172.21.15.19:0/4192541743 >> 172.21.15.19:0/4192541743 conn(0x56035f887300 msgr2=0x56035fc2fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- 172.21.15.19:0/4192541743 shutdown_connections 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- 172.21.15.19:0/4192541743 wait complete. 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- start start 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 Processor -- start 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc3dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fc45180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56035f87c790 con 0x56035f8724d0 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56035fb52890 con 0x56035f8931b0 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56035fc3c270 con 0x56035f86a2f0 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe53c20f740 10 client.0 _ll_register_callbacks cb 0x56035fb55d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe5267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.711+0000 7fe5267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc3e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54222/0 (socket says 172.21.15.19:54222) 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 -- 172.21.15.19:0/910404416 learned_addr learned my addr 172.21.15.19:0/910404416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe52effd640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc3dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fc45180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 -- 172.21.15.19:0/910404416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 msgr2=0x56035fc3dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc3dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 -- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 msgr2=0x56035fc45180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fc45180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 -- 172.21.15.19:0/910404416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035f977620 con 0x56035f8724d0 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5267fc640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc3e220 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe518012fd0 tx=0x7fe518004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724d49ae170d13c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5167fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5167fc640 1 -- 172.21.15.19:0/910404416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180046c0 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5167fc640 1 -- 172.21.15.19:0/910404416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe518004020 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe518005650 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe5167fc640 1 -- 172.21.15.19:0/910404416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe518033050 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.712+0000 7fe53c20f740 10 client.16791 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56035fc480b0 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe5167fc640 1 -- 172.21.15.19:0/910404416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe518005650 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe5167fc640 1 -- 172.21.15.19:0/910404416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe5180035f0 con 0x56035f8724d0 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 10 client.16791 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 20 client.16791 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f851d80 0x56035fc4b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 4 client.16791 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56035fba8b80 con 0x56035f851d80 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.713+0000 7fe52effd640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f851d80 0x56035fc4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.714+0000 7fe52effd640 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f851d80 0x56035fc4b600 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f857154a591517e8 server_cookie=6dabd2b920814d37 in_seq=0 out_seq=0 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.714+0000 7fe5167fc640 10 client.16791 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.714+0000 7fe5167fc640 1 -- 172.21.15.19:0/910404416 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56035fba8b80 con 0x56035f851d80 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.714+0000 7fe5167fc640 10 client.16791 handle_command_reply: tid=0 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 client.16791 shutdown 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 msgr2=0x56035fc3e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc3e220 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe518012fd0 tx=0x7fe518004e20 comp rx=0 tx=0).stop 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 shutdown_connections 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f851d80 0x56035fc4b600 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fc45180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc3e220 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 --2- 172.21.15.19:0/910404416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc3dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe5167fc640 0 client.16791 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 >> 172.21.15.19:0/910404416 conn(0x56035f887300 msgr2=0x56035fc32c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 shutdown_connections 2024-05-17T09:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 1 -- 172.21.15.19:0/910404416 wait complete. 2024-05-17T09:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.715+0000 7fe53c20f740 20 client.16791 trim_cache size 0 max 16384 2024-05-17T09:25:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.730+0000 7fe53c20f740 1 Processor -- start 2024-05-17T09:25:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.730+0000 7fe53c20f740 1 -- start start 2024-05-17T09:25:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe53c20f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301af2f0 con 0x56035f8931b0 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe53c20f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301acbc0 con 0x56035f8724d0 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe53c20f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301aaa10 con 0x56035f86a2f0 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc4fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc4fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54234/0 (socket says 172.21.15.19:54234) 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1220308493 learned_addr learned my addr 172.21.15.19:0/1220308493 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe5267fc640 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52effd640 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1220308493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 msgr2=0x56035fc388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1220308493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 msgr2=0x56035fc450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 -- 172.21.15.19:0/1220308493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035fc3c0c0 con 0x56035f8931b0 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc4fdb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe5280049d0 tx=0x7fe5280a5c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37574b5faafa3e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe524ff9640 1 -- 172.21.15.19:0/1220308493 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280a5430 con 0x56035f8931b0 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe524ff9640 1 -- 172.21.15.19:0/1220308493 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5280b2440 con 0x56035f8931b0 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.731+0000 7fe524ff9640 1 -- 172.21.15.19:0/1220308493 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280b2b00 con 0x56035f8931b0 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 -- 172.21.15.19:0/1220308493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 msgr2=0x56035fc4fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc4fdb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe5280049d0 tx=0x7fe5280a5c30 comp rx=0 tx=0).stop 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 -- 172.21.15.19:0/1220308493 shutdown_connections 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8931b0 0x56035fc4fdb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8724d0 0x56035fc450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 --2- 172.21.15.19:0/1220308493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 -- 172.21.15.19:0/1220308493 >> 172.21.15.19:0/1220308493 conn(0x56035f887300 msgr2=0x56035fc32b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 -- 172.21.15.19:0/1220308493 shutdown_connections 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.732+0000 7fe53c20f740 1 -- 172.21.15.19:0/1220308493 wait complete. 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 -- start start 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 Processor -- start 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fcb1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ab080 con 0x56035f8724d0 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ac4f0 con 0x56035f8931b0 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ab750 con 0x56035f86a2f0 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe53c20f740 10 client.0 _ll_register_callbacks cb 0x7fe518004a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc44630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54250/0 (socket says 172.21.15.19:54250) 2024-05-17T09:25:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 -- 172.21.15.19:0/3691399636 learned_addr learned my addr 172.21.15.19:0/3691399636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe52f7fe640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fcb1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe52effd640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 -- 172.21.15.19:0/3691399636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 msgr2=0x56035fc440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 -- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 msgr2=0x56035fcb1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.733+0000 7fe5267fc640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fcb1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5267fc640 1 -- 172.21.15.19:0/3691399636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035fcb21c0 con 0x56035f8724d0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5267fc640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc44630 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b470 tx=0x7fe518004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4be5fed8a2bbe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5167fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5167fc640 1 -- 172.21.15.19:0/3691399636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518004660 con 0x56035f8724d0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5167fc640 1 -- 172.21.15.19:0/3691399636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe518004800 con 0x56035f8724d0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56035fba2360 con 0x56035f8724d0 2024-05-17T09:25:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5167fc640 1 -- 172.21.15.19:0/3691399636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe518004e90 con 0x56035f8724d0 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe53c20f740 10 client.16797 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56035fcb33e0 con 0x56035f8724d0 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5167fc640 1 -- 172.21.15.19:0/3691399636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56035fba2360 con 0x56035f8724d0 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe5167fc640 1 -- 172.21.15.19:0/3691399636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe518003f20 con 0x56035f8724d0 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe53c20f740 10 client.16797 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.734+0000 7fe53c20f740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe53c20f740 20 client.16797 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe53c20f740 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f8271c0 0x56035fcb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe53c20f740 4 client.16797 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56035fb1c5f0 con 0x56035f8271c0 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe52effd640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f8271c0 0x56035fcb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe5167fc640 10 client.16797 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.735+0000 7fe52effd640 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f8271c0 0x56035fcb5e50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8b3904e86ec9a6 server_cookie=af2258b7b3564c60 in_seq=0 out_seq=0 2024-05-17T09:25:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe5167fc640 1 -- 172.21.15.19:0/3691399636 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56035fb1c5f0 con 0x56035f8271c0 2024-05-17T09:25:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe5167fc640 10 client.16797 handle_command_reply: tid=0 2024-05-17T09:25:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 client.16797 shutdown 2024-05-17T09:25:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 msgr2=0x56035fc44630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc44630 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b470 tx=0x7fe518004b40 comp rx=0 tx=0).stop 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 shutdown_connections 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035f8271c0 0x56035fcb5e50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56035f8931b0 0x56035fcb1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe5167fc640 0 client.16797 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56035f8724d0 0x56035fc44630 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 --2- 172.21.15.19:0/3691399636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56035f86a2f0 0x56035fc440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.754+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 >> 172.21.15.19:0/3691399636 conn(0x56035f887300 msgr2=0x56035fc37840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 shutdown_connections 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe53c20f740 1 -- 172.21.15.19:0/3691399636 wait complete. 2024-05-17T09:25:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe53c20f740 20 client.16797 trim_cache size 0 max 16384 2024-05-17T09:25:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:25:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:25:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe51c0b28a0 msgr2=0x7fe51c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe51c0b28a0 0x7fe51c0b4db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe520005760 tx=0x7fe520000bc0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 msgr2=0x7fe5301a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 0x7fe5301a14d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe52800eb10 tx=0x7fe52800c2d0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 shutdown_connections 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe51c0b28a0 0x7fe51c0b4db0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe530108a60 0x7fe5301a14d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe530103d60 0x7fe53019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 --2- 172.21.15.19:0/1288312610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe530102b20 0x7fe530075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 >> 172.21.15.19:0/1288312610 conn(0x7fe5300fdd10 msgr2=0x7fe530104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 shutdown_connections 2024-05-17T09:25:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:11.755+0000 7fe535284640 1 -- 172.21.15.19:0/1288312610 wait complete. 2024-05-17T09:25:12.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.37 MB/sec execute 70 sec latency 3703.249 ms 2024-05-17T09:25:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:12 smithi069 ceph-mon[35081]: pgmap v601: 129 pgs: 129 active+clean; 38 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 552 KiB/s wr, 61 op/s 2024-05-17T09:25:12.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:12 smithi019 ceph-mon[31620]: pgmap v601: 129 pgs: 129 active+clean; 38 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 552 KiB/s wr, 61 op/s 2024-05-17T09:25:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:12 smithi079 ceph-mon[35502]: pgmap v601: 129 pgs: 129 active+clean; 38 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 552 KiB/s wr, 61 op/s 2024-05-17T09:25:13.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.35 MB/sec execute 71 sec latency 4703.370 ms 2024-05-17T09:25:14.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.33 MB/sec execute 72 sec latency 5703.479 ms 2024-05-17T09:25:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:14 smithi069 ceph-mon[35081]: pgmap v602: 129 pgs: 129 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 757 KiB/s wr, 88 op/s 2024-05-17T09:25:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:14 smithi019 ceph-mon[31620]: pgmap v602: 129 pgs: 129 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 757 KiB/s wr, 88 op/s 2024-05-17T09:25:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:14 smithi079 ceph-mon[35502]: pgmap v602: 129 pgs: 129 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 757 KiB/s wr, 88 op/s 2024-05-17T09:25:15.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.31 MB/sec execute 73 sec latency 6703.558 ms 2024-05-17T09:25:16.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.29 MB/sec execute 74 sec latency 7703.646 ms 2024-05-17T09:25:16.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:16 smithi019 ceph-mon[31620]: pgmap v603: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 755 KiB/s wr, 88 op/s 2024-05-17T09:25:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:16 smithi069 ceph-mon[35081]: pgmap v603: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 755 KiB/s wr, 88 op/s 2024-05-17T09:25:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:16 smithi079 ceph-mon[35502]: pgmap v603: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 755 KiB/s wr, 88 op/s 2024-05-17T09:25:17.428 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.27 MB/sec execute 75 sec latency 8703.786 ms 2024-05-17T09:25:18.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.26 MB/sec execute 76 sec latency 9703.921 ms 2024-05-17T09:25:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:18 smithi069 ceph-mon[35081]: pgmap v604: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 702 KiB/s wr, 88 op/s 2024-05-17T09:25:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:18 smithi019 ceph-mon[31620]: pgmap v604: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 702 KiB/s wr, 88 op/s 2024-05-17T09:25:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:18 smithi079 ceph-mon[35502]: pgmap v604: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 702 KiB/s wr, 88 op/s 2024-05-17T09:25:19.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.24 MB/sec execute 77 sec latency 10704.045 ms 2024-05-17T09:25:20.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.23 MB/sec execute 78 sec latency 11704.196 ms 2024-05-17T09:25:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:20 smithi069 ceph-mon[35081]: pgmap v605: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 438 KiB/s wr, 45 op/s 2024-05-17T09:25:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:20 smithi019 ceph-mon[31620]: pgmap v605: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 438 KiB/s wr, 45 op/s 2024-05-17T09:25:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:20 smithi079 ceph-mon[35502]: pgmap v605: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 438 KiB/s wr, 45 op/s 2024-05-17T09:25:21.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.21 MB/sec execute 79 sec latency 12704.305 ms 2024-05-17T09:25:22.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.19 MB/sec execute 80 sec latency 13704.340 ms 2024-05-17T09:25:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:22 smithi069 ceph-mon[35081]: pgmap v606: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 277 KiB/s wr, 38 op/s 2024-05-17T09:25:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:22 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:25:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:22 smithi019 ceph-mon[31620]: pgmap v606: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 277 KiB/s wr, 38 op/s 2024-05-17T09:25:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:22 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:25:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:22 smithi079 ceph-mon[35502]: pgmap v606: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 277 KiB/s wr, 38 op/s 2024-05-17T09:25:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:22 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:25:23.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.18 MB/sec execute 81 sec latency 14704.416 ms 2024-05-17T09:25:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:25:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:25:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:25:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:25:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:25:23.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:25:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:25:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:25:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:25:24.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.17 MB/sec execute 82 sec latency 15704.501 ms 2024-05-17T09:25:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:24 smithi069 ceph-mon[35081]: pgmap v607: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 211 KiB/s wr, 37 op/s 2024-05-17T09:25:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:24 smithi019 ceph-mon[31620]: pgmap v607: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 211 KiB/s wr, 37 op/s 2024-05-17T09:25:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:24 smithi079 ceph-mon[35502]: pgmap v607: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 211 KiB/s wr, 37 op/s 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 -- 172.21.15.19:0/1458428686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0106ff0 msgr2=0x7f4ae0109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/1458428686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0106ff0 0x7f4ae0109410 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009900 tx=0x7f4ad402a410 comp rx=0 tx=0).stop 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 -- 172.21.15.19:0/1458428686 shutdown_connections 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/1458428686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ae0109950 0x7f4ae010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/1458428686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0106ff0 0x7f4ae0109410 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/1458428686 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ae0101850 0x7f4ae0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 -- 172.21.15.19:0/1458428686 >> 172.21.15.19:0/1458428686 conn(0x7f4ae00fd040 msgr2=0x7f4ae00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 -- 172.21.15.19:0/1458428686 shutdown_connections 2024-05-17T09:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.869+0000 7f4ae5b86640 1 -- 172.21.15.19:0/1458428686 wait complete. 2024-05-17T09:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 Processor -- start 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 -- start start 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ae0101850 0x7f4ae019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ae0106ff0 0x7f4ae019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 0x7f4ae01a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae010f5f0 con 0x7f4ae0109950 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae010f470 con 0x7f4ae0101850 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.870+0000 7f4ae5b86640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae010f770 con 0x7f4ae0106ff0 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 0x7f4ae01a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 0x7f4ae01a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48018/0 (socket says 172.21.15.19:48018) 2024-05-17T09:25:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 -- 172.21.15.19:0/918916035 learned_addr learned my addr 172.21.15.19:0/918916035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ae0101850 0x7f4ae019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adeffd640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ae0106ff0 0x7f4ae019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 -- 172.21.15.19:0/918916035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ae0106ff0 msgr2=0x7f4ae019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ae0106ff0 0x7f4ae019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 -- 172.21.15.19:0/918916035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ae0101850 msgr2=0x7f4ae019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ae0101850 0x7f4ae019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 -- 172.21.15.19:0/918916035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae01a1d10 con 0x7f4ae0109950 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adffff640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 0x7f4ae01a16a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800eaa0 tx=0x7f4ad800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adcff9640 1 -- 172.21.15.19:0/918916035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800cdb0 con 0x7f4ae0109950 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae01a1fa0 con 0x7f4ae0109950 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adcff9640 1 -- 172.21.15.19:0/918916035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ad8005020 con 0x7f4ae0109950 2024-05-17T09:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.871+0000 7f4adcff9640 1 -- 172.21.15.19:0/918916035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80116d0 con 0x7f4ae0109950 2024-05-17T09:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.872+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae01a2460 con 0x7f4ae0109950 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.874+0000 7f4aeca40740 1 Processor -- start 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.874+0000 7f4adcff9640 1 -- 172.21.15.19:0/918916035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4ad80051d0 con 0x7f4ae0109950 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 -- start start 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x5584732f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584732f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x5584732fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558473264c30 con 0x558472f511b0 2024-05-17T09:25:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5584732547d0 con 0x558472f282f0 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4aeca40740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558473258810 con 0x558472f304d0 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x5584732f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x5584732f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48022/0 (socket says 172.21.15.19:48022) 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 -- 172.21.15.19:0/206264560 learned_addr learned my addr 172.21.15.19:0/206264560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x5584732fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584732f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 -- 172.21.15.19:0/206264560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 msgr2=0x5584732fdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x5584732fdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 -- 172.21.15.19:0/206264560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 msgr2=0x5584732f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584732f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adeffd640 1 -- 172.21.15.19:0/206264560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558472f69210 con 0x558472f511b0 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.875+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584732f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4adcff9640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ac80b28a0 0x7f4ac80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4adeffd640 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x5584732f67b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ad402a930 tx=0x7f4ad4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade3d5795a1ba12e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ac80b28a0 0x7f4ac80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4adcff9640 1 -- 172.21.15.19:0/918916035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4ad80e2700 con 0x7f4ae0109950 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4ac5ffb640 1 -- 172.21.15.19:0/206264560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4035070 con 0x558472f511b0 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4ac5ffb640 1 -- 172.21.15.19:0/206264560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ad40044c0 con 0x558472f511b0 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.876+0000 7f4ac5ffb640 1 -- 172.21.15.19:0/206264560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4003c20 con 0x558472f511b0 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 -- 172.21.15.19:0/206264560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 msgr2=0x5584732f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ac80b28a0 0x7f4ac80b4db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5584732f7b90 tx=0x7f4ad000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x5584732f67b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ad402a930 tx=0x7f4ad4005fd0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 -- 172.21.15.19:0/206264560 shutdown_connections 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x5584732fdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584732f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 --2- 172.21.15.19:0/206264560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x5584732f67b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 -- 172.21.15.19:0/206264560 >> 172.21.15.19:0/206264560 conn(0x558472f45300 msgr2=0x5584732edd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 -- 172.21.15.19:0/206264560 shutdown_connections 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 -- 172.21.15.19:0/206264560 wait complete. 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.877+0000 7f4aeca40740 1 -- start start 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 Processor -- start 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473300100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473300640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f511b0 0x5584733075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558472f3a790 con 0x558472f282f0 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558473210890 con 0x558472f511b0 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5584732fe620 con 0x558472f304d0 2024-05-17T09:25:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4aeca40740 10 client.0 _ll_register_callbacks cb 0x558473213d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473300100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473300100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48030/0 (socket says 172.21.15.19:48030) 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 -- 172.21.15.19:0/2109123003 learned_addr learned my addr 172.21.15.19:0/2109123003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adeffd640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473300640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adffff640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f511b0 0x5584733075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 -- 172.21.15.19:0/2109123003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 msgr2=0x558473300640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473300640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 -- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f511b0 msgr2=0x5584733075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f511b0 0x5584733075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.878+0000 7f4adf7fe640 1 -- 172.21.15.19:0/2109123003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558473035620 con 0x558472f282f0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473300100 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0003f10 tx=0x7f4ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48546a7d9ae65191 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4abeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4abeffd640 1 -- 172.21.15.19:0/2109123003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0005220 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558473266b80 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4abeffd640 1 -- 172.21.15.19:0/2109123003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4ad00053c0 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4abeffd640 1 -- 172.21.15.19:0/2109123003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ad00055c0 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4aeca40740 10 client.16809 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55847330a5d0 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4abeffd640 1 -- 172.21.15.19:0/2109123003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558473266b80 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.879+0000 7f4abeffd640 1 -- 172.21.15.19:0/2109123003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4ad0024070 con 0x558472f282f0 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4aeca40740 10 client.16809 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4aeca40740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4aeca40740 20 client.16809 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472f0fd80 0x55847330ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4aeca40740 4 client.16809 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558473260360 con 0x558472f0fd80 2024-05-17T09:25:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4adeffd640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472f0fd80 0x55847330ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4adeffd640 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472f0fd80 0x55847330ddb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4bcef7e4d52580 server_cookie=8a18c31296fd340b in_seq=0 out_seq=0 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.880+0000 7f4abeffd640 10 client.16809 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4abeffd640 1 -- 172.21.15.19:0/2109123003 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558473260360 con 0x558472f0fd80 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4abeffd640 10 client.16809 handle_command_reply: tid=0 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 client.16809 shutdown 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 msgr2=0x558473300100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473300100 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0003f10 tx=0x7f4ad00046c0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 shutdown_connections 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472f0fd80 0x55847330ddb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f511b0 0x5584733075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473300640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4abeffd640 0 client.16809 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2109123003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473300100 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.881+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 >> 172.21.15.19:0/2109123003 conn(0x558472f45300 msgr2=0x5584732f0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.882+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 shutdown_connections 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.882+0000 7f4aeca40740 1 -- 172.21.15.19:0/2109123003 wait complete. 2024-05-17T09:25:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.882+0000 7f4aeca40740 20 client.16809 trim_cache size 0 max 16384 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 Processor -- start 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 -- start start 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584733003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473307530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x558473307a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01ad000 con 0x558472f511b0 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01aa8d0 con 0x558472f282f0 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.897+0000 7f4aeca40740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01a8720 con 0x558472f304d0 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x558473307a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x558473307a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48040/0 (socket says 172.21.15.19:48040) 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 -- 172.21.15.19:0/2021928391 learned_addr learned my addr 172.21.15.19:0/2021928391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584733003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adeffd640 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473307530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 -- 172.21.15.19:0/2021928391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 msgr2=0x558473307530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473307530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 -- 172.21.15.19:0/2021928391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 msgr2=0x5584733003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584733003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 -- 172.21.15.19:0/2021928391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584732fda90 con 0x558472f511b0 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adeffd640 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473307530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4adffff640 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x558473307a70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80e01c0 tx=0x7f4ad80a52d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd96f89f8ef44ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4ac57fa640 1 -- 172.21.15.19:0/2021928391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad801f040 con 0x558472f511b0 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4ac57fa640 1 -- 172.21.15.19:0/2021928391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ad80add20 con 0x558472f511b0 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.898+0000 7f4ac57fa640 1 -- 172.21.15.19:0/2021928391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80cc0d0 con 0x558472f511b0 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 -- 172.21.15.19:0/2021928391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 msgr2=0x558473307a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x558473307a70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80e01c0 tx=0x7f4ad80a52d0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 -- 172.21.15.19:0/2021928391 shutdown_connections 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f511b0 0x558473307a70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f304d0 0x558473307530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 --2- 172.21.15.19:0/2021928391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f282f0 0x5584733003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 -- 172.21.15.19:0/2021928391 >> 172.21.15.19:0/2021928391 conn(0x558472f45300 msgr2=0x5584732ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 -- 172.21.15.19:0/2021928391 shutdown_connections 2024-05-17T09:25:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.899+0000 7f4aeca40740 1 -- 172.21.15.19:0/2021928391 wait complete. 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 -- start start 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 Processor -- start 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473305060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f304d0 0x5584733055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f511b0 0x558473306460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01a8d90 con 0x558472f282f0 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01aa200 con 0x558472f304d0 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae01a9460 con 0x558472f511b0 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4aeca40740 10 client.0 _ll_register_callbacks cb 0x7f4ad0003bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473305060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473305060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48044/0 (socket says 172.21.15.19:48044) 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 -- 172.21.15.19:0/150375740 learned_addr learned my addr 172.21.15.19:0/150375740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adeffd640 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f304d0 0x5584733055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 -- 172.21.15.19:0/150375740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f511b0 msgr2=0x558473306460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f511b0 0x558473306460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 -- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f304d0 msgr2=0x5584733055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f304d0 0x5584733055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.900+0000 7f4adf7fe640 1 -- 172.21.15.19:0/150375740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558473306a20 con 0x558472f282f0 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4adf7fe640 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473305060 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00057c0 tx=0x7f4ad0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8d0531e06ddd6e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4abeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4abeffd640 1 -- 172.21.15.19:0/150375740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0038060 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad80add20 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4abeffd640 1 -- 172.21.15.19:0/150375740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4ad00108f0 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4abeffd640 1 -- 172.21.15.19:0/150375740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ad003f050 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4aeca40740 10 client.16815 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558473307d20 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4abeffd640 1 -- 172.21.15.19:0/150375740 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ad80add20 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.901+0000 7f4abeffd640 1 -- 172.21.15.19:0/150375740 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4ad0024040 con 0x558472f282f0 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4aeca40740 10 client.16815 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4aeca40740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4aeca40740 20 client.16815 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4aeca40740 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472ee51c0 0x558473378bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4aeca40740 4 client.16815 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:25:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584731da5f0 con 0x558472ee51c0 2024-05-17T09:25:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4adeffd640 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472ee51c0 0x558473378bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4abeffd640 10 client.16815 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.902+0000 7f4adeffd640 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472ee51c0 0x558473378bb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b2312920aae92d server_cookie=8f22e1d539a7a4ef in_seq=0 out_seq=0 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4abeffd640 1 -- 172.21.15.19:0/150375740 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5584731da5f0 con 0x558472ee51c0 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4abeffd640 10 client.16815 handle_command_reply: tid=0 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 client.16815 shutdown 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 msgr2=0x558473305060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473305060 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00057c0 tx=0x7f4ad0015000 comp rx=0 tx=0).stop 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 shutdown_connections 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558472ee51c0 0x558473378bb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558472f511b0 0x558473306460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4abeffd640 0 client.16815 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558472f304d0 0x5584733055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 --2- 172.21.15.19:0/150375740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558472f282f0 0x558473305060 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 >> 172.21.15.19:0/150375740 conn(0x558472f45300 msgr2=0x5584732ec330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 shutdown_connections 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.921+0000 7f4aeca40740 1 -- 172.21.15.19:0/150375740 wait complete. 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4aeca40740 20 client.16815 trim_cache size 0 max 16384 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ac80b28a0 msgr2=0x7f4ac80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ac80b28a0 0x7f4ac80b4db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5584732f7b90 tx=0x7f4ad000b390 comp rx=0 tx=0).stop 2024-05-17T09:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 msgr2=0x7f4ae01a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 0x7f4ae01a16a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800eaa0 tx=0x7f4ad800ef60 comp rx=0 tx=0).stop 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 shutdown_connections 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4ac80b28a0 0x7f4ac80b4db0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae0109950 0x7f4ae01a16a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ae0106ff0 0x7f4ae019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 --2- 172.21.15.19:0/918916035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ae0101850 0x7f4ae019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 >> 172.21.15.19:0/918916035 conn(0x7f4ae00fd040 msgr2=0x7f4ae0107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 shutdown_connections 2024-05-17T09:25:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:24.922+0000 7f4ae5b86640 1 -- 172.21.15.19:0/918916035 wait complete. 2024-05-17T09:25:25.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.15 MB/sec execute 83 sec latency 16704.597 ms 2024-05-17T09:25:26.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.14 MB/sec execute 84 sec latency 17704.723 ms 2024-05-17T09:25:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:26 smithi069 ceph-mon[35081]: pgmap v608: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-05-17T09:25:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:26 smithi019 ceph-mon[31620]: pgmap v608: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-05-17T09:25:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:26 smithi079 ceph-mon[35502]: pgmap v608: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-05-17T09:25:27.429 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.12 MB/sec execute 85 sec latency 18704.809 ms 2024-05-17T09:25:28.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.11 MB/sec execute 86 sec latency 19704.919 ms 2024-05-17T09:25:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:28 smithi069 ceph-mon[35081]: pgmap v609: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-05-17T09:25:28.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:28 smithi019 ceph-mon[31620]: pgmap v609: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-05-17T09:25:28.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:28 smithi079 ceph-mon[35502]: pgmap v609: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-05-17T09:25:29.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.10 MB/sec execute 87 sec latency 20705.042 ms 2024-05-17T09:25:29.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:25:29.801 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:25:30.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.09 MB/sec execute 88 sec latency 21705.195 ms 2024-05-17T09:25:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:30 smithi069 ceph-mon[35081]: pgmap v610: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-05-17T09:25:30.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:30 smithi019 ceph-mon[31620]: pgmap v610: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-05-17T09:25:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:30 smithi079 ceph-mon[35502]: pgmap v610: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-05-17T09:25:31.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.07 MB/sec execute 89 sec latency 22705.294 ms 2024-05-17T09:25:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/703556992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc0109840 msgr2=0x7f1cc01111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/703556992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc0109840 0x7f1cc01111a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800b0d0 tx=0x7f1cb800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/703556992 shutdown_connections 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/703556992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cc01045c0 0x7f1cc01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/703556992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc0109840 0x7f1cc01111a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/703556992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 0x7f1cc0109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.221+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/703556992 >> 172.21.15.19:0/703556992 conn(0x7f1cc00ffe90 msgr2=0x7f1cc01022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.222+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/703556992 shutdown_connections 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.222+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/703556992 wait complete. 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.222+0000 7f1cc6fd9700 1 Processor -- start 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.222+0000 7f1cc6fd9700 1 -- start start 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc6fd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 0x7f1cc01a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc6fd9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 0x7f1cc01a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc6fd9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cc0109840 0x7f1cc019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc6fd9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0114320 con 0x7f1cc01045c0 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc6fd9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01141a0 con 0x7f1cc0108ef0 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc6fd9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01144a0 con 0x7f1cc0109840 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 0x7f1cc01a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 0x7f1cc01a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48062/0 (socket says 172.21.15.19:48062) 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 -- 172.21.15.19:0/764316176 learned_addr learned my addr 172.21.15.19:0/764316176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc5576700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cc0109840 0x7f1cc019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cbffff700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 0x7f1cc01a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 -- 172.21.15.19:0/764316176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cc0109840 msgr2=0x7f1cc019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cc0109840 0x7f1cc019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 -- 172.21.15.19:0/764316176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 msgr2=0x7f1cc01a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 0x7f1cc01a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cc4d75700 1 -- 172.21.15.19:0/764316176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019b150 con 0x7f1cc01045c0 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.223+0000 7f1cbffff700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 0x7f1cc01a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.224+0000 7f1cc4d75700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 0x7f1cc01a0880 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c920 tx=0x7f1cb400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.224+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4007b00 con 0x7f1cc01045c0 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.224+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cb4004cc0 con 0x7f1cc01045c0 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.224+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4005440 con 0x7f1cc01045c0 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.224+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019b440 con 0x7f1cc01045c0 2024-05-17T09:25:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.224+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a7710 con 0x7f1cc01045c0 2024-05-17T09:25:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.225+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc00512b0 con 0x7f1cc01045c0 2024-05-17T09:25:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.228+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1cb4004750 con 0x7f1cc01045c0 2024-05-17T09:25:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.228+0000 7f1cbdffb700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1cb00b2880 0x7f1cb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.229+0000 7f1cbffff700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1cb00b2880 0x7f1cb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.229+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1cb40e2570 con 0x7f1cc01045c0 2024-05-17T09:25:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.229+0000 7f1cbffff700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1cb00b2880 0x7f1cb00b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1cac005e10 tx=0x7f1cac005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.232+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cb4014030 con 0x7f1cc01045c0 2024-05-17T09:25:32.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.06 MB/sec execute 90 sec latency 23705.361 ms 2024-05-17T09:25:32.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.505+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cc019bfd0 con 0x7f1cc01045c0 2024-05-17T09:25:32.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.505+0000 7f1cbdffb700 1 -- 172.21.15.19:0/764316176 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f1cb40a9ee0 con 0x7f1cc01045c0 2024-05-17T09:25:32.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:25:32.506 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1cb00b2880 msgr2=0x7f1cb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1cb00b2880 0x7f1cb00b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1cac005e10 tx=0x7f1cac005d60 comp rx=0 tx=0).stop 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 msgr2=0x7f1cc01a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 0x7f1cc01a0880 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c920 tx=0x7f1cb400cc30 comp rx=0 tx=0).stop 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 shutdown_connections 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1cb00b2880 0x7f1cb00b4d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cc0109840 0x7f1cc019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc01045c0 0x7f1cc01a0880 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 --2- 172.21.15.19:0/764316176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc0108ef0 0x7f1cc01a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 >> 172.21.15.19:0/764316176 conn(0x7f1cc00ffe90 msgr2=0x7f1cc0108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 shutdown_connections 2024-05-17T09:25:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:32.508+0000 7f1cc6fd9700 1 -- 172.21.15.19:0/764316176 wait complete. 2024-05-17T09:25:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:25:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:32 smithi069 ceph-mon[35081]: pgmap v611: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-05-17T09:25:32.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:32 smithi019 ceph-mon[31620]: pgmap v611: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-05-17T09:25:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:32 smithi079 ceph-mon[35502]: pgmap v611: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-05-17T09:25:33.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 863fb380 --release --await' 2024-05-17T09:25:33.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 863fb380 --release --await 2024-05-17T09:25:33.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.05 MB/sec execute 91 sec latency 24705.392 ms 2024-05-17T09:25:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:25:33.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:33 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/764316176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:33.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:33 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/764316176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:33.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:33 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/764316176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:25:34.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.04 MB/sec execute 92 sec latency 25705.435 ms 2024-05-17T09:25:34.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:34 smithi019 ceph-mon[31620]: pgmap v612: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-05-17T09:25:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:34 smithi069 ceph-mon[35081]: pgmap v612: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-05-17T09:25:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:34 smithi079 ceph-mon[35502]: pgmap v612: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-05-17T09:25:35.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60602 1.03 MB/sec execute 93 sec latency 26705.485 ms 2024-05-17T09:25:35.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 -- 172.21.15.19:0/3922016629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 msgr2=0x7fd0341100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/3922016629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341100b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00b0d0 tx=0x7fd02c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 -- 172.21.15.19:0/3922016629 shutdown_connections 2024-05-17T09:25:35.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/3922016629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0341034d0 0x7fd0341038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/3922016629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341100b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/3922016629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034107e00 0x7fd034108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 -- 172.21.15.19:0/3922016629 >> 172.21.15.19:0/3922016629 conn(0x7fd0340fef40 msgr2=0x7fd034101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 -- 172.21.15.19:0/3922016629 shutdown_connections 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.492+0000 7fd03a1e3700 1 -- 172.21.15.19:0/3922016629 wait complete. 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.493+0000 7fd03a1e3700 1 Processor -- start 2024-05-17T09:25:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.493+0000 7fd03a1e3700 1 -- start start 2024-05-17T09:25:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd03a1e3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0341034d0 0x7fd03419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd03a1e3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd034107e00 0x7fd03419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd03a1e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd03a1e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd034113280 con 0x7fd034108750 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd03a1e3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd034113100 con 0x7fd0341034d0 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd03a1e3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd034113400 con 0x7fd034107e00 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41460/0 (socket says 172.21.15.19:41460) 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 -- 172.21.15.19:0/1116987734 learned_addr learned my addr 172.21.15.19:0/1116987734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0341034d0 0x7fd03419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd02affd700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd034107e00 0x7fd03419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 -- 172.21.15.19:0/1116987734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd034107e00 msgr2=0x7fd03419c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd034107e00 0x7fd03419c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 -- 172.21.15.19:0/1116987734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0341034d0 msgr2=0x7fd03419bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0341034d0 0x7fd03419bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 -- 172.21.15.19:0/1116987734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03419c9b0 con 0x7fd034108750 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.494+0000 7fd033fff700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341a6c40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007d10 tx=0x7fd02c007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.495+0000 7fd0317fa700 1 -- 172.21.15.19:0/1116987734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c01f070 con 0x7fd034108750 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.495+0000 7fd0317fa700 1 -- 172.21.15.19:0/1116987734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd02c003e10 con 0x7fd034108750 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.495+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0341a7240 con 0x7fd034108750 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.495+0000 7fd0317fa700 1 -- 172.21.15.19:0/1116987734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c01b720 con 0x7fd034108750 2024-05-17T09:25:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.495+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a7690 con 0x7fd034108750 2024-05-17T09:25:35.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.498+0000 7fd0317fa700 1 -- 172.21.15.19:0/1116987734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd02c022070 con 0x7fd034108750 2024-05-17T09:25:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd0317fa700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0200b2900 0x7fd0200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0200b2900 0x7fd0200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd048bf8b80 1 Processor -- start 2024-05-17T09:25:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd0317fa700 1 -- 172.21.15.19:0/1116987734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd02c00f020 con 0x7fd034108750 2024-05-17T09:25:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd048bf8b80 1 -- start start 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0200b2900 0x7fd0200b4e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd024005950 tx=0x7fd02400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.499+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50091560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 0x562b50091aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b5008ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b4ffb9d90 con 0x562b4ff0cf40 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b4ffa9930 con 0x562b5008b620 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562b4ffad970 con 0x562b4ff8a4d0 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 0x562b50091aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 0x562b50091aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41472/0 (socket says 172.21.15.19:41472) 2024-05-17T09:25:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 -- 172.21.15.19:0/2600000993 learned_addr learned my addr 172.21.15.19:0/2600000993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd0337fe700 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50091560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd033fff700 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b5008ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 -- 172.21.15.19:0/2600000993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 msgr2=0x562b50091560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50091560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 -- 172.21.15.19:0/2600000993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 msgr2=0x562b5008ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b5008ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 -- 172.21.15.19:0/2600000993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4fef99d0 con 0x562b4ff0cf40 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd033fff700 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b5008ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd02affd700 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 0x562b50091aa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00bc70 tx=0x7fd01c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5fe09801e20db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd029ffb700 1 -- 172.21.15.19:0/2600000993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c010be0 con 0x562b4ff0cf40 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd029ffb700 1 -- 172.21.15.19:0/2600000993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd01c0044c0 con 0x562b4ff0cf40 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.500+0000 7fd029ffb700 1 -- 172.21.15.19:0/2600000993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c005020 con 0x562b4ff0cf40 2024-05-17T09:25:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 -- 172.21.15.19:0/2600000993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 msgr2=0x562b50091aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 0x562b50091aa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00bc70 tx=0x7fd01c00bf80 comp rx=0 tx=0).stop 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 -- 172.21.15.19:0/2600000993 shutdown_connections 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50091560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b5008ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/2600000993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff0cf40 0x562b50091aa0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 -- 172.21.15.19:0/2600000993 >> 172.21.15.19:0/2600000993 conn(0x562b4ff4bd80 msgr2=0x562b50086520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 -- 172.21.15.19:0/2600000993 shutdown_connections 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.501+0000 7fd048bf8b80 1 -- 172.21.15.19:0/2600000993 wait complete. 2024-05-17T09:25:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 -- start start 2024-05-17T09:25:35.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 Processor -- start 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b500951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 0x562b50095710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b50095c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b4feb00a0 con 0x562b4ff8a4d0 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b4ff4b8c0 con 0x562b5008b620 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.502+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562b500a1400 con 0x562b4ff0cf40 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 0x562b50095710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd048bf8b80 10 client.0 _ll_register_callbacks cb 0x562b4ff44a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 0x562b50095710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41480/0 (socket says 172.21.15.19:41480) 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 -- 172.21.15.19:0/1686270585 learned_addr learned my addr 172.21.15.19:0/1686270585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b500951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd033fff700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b50095c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 -- 172.21.15.19:0/1686270585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 msgr2=0x562b500951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b500951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 -- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 msgr2=0x562b50095c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b50095c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 -- 172.21.15.19:0/1686270585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4ff48790 con 0x562b4ff8a4d0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b500951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd033fff700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b50095c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd02affd700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 0x562b50095710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd01c010c10 tx=0x7fd01c010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea449d2ad445f038 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd012ffd700 1 -- 172.21.15.19:0/1686270585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c011650 con 0x562b4ff8a4d0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01c0044c0 con 0x562b4ff8a4d0 2024-05-17T09:25:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd012ffd700 1 -- 172.21.15.19:0/1686270585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd01c0117f0 con 0x562b4ff8a4d0 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.503+0000 7fd012ffd700 1 -- 172.21.15.19:0/1686270585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd01c02f050 con 0x562b4ff8a4d0 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 10 client.16833 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b500a2cf0 con 0x562b4ff8a4d0 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd012ffd700 1 -- 172.21.15.19:0/1686270585 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd01c0044c0 con 0x562b4ff8a4d0 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd012ffd700 1 -- 172.21.15.19:0/1686270585 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd01c026990 con 0x562b4ff8a4d0 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 10 client.16833 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 20 client.16833 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b500a5270 0x562b500a7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 4 client.16833 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4ffbbce0 con 0x562b500a5270 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.504+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b500a5270 0x562b500a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.505+0000 7fd012ffd700 10 client.16833 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.505+0000 7fd0337fe700 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b500a5270 0x562b500a7690 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2acf485571ef586d server_cookie=5239f50d6d281958 in_seq=0 out_seq=0 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.505+0000 7fd012ffd700 1 -- 172.21.15.19:0/1686270585 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562b4ffbbce0 con 0x562b500a5270 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.505+0000 7fd012ffd700 10 client.16833 handle_command_reply: tid=0 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 client.16833 shutdown 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 msgr2=0x562b50095710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 0x562b50095710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd01c010c10 tx=0x7fd01c010cf0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 shutdown_connections 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b500951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b500a5270 0x562b500a7690 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b5008b620 0x562b50095c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/1686270585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b4ff8a4d0 0x562b50095710 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 >> 172.21.15.19:0/1686270585 conn(0x562b4ff4bd80 msgr2=0x562b50080280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd012ffd700 0 client.16833 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:35.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 shutdown_connections 2024-05-17T09:25:35.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 1 -- 172.21.15.19:0/1686270585 wait complete. 2024-05-17T09:25:35.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.506+0000 7fd048bf8b80 20 client.16833 trim_cache size 0 max 16384 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 Processor -- start 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 -- start start 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b5008b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff8a4d0 0x562b50096880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50096dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341b2ab0 con 0x562b5008b620 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341b0340 con 0x562b4ff8a4d0 2024-05-17T09:25:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341ae190 con 0x562b4ff0cf40 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50096dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50096dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41494/0 (socket says 172.21.15.19:41494) 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 -- 172.21.15.19:0/232911043 learned_addr learned my addr 172.21.15.19:0/232911043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd02affd700 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff8a4d0 0x562b50096880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd0337fe700 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b5008b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 -- 172.21.15.19:0/232911043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 msgr2=0x562b5008b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b5008b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 -- 172.21.15.19:0/232911043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff8a4d0 msgr2=0x562b50096880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff8a4d0 0x562b50096880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd033fff700 1 -- 172.21.15.19:0/232911043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b5008c630 con 0x562b5008b620 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.527+0000 7fd02affd700 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff8a4d0 0x562b50096880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.528+0000 7fd033fff700 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50096dc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0e3a10 tx=0x7fd02c0aa6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de27832a6f8e1e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.528+0000 7fd0297fa700 1 -- 172.21.15.19:0/232911043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0aa730 con 0x562b5008b620 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.528+0000 7fd0297fa700 1 -- 172.21.15.19:0/232911043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd02c0ab320 con 0x562b5008b620 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.528+0000 7fd0297fa700 1 -- 172.21.15.19:0/232911043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c005640 con 0x562b5008b620 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.528+0000 7fd048bf8b80 1 -- 172.21.15.19:0/232911043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 msgr2=0x562b50096dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.528+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50096dc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0e3a10 tx=0x7fd02c0aa6a0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 -- 172.21.15.19:0/232911043 shutdown_connections 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff0cf40 0x562b5008b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50096dc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/232911043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff8a4d0 0x562b50096880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 -- 172.21.15.19:0/232911043 >> 172.21.15.19:0/232911043 conn(0x562b4ff4bd80 msgr2=0x562b50090970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 -- 172.21.15.19:0/232911043 shutdown_connections 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 -- 172.21.15.19:0/232911043 wait complete. 2024-05-17T09:25:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 -- start start 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.529+0000 7fd048bf8b80 1 Processor -- start 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff0cf40 0x562b50107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50103a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341ae800 con 0x562b5008b620 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341afc70 con 0x562b4ff0cf40 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341aeed0 con 0x562b4ff8a4d0 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 10 client.0 _ll_register_callbacks cb 0x562b4ff44a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41510/0 (socket says 172.21.15.19:41510) 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 -- 172.21.15.19:0/333212821 learned_addr learned my addr 172.21.15.19:0/333212821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd0337fe700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff0cf40 0x562b50107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd02affd700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 -- 172.21.15.19:0/333212821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 msgr2=0x562b50108430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50108430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 -- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff0cf40 msgr2=0x562b50107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff0cf40 0x562b50107ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 -- 172.21.15.19:0/333212821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b501041c0 con 0x562b5008b620 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd02affd700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd033fff700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50103a70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0a6ca0 tx=0x7fd02c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741e64cb1e5ec308 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd012ffd700 1 -- 172.21.15.19:0/333212821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0aac20 con 0x562b5008b620 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd012ffd700 1 -- 172.21.15.19:0/333212821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd02c0ab090 con 0x562b5008b620 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.530+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02c0ab320 con 0x562b5008b620 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd012ffd700 1 -- 172.21.15.19:0/333212821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd02c0b3560 con 0x562b5008b620 2024-05-17T09:25:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 10 client.16839 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b50110570 con 0x562b5008b620 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd012ffd700 1 -- 172.21.15.19:0/333212821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd02c0ab320 con 0x562b5008b620 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd012ffd700 1 -- 172.21.15.19:0/333212821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd02c011070 con 0x562b5008b620 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 10 client.16839 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 10 client.16839 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 20 client.16839 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b501109d0 0x562b50112df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 4 client.16839 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "863fb380", "release": true, "await": true}] 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "863fb380", "release": true, "await": true}) -- 0x562b4ffb54c0 con 0x562b501109d0 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.531+0000 7fd0337fe700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b501109d0 0x562b50112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.532+0000 7fd012ffd700 10 client.16839 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.532+0000 7fd0337fe700 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b501109d0 0x562b50112df0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f0cf46fbaef15c server_cookie=dffd19ce43619770 in_seq=0 out_seq=0 2024-05-17T09:25:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.537+0000 7fd012ffd700 1 -- 172.21.15.19:0/333212821 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562b4ffb54c0 con 0x562b501109d0 2024-05-17T09:25:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.537+0000 7fd012ffd700 10 client.16839 handle_command_reply: tid=0 2024-05-17T09:25:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.537+0000 7fd048bf8b80 1 client.16839 shutdown 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 msgr2=0x562b50103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50103a70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0a6ca0 tx=0x7fd02c005850 comp rx=0 tx=0).stop 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 shutdown_connections 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562b501109d0 0x562b50112df0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b5008b620 0x562b50103a70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b4ff0cf40 0x562b50107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 --2- 172.21.15.19:0/333212821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b4ff8a4d0 0x562b50108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd012ffd700 0 client.16839 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 >> 172.21.15.19:0/333212821 conn(0x562b4ff4bd80 msgr2=0x562b5008ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 shutdown_connections 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 1 -- 172.21.15.19:0/333212821 wait complete. 2024-05-17T09:25:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd048bf8b80 20 client.16839 trim_cache size 0 max 16384 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 24, 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "863fb380": { 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 24, 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:25:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 146.7, 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:25:35.541 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.538+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0200b2900 msgr2=0x7fd0200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0200b2900 0x7fd0200b4e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd024005950 tx=0x7fd02400a300 comp rx=0 tx=0).stop 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 msgr2=0x7fd0341a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341a6c40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007d10 tx=0x7fd02c007d40 comp rx=0 tx=0).stop 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 shutdown_connections 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd034108750 0x7fd0341a6c40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0341034d0 0x7fd03419bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd0200b2900 0x7fd0200b4e10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 --2- 172.21.15.19:0/1116987734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd034107e00 0x7fd03419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 >> 172.21.15.19:0/1116987734 conn(0x7fd0340fef40 msgr2=0x7fd034107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 shutdown_connections 2024-05-17T09:25:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:25:35.539+0000 7fd03a1e3700 1 -- 172.21.15.19:0/1116987734 wait complete. 2024-05-17T09:25:36.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60815 1.02 MB/sec execute 94 sec latency 26815.563 ms 2024-05-17T09:25:36.436 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '863fb380', total seconds elapsed: 33.6 2024-05-17T09:25:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:36 smithi069 ceph-mon[35081]: pgmap v613: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-17T09:25:36.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:36 smithi019 ceph-mon[31620]: pgmap v613: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-17T09:25:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:36 smithi079 ceph-mon[35502]: pgmap v613: 129 pgs: 129 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-17T09:25:37.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 61249 1.02 MB/sec execute 95 sec latency 12.711 ms 2024-05-17T09:25:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/3026580788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 msgr2=0x7fa6b410ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/3026580788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b410ae70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009ca0 tx=0x7fa6ac02a8c0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/3026580788 shutdown_connections 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/3026580788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b410ae70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/3026580788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6b4103d50 0x7fa6b4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/3026580788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/3026580788 >> 172.21.15.19:0/3026580788 conn(0x7fa6b40fdd20 msgr2=0x7fa6b41001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/3026580788 shutdown_connections 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.031+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/3026580788 wait complete. 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 Processor -- start 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 -- start start 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6b4103d50 0x7fa6b41995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410e540 con 0x7fa6b4108a50 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410e3c0 con 0x7fa6b4102b10 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6bb2f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410e6c0 con 0x7fa6b4103d50 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6b9095640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6b9095640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41468/0 (socket says 172.21.15.19:41468) 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6b9896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6b8894640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6b4103d50 0x7fa6b41995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.032+0000 7fa6b9095640 1 -- 172.21.15.19:0/4149809486 learned_addr learned my addr 172.21.15.19:0/4149809486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9095640 1 -- 172.21.15.19:0/4149809486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6b4103d50 msgr2=0x7fa6b41995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9095640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6b4103d50 0x7fa6b41995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9095640 1 -- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 msgr2=0x7fa6b41a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9095640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b41a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9095640 1 -- 172.21.15.19:0/4149809486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b41a0bd0 con 0x7fa6b4102b10 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9896640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6b9095640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4199090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000e9f0 tx=0x7fa6b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6aa7fc640 1 -- 172.21.15.19:0/4149809486 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000ccc0 con 0x7fa6b4102b10 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6aa7fc640 1 -- 172.21.15.19:0/4149809486 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6b00044b0 con 0x7fa6b4102b10 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b41a0ec0 con 0x7fa6b4102b10 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6aa7fc640 1 -- 172.21.15.19:0/4149809486 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0010430 con 0x7fa6b4102b10 2024-05-17T09:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.033+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a1320 con 0x7fa6b4102b10 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.036+0000 7fa6c21a9740 1 Processor -- start 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.036+0000 7fa6c21a9740 1 -- start start 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.036+0000 7fa6aa7fc640 1 -- 172.21.15.19:0/4149809486 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa6b0003f90 con 0x7fa6b4102b10 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.036+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.036+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e44b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f64db4c30 con 0x563f64a804d0 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f64da47d0 con 0x563f64a782f0 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f64da8810 con 0x563f64aa11b0 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b9095640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e44b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41538/0 (socket says 172.21.15.19:41538) 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 -- 172.21.15.19:0/1960920281 learned_addr learned my addr 172.21.15.19:0/1960920281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b9095640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e44b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41470/0 (socket says 172.21.15.19:41470) 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b9896640 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 -- 172.21.15.19:0/1960920281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 msgr2=0x563f64e445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 -- 172.21.15.19:0/1960920281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 msgr2=0x563f64e44b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e44b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 -- 172.21.15.19:0/1960920281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f64ab9210 con 0x563f64a804d0 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b9095640 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e44b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b9896640 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6b8894640 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4ba30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4004650 tx=0x7fa6a400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52a3554978102a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa6aa7fc640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a00b28a0 0x7fa6a00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa6b9896640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a00b28a0 0x7fa6a00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa6aa7fc640 1 -- 172.21.15.19:0/4149809486 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6b00e2900 con 0x7fa6b4102b10 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.037+0000 7fa69f7fe640 1 -- 172.21.15.19:0/1960920281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4003bd0 con 0x563f64a804d0 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa69f7fe640 1 -- 172.21.15.19:0/1960920281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6a4015070 con 0x563f64a804d0 2024-05-17T09:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa69f7fe640 1 -- 172.21.15.19:0/1960920281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4004c70 con 0x563f64a804d0 2024-05-17T09:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1960920281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 msgr2=0x563f64e4ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4ba30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4004650 tx=0x7fa6a400bba0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.038+0000 7fa6b9896640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a00b28a0 0x7fa6a00b4db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x563f64e45780 tx=0x7fa6ac031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1960920281 shutdown_connections 2024-05-17T09:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4ba30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e44b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1960920281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1960920281 >> 172.21.15.19:0/1960920281 conn(0x563f64a95300 msgr2=0x563f64e3dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1960920281 shutdown_connections 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1960920281 wait complete. 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 -- start start 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.039+0000 7fa6c21a9740 1 Processor -- start 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f64a8a790 con 0x563f64a804d0 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f64d60890 con 0x563f64a782f0 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f64e4c480 con 0x563f64aa11b0 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6c21a9740 10 client.0 _ll_register_callbacks cb 0x563f64d63d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41544/0 (socket says 172.21.15.19:41544) 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 -- 172.21.15.19:0/2459525011 learned_addr learned my addr 172.21.15.19:0/2459525011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b9095640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b9896640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 -- 172.21.15.19:0/2459525011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 msgr2=0x563f64e553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 -- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 msgr2=0x563f64e4df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e4df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 -- 172.21.15.19:0/2459525011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f64b85620 con 0x563f64a804d0 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b9896640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b9095640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa6b8894640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4e460 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c000 tx=0x7fa6a40043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba9bd067acebb3f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa69cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.040+0000 7fa69cff9640 1 -- 172.21.15.19:0/2459525011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4011440 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa69cff9640 1 -- 172.21.15.19:0/2459525011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6a40115e0 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4015070 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa69cff9640 1 -- 172.21.15.19:0/2459525011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa6a4030050 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 10 client.16851 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f64e58370 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa69cff9640 1 -- 172.21.15.19:0/2459525011 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6a4015070 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa69cff9640 1 -- 172.21.15.19:0/2459525011 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa6a40257a0 con 0x563f64a804d0 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 10 client.16851 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 20 client.16851 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64a5fd80 0x563f64e5bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 4 client.16851 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f64db6b80 con 0x563f64a5fd80 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.041+0000 7fa6b9095640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64a5fd80 0x563f64e5bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.042+0000 7fa69cff9640 10 client.16851 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.042+0000 7fa6b9095640 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64a5fd80 0x563f64e5bef0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd06ee7e7048d23 server_cookie=9a54bbf578af5f50 in_seq=0 out_seq=0 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.043+0000 7fa69cff9640 1 -- 172.21.15.19:0/2459525011 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563f64db6b80 con 0x563f64a5fd80 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.043+0000 7fa69cff9640 10 client.16851 handle_command_reply: tid=0 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.043+0000 7fa6c21a9740 1 client.16851 shutdown 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.043+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 msgr2=0x563f64e4e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.043+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4e460 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c000 tx=0x7fa6a40043a0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 shutdown_connections 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64a5fd80 0x563f64e5bef0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64aa11b0 0x563f64e553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e4e460 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa69cff9640 0 client.16851 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2459525011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 >> 172.21.15.19:0/2459525011 conn(0x563f64a95300 msgr2=0x563f64e40c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 shutdown_connections 2024-05-17T09:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2459525011 wait complete. 2024-05-17T09:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.044+0000 7fa6c21a9740 20 client.16851 trim_cache size 0 max 16384 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 Processor -- start 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 -- start start 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e3dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e55820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41ad000 con 0x563f64a804d0 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41aa8d0 con 0x563f64a782f0 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a8720 con 0x563f64a5fd80 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6b9896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e55820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6b9896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e55820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41558/0 (socket says 172.21.15.19:41558) 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6b9896640 1 -- 172.21.15.19:0/2397840892 learned_addr learned my addr 172.21.15.19:0/2397840892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.062+0000 7fa6b8894640 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9095640 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e3dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9896640 1 -- 172.21.15.19:0/2397840892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 msgr2=0x563f64e3dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9896640 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e3dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9896640 1 -- 172.21.15.19:0/2397840892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 msgr2=0x563f64e552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9896640 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9896640 1 -- 172.21.15.19:0/2397840892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f64e44160 con 0x563f64a804d0 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b8894640 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa6b9896640 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e55820 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009ca0 tx=0x7fa6ac041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ceb380499aadae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa69effd640 1 -- 172.21.15.19:0/2397840892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004040 con 0x563f64a804d0 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa69effd640 1 -- 172.21.15.19:0/2397840892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6ac036050 con 0x563f64a804d0 2024-05-17T09:25:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.063+0000 7fa69effd640 1 -- 172.21.15.19:0/2397840892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac030a40 con 0x563f64a804d0 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2397840892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 msgr2=0x563f64e55820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e55820 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009ca0 tx=0x7fa6ac041000 comp rx=0 tx=0).stop 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2397840892 shutdown_connections 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a804d0 0x563f64e55820 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a782f0 0x563f64e552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/2397840892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e3dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2397840892 >> 172.21.15.19:0/2397840892 conn(0x563f64a95300 msgr2=0x563f64e41560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2397840892 shutdown_connections 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.064+0000 7fa6c21a9740 1 -- 172.21.15.19:0/2397840892 wait complete. 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 -- start start 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 Processor -- start 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 0x563f64e525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a804d0 0x563f64e52b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a8d90 con 0x563f64a782f0 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41aa200 con 0x563f64a804d0 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a9460 con 0x563f64a5fd80 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6c21a9740 10 client.0 _ll_register_callbacks cb 0x7fa6a4005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6b8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 0x563f64e525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6b8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 0x563f64e525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41568/0 (socket says 172.21.15.19:41568) 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6b8894640 1 -- 172.21.15.19:0/1085058861 learned_addr learned my addr 172.21.15.19:0/1085058861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.065+0000 7fa6b9896640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a804d0 0x563f64e52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9095640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9896640 1 -- 172.21.15.19:0/1085058861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 msgr2=0x563f64e559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9896640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9896640 1 -- 172.21.15.19:0/1085058861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 msgr2=0x563f64e525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9896640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 0x563f64e525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9896640 1 -- 172.21.15.19:0/1085058861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f64ec25b0 con 0x563f64a804d0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b8894640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 0x563f64e525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6b9896640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a804d0 0x563f64e52b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac031920 tx=0x7fa6ac0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748596620a935e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa69cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa69cff9640 1 -- 172.21.15.19:0/1085058861 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac061060 con 0x563f64a804d0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa69cff9640 1 -- 172.21.15.19:0/1085058861 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6ac004000 con 0x563f64a804d0 2024-05-17T09:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ac036050 con 0x563f64a804d0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa69cff9640 1 -- 172.21.15.19:0/1085058861 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa6ac068050 con 0x563f64a804d0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6c21a9740 10 client.26501 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.066+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f64ec3710 con 0x563f64a804d0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa69cff9640 1 -- 172.21.15.19:0/1085058861 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6ac036050 con 0x563f64a804d0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa69cff9640 1 -- 172.21.15.19:0/1085058861 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa6ac0498b0 con 0x563f64a804d0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6c21a9740 10 client.26501 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6c21a9740 10 client.26501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6c21a9740 20 client.26501 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64aa11b0 0x563f64ec60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6c21a9740 4 client.26501 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f64db0360 con 0x563f64aa11b0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6b9095640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64aa11b0 0x563f64ec60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa69cff9640 10 client.26501 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:38.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.067+0000 7fa6b9095640 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64aa11b0 0x563f64ec60d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f03d0d54b0e806 server_cookie=33baa497762dd2a7 in_seq=0 out_seq=0 2024-05-17T09:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa69cff9640 1 -- 172.21.15.19:0/1085058861 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563f64db0360 con 0x563f64aa11b0 2024-05-17T09:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa69cff9640 10 client.26501 handle_command_reply: tid=0 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 client.26501 shutdown 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a804d0 msgr2=0x563f64e52b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a804d0 0x563f64e52b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac031920 tx=0x7fa6ac0095c0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 shutdown_connections 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f64aa11b0 0x563f64ec60d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f64a804d0 0x563f64e52b00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f64a782f0 0x563f64e525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa69cff9640 0 client.26501 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 --2- 172.21.15.19:0/1085058861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f64a5fd80 0x563f64e559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 >> 172.21.15.19:0/1085058861 conn(0x563f64a95300 msgr2=0x563f64e3cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 shutdown_connections 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 1 -- 172.21.15.19:0/1085058861 wait complete. 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.086+0000 7fa6c21a9740 20 client.26501 trim_cache size 0 max 16384 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a00b28a0 msgr2=0x7fa6a00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a00b28a0 0x7fa6a00b4db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x563f64e45780 tx=0x7fa6ac031000 comp rx=0 tx=0).stop 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 msgr2=0x7fa6b4199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4199090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000e9f0 tx=0x7fa6b000eeb0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 shutdown_connections 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa6a00b28a0 0x7fa6a00b4db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6b4108a50 0x7fa6b41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6b4103d50 0x7fa6b41995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 --2- 172.21.15.19:0/4149809486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6b4102b10 0x7fa6b4199090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 >> 172.21.15.19:0/4149809486 conn(0x7fa6b40fdd20 msgr2=0x7fa6b4104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 shutdown_connections 2024-05-17T09:25:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:38.087+0000 7fa6bb2f9640 1 -- 172.21.15.19:0/4149809486 wait complete. 2024-05-17T09:25:38.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 61797 1.05 MB/sec execute 96 sec latency 15.971 ms 2024-05-17T09:25:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:38 smithi069 ceph-mon[35081]: pgmap v614: 129 pgs: 129 active+clean; 40 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-05-17T09:25:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:38 smithi019 ceph-mon[31620]: pgmap v614: 129 pgs: 129 active+clean; 40 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-05-17T09:25:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:38 smithi079 ceph-mon[35502]: pgmap v614: 129 pgs: 129 active+clean; 40 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-05-17T09:25:39.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 62376 1.05 MB/sec execute 97 sec latency 17.463 ms 2024-05-17T09:25:40.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 62889 1.06 MB/sec execute 98 sec latency 14.889 ms 2024-05-17T09:25:40.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:40 smithi069 ceph-mon[35081]: pgmap v615: 129 pgs: 129 active+clean; 40 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-05-17T09:25:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:40 smithi019 ceph-mon[31620]: pgmap v615: 129 pgs: 129 active+clean; 40 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-05-17T09:25:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:40 smithi079 ceph-mon[35502]: pgmap v615: 129 pgs: 129 active+clean; 40 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 104 KiB/s wr, 23 op/s 2024-05-17T09:25:41.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 63617 1.11 MB/sec execute 99 sec latency 15.242 ms 2024-05-17T09:25:42.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 64010 1.14 MB/sec execute 100 sec latency 16.458 ms 2024-05-17T09:25:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:42 smithi069 ceph-mon[35081]: pgmap v616: 129 pgs: 129 active+clean; 40 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 119 KiB/s wr, 24 op/s 2024-05-17T09:25:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:42 smithi019 ceph-mon[31620]: pgmap v616: 129 pgs: 129 active+clean; 40 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 119 KiB/s wr, 24 op/s 2024-05-17T09:25:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:42 smithi079 ceph-mon[35502]: pgmap v616: 129 pgs: 129 active+clean; 40 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 119 KiB/s wr, 24 op/s 2024-05-17T09:25:43.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 64285 1.14 MB/sec execute 101 sec latency 14.153 ms 2024-05-17T09:25:44.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 64611 1.13 MB/sec execute 102 sec latency 12.586 ms 2024-05-17T09:25:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:44 smithi069 ceph-mon[35081]: pgmap v617: 129 pgs: 129 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 500 KiB/s wr, 78 op/s 2024-05-17T09:25:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:44 smithi019 ceph-mon[31620]: pgmap v617: 129 pgs: 129 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 500 KiB/s wr, 78 op/s 2024-05-17T09:25:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:44 smithi079 ceph-mon[35502]: pgmap v617: 129 pgs: 129 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 500 KiB/s wr, 78 op/s 2024-05-17T09:25:45.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 65090 1.14 MB/sec execute 103 sec latency 17.432 ms 2024-05-17T09:25:46.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 65698 1.16 MB/sec execute 104 sec latency 13.466 ms 2024-05-17T09:25:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:46 smithi069 ceph-mon[35081]: pgmap v618: 129 pgs: 129 active+clean; 41 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 701 KiB/s wr, 82 op/s 2024-05-17T09:25:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:46 smithi019 ceph-mon[31620]: pgmap v618: 129 pgs: 129 active+clean; 41 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 701 KiB/s wr, 82 op/s 2024-05-17T09:25:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:46 smithi079 ceph-mon[35502]: pgmap v618: 129 pgs: 129 active+clean; 41 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 701 KiB/s wr, 82 op/s 2024-05-17T09:25:47.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 66256 1.16 MB/sec execute 105 sec latency 23.645 ms 2024-05-17T09:25:48.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 66843 1.20 MB/sec execute 106 sec latency 15.943 ms 2024-05-17T09:25:48.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:48 smithi069 ceph-mon[35081]: pgmap v619: 129 pgs: 129 active+clean; 42 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 913 KiB/s wr, 111 op/s 2024-05-17T09:25:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:48 smithi019 ceph-mon[31620]: pgmap v619: 129 pgs: 129 active+clean; 42 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 913 KiB/s wr, 111 op/s 2024-05-17T09:25:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:48 smithi079 ceph-mon[35502]: pgmap v619: 129 pgs: 129 active+clean; 42 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 913 KiB/s wr, 111 op/s 2024-05-17T09:25:49.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 67377 1.24 MB/sec execute 107 sec latency 11.174 ms 2024-05-17T09:25:50.430 INFO:tasks.workunit.client.0.smithi019.stdout: 1 67739 1.24 MB/sec execute 108 sec latency 16.021 ms 2024-05-17T09:25:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:50 smithi069 ceph-mon[35081]: pgmap v620: 129 pgs: 129 active+clean; 43 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:25:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:50 smithi019 ceph-mon[31620]: pgmap v620: 129 pgs: 129 active+clean; 43 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:25:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:50 smithi079 ceph-mon[35502]: pgmap v620: 129 pgs: 129 active+clean; 43 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:25:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 -- 172.21.15.19:0/2803703099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 msgr2=0x7fd618103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 --2- 172.21.15.19:0/2803703099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd618103860 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd608009900 tx=0x7fd60802a410 comp rx=0 tx=0).stop 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 -- 172.21.15.19:0/2803703099 shutdown_connections 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 --2- 172.21.15.19:0/2803703099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 0x7fd61810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 --2- 172.21.15.19:0/2803703099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 0x7fd6181041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 --2- 172.21.15.19:0/2803703099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd618103860 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 -- 172.21.15.19:0/2803703099 >> 172.21.15.19:0/2803703099 conn(0x7fd6180fdd50 msgr2=0x7fd618100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 -- 172.21.15.19:0/2803703099 shutdown_connections 2024-05-17T09:25:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.194+0000 7fd61c957640 1 -- 172.21.15.19:0/2803703099 wait complete. 2024-05-17T09:25:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 Processor -- start 2024-05-17T09:25:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 -- start start 2024-05-17T09:25:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd6180ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 0x7fd6180fe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 0x7fd6180fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810e4f0 con 0x7fd618102b60 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810e370 con 0x7fd618103da0 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61c957640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810e670 con 0x7fd618108aa0 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd6180ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.195+0000 7fd61659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd6180ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55772/0 (socket says 172.21.15.19:55772) 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd61659c640 1 -- 172.21.15.19:0/1939295681 learned_addr learned my addr 172.21.15.19:0/1939295681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd616d9d640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 0x7fd6180fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 0x7fd6180fe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 -- 172.21.15.19:0/1939295681 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 msgr2=0x7fd6180fe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 0x7fd6180fe9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 -- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 msgr2=0x7fd6180ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd6180ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 -- 172.21.15.19:0/1939295681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6180ff120 con 0x7fd618103da0 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd616d9d640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 0x7fd6180fe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd61659c640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd6180ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 0x7fd6180fe4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e9f0 tx=0x7fd60c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd6077fe640 1 -- 172.21.15.19:0/1939295681 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00ccc0 con 0x7fd618103da0 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.196+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6181a42d0 con 0x7fd618103da0 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.197+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6181a47c0 con 0x7fd618103da0 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.197+0000 7fd6077fe640 1 -- 172.21.15.19:0/1939295681 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd60c0108c0 con 0x7fd618103da0 2024-05-17T09:25:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.197+0000 7fd6077fe640 1 -- 172.21.15.19:0/1939295681 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c018660 con 0x7fd618103da0 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 Processor -- start 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 -- start start 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd6077fe640 1 -- 172.21.15.19:0/1939295681 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd60c018800 con 0x7fd618103da0 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb5f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb66820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55955bad1c30 con 0x55955b7952f0 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55955bac17d0 con 0x55955b7be1b0 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.199+0000 7fd623806740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55955bac5810 con 0x55955b79d4d0 2024-05-17T09:25:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd61659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb5f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd61659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb5f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55788/0 (socket says 172.21.15.19:55788) 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd61659c640 1 -- 172.21.15.19:0/3697447892 learned_addr learned my addr 172.21.15.19:0/3697447892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd615d9b640 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb66820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb5f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51792/0 (socket says 172.21.15.19:51792) 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 -- 172.21.15.19:0/3697447892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 msgr2=0x55955bb66820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb66820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 -- 172.21.15.19:0/3697447892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 msgr2=0x55955bb5f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb5f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 -- 172.21.15.19:0/3697447892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55955b7d6210 con 0x55955b7be1b0 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd61659c640 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb5f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd616d9d640 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb5f3a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd610005fd0 tx=0x7fd610011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30abff29154a7213 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd604ff9640 1 -- 172.21.15.19:0/3697447892 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610004d20 con 0x55955b7be1b0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd604ff9640 1 -- 172.21.15.19:0/3697447892 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd61000f9a0 con 0x55955b7be1b0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.200+0000 7fd604ff9640 1 -- 172.21.15.19:0/3697447892 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6100043d0 con 0x55955b7be1b0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd6077fe640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd6000b28a0 0x7fd6000b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd61659c640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd6000b28a0 0x7fd6000b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd6077fe640 1 -- 172.21.15.19:0/1939295681 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd60c0e28b0 con 0x7fd618103da0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd623806740 1 -- 172.21.15.19:0/3697447892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 msgr2=0x55955bb5f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd623806740 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb5f3a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd610005fd0 tx=0x7fd610011bf0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd61659c640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd6000b28a0 0x7fd6000b4db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6080319f0 tx=0x7fd608031910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd623806740 1 -- 172.21.15.19:0/3697447892 shutdown_connections 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.201+0000 7fd623806740 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb66820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb5f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 --2- 172.21.15.19:0/3697447892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb5f3a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 -- 172.21.15.19:0/3697447892 >> 172.21.15.19:0/3697447892 conn(0x55955b7b2300 msgr2=0x55955bb5ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 -- 172.21.15.19:0/3697447892 shutdown_connections 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 -- 172.21.15.19:0/3697447892 wait complete. 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 -- start start 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.202+0000 7fd623806740 1 Processor -- start 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 0x55955bb68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7be1b0 0x55955bb701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55955b7a7790 con 0x55955b7be1b0 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55955ba7d890 con 0x55955b7952f0 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55955bb67270 con 0x55955b79d4d0 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 0x55955bb68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd623806740 10 client.0 _ll_register_callbacks cb 0x55955ba80d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 0x55955bb68d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51808/0 (socket says 172.21.15.19:51808) 2024-05-17T09:25:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 -- 172.21.15.19:0/1528117444 learned_addr learned my addr 172.21.15.19:0/1528117444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd616d9d640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7be1b0 0x55955bb701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 -- 172.21.15.19:0/1528117444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 msgr2=0x55955bb69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 -- 172.21.15.19:0/1528117444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7be1b0 msgr2=0x55955bb701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7be1b0 0x55955bb701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.203+0000 7fd61659c640 1 -- 172.21.15.19:0/1528117444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55955b8a2620 con 0x55955b7952f0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd61659c640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 0x55955bb68d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd608000c00 tx=0x7fd608009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62627309515488c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd5fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd5fdffb640 1 -- 172.21.15.19:0/1528117444 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608035040 con 0x55955b7952f0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61000f9a0 con 0x55955b7952f0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd5fdffb640 1 -- 172.21.15.19:0/1528117444 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd608003a00 con 0x55955b7952f0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd5fdffb640 1 -- 172.21.15.19:0/1528117444 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd608003c00 con 0x55955b7952f0 2024-05-17T09:25:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd623806740 10 client.26513 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55955bb731b0 con 0x55955b7952f0 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd5fdffb640 1 -- 172.21.15.19:0/1528117444 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd61000f9a0 con 0x55955b7952f0 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.204+0000 7fd5fdffb640 1 -- 172.21.15.19:0/1528117444 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd608044030 con 0x55955b7952f0 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd623806740 10 client.26513 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd623806740 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd623806740 20 client.26513 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd623806740 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b77cd80 0x55955bb768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd623806740 4 client.26513 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55955bad3b80 con 0x55955b77cd80 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b77cd80 0x55955bb768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd5fdffb640 10 client.26513 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.205+0000 7fd615d9b640 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b77cd80 0x55955bb768d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accc47417b85dbdf server_cookie=bfe5b163022e034b in_seq=0 out_seq=0 2024-05-17T09:25:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd5fdffb640 1 -- 172.21.15.19:0/1528117444 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55955bad3b80 con 0x55955b77cd80 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd5fdffb640 10 client.26513 handle_command_reply: tid=0 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 client.26513 shutdown 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 msgr2=0x55955bb68d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 0x55955bb68d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd608000c00 tx=0x7fd608009260 comp rx=0 tx=0).stop 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 shutdown_connections 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b77cd80 0x55955bb768d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7be1b0 0x55955bb701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b79d4d0 0x55955bb69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd5fdffb640 0 client.26513 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 --2- 172.21.15.19:0/1528117444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7952f0 0x55955bb68d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 >> 172.21.15.19:0/1528117444 conn(0x55955b7b2300 msgr2=0x55955bb59d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 shutdown_connections 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 1 -- 172.21.15.19:0/1528117444 wait complete. 2024-05-17T09:25:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.206+0000 7fd623806740 20 client.26513 trim_cache size 0 max 16384 2024-05-17T09:25:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 Processor -- start 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 -- start start 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb63890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b79d4d0 0x55955bb70150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7be1b0 0x55955bb5e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8012510 con 0x55955b7952f0 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800fde0 con 0x55955b79d4d0 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.223+0000 7fd623806740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800dc30 con 0x55955b7be1b0 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb63890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd616d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7be1b0 0x55955bb5e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb63890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55818/0 (socket says 172.21.15.19:55818) 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd615d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b79d4d0 0x55955bb70150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 -- 172.21.15.19:0/2248407090 learned_addr learned my addr 172.21.15.19:0/2248407090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 -- 172.21.15.19:0/2248407090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7be1b0 msgr2=0x55955bb5e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7be1b0 0x55955bb5e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 -- 172.21.15.19:0/2248407090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b79d4d0 msgr2=0x55955bb70150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b79d4d0 0x55955bb70150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 -- 172.21.15.19:0/2248407090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55955bb690c0 con 0x55955b7952f0 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd615d9b640 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b79d4d0 0x55955bb70150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd616d9d640 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7be1b0 0x55955bb5e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.224+0000 7fd61659c640 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb63890 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd60802a930 tx=0x7fd60802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb179e7dcb1c3ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.225+0000 7fd5fffff640 1 -- 172.21.15.19:0/2248407090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608035040 con 0x55955b7952f0 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.225+0000 7fd5fffff640 1 -- 172.21.15.19:0/2248407090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd60804c070 con 0x55955b7952f0 2024-05-17T09:25:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.225+0000 7fd5fffff640 1 -- 172.21.15.19:0/2248407090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60803d6b0 con 0x55955b7952f0 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 -- 172.21.15.19:0/2248407090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 msgr2=0x55955bb63890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb63890 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd60802a930 tx=0x7fd60802a410 comp rx=0 tx=0).stop 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 -- 172.21.15.19:0/2248407090 shutdown_connections 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7be1b0 0x55955bb5e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b79d4d0 0x55955bb70150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 --2- 172.21.15.19:0/2248407090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b7952f0 0x55955bb63890 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 -- 172.21.15.19:0/2248407090 >> 172.21.15.19:0/2248407090 conn(0x55955b7b2300 msgr2=0x55955bb680d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 -- 172.21.15.19:0/2248407090 shutdown_connections 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 -- 172.21.15.19:0/2248407090 wait complete. 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 -- start start 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.226+0000 7fd623806740 1 Processor -- start 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 0x55955bb6d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 0x55955bb6dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800e2a0 con 0x55955b79d4d0 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800f710 con 0x55955b7be1b0 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800e970 con 0x55955b7952f0 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 10 client.0 _ll_register_callbacks cb 0x7fd608009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd615d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 0x55955bb6dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 0x55955bb6d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd615d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 0x55955bb6dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55828/0 (socket says 172.21.15.19:55828) 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 0x55955bb6d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:45972/0 (socket says 172.21.15.19:45972) 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd615d9b640 1 -- 172.21.15.19:0/185913160 learned_addr learned my addr 172.21.15.19:0/185913160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd616d9d640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 -- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 msgr2=0x55955bb6eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb6eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 -- 172.21.15.19:0/185913160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 msgr2=0x55955bb6dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 0x55955bb6dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 -- 172.21.15.19:0/185913160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55955bb6f0c0 con 0x55955b7952f0 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd615d9b640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 0x55955bb6dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd61659c640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 0x55955bb6d690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55955bb76ed0 tx=0x7fd608051aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15325ddae6bfd32 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd5fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:25:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd5fdffb640 1 -- 172.21.15.19:0/185913160 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608035040 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd5fdffb640 1 -- 172.21.15.19:0/185913160 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd60803d450 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.227+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60804c070 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd5fdffb640 1 -- 172.21.15.19:0/185913160 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd60806f050 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 10 client.26497 fetch_fsmap learned FSMap version 32 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55955bb70350 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd5fdffb640 1 -- 172.21.15.19:0/185913160 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd60804c070 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd5fdffb640 1 -- 172.21.15.19:0/185913160 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd608065030 con 0x55955b7952f0 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 10 client.26497 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 10 client.26497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 20 client.26497 populate_metadata read hostname 'smithi019' 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b7521c0 0x55955bbe14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:25:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 4 client.26497 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:25:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55955bacd360 con 0x55955b7521c0 2024-05-17T09:25:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.228+0000 7fd615d9b640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b7521c0 0x55955bbe14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:25:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.229+0000 7fd5fdffb640 10 client.26497 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.229+0000 7fd615d9b640 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b7521c0 0x55955bbe14d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135949a84880d9b5 server_cookie=6ee10d4cb6ddb797 in_seq=0 out_seq=0 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd5fdffb640 1 -- 172.21.15.19:0/185913160 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55955bacd360 con 0x55955b7521c0 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd5fdffb640 10 client.26497 handle_command_reply: tid=0 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 client.26497 shutdown 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 msgr2=0x55955bb6d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 0x55955bb6d690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55955bb76ed0 tx=0x7fd608051aa0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 shutdown_connections 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55955b7521c0 0x55955bbe14d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55955b7be1b0 0x55955bb6eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd5fdffb640 0 client.26497 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55955b79d4d0 0x55955bb6dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 --2- 172.21.15.19:0/185913160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55955b7952f0 0x55955bb6d690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 >> 172.21.15.19:0/185913160 conn(0x55955b7b2300 msgr2=0x55955bb67e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 shutdown_connections 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 1 -- 172.21.15.19:0/185913160 wait complete. 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.246+0000 7fd623806740 20 client.26497 trim_cache size 0 max 16384 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:25:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:25:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd6000b28a0 msgr2=0x7fd6000b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd6000b28a0 0x7fd6000b4db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6080319f0 tx=0x7fd608031910 comp rx=0 tx=0).stop 2024-05-17T09:25:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 msgr2=0x7fd6180fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:25:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 0x7fd6180fe4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e9f0 tx=0x7fd60c00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 shutdown_connections 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618108aa0 0x7fd6180fe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd618103da0 0x7fd6180fe4b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd6000b28a0 0x7fd6000b4db0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 --2- 172.21.15.19:0/1939295681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102b60 0x7fd6180ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 >> 172.21.15.19:0/1939295681 conn(0x7fd6180fdd50 msgr2=0x7fd61806b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 shutdown_connections 2024-05-17T09:25:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:25:51.247+0000 7fd61c957640 1 -- 172.21.15.19:0/1939295681 wait complete. 2024-05-17T09:25:51.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 68017 1.23 MB/sec execute 109 sec latency 11.760 ms 2024-05-17T09:25:52.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 68488 1.24 MB/sec execute 110 sec latency 16.412 ms 2024-05-17T09:25:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:52 smithi069 ceph-mon[35081]: pgmap v621: 129 pgs: 129 active+clean; 43 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-17T09:25:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:52 smithi019 ceph-mon[31620]: pgmap v621: 129 pgs: 129 active+clean; 43 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-17T09:25:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:52 smithi079 ceph-mon[35502]: pgmap v621: 129 pgs: 129 active+clean; 43 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-17T09:25:53.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 69036 1.26 MB/sec execute 111 sec latency 15.994 ms 2024-05-17T09:25:54.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 69713 1.26 MB/sec execute 112 sec latency 15.749 ms 2024-05-17T09:25:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:54 smithi069 ceph-mon[35081]: pgmap v622: 129 pgs: 129 active+clean; 40 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-17T09:25:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:54 smithi019 ceph-mon[31620]: pgmap v622: 129 pgs: 129 active+clean; 40 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-17T09:25:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:54 smithi079 ceph-mon[35502]: pgmap v622: 129 pgs: 129 active+clean; 40 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-17T09:25:55.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 70197 1.28 MB/sec execute 113 sec latency 16.021 ms 2024-05-17T09:25:56.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 70747 1.30 MB/sec execute 114 sec latency 15.836 ms 2024-05-17T09:25:56.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:56 smithi069 ceph-mon[35081]: pgmap v623: 129 pgs: 129 active+clean; 40 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:25:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:56 smithi019 ceph-mon[31620]: pgmap v623: 129 pgs: 129 active+clean; 40 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:25:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:56 smithi079 ceph-mon[35502]: pgmap v623: 129 pgs: 129 active+clean; 40 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:25:57.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 71212 1.33 MB/sec execute 115 sec latency 13.496 ms 2024-05-17T09:25:58.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 71448 1.32 MB/sec execute 116 sec latency 15.102 ms 2024-05-17T09:25:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:25:58 smithi069 ceph-mon[35081]: pgmap v624: 129 pgs: 129 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:25:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:25:58 smithi019 ceph-mon[31620]: pgmap v624: 129 pgs: 129 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:25:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:25:58 smithi079 ceph-mon[35502]: pgmap v624: 129 pgs: 129 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:25:59.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 71883 1.32 MB/sec execute 117 sec latency 12.324 ms 2024-05-17T09:26:00.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 72436 1.34 MB/sec execute 118 sec latency 16.842 ms 2024-05-17T09:26:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:00 smithi069 ceph-mon[35081]: pgmap v625: 129 pgs: 129 active+clean; 41 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-17T09:26:00.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:00 smithi019 ceph-mon[31620]: pgmap v625: 129 pgs: 129 active+clean; 41 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-17T09:26:00.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:00 smithi079 ceph-mon[35502]: pgmap v625: 129 pgs: 129 active+clean; 41 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-17T09:26:01.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73068 1.34 MB/sec execute 119 sec latency 13.154 ms 2024-05-17T09:26:02.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73573 1.34 MB/sec execute 120 sec latency 15.432 ms 2024-05-17T09:26:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:02 smithi069 ceph-mon[35081]: pgmap v626: 129 pgs: 129 active+clean; 41 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:26:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:02 smithi019 ceph-mon[31620]: pgmap v626: 129 pgs: 129 active+clean; 41 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:26:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:02 smithi079 ceph-mon[35502]: pgmap v626: 129 pgs: 129 active+clean; 41 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:26:03.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 74288 1.39 MB/sec execute 121 sec latency 15.466 ms 2024-05-17T09:26:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.367+0000 7f65af30d640 1 -- 172.21.15.19:0/2554519663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a8106d60 msgr2=0x7f65a8109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.367+0000 7f65af30d640 1 --2- 172.21.15.19:0/2554519663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a8106d60 0x7f65a8109240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6598009900 tx=0x7f659802a410 comp rx=0 tx=0).stop 2024-05-17T09:26:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 -- 172.21.15.19:0/2554519663 shutdown_connections 2024-05-17T09:26:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 --2- 172.21.15.19:0/2554519663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a81098d0 0x7f65a810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 --2- 172.21.15.19:0/2554519663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a8106d60 0x7f65a8109240 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 --2- 172.21.15.19:0/2554519663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 0x7f65a8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 -- 172.21.15.19:0/2554519663 >> 172.21.15.19:0/2554519663 conn(0x7f65a80fd4d0 msgr2=0x7f65a80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 -- 172.21.15.19:0/2554519663 shutdown_connections 2024-05-17T09:26:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.368+0000 7f65af30d640 1 -- 172.21.15.19:0/2554519663 wait complete. 2024-05-17T09:26:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 Processor -- start 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 -- start start 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 0x7f65a819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a8106d60 0x7f65a819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 0x7f65a81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a810f5f0 con 0x7f65a81098d0 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65a810f470 con 0x7f65a8106d60 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65af30d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65a810f770 con 0x7f65a8101f80 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65ad8aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 0x7f65a81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65ad8aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 0x7f65a81a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38982/0 (socket says 172.21.15.19:38982) 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65ad8aa640 1 -- 172.21.15.19:0/3228638710 learned_addr learned my addr 172.21.15.19:0/3228638710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a8106d60 0x7f65a819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.369+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 0x7f65a819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3228638710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 msgr2=0x7f65a819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 0x7f65a819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 msgr2=0x7f65a81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 0x7f65a81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3228638710 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a81a1cd0 con 0x7f65a8106d60 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 0x7f65a81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 0x7f65a819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a8106d60 0x7f65a819a6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6598005760 tx=0x7f6598005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f659e7fc640 1 -- 172.21.15.19:0/3228638710 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659803f070 con 0x7f65a8106d60 2024-05-17T09:26:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f659e7fc640 1 -- 172.21.15.19:0/3228638710 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6598003a80 con 0x7f65a8106d60 2024-05-17T09:26:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f659e7fc640 1 -- 172.21.15.19:0/3228638710 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598035070 con 0x7f65a8106d60 2024-05-17T09:26:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a81a1f60 con 0x7f65a8106d60 2024-05-17T09:26:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.370+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a81a2370 con 0x7f65a8106d60 2024-05-17T09:26:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.373+0000 7f65b61c2740 1 Processor -- start 2024-05-17T09:26:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.373+0000 7f659e7fc640 1 -- 172.21.15.19:0/3228638710 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f65980035f0 con 0x7f65a8106d60 2024-05-17T09:26:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 -- start start 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 0x55975abbf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abbf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55975ab31c30 con 0x55975a7fd4d0 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55975ab217d0 con 0x55975a81e1b0 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65b61c2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55975ab25810 con 0x55975a7f52f0 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad8aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 0x55975abbf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad8aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 0x55975abbf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45304/0 (socket says 172.21.15.19:45304) 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad8aa640 1 -- 172.21.15.19:0/248530820 learned_addr learned my addr 172.21.15.19:0/248530820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abbf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad0a9640 1 -- 172.21.15.19:0/248530820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 msgr2=0x55975abbf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 0x55975abbf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad0a9640 1 -- 172.21.15.19:0/248530820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 msgr2=0x55975abc6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.374+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abc6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f65ad0a9640 1 -- 172.21.15.19:0/248530820 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975a836210 con 0x55975a7f52f0 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 0x55975abbf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f659e7fc640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65940b28a0 0x7f65940b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65940b28a0 0x7f65940b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f659e7fc640 1 -- 172.21.15.19:0/3228638710 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f65981026e0 con 0x7f65a8106d60 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abbf920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009fa0 tx=0x7f65a4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb8f84c74874aba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.375+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65940b28a0 0x7f65940b4db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55975abc0570 tx=0x7f65a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.376+0000 7f65937fe640 1 -- 172.21.15.19:0/248530820 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4004220 con 0x55975a7f52f0 2024-05-17T09:26:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.376+0000 7f65937fe640 1 -- 172.21.15.19:0/248530820 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65a40043c0 con 0x55975a7f52f0 2024-05-17T09:26:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.376+0000 7f65937fe640 1 -- 172.21.15.19:0/248530820 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40035f0 con 0x55975a7f52f0 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 -- 172.21.15.19:0/248530820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 msgr2=0x55975abbf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abbf920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009fa0 tx=0x7f65a4011810 comp rx=0 tx=0).stop 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 -- 172.21.15.19:0/248530820 shutdown_connections 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abc6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abbf920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 --2- 172.21.15.19:0/248530820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a81e1b0 0x55975abbf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 -- 172.21.15.19:0/248530820 >> 172.21.15.19:0/248530820 conn(0x55975a812300 msgr2=0x55975abbad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 -- 172.21.15.19:0/248530820 shutdown_connections 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.377+0000 7f65b61c2740 1 -- 172.21.15.19:0/248530820 wait complete. 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.378+0000 7f65b61c2740 1 -- start start 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.378+0000 7f65b61c2740 1 Processor -- start 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abcd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abcd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a81e1b0 0x55975abd45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55975a807790 con 0x55975a81e1b0 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55975aadd890 con 0x55975a7fd4d0 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55975abcb680 con 0x55975a7f52f0 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abcd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abcd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45318/0 (socket says 172.21.15.19:45318) 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3743109739 learned_addr learned my addr 172.21.15.19:0/3743109739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65b61c2740 10 client.0 _ll_register_callbacks cb 0x55975aae0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a81e1b0 0x55975abd45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abcd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3743109739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 msgr2=0x55975abcd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abcd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3743109739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a81e1b0 msgr2=0x55975abd45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a81e1b0 0x55975abd45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.379+0000 7f65ac8a8640 1 -- 172.21.15.19:0/3743109739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975a902620 con 0x55975a7fd4d0 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a81e1b0 0x55975abd45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abcd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:26:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abcd690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f65980c6c70 tx=0x7f65980057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=203eaea0e57b4044 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f6590ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f6590ff9640 1 -- 172.21.15.19:0/3743109739 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598048070 con 0x55975a7fd4d0 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f6590ff9640 1 -- 172.21.15.19:0/3743109739 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f65980c5c90 con 0x55975a7fd4d0 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.380+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a40043c0 con 0x55975a7fd4d0 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f6590ff9640 1 -- 172.21.15.19:0/3743109739 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65980c97d0 con 0x55975a7fd4d0 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 10 client.26531 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55975abd7310 con 0x55975a7fd4d0 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f6590ff9640 1 -- 172.21.15.19:0/3743109739 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65a40043c0 con 0x55975a7fd4d0 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f6590ff9640 1 -- 172.21.15.19:0/3743109739 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6598101070 con 0x55975a7fd4d0 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 10 client.26531 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 20 client.26531 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7dcd80 0x55975abdab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 4 client.26531 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.381+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55975ab33b80 con 0x55975a7dcd80 2024-05-17T09:26:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.382+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7dcd80 0x55975abdab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.382+0000 7f6590ff9640 10 client.26531 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.382+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7dcd80 0x55975abdab50 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545d26b88dbef7f7 server_cookie=450769ccbf498804 in_seq=0 out_seq=0 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f6590ff9640 1 -- 172.21.15.19:0/3743109739 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55975ab33b80 con 0x55975a7dcd80 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f6590ff9640 10 client.26531 handle_command_reply: tid=0 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 client.26531 shutdown 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 msgr2=0x55975abcd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abcd690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f65980c6c70 tx=0x7f65980057c0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 shutdown_connections 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7dcd80 0x55975abdab50 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a81e1b0 0x55975abd45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abcd690 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 --2- 172.21.15.19:0/3743109739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a7f52f0 0x55975abcd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f6590ff9640 0 client.26531 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.383+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 >> 172.21.15.19:0/3743109739 conn(0x55975a812300 msgr2=0x55975abbdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.384+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 shutdown_connections 2024-05-17T09:26:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.384+0000 7f65b61c2740 1 -- 172.21.15.19:0/3743109739 wait complete. 2024-05-17T09:26:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.384+0000 7f65b61c2740 20 client.26531 trim_cache size 0 max 16384 2024-05-17T09:26:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 Processor -- start 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 -- start start 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 0x55975abc38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abd4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975abdf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a81acf90 con 0x55975a7fd4d0 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65a81aa860 con 0x55975a7f52f0 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65b61c2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65a81a86b0 con 0x55975a81e1b0 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.402+0000 7f65ac8a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abd4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 0x55975abc38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 0x55975abc38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45328/0 (socket says 172.21.15.19:45328) 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 -- 172.21.15.19:0/2274667249 learned_addr learned my addr 172.21.15.19:0/2274667249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975abdf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 -- 172.21.15.19:0/2274667249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 msgr2=0x55975abdf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975abdf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 -- 172.21.15.19:0/2274667249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 msgr2=0x55975abd4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abd4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 -- 172.21.15.19:0/2274667249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975abb4fd0 con 0x55975a7f52f0 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 0x55975abc38b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65a4014040 tx=0x7f65a4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c98f7ba8afd70fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f6592ffd640 1 -- 172.21.15.19:0/2274667249 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a401d040 con 0x55975a7f52f0 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f6592ffd640 1 -- 172.21.15.19:0/2274667249 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65a40044d0 con 0x55975a7f52f0 2024-05-17T09:26:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.403+0000 7f6592ffd640 1 -- 172.21.15.19:0/2274667249 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a400fc90 con 0x55975a7f52f0 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 -- 172.21.15.19:0/2274667249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 msgr2=0x55975abc38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 0x55975abc38b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65a4014040 tx=0x7f65a4009310 comp rx=0 tx=0).stop 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 -- 172.21.15.19:0/2274667249 shutdown_connections 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975abdf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7fd4d0 0x55975abd4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 --2- 172.21.15.19:0/2274667249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7f52f0 0x55975abc38b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 -- 172.21.15.19:0/2274667249 >> 172.21.15.19:0/2274667249 conn(0x55975a812300 msgr2=0x55975abbe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 -- 172.21.15.19:0/2274667249 shutdown_connections 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 -- 172.21.15.19:0/2274667249 wait complete. 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 -- start start 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.405+0000 7f65b61c2740 1 Processor -- start 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 0x55975abd3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abd3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975ac41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65a81a8d20 con 0x55975a7f52f0 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65a81aa190 con 0x55975a7fd4d0 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65a81a93f0 con 0x55975a81e1b0 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65b61c2740 10 client.0 _ll_register_callbacks cb 0x7f6598108240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 0x55975abd3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 0x55975abd3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39018/0 (socket says 172.21.15.19:39018) 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 -- 172.21.15.19:0/548631251 learned_addr learned my addr 172.21.15.19:0/548631251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abd3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad8aa640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975ac41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 -- 172.21.15.19:0/548631251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 msgr2=0x55975ac41210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975ac41210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 -- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 msgr2=0x55975abd3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abd3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ad0a9640 1 -- 172.21.15.19:0/548631251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975ac41750 con 0x55975a7f52f0 2024-05-17T09:26:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.406+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abd3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f65ad0a9640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 0x55975abd3520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f65a401b040 tx=0x7f65a4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df664de5ed7b643 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f6590ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f6590ff9640 1 -- 172.21.15.19:0/548631251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a401d030 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f6590ff9640 1 -- 172.21.15.19:0/548631251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f65a400fa10 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a40044d0 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f6590ff9640 1 -- 172.21.15.19:0/548631251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65a400f640 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f65b61c2740 10 client.16893 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55975ac42900 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.407+0000 7f6590ff9640 1 -- 172.21.15.19:0/548631251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65a40044d0 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f6590ff9640 1 -- 172.21.15.19:0/548631251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f65a4024430 con 0x55975a7f52f0 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65b61c2740 10 client.16893 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65b61c2740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65b61c2740 20 client.16893 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65b61c2740 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7b21c0 0x55975ac45100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65b61c2740 4 client.16893 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55975ab2d360 con 0x55975a7b21c0 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7b21c0 0x55975ac45100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f6590ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.408+0000 7f65ac8a8640 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7b21c0 0x55975ac45100 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f505b61f3fdc9113 server_cookie=45258b33b316e4e9 in_seq=0 out_seq=0 2024-05-17T09:26:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.424+0000 7f6590ff9640 1 -- 172.21.15.19:0/548631251 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55975ab2d360 con 0x55975a7b21c0 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.424+0000 7f6590ff9640 10 client.16893 handle_command_reply: tid=0 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.424+0000 7f65b61c2740 1 client.16893 shutdown 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.424+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 msgr2=0x55975abd3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.424+0000 7f65b61c2740 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 0x55975abd3520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f65a401b040 tx=0x7f65a4005dc0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 shutdown_connections 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55975a7b21c0 0x55975ac45100 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55975a81e1b0 0x55975ac41210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f6590ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55975a7fd4d0 0x55975abd3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 --2- 172.21.15.19:0/548631251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55975a7f52f0 0x55975abd3520 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 >> 172.21.15.19:0/548631251 conn(0x55975a812300 msgr2=0x55975abbe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:26:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:26:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 shutdown_connections 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 1 -- 172.21.15.19:0/548631251 wait complete. 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.425+0000 7f65b61c2740 20 client.16893 trim_cache size 0 max 16384 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65940b28a0 msgr2=0x7f65940b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65940b28a0 0x7f65940b4db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55975abc0570 tx=0x7f65a0009500 comp rx=0 tx=0).stop 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a8106d60 msgr2=0x7f65a819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a8106d60 0x7f65a819a6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6598005760 tx=0x7f6598005e20 comp rx=0 tx=0).stop 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 shutdown_connections 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65940b28a0 0x7f65940b4db0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65a81098d0 0x7f65a81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65a8106d60 0x7f65a819a6d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 --2- 172.21.15.19:0/3228638710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a8101f80 0x7f65a819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 >> 172.21.15.19:0/3228638710 conn(0x7f65a80fd4d0 msgr2=0x7f65a81078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 shutdown_connections 2024-05-17T09:26:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:04.426+0000 7f65af30d640 1 -- 172.21.15.19:0/3228638710 wait complete. 2024-05-17T09:26:04.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 74680 1.40 MB/sec execute 122 sec latency 16.655 ms 2024-05-17T09:26:04.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:04 smithi069 ceph-mon[35081]: pgmap v627: 129 pgs: 129 active+clean; 40 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-17T09:26:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:04 smithi019 ceph-mon[31620]: pgmap v627: 129 pgs: 129 active+clean; 40 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-17T09:26:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:04 smithi079 ceph-mon[35502]: pgmap v627: 129 pgs: 129 active+clean; 40 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-17T09:26:05.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 74958 1.40 MB/sec execute 123 sec latency 13.898 ms 2024-05-17T09:26:06.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 75299 1.39 MB/sec execute 124 sec latency 12.505 ms 2024-05-17T09:26:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:06 smithi069 ceph-mon[35081]: pgmap v628: 129 pgs: 129 active+clean; 42 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:26:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:06 smithi019 ceph-mon[31620]: pgmap v628: 129 pgs: 129 active+clean; 42 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:26:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:06 smithi079 ceph-mon[35502]: pgmap v628: 129 pgs: 129 active+clean; 42 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:26:07.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 75781 1.40 MB/sec execute 125 sec latency 15.615 ms 2024-05-17T09:26:08.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 76390 1.41 MB/sec execute 126 sec latency 17.501 ms 2024-05-17T09:26:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:08 smithi069 ceph-mon[35081]: pgmap v629: 129 pgs: 129 active+clean; 42 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-05-17T09:26:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:08 smithi019 ceph-mon[31620]: pgmap v629: 129 pgs: 129 active+clean; 42 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-05-17T09:26:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:08 smithi079 ceph-mon[35502]: pgmap v629: 129 pgs: 129 active+clean; 42 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-05-17T09:26:09.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 76945 1.41 MB/sec execute 127 sec latency 18.336 ms 2024-05-17T09:26:10.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 77544 1.45 MB/sec execute 128 sec latency 15.537 ms 2024-05-17T09:26:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:10 smithi069 ceph-mon[35081]: pgmap v630: 129 pgs: 129 active+clean; 43 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-17T09:26:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:10 smithi019 ceph-mon[31620]: pgmap v630: 129 pgs: 129 active+clean; 43 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-17T09:26:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:10 smithi079 ceph-mon[35502]: pgmap v630: 129 pgs: 129 active+clean; 43 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-17T09:26:11.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 78077 1.47 MB/sec execute 129 sec latency 15.232 ms 2024-05-17T09:26:12.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 78447 1.47 MB/sec execute 130 sec latency 15.516 ms 2024-05-17T09:26:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:12 smithi069 ceph-mon[35081]: pgmap v631: 129 pgs: 129 active+clean; 42 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-05-17T09:26:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:12 smithi019 ceph-mon[31620]: pgmap v631: 129 pgs: 129 active+clean; 42 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-05-17T09:26:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:12 smithi079 ceph-mon[35502]: pgmap v631: 129 pgs: 129 active+clean; 42 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-05-17T09:26:13.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 78713 1.46 MB/sec execute 131 sec latency 16.501 ms 2024-05-17T09:26:14.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 79208 1.47 MB/sec execute 132 sec latency 16.666 ms 2024-05-17T09:26:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:14 smithi069 ceph-mon[35081]: pgmap v632: 129 pgs: 129 active+clean; 43 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:26:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:14 smithi019 ceph-mon[31620]: pgmap v632: 129 pgs: 129 active+clean; 43 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:26:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:14 smithi079 ceph-mon[35502]: pgmap v632: 129 pgs: 129 active+clean; 43 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:26:15.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 79778 1.48 MB/sec execute 133 sec latency 20.797 ms 2024-05-17T09:26:16.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 80433 1.48 MB/sec execute 134 sec latency 22.345 ms 2024-05-17T09:26:16.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:16 smithi069 ceph-mon[35081]: pgmap v633: 129 pgs: 129 active+clean; 45 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:26:16.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:16 smithi019 ceph-mon[31620]: pgmap v633: 129 pgs: 129 active+clean; 45 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:26:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:16 smithi079 ceph-mon[35502]: pgmap v633: 129 pgs: 129 active+clean; 45 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:26:17.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 81021 1.51 MB/sec execute 135 sec latency 15.879 ms 2024-05-17T09:26:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 -- 172.21.15.19:0/1607622916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 msgr2=0x7f1f78109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1607622916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78109230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009900 tx=0x7f1f6802a410 comp rx=0 tx=0).stop 2024-05-17T09:26:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 -- 172.21.15.19:0/1607622916 shutdown_connections 2024-05-17T09:26:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1607622916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f781098c0 0x7f1f7810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1607622916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78109230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1607622916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f78104340 0x7f1f78106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 -- 172.21.15.19:0/1607622916 >> 172.21.15.19:0/1607622916 conn(0x7f1f780fdd20 msgr2=0x7f1f781001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 -- 172.21.15.19:0/1607622916 shutdown_connections 2024-05-17T09:26:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.539+0000 7f1f802a8640 1 -- 172.21.15.19:0/1607622916 wait complete. 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.540+0000 7f1f802a8640 1 Processor -- start 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.540+0000 7f1f802a8640 1 -- start start 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.540+0000 7f1f802a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f78104340 0x7f1f78103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.540+0000 7f1f802a8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f802a8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f781098c0 0x7f1f78102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f802a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810f630 con 0x7f1f78104340 2024-05-17T09:26:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f802a8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810f4b0 con 0x7f1f781098c0 2024-05-17T09:26:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f802a8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810f7b0 con 0x7f1f78106d50 2024-05-17T09:26:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f7d843640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f78104340 0x7f1f78103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f7d843640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78102200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56702/0 (socket says 172.21.15.19:56702) 2024-05-17T09:26:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f7d843640 1 -- 172.21.15.19:0/1935072161 learned_addr learned my addr 172.21.15.19:0/1935072161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.541+0000 7f1f7e845640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f781098c0 0x7f1f78102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e845640 1 -- 172.21.15.19:0/1935072161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 msgr2=0x7f1f78102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e845640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e845640 1 -- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f78104340 msgr2=0x7f1f78103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e845640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f78104340 0x7f1f78103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e845640 1 -- 172.21.15.19:0/1935072161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f78102e20 con 0x7f1f781098c0 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e044640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f78104340 0x7f1f78103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.542+0000 7f1f7e845640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f781098c0 0x7f1f78102740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009fd0 tx=0x7f1f7000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.543+0000 7f1f6f7fe640 1 -- 172.21.15.19:0/1935072161 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000ce20 con 0x7f1f781098c0 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.543+0000 7f1f6f7fe640 1 -- 172.21.15.19:0/1935072161 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f70005020 con 0x7f1f781098c0 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.543+0000 7f1f6f7fe640 1 -- 172.21.15.19:0/1935072161 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70010690 con 0x7f1f781098c0 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.543+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f78073880 con 0x7f1f781098c0 2024-05-17T09:26:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.543+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f78073d20 con 0x7f1f781098c0 2024-05-17T09:26:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.547+0000 7f1f87239740 1 Processor -- start 2024-05-17T09:26:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.548+0000 7f1f6f7fe640 1 -- 172.21.15.19:0/1935072161 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f70004a40 con 0x7f1f781098c0 2024-05-17T09:26:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 -- start start 2024-05-17T09:26:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d20181b0 0x55c6d23b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6d232bc30 con 0x55c6d20181b0 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c6d231b7d0 con 0x55c6d1ff74d0 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f87239740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c6d231f810 con 0x55c6d1fef2f0 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f7d843640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d20181b0 0x55c6d23b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56718/0 (socket says 172.21.15.19:56718) 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f7e044640 1 -- 172.21.15.19:0/2533440421 learned_addr learned my addr 172.21.15.19:0/2533440421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.549+0000 7f1f7d843640 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7e044640 1 -- 172.21.15.19:0/2533440421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 msgr2=0x55c6d23c0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7e044640 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7e044640 1 -- 172.21.15.19:0/2533440421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d20181b0 msgr2=0x55c6d23b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7e044640 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d20181b0 0x55c6d23b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7e044640 1 -- 172.21.15.19:0/2533440421 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d2030210 con 0x55c6d1fef2f0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7d843640 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d20181b0 0x55c6d23b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f6f7fe640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f640b29a0 0x7f1f640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7d843640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f640b29a0 0x7f1f640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f7e044640 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b9920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f740114f0 tx=0x7f1f740119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82a2773c65fc7ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f6f7fe640 1 -- 172.21.15.19:0/1935072161 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1f700e2d50 con 0x7f1f781098c0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f6cbfe640 1 -- 172.21.15.19:0/2533440421 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7401e070 con 0x55c6d1fef2f0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f6cbfe640 1 -- 172.21.15.19:0/2533440421 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f7400f9f0 con 0x55c6d1fef2f0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.550+0000 7f1f6cbfe640 1 -- 172.21.15.19:0/2533440421 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74019a20 con 0x55c6d1fef2f0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 -- 172.21.15.19:0/2533440421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 msgr2=0x55c6d23b9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f7d843640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f640b29a0 0x7f1f640b4eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005f70 tx=0x7f1f68009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b9920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f740114f0 tx=0x7f1f740119b0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 -- 172.21.15.19:0/2533440421 shutdown_connections 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b9920 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 --2- 172.21.15.19:0/2533440421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d20181b0 0x55c6d23b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 -- 172.21.15.19:0/2533440421 >> 172.21.15.19:0/2533440421 conn(0x55c6d200c300 msgr2=0x55c6d23b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 -- 172.21.15.19:0/2533440421 shutdown_connections 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.551+0000 7f1f87239740 1 -- 172.21.15.19:0/2533440421 wait complete. 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 -- start start 2024-05-17T09:26:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 Processor -- start 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d20181b0 0x55c6d23ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6d2001790 con 0x55c6d1ff74d0 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c6d22d7890 con 0x55c6d20181b0 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c6d23c1270 con 0x55c6d1fef2f0 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f7d843640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f87239740 10 client.0 _ll_register_callbacks cb 0x55c6d22dad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f7e845640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d20181b0 0x55c6d23ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f7d843640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51004/0 (socket says 172.21.15.19:51004) 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f7d843640 1 -- 172.21.15.19:0/654573764 learned_addr learned my addr 172.21.15.19:0/654573764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.552+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56724/0 (socket says 172.21.15.19:56724) 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e044640 1 -- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d20181b0 msgr2=0x55c6d23ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e044640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d20181b0 0x55c6d23ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e044640 1 -- 172.21.15.19:0/654573764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 msgr2=0x55c6d23c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e044640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e044640 1 -- 172.21.15.19:0/654573764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d20fc620 con 0x55c6d1fef2f0 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7d843640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e845640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d20181b0 0x55c6d23ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f7e044640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c2c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1f74000c00 tx=0x7f1f7400fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c290c59252f62a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f61ffb640 1 -- 172.21.15.19:0/654573764 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7401e040 con 0x55c6d1fef2f0 2024-05-17T09:26:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f7400f9f0 con 0x55c6d1fef2f0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f61ffb640 1 -- 172.21.15.19:0/654573764 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1f74004040 con 0x55c6d1fef2f0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f61ffb640 1 -- 172.21.15.19:0/654573764 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f74032050 con 0x55c6d1fef2f0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f87239740 10 client.26524 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.553+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c6d23cd0a0 con 0x55c6d1fef2f0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f61ffb640 1 -- 172.21.15.19:0/654573764 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f7400f9f0 con 0x55c6d1fef2f0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f61ffb640 1 -- 172.21.15.19:0/654573764 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1f74021030 con 0x55c6d1fef2f0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f87239740 10 client.26524 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f87239740 10 client.26524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f87239740 20 client.26524 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f87239740 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d1fd6d80 0x55c6d23d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f87239740 4 client.26524 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d232db80 con 0x55c6d1fd6d80 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.554+0000 7f1f7d843640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d1fd6d80 0x55c6d23d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.555+0000 7f1f61ffb640 10 client.26524 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.555+0000 7f1f7d843640 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d1fd6d80 0x55c6d23d0580 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5e05141bc90d41 server_cookie=9dd8d3c9951003d9 in_seq=0 out_seq=0 2024-05-17T09:26:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f61ffb640 1 -- 172.21.15.19:0/654573764 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c6d232db80 con 0x55c6d1fd6d80 2024-05-17T09:26:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f61ffb640 10 client.26524 handle_command_reply: tid=0 2024-05-17T09:26:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 client.26524 shutdown 2024-05-17T09:26:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 msgr2=0x55c6d23c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c2c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1f74000c00 tx=0x7f1f7400fdd0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 shutdown_connections 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d1fd6d80 0x55c6d23d0580 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d20181b0 0x55c6d23ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f61ffb640 0 client.26524 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1ff74d0 0x55c6d23c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 --2- 172.21.15.19:0/654573764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c2c80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 >> 172.21.15.19:0/654573764 conn(0x55c6d200c300 msgr2=0x55c6d23b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 shutdown_connections 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 1 -- 172.21.15.19:0/654573764 wait complete. 2024-05-17T09:26:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.556+0000 7f1f87239740 20 client.26524 trim_cache size 0 max 16384 2024-05-17T09:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.573+0000 7f1f87239740 1 Processor -- start 2024-05-17T09:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.573+0000 7f1f87239740 1 -- start start 2024-05-17T09:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f87239740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f87239740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f87239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f87239740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781be0d0 con 0x55c6d1fd6d80 2024-05-17T09:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f87239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781bb9a0 con 0x55c6d1ff74d0 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f87239740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781b97f0 con 0x55c6d1fef2f0 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51008/0 (socket says 172.21.15.19:51008) 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 -- 172.21.15.19:0/3298485710 learned_addr learned my addr 172.21.15.19:0/3298485710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7d843640 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e845640 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 -- 172.21.15.19:0/3298485710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 msgr2=0x55c6d23b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 -- 172.21.15.19:0/3298485710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 msgr2=0x55c6d23b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 -- 172.21.15.19:0/3298485710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d23b4340 con 0x55c6d1fd6d80 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7d843640 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.574+0000 7f1f7e044640 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c3280 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1f74019d20 tx=0x7f1f74005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99810a8467b140cd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.575+0000 7f1f63fff640 1 -- 172.21.15.19:0/3298485710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7401e070 con 0x55c6d1fd6d80 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.575+0000 7f1f63fff640 1 -- 172.21.15.19:0/3298485710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f74029070 con 0x55c6d1fd6d80 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.575+0000 7f1f63fff640 1 -- 172.21.15.19:0/3298485710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74003d60 con 0x55c6d1fd6d80 2024-05-17T09:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 -- 172.21.15.19:0/3298485710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 msgr2=0x55c6d23c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c3280 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1f74019d20 tx=0x7f1f74005740 comp rx=0 tx=0).stop 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 -- 172.21.15.19:0/3298485710 shutdown_connections 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1ff74d0 0x55c6d23b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1fef2f0 0x55c6d23b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 --2- 172.21.15.19:0/3298485710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c3280 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 -- 172.21.15.19:0/3298485710 >> 172.21.15.19:0/3298485710 conn(0x55c6d200c300 msgr2=0x55c6d23b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 -- 172.21.15.19:0/3298485710 shutdown_connections 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 -- 172.21.15.19:0/3298485710 wait complete. 2024-05-17T09:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 -- start start 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 Processor -- start 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.576+0000 7f1f87239740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 0x55c6d2436c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781b9e60 con 0x55c6d1fef2f0 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781bb2d0 con 0x55c6d1fd6d80 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781ba530 con 0x55c6d1ff74d0 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 10 client.0 _ll_register_callbacks cb 0x7f1f74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e845640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 0x55c6d2436c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e044640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55688/0 (socket says 172.21.15.19:55688) 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e845640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 0x55c6d2436c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:56738/0 (socket says 172.21.15.19:56738) 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e044640 1 -- 172.21.15.19:0/748622226 learned_addr learned my addr 172.21.15.19:0/748622226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 -- 172.21.15.19:0/748622226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 msgr2=0x55c6d2436c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 0x55c6d2436c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 -- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 msgr2=0x55c6d23c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 -- 172.21.15.19:0/748622226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d23cab30 con 0x55c6d1fef2f0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e044640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7e845640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 0x55c6d2436c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f7d843640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c7b70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55c6d23d0d00 tx=0x7f1f68005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6e6503139a5f68 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f61ffb640 1 -- 172.21.15.19:0/748622226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68059070 con 0x55c6d1fef2f0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f61ffb640 1 -- 172.21.15.19:0/748622226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1f68031900 con 0x55c6d1fef2f0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.577+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f74029070 con 0x55c6d1fef2f0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f61ffb640 1 -- 172.21.15.19:0/748622226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f68062050 con 0x55c6d1fef2f0 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 10 client.16911 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c6d24381c0 con 0x55c6d1fef2f0 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f61ffb640 1 -- 172.21.15.19:0/748622226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f74029070 con 0x55c6d1fef2f0 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f61ffb640 1 -- 172.21.15.19:0/748622226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1f6805c020 con 0x55c6d1fef2f0 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 10 client.16911 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 20 client.16911 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d20181b0 0x55c6d243abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 4 client.16911 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6d2327360 con 0x55c6d20181b0 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.578+0000 7f1f7e044640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d20181b0 0x55c6d243abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.579+0000 7f1f61ffb640 10 client.16911 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.579+0000 7f1f7e044640 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d20181b0 0x55c6d243abb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0947cf346e01f54 server_cookie=dbf174a72280e104 in_seq=0 out_seq=0 2024-05-17T09:26:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f61ffb640 1 -- 172.21.15.19:0/748622226 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c6d2327360 con 0x55c6d20181b0 2024-05-17T09:26:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f61ffb640 10 client.16911 handle_command_reply: tid=0 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 client.16911 shutdown 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fef2f0 msgr2=0x55c6d23c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c7b70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55c6d23d0d00 tx=0x7f1f68005720 comp rx=0 tx=0).stop 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 shutdown_connections 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c6d20181b0 0x55c6d243abb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6d1ff74d0 0x55c6d2436c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f61ffb640 0 client.16911 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6d1fef2f0 0x55c6d23c7b70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 --2- 172.21.15.19:0/748622226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6d1fd6d80 0x55c6d23c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 >> 172.21.15.19:0/748622226 conn(0x55c6d200c300 msgr2=0x55c6d23c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 shutdown_connections 2024-05-17T09:26:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 1 -- 172.21.15.19:0/748622226 wait complete. 2024-05-17T09:26:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.599+0000 7f1f87239740 20 client.16911 trim_cache size 0 max 16384 2024-05-17T09:26:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:26:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:26:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f640b29a0 msgr2=0x7f1f640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f640b29a0 0x7f1f640b4eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005f70 tx=0x7f1f68009560 comp rx=0 tx=0).stop 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f781098c0 msgr2=0x7f1f78102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f781098c0 0x7f1f78102740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009fd0 tx=0x7f1f7000ef10 comp rx=0 tx=0).stop 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 shutdown_connections 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f781098c0 0x7f1f78102740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f78106d50 0x7f1f78102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1f640b29a0 0x7f1f640b4eb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 --2- 172.21.15.19:0/1935072161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f78104340 0x7f1f78103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 >> 172.21.15.19:0/1935072161 conn(0x7f1f780fdd20 msgr2=0x7f1f78104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 shutdown_connections 2024-05-17T09:26:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:17.600+0000 7f1f802a8640 1 -- 172.21.15.19:0/1935072161 wait complete. 2024-05-17T09:26:18.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 81474 1.52 MB/sec execute 136 sec latency 18.857 ms 2024-05-17T09:26:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:18 smithi069 ceph-mon[35081]: pgmap v634: 129 pgs: 129 active+clean; 45 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:26:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:18 smithi019 ceph-mon[31620]: pgmap v634: 129 pgs: 129 active+clean; 45 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:26:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:18 smithi079 ceph-mon[35502]: pgmap v634: 129 pgs: 129 active+clean; 45 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:26:19.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 81940 1.54 MB/sec execute 137 sec latency 13.998 ms 2024-05-17T09:26:20.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 82218 1.53 MB/sec execute 138 sec latency 11.994 ms 2024-05-17T09:26:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:20 smithi069 ceph-mon[35081]: pgmap v635: 129 pgs: 129 active+clean; 46 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-05-17T09:26:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:20 smithi019 ceph-mon[31620]: pgmap v635: 129 pgs: 129 active+clean; 46 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-05-17T09:26:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:20 smithi079 ceph-mon[35502]: pgmap v635: 129 pgs: 129 active+clean; 46 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-05-17T09:26:21.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 82673 1.53 MB/sec execute 139 sec latency 41.120 ms 2024-05-17T09:26:22.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 83229 1.54 MB/sec execute 140 sec latency 15.273 ms 2024-05-17T09:26:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:22 smithi069 ceph-mon[35081]: pgmap v636: 129 pgs: 129 active+clean; 46 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:26:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:22 smithi019 ceph-mon[31620]: pgmap v636: 129 pgs: 129 active+clean; 46 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:26:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:22 smithi079 ceph-mon[35502]: pgmap v636: 129 pgs: 129 active+clean; 46 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:26:23.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 83820 1.54 MB/sec execute 141 sec latency 16.326 ms 2024-05-17T09:26:23.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:23 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:26:23.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:23 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:26:23.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:23 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:26:24.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 84252 1.54 MB/sec execute 142 sec latency 16.231 ms 2024-05-17T09:26:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:24 smithi069 ceph-mon[35081]: pgmap v637: 129 pgs: 129 active+clean; 47 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:26:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:26:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:26:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:26:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:24 smithi019 ceph-mon[31620]: pgmap v637: 129 pgs: 129 active+clean; 47 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:26:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:26:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:26:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:26:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:24 smithi079 ceph-mon[35502]: pgmap v637: 129 pgs: 129 active+clean; 47 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:26:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:26:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:26:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:26:25.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 84952 1.57 MB/sec execute 143 sec latency 15.870 ms 2024-05-17T09:26:26.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 85417 1.59 MB/sec execute 144 sec latency 14.651 ms 2024-05-17T09:26:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:26 smithi069 ceph-mon[35081]: pgmap v638: 129 pgs: 129 active+clean; 45 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:26:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:26 smithi019 ceph-mon[31620]: pgmap v638: 129 pgs: 129 active+clean; 45 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:26:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:26 smithi079 ceph-mon[35502]: pgmap v638: 129 pgs: 129 active+clean; 45 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:26:27.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 85653 1.58 MB/sec execute 145 sec latency 33.428 ms 2024-05-17T09:26:28.431 INFO:tasks.workunit.client.0.smithi019.stdout: 1 86060 1.58 MB/sec execute 146 sec latency 12.392 ms 2024-05-17T09:26:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:28 smithi069 ceph-mon[35081]: pgmap v639: 129 pgs: 129 active+clean; 47 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:26:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:28 smithi019 ceph-mon[31620]: pgmap v639: 129 pgs: 129 active+clean; 47 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:26:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:28 smithi079 ceph-mon[35502]: pgmap v639: 129 pgs: 129 active+clean; 47 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:26:29.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 86586 1.60 MB/sec execute 147 sec latency 18.084 ms 2024-05-17T09:26:30.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 87164 1.59 MB/sec execute 148 sec latency 14.808 ms 2024-05-17T09:26:30.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:30 smithi069 ceph-mon[35081]: pgmap v640: 129 pgs: 129 active+clean; 48 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:26:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:30 smithi019 ceph-mon[31620]: pgmap v640: 129 pgs: 129 active+clean; 48 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 -- 172.21.15.19:0/1844748705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30103d90 msgr2=0x7fde301041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 --2- 172.21.15.19:0/1844748705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30103d90 0x7fde301041a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde20009900 tx=0x7fde2002a410 comp rx=0 tx=0).stop 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 -- 172.21.15.19:0/1844748705 shutdown_connections 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 --2- 172.21.15.19:0/1844748705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 0x7fde3010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 --2- 172.21.15.19:0/1844748705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30103d90 0x7fde301041a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 --2- 172.21.15.19:0/1844748705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30102b50 0x7fde30103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 -- 172.21.15.19:0/1844748705 >> 172.21.15.19:0/1844748705 conn(0x7fde300fdd40 msgr2=0x7fde301001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 -- 172.21.15.19:0/1844748705 shutdown_connections 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.714+0000 7fde37bd5640 1 -- 172.21.15.19:0/1844748705 wait complete. 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 Processor -- start 2024-05-17T09:26:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 -- start start 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 0x7fde3006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30103d90 0x7fde3019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 0x7fde301a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde3010e530 con 0x7fde30102b50 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde3010e3b0 con 0x7fde30103d90 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.715+0000 7fde37bd5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde3010e6b0 con 0x7fde30108a90 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 0x7fde3006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 0x7fde3006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38720/0 (socket says 172.21.15.19:38720) 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 -- 172.21.15.19:0/2216033037 learned_addr learned my addr 172.21.15.19:0/2216033037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35170640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30103d90 0x7fde3019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde36172640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 0x7fde301a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 -- 172.21.15.19:0/2216033037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 msgr2=0x7fde301a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 0x7fde301a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 -- 172.21.15.19:0/2216033037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30103d90 msgr2=0x7fde3019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30103d90 0x7fde3019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 -- 172.21.15.19:0/2216033037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde301a1cf0 con 0x7fde30102b50 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde36172640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 0x7fde301a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35170640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30103d90 0x7fde3019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde35971640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 0x7fde3006bd90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00e9f0 tx=0x7fde2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde26ffd640 1 -- 172.21.15.19:0/2216033037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00ccc0 con 0x7fde30102b50 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde26ffd640 1 -- 172.21.15.19:0/2216033037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2c0044b0 con 0x7fde30102b50 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde301a1fe0 con 0x7fde30102b50 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde26ffd640 1 -- 172.21.15.19:0/2216033037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c010430 con 0x7fde30102b50 2024-05-17T09:26:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.716+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde301a2440 con 0x7fde30102b50 2024-05-17T09:26:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.719+0000 7fde26ffd640 1 -- 172.21.15.19:0/2216033037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fde2c003f90 con 0x7fde30102b50 2024-05-17T09:26:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.719+0000 7fde3eb65740 1 Processor -- start 2024-05-17T09:26:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.719+0000 7fde3eb65740 1 -- start start 2024-05-17T09:26:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed7353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 0x55a8ed735920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde3eb65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8ed6a7c30 con 0x55a8ed36b2f0 2024-05-17T09:26:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde3eb65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8ed6977d0 con 0x55a8ed3734d0 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde3eb65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8ed69b810 con 0x55a8ed3941b0 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 0x55a8ed735920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 0x55a8ed735920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38730/0 (socket says 172.21.15.19:38730) 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55084/0 (socket says 172.21.15.19:55084) 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35971640 1 -- 172.21.15.19:0/2290277166 learned_addr learned my addr 172.21.15.19:0/2290277166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde36172640 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed7353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 -- 172.21.15.19:0/2290277166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 msgr2=0x55a8ed7353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed7353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 -- 172.21.15.19:0/2290277166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 msgr2=0x55a8ed735920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 0x55a8ed735920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35170640 1 -- 172.21.15.19:0/2290277166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ed3ac210 con 0x55a8ed3734d0 2024-05-17T09:26:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde35971640 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 0x55a8ed735920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde36172640 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed7353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.720+0000 7fde26ffd640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde1c0b28a0 0x7fde1c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde36172640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde1c0b28a0 0x7fde1c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde26ffd640 1 -- 172.21.15.19:0/2216033037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde2c0e2990 con 0x7fde30102b50 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde35170640 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73c820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde20005760 tx=0x7fde20005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5633557baf9688dd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde1bfff640 1 -- 172.21.15.19:0/2290277166 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20035030 con 0x55a8ed3734d0 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde36172640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde1c0b28a0 0x7fde1c0b4db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55a8ed736570 tx=0x7fde28009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde1bfff640 1 -- 172.21.15.19:0/2290277166 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde20040070 con 0x55a8ed3734d0 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.721+0000 7fde1bfff640 1 -- 172.21.15.19:0/2290277166 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20003a80 con 0x55a8ed3734d0 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 -- 172.21.15.19:0/2290277166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 msgr2=0x55a8ed73c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73c820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde20005760 tx=0x7fde20005d50 comp rx=0 tx=0).stop 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 -- 172.21.15.19:0/2290277166 shutdown_connections 2024-05-17T09:26:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73c820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed36b2f0 0x55a8ed735920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 --2- 172.21.15.19:0/2290277166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed7353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 -- 172.21.15.19:0/2290277166 >> 172.21.15.19:0/2290277166 conn(0x55a8ed388300 msgr2=0x55a8ed730d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 -- 172.21.15.19:0/2290277166 shutdown_connections 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.722+0000 7fde3eb65740 1 -- 172.21.15.19:0/2290277166 wait complete. 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 -- start start 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 Processor -- start 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8ed37d790 con 0x55a8ed3941b0 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8ed653890 con 0x55a8ed3734d0 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8ed73d210 con 0x55a8ed36b2f0 2024-05-17T09:26:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde3eb65740 10 client.0 _ll_register_callbacks cb 0x55a8ed656d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde36172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde35170640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde35170640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55098/0 (socket says 172.21.15.19:55098) 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde36172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38734/0 (socket says 172.21.15.19:38734) 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.723+0000 7fde35170640 1 -- 172.21.15.19:0/3376827044 learned_addr learned my addr 172.21.15.19:0/3376827044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35971640 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35170640 1 -- 172.21.15.19:0/3376827044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed36b2f0 msgr2=0x55a8ed73ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35170640 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35170640 1 -- 172.21.15.19:0/3376827044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 msgr2=0x55a8ed7460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35170640 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35170640 1 -- 172.21.15.19:0/3376827044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ed478620 con 0x55a8ed3734d0 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde35170640 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73f160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde200057c0 tx=0x7fde20031ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d2afd03ede79552 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde197fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde197fa640 1 -- 172.21.15.19:0/3376827044 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20052060 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20040070 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde197fa640 1 -- 172.21.15.19:0/3376827044 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde2003c650 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde197fa640 1 -- 172.21.15.19:0/3376827044 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde20059050 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde3eb65740 10 client.26567 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.724+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a8ed748ff0 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde197fa640 1 -- 172.21.15.19:0/3376827044 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde20040070 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde197fa640 1 -- 172.21.15.19:0/3376827044 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fde20044770 con 0x55a8ed3734d0 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde3eb65740 10 client.26567 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde3eb65740 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde3eb65740 20 client.26567 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde3eb65740 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed352d80 0x55a8ed74c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde3eb65740 4 client.26567 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:26:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8ed6a9b80 con 0x55a8ed352d80 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde35971640 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed352d80 0x55a8ed74c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.725+0000 7fde197fa640 10 client.26567 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.726+0000 7fde35971640 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed352d80 0x55a8ed74c350 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35c1ae87a6785e0 server_cookie=e78fffea1f98774d in_seq=0 out_seq=0 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde197fa640 1 -- 172.21.15.19:0/3376827044 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8ed6a9b80 con 0x55a8ed352d80 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde197fa640 10 client.26567 handle_command_reply: tid=0 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 client.26567 shutdown 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 msgr2=0x55a8ed73f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73f160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde200057c0 tx=0x7fde20031ca0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 shutdown_connections 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed352d80 0x55a8ed74c350 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed3734d0 0x55a8ed73f160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde197fa640 0 client.26567 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 --2- 172.21.15.19:0/3376827044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 >> 172.21.15.19:0/3376827044 conn(0x55a8ed388300 msgr2=0x55a8ed72fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 shutdown_connections 2024-05-17T09:26:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 1 -- 172.21.15.19:0/3376827044 wait complete. 2024-05-17T09:26:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.727+0000 7fde3eb65740 20 client.26567 trim_cache size 0 max 16384 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 Processor -- start 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 -- start start 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 0x55a8ed745fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed73db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde301af2a0 con 0x55a8ed3734d0 2024-05-17T09:26:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde301acb70 con 0x55a8ed36b2f0 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde3eb65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde301aa9c0 con 0x55a8ed3941b0 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 0x55a8ed745fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 0x55a8ed745fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38750/0 (socket says 172.21.15.19:38750) 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 -- 172.21.15.19:0/1049768368 learned_addr learned my addr 172.21.15.19:0/1049768368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35971640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde36172640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed73db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 -- 172.21.15.19:0/1049768368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 msgr2=0x55a8ed73db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed73db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 -- 172.21.15.19:0/1049768368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 msgr2=0x55a8ed73d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 -- 172.21.15.19:0/1049768368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ed7365a0 con 0x55a8ed3734d0 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde36172640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed73db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:26:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35971640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.745+0000 7fde35170640 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 0x55a8ed745fe0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fde2002a930 tx=0x7fde20005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f972d4b5de10bf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.746+0000 7fde1b7fe640 1 -- 172.21.15.19:0/1049768368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20052040 con 0x55a8ed3734d0 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.746+0000 7fde1b7fe640 1 -- 172.21.15.19:0/1049768368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde20040070 con 0x55a8ed3734d0 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.746+0000 7fde1b7fe640 1 -- 172.21.15.19:0/1049768368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20035040 con 0x55a8ed3734d0 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 -- 172.21.15.19:0/1049768368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 msgr2=0x55a8ed745fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 0x55a8ed745fe0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fde2002a930 tx=0x7fde20005720 comp rx=0 tx=0).stop 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 -- 172.21.15.19:0/1049768368 shutdown_connections 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3941b0 0x55a8ed73db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3734d0 0x55a8ed745fe0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 --2- 172.21.15.19:0/1049768368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed73d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 -- 172.21.15.19:0/1049768368 >> 172.21.15.19:0/1049768368 conn(0x55a8ed388300 msgr2=0x55a8ed72fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 -- 172.21.15.19:0/1049768368 shutdown_connections 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 -- 172.21.15.19:0/1049768368 wait complete. 2024-05-17T09:26:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 -- start start 2024-05-17T09:26:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.747+0000 7fde3eb65740 1 Processor -- start 2024-05-17T09:26:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed743080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3734d0 0x55a8ed7435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde301ab030 con 0x55a8ed3941b0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde301ac4a0 con 0x55a8ed36b2f0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fde301ab700 con 0x55a8ed3734d0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde3eb65740 10 client.0 _ll_register_callbacks cb 0x7fde20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38766/0 (socket says 172.21.15.19:38766) 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 -- 172.21.15.19:0/905343821 learned_addr learned my addr 172.21.15.19:0/905343821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde35971640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed743080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde35170640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3734d0 0x55a8ed7435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 -- 172.21.15.19:0/905343821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3734d0 msgr2=0x55a8ed7435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3734d0 0x55a8ed7435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 -- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 msgr2=0x55a8ed743080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed743080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 -- 172.21.15.19:0/905343821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ed7b2ec0 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde35971640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed743080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde35170640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3734d0 0x55a8ed7435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde36172640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7b2980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fde28010ce0 tx=0x7fde28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5efd93ad3ab98 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.748+0000 7fde197fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde197fa640 1 -- 172.21.15.19:0/905343821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2802d060 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde197fa640 1 -- 172.21.15.19:0/905343821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fde2801e9a0 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20040070 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde197fa640 1 -- 172.21.15.19:0/905343821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde28034050 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 10 client.16929 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a8ed7b4120 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde197fa640 1 -- 172.21.15.19:0/905343821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde20040070 con 0x55a8ed3941b0 2024-05-17T09:26:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde197fa640 1 -- 172.21.15.19:0/905343821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fde28028c50 con 0x55a8ed3941b0 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 10 client.16929 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 20 client.16929 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed3281c0 0x55a8ed7b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 4 client.16929 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8ed6a3360 con 0x55a8ed3281c0 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.749+0000 7fde35971640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed3281c0 0x55a8ed7b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.750+0000 7fde197fa640 10 client.16929 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.750+0000 7fde35971640 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed3281c0 0x55a8ed7b6b70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a388b8b03c0d3ba server_cookie=fc31a482cd8c6bb1 in_seq=0 out_seq=0 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde197fa640 1 -- 172.21.15.19:0/905343821 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a8ed6a3360 con 0x55a8ed3281c0 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde197fa640 10 client.16929 handle_command_reply: tid=0 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 client.16929 shutdown 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 msgr2=0x55a8ed7b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7b2980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fde28010ce0 tx=0x7fde28004a40 comp rx=0 tx=0).stop 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 shutdown_connections 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a8ed3281c0 0x55a8ed7b6b70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8ed3941b0 0x55a8ed7b2980 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8ed3734d0 0x55a8ed7435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 --2- 172.21.15.19:0/905343821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ed36b2f0 0x55a8ed743080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde197fa640 0 client.16929 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.769+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 >> 172.21.15.19:0/905343821 conn(0x55a8ed388300 msgr2=0x55a8ed72fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 shutdown_connections 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde3eb65740 1 -- 172.21.15.19:0/905343821 wait complete. 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde3eb65740 20 client.16929 trim_cache size 0 max 16384 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:26:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde1c0b28a0 msgr2=0x7fde1c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde1c0b28a0 0x7fde1c0b4db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55a8ed736570 tx=0x7fde28009500 comp rx=0 tx=0).stop 2024-05-17T09:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 msgr2=0x7fde3006bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 0x7fde3006bd90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00e9f0 tx=0x7fde2c00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 shutdown_connections 2024-05-17T09:26:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde30108a90 0x7fde301a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde30103d90 0x7fde3019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fde1c0b28a0 0x7fde1c0b4db0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 --2- 172.21.15.19:0/2216033037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde30102b50 0x7fde3006bd90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.770+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 >> 172.21.15.19:0/2216033037 conn(0x7fde300fdd40 msgr2=0x7fde30104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.771+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 shutdown_connections 2024-05-17T09:26:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:30.771+0000 7fde37bd5640 1 -- 172.21.15.19:0/2216033037 wait complete. 2024-05-17T09:26:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:30 smithi079 ceph-mon[35502]: pgmap v640: 129 pgs: 129 active+clean; 48 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-17T09:26:31.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 87664 1.59 MB/sec execute 149 sec latency 16.600 ms 2024-05-17T09:26:32.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 88384 1.62 MB/sec execute 150 sec latency 18.592 ms 2024-05-17T09:26:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:32 smithi069 ceph-mon[35081]: pgmap v641: 129 pgs: 129 active+clean; 48 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-17T09:26:32.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:32 smithi019 ceph-mon[31620]: pgmap v641: 129 pgs: 129 active+clean; 48 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-17T09:26:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:32 smithi079 ceph-mon[35502]: pgmap v641: 129 pgs: 129 active+clean; 48 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-05-17T09:26:33.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 88809 1.64 MB/sec execute 151 sec latency 15.911 ms 2024-05-17T09:26:34.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 89118 1.64 MB/sec execute 152 sec latency 13.053 ms 2024-05-17T09:26:34.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:34 smithi069 ceph-mon[35081]: pgmap v642: 129 pgs: 129 active+clean; 48 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-05-17T09:26:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:34 smithi019 ceph-mon[31620]: pgmap v642: 129 pgs: 129 active+clean; 48 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-05-17T09:26:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:34 smithi079 ceph-mon[35502]: pgmap v642: 129 pgs: 129 active+clean; 48 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-05-17T09:26:35.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 89447 1.63 MB/sec execute 153 sec latency 11.942 ms 2024-05-17T09:26:36.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 89937 1.63 MB/sec execute 154 sec latency 12.990 ms 2024-05-17T09:26:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:36 smithi069 ceph-mon[35081]: pgmap v643: 129 pgs: 129 active+clean; 48 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:26:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:36 smithi019 ceph-mon[31620]: pgmap v643: 129 pgs: 129 active+clean; 48 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:26:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:36 smithi079 ceph-mon[35502]: pgmap v643: 129 pgs: 129 active+clean; 48 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:26:37.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 90538 1.64 MB/sec execute 155 sec latency 15.902 ms 2024-05-17T09:26:38.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91098 1.64 MB/sec execute 156 sec latency 24.705 ms 2024-05-17T09:26:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:38 smithi069 ceph-mon[35081]: pgmap v644: 129 pgs: 129 active+clean; 50 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-17T09:26:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:38 smithi019 ceph-mon[31620]: pgmap v644: 129 pgs: 129 active+clean; 50 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-17T09:26:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:38 smithi079 ceph-mon[35502]: pgmap v644: 129 pgs: 129 active+clean; 50 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-17T09:26:39.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91712 1.67 MB/sec execute 157 sec latency 15.622 ms 2024-05-17T09:26:40.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 92249 1.69 MB/sec execute 158 sec latency 19.006 ms 2024-05-17T09:26:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:40 smithi069 ceph-mon[35081]: pgmap v645: 129 pgs: 129 active+clean; 51 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:26:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:40 smithi019 ceph-mon[31620]: pgmap v645: 129 pgs: 129 active+clean; 51 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:26:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:40 smithi079 ceph-mon[35502]: pgmap v645: 129 pgs: 129 active+clean; 51 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:26:41.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 92614 1.69 MB/sec execute 159 sec latency 16.374 ms 2024-05-17T09:26:42.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 92873 1.68 MB/sec execute 160 sec latency 15.380 ms 2024-05-17T09:26:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:42 smithi069 ceph-mon[35081]: pgmap v646: 129 pgs: 129 active+clean; 50 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-17T09:26:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:42 smithi019 ceph-mon[31620]: pgmap v646: 129 pgs: 129 active+clean; 50 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-17T09:26:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:42 smithi079 ceph-mon[35502]: pgmap v646: 129 pgs: 129 active+clean; 50 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-17T09:26:43.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 93329 1.68 MB/sec execute 161 sec latency 15.892 ms 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 -- 172.21.15.19:0/4007301339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c103500 msgr2=0x7f1e8c103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 --2- 172.21.15.19:0/4007301339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c103500 0x7f1e8c103910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400aa10 tx=0x7f1e7402b720 comp rx=0 tx=0).stop 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 -- 172.21.15.19:0/4007301339 shutdown_connections 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 --2- 172.21.15.19:0/4007301339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c108200 0x7f1e8c10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 --2- 172.21.15.19:0/4007301339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c103500 0x7f1e8c103910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 --2- 172.21.15.19:0/4007301339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 -- 172.21.15.19:0/4007301339 >> 172.21.15.19:0/4007301339 conn(0x7f1e8c0fd4d0 msgr2=0x7f1e8c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 -- 172.21.15.19:0/4007301339 shutdown_connections 2024-05-17T09:26:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.879+0000 7f1e91f34640 1 -- 172.21.15.19:0/4007301339 wait complete. 2024-05-17T09:26:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 Processor -- start 2024-05-17T09:26:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 -- start start 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c103500 0x7f1e8c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e8c108200 0x7f1e8c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10d400 con 0x7f1e8c108200 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10d280 con 0x7f1e8c1022c0 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.880+0000 7f1e91f34640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10d580 con 0x7f1e8c103500 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33322/0 (socket says 172.21.15.19:33322) 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e8c108200 0x7f1e8c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/639945664 learned_addr learned my addr 172.21.15.19:0/639945664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e83fff640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c103500 0x7f1e8c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/639945664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c103500 msgr2=0x7f1e8c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c103500 0x7f1e8c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e8c108200 msgr2=0x7f1e8c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e8c108200 0x7f1e8c1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/639945664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c1a1d00 con 0x7f1e8c1022c0 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e83fff640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c103500 0x7f1e8c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c19a1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e980 tx=0x7f1e7c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e897fa640 1 -- 172.21.15.19:0/639945664 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00cd80 con 0x7f1e8c1022c0 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e897fa640 1 -- 172.21.15.19:0/639945664 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e7c0108c0 con 0x7f1e8c1022c0 2024-05-17T09:26:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c1a1ff0 con 0x7f1e8c1022c0 2024-05-17T09:26:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e897fa640 1 -- 172.21.15.19:0/639945664 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c018840 con 0x7f1e8c1022c0 2024-05-17T09:26:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.881+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1a24b0 con 0x7f1e8c1022c0 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 Processor -- start 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 -- start start 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0334780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c0334d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0331430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e8c02a3c30 con 0x55e8bff672f0 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e8c02937d0 con 0x55e8bff901b0 2024-05-17T09:26:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e98de2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0297810 con 0x55e8bff6f4d0 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e897fa640 1 -- 172.21.15.19:0/639945664 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e7c01f020 con 0x7f1e8c1022c0 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e83fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0331430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e83fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0331430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:48924/0 (socket says 172.21.15.19:48924) 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.885+0000 7f1e83fff640 1 -- 172.21.15.19:0/4169747485 learned_addr learned my addr 172.21.15.19:0/4169747485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c0334d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e8bfff640 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0334780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e83fff640 1 -- 172.21.15.19:0/4169747485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 msgr2=0x55e8c0334780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e83fff640 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0334780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e83fff640 1 -- 172.21.15.19:0/4169747485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 msgr2=0x55e8c0334d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e83fff640 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c0334d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e83fff640 1 -- 172.21.15.19:0/4169747485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bffa8210 con 0x55e8bff6f4d0 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.886+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c0334d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e83fff640 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0331430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400aa10 tx=0x7f1e74005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11108ecf67ddc9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e81ffb640 1 -- 172.21.15.19:0/4169747485 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74036040 con 0x55e8bff6f4d0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e81ffb640 1 -- 172.21.15.19:0/4169747485 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e740044c0 con 0x55e8bff6f4d0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e81ffb640 1 -- 172.21.15.19:0/4169747485 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74003d60 con 0x55e8bff6f4d0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e897fa640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1e780b2920 0x7f1e780b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e8bfff640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1e780b2920 0x7f1e780b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.887+0000 7f1e897fa640 1 -- 172.21.15.19:0/639945664 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1e7c0e2610 con 0x7f1e8c1022c0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 -- 172.21.15.19:0/4169747485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 msgr2=0x55e8c0331430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e8bfff640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1e780b2920 0x7f1e780b4e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e8c03324f0 tx=0x7f1e84009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0331430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400aa10 tx=0x7f1e74005de0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 -- 172.21.15.19:0/4169747485 shutdown_connections 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0331430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c0334d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 --2- 172.21.15.19:0/4169747485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0334780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 -- 172.21.15.19:0/4169747485 >> 172.21.15.19:0/4169747485 conn(0x55e8bff84300 msgr2=0x55e8c032cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 -- 172.21.15.19:0/4169747485 shutdown_connections 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.888+0000 7f1e98de2740 1 -- 172.21.15.19:0/4169747485 wait complete. 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 -- start start 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 Processor -- start 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 0x55e8c0343460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c03439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff901b0 0x55e8c034a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e8bff79790 con 0x55e8bff901b0 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e8c024f890 con 0x55e8bff672f0 2024-05-17T09:26:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e8c0331ed0 con 0x55e8bff6f4d0 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.889+0000 7f1e98de2740 10 client.0 _ll_register_callbacks cb 0x55e8c0252d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 0x55e8c0343460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff901b0 0x55e8c034a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 0x55e8c0343460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33344/0 (socket says 172.21.15.19:33344) 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/3930994389 learned_addr learned my addr 172.21.15.19:0/3930994389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff901b0 0x55e8c034a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57372/0 (socket says 172.21.15.19:57372) 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e83fff640 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c03439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/3930994389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 msgr2=0x55e8c03439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c03439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/3930994389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff901b0 msgr2=0x55e8c034a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff901b0 0x55e8c034a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/3930994389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c0074620 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 0x55e8c0343460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0a6af0 tx=0x7f1e7c0095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b865353317a3bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e6affd640 1 -- 172.21.15.19:0/3930994389 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0a91d0 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e8c02a5b80 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.890+0000 7f1e6affd640 1 -- 172.21.15.19:0/3930994389 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1e7c010a70 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e6affd640 1 -- 172.21.15.19:0/3930994389 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c0a8ff0 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 10 client.26585 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8c034d8a0 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e6affd640 1 -- 172.21.15.19:0/3930994389 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e8c02a5b80 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e6affd640 1 -- 172.21.15.19:0/3930994389 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1e7c0b2250 con 0x55e8bff672f0 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 10 client.26585 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 10 client.26585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 20 client.26585 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff4ed80 0x55e8c0350f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 4 client.26585 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8c029f360 con 0x55e8bff4ed80 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.891+0000 7f1e83fff640 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff4ed80 0x55e8c0350f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.892+0000 7f1e6affd640 10 client.26585 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.892+0000 7f1e83fff640 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff4ed80 0x55e8c0350f90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619544c597589dee server_cookie=acc2cc9e9f06ffb7 in_seq=0 out_seq=0 2024-05-17T09:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e6affd640 1 -- 172.21.15.19:0/3930994389 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e8c029f360 con 0x55e8bff4ed80 2024-05-17T09:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e6affd640 10 client.26585 handle_command_reply: tid=0 2024-05-17T09:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 client.26585 shutdown 2024-05-17T09:26:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 msgr2=0x55e8c0343460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 0x55e8c0343460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0a6af0 tx=0x7f1e7c0095b0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 shutdown_connections 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff4ed80 0x55e8c0350f90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff901b0 0x55e8c034a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c03439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 --2- 172.21.15.19:0/3930994389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff672f0 0x55e8c0343460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e6affd640 0 client.26585 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 >> 172.21.15.19:0/3930994389 conn(0x55e8bff84300 msgr2=0x55e8c032bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 shutdown_connections 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 1 -- 172.21.15.19:0/3930994389 wait complete. 2024-05-17T09:26:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.893+0000 7f1e98de2740 20 client.26585 trim_cache size 0 max 16384 2024-05-17T09:26:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 Processor -- start 2024-05-17T09:26:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 -- start start 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff672f0 0x55e8c033e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 0x55e8c034a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c032b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ad130 con 0x55e8bff6f4d0 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1aaa00 con 0x55e8bff901b0 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.910+0000 7f1e98de2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1a8850 con 0x55e8bff672f0 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 0x55e8c034a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 0x55e8c034a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57386/0 (socket says 172.21.15.19:57386) 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c032b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 -- 172.21.15.19:0/1626778188 learned_addr learned my addr 172.21.15.19:0/1626778188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff672f0 0x55e8c033e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 -- 172.21.15.19:0/1626778188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff672f0 msgr2=0x55e8c033e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff672f0 0x55e8c033e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 -- 172.21.15.19:0/1626778188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 msgr2=0x55e8c032b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c032b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 -- 172.21.15.19:0/1626778188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c0335070 con 0x55e8bff6f4d0 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff672f0 0x55e8c033e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e83fff640 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 0x55e8c034a890 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55e8c032dac0 tx=0x7f1e7402b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caac2d4bb0f61ab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e817fa640 1 -- 172.21.15.19:0/1626778188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74057070 con 0x55e8bff6f4d0 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e817fa640 1 -- 172.21.15.19:0/1626778188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e74030910 con 0x55e8bff6f4d0 2024-05-17T09:26:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.911+0000 7f1e817fa640 1 -- 172.21.15.19:0/1626778188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74036040 con 0x55e8bff6f4d0 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 -- 172.21.15.19:0/1626778188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 msgr2=0x55e8c034a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 0x55e8c034a890 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55e8c032dac0 tx=0x7f1e7402b720 comp rx=0 tx=0).stop 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 -- 172.21.15.19:0/1626778188 shutdown_connections 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c032b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff6f4d0 0x55e8c034a890 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 --2- 172.21.15.19:0/1626778188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff672f0 0x55e8c033e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 -- 172.21.15.19:0/1626778188 >> 172.21.15.19:0/1626778188 conn(0x55e8bff84300 msgr2=0x55e8c033dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 -- 172.21.15.19:0/1626778188 shutdown_connections 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.912+0000 7f1e98de2740 1 -- 172.21.15.19:0/1626778188 wait complete. 2024-05-17T09:26:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 -- start start 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 Processor -- start 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c03479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0347f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0348e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1a8ec0 con 0x55e8bff672f0 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1aa330 con 0x55e8bff901b0 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1a9590 con 0x55e8bff6f4d0 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e98de2740 10 client.0 _ll_register_callbacks cb 0x7f1e7c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e8b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c03479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e8b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c03479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57396/0 (socket says 172.21.15.19:57396) 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/80745350 learned_addr learned my addr 172.21.15.19:0/80745350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e8bfff640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0348e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.913+0000 7f1e83fff640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0347f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/80745350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 msgr2=0x55e8c0347f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0347f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 msgr2=0x55e8c0348e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0348e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e8b7fe640 1 -- 172.21.15.19:0/80745350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c0349410 con 0x55e8bff672f0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e83fff640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0347f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e8b7fe640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c03479e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009230 tx=0x7f1e7c0e2540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255c040d62ac7c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e6affd640 1 -- 172.21.15.19:0/80745350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0a91d0 con 0x55e8bff672f0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e6affd640 1 -- 172.21.15.19:0/80745350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1e7c0a5e70 con 0x55e8bff672f0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e74030910 con 0x55e8bff672f0 2024-05-17T09:26:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e6affd640 1 -- 172.21.15.19:0/80745350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c0c5050 con 0x55e8bff672f0 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e98de2740 10 client.16947 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8c034a710 con 0x55e8bff672f0 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.914+0000 7f1e6affd640 1 -- 172.21.15.19:0/80745350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e74030910 con 0x55e8bff672f0 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e6affd640 1 -- 172.21.15.19:0/80745350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1e7c0cab30 con 0x55e8bff672f0 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e98de2740 10 client.16947 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e98de2740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e98de2740 20 client.16947 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e98de2740 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff241c0 0x55e8c03bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e98de2740 4 client.16947 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8c02195f0 con 0x55e8bff241c0 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e83fff640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff241c0 0x55e8c03bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e6affd640 10 client.16947 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.915+0000 7f1e83fff640 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff241c0 0x55e8c03bb990 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6813f50ecc2cf4b1 server_cookie=4d502501dc1c0e7b in_seq=0 out_seq=0 2024-05-17T09:26:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e6affd640 1 -- 172.21.15.19:0/80745350 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e8c02195f0 con 0x55e8bff241c0 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e6affd640 10 client.16947 handle_command_reply: tid=0 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 client.16947 shutdown 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 msgr2=0x55e8c03479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c03479e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009230 tx=0x7f1e7c0e2540 comp rx=0 tx=0).stop 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 shutdown_connections 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e8bff241c0 0x55e8c03bb990 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8bff901b0 0x55e8c0348e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8bff6f4d0 0x55e8c0347f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 --2- 172.21.15.19:0/80745350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8bff672f0 0x55e8c03479e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e6affd640 0 client.16947 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 >> 172.21.15.19:0/80745350 conn(0x55e8bff84300 msgr2=0x55e8c03437c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 shutdown_connections 2024-05-17T09:26:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 1 -- 172.21.15.19:0/80745350 wait complete. 2024-05-17T09:26:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.935+0000 7f1e98de2740 20 client.16947 trim_cache size 0 max 16384 2024-05-17T09:26:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:26:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:26:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1e780b2920 msgr2=0x7f1e780b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1e780b2920 0x7f1e780b4e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e8c03324f0 tx=0x7f1e84009500 comp rx=0 tx=0).stop 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 msgr2=0x7f1e8c19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c19a1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e980 tx=0x7f1e7c00ee40 comp rx=0 tx=0).stop 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 shutdown_connections 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1e780b2920 0x7f1e780b4e30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e8c108200 0x7f1e8c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e8c103500 0x7f1e8c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 --2- 172.21.15.19:0/639945664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e8c1022c0 0x7f1e8c19a1c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 >> 172.21.15.19:0/639945664 conn(0x7f1e8c0fd4d0 msgr2=0x7f1e8c1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 shutdown_connections 2024-05-17T09:26:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:43.936+0000 7f1e91f34640 1 -- 172.21.15.19:0/639945664 wait complete. 2024-05-17T09:26:44.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 93885 1.69 MB/sec execute 162 sec latency 23.297 ms 2024-05-17T09:26:44.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:44 smithi069 ceph-mon[35081]: pgmap v647: 129 pgs: 129 active+clean; 51 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:26:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:44 smithi019 ceph-mon[31620]: pgmap v647: 129 pgs: 129 active+clean; 51 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:26:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:44 smithi079 ceph-mon[35502]: pgmap v647: 129 pgs: 129 active+clean; 51 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:26:45.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 94565 1.69 MB/sec execute 163 sec latency 15.975 ms 2024-05-17T09:26:46.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 95147 1.70 MB/sec execute 164 sec latency 12.084 ms 2024-05-17T09:26:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:46 smithi019 ceph-mon[31620]: pgmap v648: 129 pgs: 129 active+clean; 52 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:26:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:46 smithi079 ceph-mon[35502]: pgmap v648: 129 pgs: 129 active+clean; 52 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:26:46.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:46 smithi069 ceph-mon[35081]: pgmap v648: 129 pgs: 129 active+clean; 52 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:26:47.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 95620 1.71 MB/sec execute 165 sec latency 15.013 ms 2024-05-17T09:26:48.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 96101 1.73 MB/sec execute 166 sec latency 15.884 ms 2024-05-17T09:26:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:48 smithi019 ceph-mon[31620]: pgmap v649: 129 pgs: 129 active+clean; 51 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:26:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:48 smithi079 ceph-mon[35502]: pgmap v649: 129 pgs: 129 active+clean; 51 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:26:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:48 smithi069 ceph-mon[35081]: pgmap v649: 129 pgs: 129 active+clean; 51 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:26:49.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 96372 1.72 MB/sec execute 167 sec latency 16.142 ms 2024-05-17T09:26:50.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 96820 1.72 MB/sec execute 168 sec latency 12.228 ms 2024-05-17T09:26:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:50 smithi019 ceph-mon[31620]: pgmap v650: 129 pgs: 129 active+clean; 48 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:26:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:50 smithi079 ceph-mon[35502]: pgmap v650: 129 pgs: 129 active+clean; 48 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:26:50.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:50 smithi069 ceph-mon[35081]: pgmap v650: 129 pgs: 129 active+clean; 48 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:26:51.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 97377 1.73 MB/sec execute 169 sec latency 15.679 ms 2024-05-17T09:26:52.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 97999 1.72 MB/sec execute 170 sec latency 16.368 ms 2024-05-17T09:26:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:52 smithi019 ceph-mon[31620]: pgmap v651: 129 pgs: 129 active+clean; 49 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-17T09:26:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:52 smithi079 ceph-mon[35502]: pgmap v651: 129 pgs: 129 active+clean; 49 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-17T09:26:52.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:52 smithi069 ceph-mon[35081]: pgmap v651: 129 pgs: 129 active+clean; 49 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-17T09:26:53.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 98570 1.74 MB/sec execute 171 sec latency 13.282 ms 2024-05-17T09:26:54.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 99146 1.75 MB/sec execute 172 sec latency 12.769 ms 2024-05-17T09:26:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:54 smithi019 ceph-mon[31620]: pgmap v652: 129 pgs: 129 active+clean; 50 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-17T09:26:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:54 smithi079 ceph-mon[35502]: pgmap v652: 129 pgs: 129 active+clean; 50 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-17T09:26:54.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:54 smithi069 ceph-mon[35081]: pgmap v652: 129 pgs: 129 active+clean; 50 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-17T09:26:55.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 99624 1.77 MB/sec execute 173 sec latency 12.984 ms 2024-05-17T09:26:56.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 99888 1.76 MB/sec execute 174 sec latency 12.985 ms 2024-05-17T09:26:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:56 smithi079 ceph-mon[35502]: pgmap v653: 129 pgs: 129 active+clean; 50 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:26:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:56 smithi069 ceph-mon[35081]: pgmap v653: 129 pgs: 129 active+clean; 50 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:26:56.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:56 smithi019 ceph-mon[31620]: pgmap v653: 129 pgs: 129 active+clean; 50 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 -- 172.21.15.19:0/1682574793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 msgr2=0x7f8894106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 --2- 172.21.15.19:0/1682574793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f8894106790 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8890009900 tx=0x7f889002a410 comp rx=0 tx=0).stop 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 -- 172.21.15.19:0/1682574793 shutdown_connections 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 --2- 172.21.15.19:0/1682574793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88941098d0 0x7f889410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 --2- 172.21.15.19:0/1682574793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8894106d60 0x7f8894109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 --2- 172.21.15.19:0/1682574793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f8894106790 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 -- 172.21.15.19:0/1682574793 >> 172.21.15.19:0/1682574793 conn(0x7f88940fdd30 msgr2=0x7f88941001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 -- 172.21.15.19:0/1682574793 shutdown_connections 2024-05-17T09:26:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.044+0000 7f889b2e0640 1 -- 172.21.15.19:0/1682574793 wait complete. 2024-05-17T09:26:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 Processor -- start 2024-05-17T09:26:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 -- start start 2024-05-17T09:26:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f889419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8894106d60 0x7f889419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88941098d0 0x7f88941a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f889410f5c0 con 0x7f8894104370 2024-05-17T09:26:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f889410f440 con 0x7f8894106d60 2024-05-17T09:26:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889b2e0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f889410f740 con 0x7f88941098d0 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889907c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f889419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.045+0000 7f889907c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f889419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42512/0 (socket says 172.21.15.19:42512) 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889907c640 1 -- 172.21.15.19:0/55733898 learned_addr learned my addr 172.21.15.19:0/55733898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8894106d60 0x7f889419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889987d640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88941098d0 0x7f88941a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 -- 172.21.15.19:0/55733898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88941098d0 msgr2=0x7f88941a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88941098d0 0x7f88941a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 -- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 msgr2=0x7f889419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f889419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 -- 172.21.15.19:0/55733898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88941a1d30 con 0x7f8894106d60 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889907c640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f889419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889887b640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8894106d60 0x7f889419a730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9f0 tx=0x7f888400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f888a7fc640 1 -- 172.21.15.19:0/55733898 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888400ccc0 con 0x7f8894106d60 2024-05-17T09:26:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f888a7fc640 1 -- 172.21.15.19:0/55733898 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88840108c0 con 0x7f8894106d60 2024-05-17T09:26:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88941a1fc0 con 0x7f8894106d60 2024-05-17T09:26:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f888a7fc640 1 -- 172.21.15.19:0/55733898 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840187e0 con 0x7f8894106d60 2024-05-17T09:26:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.046+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88941a24b0 con 0x7f8894106d60 2024-05-17T09:26:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 Processor -- start 2024-05-17T09:26:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 -- start start 2024-05-17T09:26:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f888a7fc640 1 -- 172.21.15.19:0/55733898 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8884010430 con 0x7f8894106d60 2024-05-17T09:26:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfb3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f4bf1ec30 con 0x558f4bbe22f0 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f4bf0e7d0 con 0x558f4bbea4d0 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.049+0000 7f88a218b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558f4bf12810 con 0x558f4bc0b1b0 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889907c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889887b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfb3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:59762/0 (socket says 172.21.15.19:59762) 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 -- 172.21.15.19:0/226685671 learned_addr learned my addr 172.21.15.19:0/226685671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 -- 172.21.15.19:0/226685671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 msgr2=0x558f4bfb3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfb3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 -- 172.21.15.19:0/226685671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 msgr2=0x558f4bfac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 -- 172.21.15.19:0/226685671 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f4bc23210 con 0x558f4bc0b1b0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889887b640 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfb3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f889987d640 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfac3a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f888c009fd0 tx=0x7f888c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8bbef8772016f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f887f7fe640 1 -- 172.21.15.19:0/226685671 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c011700 con 0x558f4bc0b1b0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f888a7fc640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88800b28a0 0x7f88800b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f887f7fe640 1 -- 172.21.15.19:0/226685671 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f888c011d30 con 0x558f4bc0b1b0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.050+0000 7f887f7fe640 1 -- 172.21.15.19:0/226685671 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c01cd80 con 0x558f4bc0b1b0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f889907c640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88800b28a0 0x7f88800b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f888a7fc640 1 -- 172.21.15.19:0/55733898 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f88840e2960 con 0x7f8894106d60 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 -- 172.21.15.19:0/226685671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 msgr2=0x558f4bfac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f889907c640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88800b28a0 0x7f88800b4db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8890000c00 tx=0x7f88900046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfac3a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f888c009fd0 tx=0x7f888c010860 comp rx=0 tx=0).stop 2024-05-17T09:26:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 -- 172.21.15.19:0/226685671 shutdown_connections 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfb3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 --2- 172.21.15.19:0/226685671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfac3a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 -- 172.21.15.19:0/226685671 >> 172.21.15.19:0/226685671 conn(0x558f4bbff300 msgr2=0x558f4bfa7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 -- 172.21.15.19:0/226685671 shutdown_connections 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.051+0000 7f88a218b740 1 -- 172.21.15.19:0/226685671 wait complete. 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 -- start start 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 Processor -- start 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfb5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bbea4d0 0x558f4bfb6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 0x558f4bfbd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f4bbf4790 con 0x558f4bc0b1b0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558f4beca890 con 0x558f4bbe22f0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558f4bfb4270 con 0x558f4bbea4d0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.052+0000 7f88a218b740 10 client.0 _ll_register_callbacks cb 0x558f4becdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889987d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 0x558f4bfbd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bbea4d0 0x558f4bfb6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889907c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfb5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889987d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 0x558f4bfbd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42540/0 (socket says 172.21.15.19:42540) 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889987d640 1 -- 172.21.15.19:0/3591675853 learned_addr learned my addr 172.21.15.19:0/3591675853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 -- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 msgr2=0x558f4bfb5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfb5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 -- 172.21.15.19:0/3591675853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 msgr2=0x558f4bfbd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 0x558f4bfbd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 -- 172.21.15.19:0/3591675853 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f4bcef620 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889987d640 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 0x558f4bfbd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f889887b640 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bbea4d0 0x558f4bfb6250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88840e0100 tx=0x7f88840a9a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1faa18df321c4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f887cff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f887cff9640 1 -- 172.21.15.19:0/3591675853 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884010720 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f887cff9640 1 -- 172.21.15.19:0/3591675853 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f88840a8e30 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f888c011d30 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f887cff9640 1 -- 172.21.15.19:0/3591675853 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88840a5360 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f88a218b740 10 client.26578 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.053+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558f4bfc0150 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f887cff9640 1 -- 172.21.15.19:0/3591675853 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f888c011d30 con 0x558f4bbea4d0 2024-05-17T09:26:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f887cff9640 1 -- 172.21.15.19:0/3591675853 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f88840a5040 con 0x558f4bbea4d0 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f88a218b740 10 client.26578 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f88a218b740 10 client.26578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f88a218b740 20 client.26578 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f88a218b740 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bbc9d80 0x558f4bfc3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f88a218b740 4 client.26578 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f4bf20b80 con 0x558f4bbc9d80 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f889907c640 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bbc9d80 0x558f4bfc3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f887cff9640 10 client.26578 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.054+0000 7f889907c640 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bbc9d80 0x558f4bfc3830 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249ec3eed428574d server_cookie=d2151d1c880367b8 in_seq=0 out_seq=0 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f887cff9640 1 -- 172.21.15.19:0/3591675853 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558f4bf20b80 con 0x558f4bbc9d80 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f887cff9640 10 client.26578 handle_command_reply: tid=0 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 client.26578 shutdown 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bbea4d0 msgr2=0x558f4bfb6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bbea4d0 0x558f4bfb6250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88840e0100 tx=0x7f88840a9a60 comp rx=0 tx=0).stop 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 shutdown_connections 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bbc9d80 0x558f4bfc3830 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bc0b1b0 0x558f4bfbd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f887cff9640 0 client.26578 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bbea4d0 0x558f4bfb6250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 --2- 172.21.15.19:0/3591675853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfb5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 >> 172.21.15.19:0/3591675853 conn(0x558f4bbff300 msgr2=0x558f4bfaac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 shutdown_connections 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 1 -- 172.21.15.19:0/3591675853 wait complete. 2024-05-17T09:26:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.056+0000 7f88a218b740 20 client.26578 trim_cache size 0 max 16384 2024-05-17T09:26:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.073+0000 7f88a218b740 1 Processor -- start 2024-05-17T09:26:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.073+0000 7f88a218b740 1 -- start start 2024-05-17T09:26:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f88a218b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfb08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f88a218b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfbd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f88a218b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfb5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f88a218b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88941af350 con 0x558f4bbe22f0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f88a218b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88941acc20 con 0x558f4bbea4d0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889907c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfb08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f88a218b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f88941aaa70 con 0x558f4bc0b1b0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889907c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfb08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42544/0 (socket says 172.21.15.19:42544) 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889907c640 1 -- 172.21.15.19:0/3761827532 learned_addr learned my addr 172.21.15.19:0/3761827532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889987d640 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfb5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889887b640 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfbd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889987d640 1 -- 172.21.15.19:0/3761827532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 msgr2=0x558f4bfbd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889987d640 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfbd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889987d640 1 -- 172.21.15.19:0/3761827532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 msgr2=0x558f4bfb08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889987d640 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfb08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889987d640 1 -- 172.21.15.19:0/3761827532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f4bfb4150 con 0x558f4bc0b1b0 2024-05-17T09:26:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.074+0000 7f889907c640 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfb08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.075+0000 7f889987d640 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfb5ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f888c011010 tx=0x7f888c009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb556fecdefb6edb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.075+0000 7f887effd640 1 -- 172.21.15.19:0/3761827532 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c004be0 con 0x558f4bc0b1b0 2024-05-17T09:26:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.075+0000 7f887effd640 1 -- 172.21.15.19:0/3761827532 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f888c003e00 con 0x558f4bc0b1b0 2024-05-17T09:26:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.075+0000 7f887effd640 1 -- 172.21.15.19:0/3761827532 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c01ce40 con 0x558f4bc0b1b0 2024-05-17T09:26:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 -- 172.21.15.19:0/3761827532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 msgr2=0x558f4bfb5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfb5ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f888c011010 tx=0x7f888c009be0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 -- 172.21.15.19:0/3761827532 shutdown_connections 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4bfb5ff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbea4d0 0x558f4bfbd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 --2- 172.21.15.19:0/3761827532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbe22f0 0x558f4bfb08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 -- 172.21.15.19:0/3761827532 >> 172.21.15.19:0/3761827532 conn(0x558f4bbff300 msgr2=0x558f4bfb58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 -- 172.21.15.19:0/3761827532 shutdown_connections 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 -- 172.21.15.19:0/3761827532 wait complete. 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 -- start start 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.076+0000 7f88a218b740 1 Processor -- start 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfbc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 0x558f4bfbc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4c029e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88941ab0e0 con 0x558f4bbea4d0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88941ac550 con 0x558f4bbe22f0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f88941ab7b0 con 0x558f4bc0b1b0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f88a218b740 10 client.0 _ll_register_callbacks cb 0x7f88840a5b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 0x558f4bfbc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 0x558f4bfbc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42548/0 (socket says 172.21.15.19:42548) 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 -- 172.21.15.19:0/2822145908 learned_addr learned my addr 172.21.15.19:0/2822145908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889987d640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4c029e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889907c640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfbc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 -- 172.21.15.19:0/2822145908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 msgr2=0x558f4c029e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4c029e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 -- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 msgr2=0x558f4bfbc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:26:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfbc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 -- 172.21.15.19:0/2822145908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f4c02a440 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889987d640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4c029e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f889887b640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 0x558f4bfbc700 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f88840a9b60 tx=0x7f8884010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e83a82cd155bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.077+0000 7f887cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f887cff9640 1 -- 172.21.15.19:0/2822145908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884010720 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f887cff9640 1 -- 172.21.15.19:0/2822145908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f88840a8ab0 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f888c003e00 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f887cff9640 1 -- 172.21.15.19:0/2822145908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88840a96e0 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 10 client.16965 fetch_fsmap learned FSMap version 32 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558f4c02b6b0 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f887cff9640 1 -- 172.21.15.19:0/2822145908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f888c003e00 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f887cff9640 1 -- 172.21.15.19:0/2822145908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f88840a5040 con 0x558f4bbea4d0 2024-05-17T09:26:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 10 client.16965 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 20 client.16965 populate_metadata read hostname 'smithi019' 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bb9f1c0 0x558f4c02df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 4 client.16965 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f4bf1a360 con 0x558f4bb9f1c0 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.078+0000 7f889907c640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bb9f1c0 0x558f4c02df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.079+0000 7f887cff9640 10 client.16965 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.079+0000 7f889907c640 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bb9f1c0 0x558f4c02df60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680998e1fc038a4a server_cookie=8527ff1b751d2494 in_seq=0 out_seq=0 2024-05-17T09:26:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f887cff9640 1 -- 172.21.15.19:0/2822145908 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f4bf1a360 con 0x558f4bb9f1c0 2024-05-17T09:26:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f887cff9640 10 client.16965 handle_command_reply: tid=0 2024-05-17T09:26:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 client.16965 shutdown 2024-05-17T09:26:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 msgr2=0x558f4bfbc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 0x558f4bfbc700 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f88840a9b60 tx=0x7f8884010c80 comp rx=0 tx=0).stop 2024-05-17T09:26:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 shutdown_connections 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558f4bb9f1c0 0x558f4c02df60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558f4bc0b1b0 0x558f4c029e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f4bbea4d0 0x558f4bfbc700 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f887cff9640 0 client.16965 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 --2- 172.21.15.19:0/2822145908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f4bbe22f0 0x558f4bfbc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 >> 172.21.15.19:0/2822145908 conn(0x558f4bbff300 msgr2=0x558f4bfab2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.096+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 shutdown_connections 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f88a218b740 1 -- 172.21.15.19:0/2822145908 wait complete. 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f88a218b740 20 client.16965 trim_cache size 0 max 16384 2024-05-17T09:26:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:26:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88800b28a0 msgr2=0x7f88800b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88800b28a0 0x7f88800b4db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8890000c00 tx=0x7f88900046c0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8894106d60 msgr2=0x7f889419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8894106d60 0x7f889419a730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9f0 tx=0x7f888400eeb0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 shutdown_connections 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88941098d0 0x7f88941a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8894106d60 0x7f889419a730 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f88800b28a0 0x7f88800b4db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 --2- 172.21.15.19:0/55733898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8894104370 0x7f889419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 >> 172.21.15.19:0/55733898 conn(0x7f88940fdd30 msgr2=0x7f88941052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 shutdown_connections 2024-05-17T09:26:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:26:57.097+0000 7f889b2e0640 1 -- 172.21.15.19:0/55733898 wait complete. 2024-05-17T09:26:57.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 100303 1.76 MB/sec execute 175 sec latency 15.831 ms 2024-05-17T09:26:58.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 100854 1.77 MB/sec execute 176 sec latency 16.551 ms 2024-05-17T09:26:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:26:58 smithi079 ceph-mon[35502]: pgmap v654: 129 pgs: 129 active+clean; 51 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:26:58.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:26:58 smithi069 ceph-mon[35081]: pgmap v654: 129 pgs: 129 active+clean; 51 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:26:58.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:26:58 smithi019 ceph-mon[31620]: pgmap v654: 129 pgs: 129 active+clean; 51 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:26:59.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 101457 1.76 MB/sec execute 177 sec latency 16.654 ms 2024-05-17T09:27:00.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 101950 1.77 MB/sec execute 178 sec latency 16.973 ms 2024-05-17T09:27:00.731 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:00 smithi069 ceph-mon[35081]: pgmap v655: 129 pgs: 129 active+clean; 52 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:27:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:00 smithi079 ceph-mon[35502]: pgmap v655: 129 pgs: 129 active+clean; 52 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:27:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:00 smithi019 ceph-mon[31620]: pgmap v655: 129 pgs: 129 active+clean; 52 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:27:01.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 102648 1.79 MB/sec execute 179 sec latency 67.730 ms 2024-05-17T09:27:02.432 INFO:tasks.workunit.client.0.smithi019.stdout: 1 103045 1.80 MB/sec execute 180 sec latency 15.438 ms 2024-05-17T09:27:02.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:02 smithi019 ceph-mon[31620]: pgmap v656: 129 pgs: 129 active+clean; 51 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-05-17T09:27:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:02 smithi079 ceph-mon[35502]: pgmap v656: 129 pgs: 129 active+clean; 51 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-05-17T09:27:02.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:02 smithi069 ceph-mon[35081]: pgmap v656: 129 pgs: 129 active+clean; 51 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-05-17T09:27:03.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 103339 1.80 MB/sec execute 181 sec latency 14.207 ms 2024-05-17T09:27:04.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 103712 1.79 MB/sec execute 182 sec latency 15.138 ms 2024-05-17T09:27:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:04 smithi079 ceph-mon[35502]: pgmap v657: 129 pgs: 129 active+clean; 51 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:27:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:04 smithi069 ceph-mon[35081]: pgmap v657: 129 pgs: 129 active+clean; 51 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:27:04.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:04 smithi019 ceph-mon[31620]: pgmap v657: 129 pgs: 129 active+clean; 51 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:27:05.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 104193 1.79 MB/sec execute 183 sec latency 15.589 ms 2024-05-17T09:27:06.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 104814 1.80 MB/sec execute 184 sec latency 17.702 ms 2024-05-17T09:27:06.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:06 smithi079 ceph-mon[35502]: pgmap v658: 129 pgs: 129 active+clean; 52 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:27:06.932 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:06 smithi069 ceph-mon[35081]: pgmap v658: 129 pgs: 129 active+clean; 52 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:27:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:06 smithi019 ceph-mon[31620]: pgmap v658: 129 pgs: 129 active+clean; 52 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:27:07.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 105363 1.80 MB/sec execute 185 sec latency 15.845 ms 2024-05-17T09:27:08.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 106104 1.82 MB/sec execute 186 sec latency 16.110 ms 2024-05-17T09:27:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:08 smithi079 ceph-mon[35502]: pgmap v659: 129 pgs: 129 active+clean; 52 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:27:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:08 smithi069 ceph-mon[35081]: pgmap v659: 129 pgs: 129 active+clean; 52 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:27:08.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:08 smithi019 ceph-mon[31620]: pgmap v659: 129 pgs: 129 active+clean; 52 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:27:09.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 106519 1.83 MB/sec execute 187 sec latency 17.333 ms 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 -- 172.21.15.19:0/3306718958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 msgr2=0x7f63b010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 --2- 172.21.15.19:0/3306718958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b010aea0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009ca0 tx=0x7f63a802a8c0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 -- 172.21.15.19:0/3306718958 shutdown_connections 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 --2- 172.21.15.19:0/3306718958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b010aea0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 --2- 172.21.15.19:0/3306718958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0103d80 0x7f63b0104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 --2- 172.21.15.19:0/3306718958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0102b40 0x7f63b0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 -- 172.21.15.19:0/3306718958 >> 172.21.15.19:0/3306718958 conn(0x7f63b00fdd30 msgr2=0x7f63b01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 -- 172.21.15.19:0/3306718958 shutdown_connections 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.203+0000 7f63b7677640 1 -- 172.21.15.19:0/3306718958 wait complete. 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 Processor -- start 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 -- start start 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 0x7f63b019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0103d80 0x7f63b019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b01a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010e570 con 0x7f63b0108a80 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010e3f0 con 0x7f63b0102b40 2024-05-17T09:27:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b7677640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010e6f0 con 0x7f63b0103d80 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b5c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b01a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.204+0000 7f63b5413640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 0x7f63b019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b01a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39140/0 (socket says 172.21.15.19:39140) 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5413640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 0x7f63b019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59348/0 (socket says 172.21.15.19:59348) 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 -- 172.21.15.19:0/2912752343 learned_addr learned my addr 172.21.15.19:0/2912752343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b4c12640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0103d80 0x7f63b019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 -- 172.21.15.19:0/2912752343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0103d80 msgr2=0x7f63b019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0103d80 0x7f63b019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 -- 172.21.15.19:0/2912752343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 msgr2=0x7f63b019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 0x7f63b019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 -- 172.21.15.19:0/2912752343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b01a1cc0 con 0x7f63b0108a80 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5413640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 0x7f63b019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b4c12640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0103d80 0x7f63b019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b5c14640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b01a1650 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005e30 tx=0x7f63a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63a67fc640 1 -- 172.21.15.19:0/2912752343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8032e90 con 0x7f63b0108a80 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63a67fc640 1 -- 172.21.15.19:0/2912752343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a80044d0 con 0x7f63b0108a80 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01a1f50 con 0x7f63b0108a80 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63a67fc640 1 -- 172.21.15.19:0/2912752343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8030ce0 con 0x7f63b0108a80 2024-05-17T09:27:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.205+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a23b0 con 0x7f63b0108a80 2024-05-17T09:27:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.208+0000 7f63be605740 1 Processor -- start 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.208+0000 7f63a67fc640 1 -- 172.21.15.19:0/2912752343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f63a8004040 con 0x7f63b0108a80 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.208+0000 7f63be605740 1 -- start start 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63be605740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817c31b0 0x55a381b643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63be605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 0x55a381b64920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63be605740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b6b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63be605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a381ad6c30 con 0x55a38179a2f0 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63be605740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a381ac67d0 con 0x55a3817a24d0 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63be605740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a381aca810 con 0x55a3817c31b0 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b5413640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 0x55a381b64920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b5413640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 0x55a381b64920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39148/0 (socket says 172.21.15.19:39148) 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b5413640 1 -- 172.21.15.19:0/1401143453 learned_addr learned my addr 172.21.15.19:0/1401143453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b6b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817c31b0 0x55a381b643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 -- 172.21.15.19:0/1401143453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817c31b0 msgr2=0x55a381b643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817c31b0 0x55a381b643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 -- 172.21.15.19:0/1401143453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 msgr2=0x55a381b64920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 0x55a381b64920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 -- 172.21.15.19:0/1401143453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3817db210 con 0x55a3817a24d0 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b5413640 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 0x55a381b64920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63a67fc640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f639c0b28a0 0x7f639c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.209+0000 7f63b4c12640 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b6b820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63b019b220 tx=0x7f63a000bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5450aa1f2660670b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.210+0000 7f63b5413640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f639c0b28a0 0x7f639c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.210+0000 7f639b7fe640 1 -- 172.21.15.19:0/1401143453 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004020 con 0x55a3817a24d0 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.210+0000 7f63a67fc640 1 -- 172.21.15.19:0/2912752343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f63a8103ab0 con 0x7f63b0108a80 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.210+0000 7f639b7fe640 1 -- 172.21.15.19:0/1401143453 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a00054b0 con 0x55a3817a24d0 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.210+0000 7f639b7fe640 1 -- 172.21.15.19:0/1401143453 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0010ba0 con 0x55a3817a24d0 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 -- 172.21.15.19:0/1401143453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 msgr2=0x55a381b6b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b6b820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63b019b220 tx=0x7f63a000bb30 comp rx=0 tx=0).stop 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63b5413640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f639c0b28a0 0x7f639c0b4db0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55a381b65780 tx=0x7f63ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 -- 172.21.15.19:0/1401143453 shutdown_connections 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b6b820 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a38179a2f0 0x55a381b64920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 --2- 172.21.15.19:0/1401143453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817c31b0 0x55a381b643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 -- 172.21.15.19:0/1401143453 >> 172.21.15.19:0/1401143453 conn(0x55a3817b7300 msgr2=0x55a381b5fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 -- 172.21.15.19:0/1401143453 shutdown_connections 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 -- 172.21.15.19:0/1401143453 wait complete. 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 -- start start 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.211+0000 7f63be605740 1 Processor -- start 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 0x55a381b6dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817a24d0 0x55a381b6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b75150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a3817ac790 con 0x55a3817c31b0 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a381a82890 con 0x55a38179a2f0 2024-05-17T09:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a381b6c270 con 0x55a3817a24d0 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63b5413640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 0x55a381b6dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63b5413640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 0x55a381b6dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59366/0 (socket says 172.21.15.19:59366) 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63b5413640 1 -- 172.21.15.19:0/2607509527 learned_addr learned my addr 172.21.15.19:0/2607509527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63b5c14640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b75150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63be605740 10 client.0 _ll_register_callbacks cb 0x55a381a85d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.212+0000 7f63b4c12640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817a24d0 0x55a381b6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63b5413640 1 -- 172.21.15.19:0/2607509527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817a24d0 msgr2=0x55a381b6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63b5413640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817a24d0 0x55a381b6e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63b5413640 1 -- 172.21.15.19:0/2607509527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 msgr2=0x55a381b75150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63b5413640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b75150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63b5413640 1 -- 172.21.15.19:0/2607509527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3818a7620 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63b5413640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 0x55a381b6dcb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00ffa0 tx=0x7f63ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44872a50f9e4866 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f6398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f6398ff9640 1 -- 172.21.15.19:0/2607509527 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac02e060 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a00054b0 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f6398ff9640 1 -- 172.21.15.19:0/2607509527 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f63ac018030 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f6398ff9640 1 -- 172.21.15.19:0/2607509527 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f63ac035050 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63be605740 10 client.26621 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.213+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a381b78180 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f6398ff9640 1 -- 172.21.15.19:0/2607509527 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a00054b0 con 0x55a38179a2f0 2024-05-17T09:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f6398ff9640 1 -- 172.21.15.19:0/2607509527 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f63ac01c7a0 con 0x55a38179a2f0 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63be605740 10 client.26621 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63be605740 10 client.26621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63be605740 20 client.26621 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63be605740 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a381781d80 0x55a381b7b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63be605740 4 client.26621 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a381ad8b80 con 0x55a381781d80 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.214+0000 7f63b4c12640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a381781d80 0x55a381b7b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.215+0000 7f63b4c12640 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a381781d80 0x55a381b7b8c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1deb84102addd24 server_cookie=e216279630aced67 in_seq=0 out_seq=0 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.215+0000 7f6398ff9640 10 client.26621 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.215+0000 7f6398ff9640 1 -- 172.21.15.19:0/2607509527 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a381ad8b80 con 0x55a381781d80 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.215+0000 7f6398ff9640 10 client.26621 handle_command_reply: tid=0 2024-05-17T09:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.215+0000 7f63be605740 1 client.26621 shutdown 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 msgr2=0x55a381b6dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 0x55a381b6dcb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00ffa0 tx=0x7f63ac004730 comp rx=0 tx=0).stop 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 shutdown_connections 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a381781d80 0x55a381b7b8c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b75150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f6398ff9640 0 client.26621 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a3817a24d0 0x55a381b6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 --2- 172.21.15.19:0/2607509527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38179a2f0 0x55a381b6dcb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 >> 172.21.15.19:0/2607509527 conn(0x55a3817b7300 msgr2=0x55a381b62c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 shutdown_connections 2024-05-17T09:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 1 -- 172.21.15.19:0/2607509527 wait complete. 2024-05-17T09:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.216+0000 7f63be605740 20 client.26621 trim_cache size 0 max 16384 2024-05-17T09:27:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.233+0000 7f63be605740 1 Processor -- start 2024-05-17T09:27:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.233+0000 7f63be605740 1 -- start start 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63be605740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b63240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63be605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 0x55a381b63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63be605740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817c31b0 0x55a381b750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63be605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01af210 con 0x55a3817a24d0 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63be605740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01acae0 con 0x55a3817c31b0 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63be605740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aa930 con 0x55a38179a2f0 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 0x55a381b63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b5413640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 0x55a381b63780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39158/0 (socket says 172.21.15.19:39158) 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 -- 172.21.15.19:0/3494815655 learned_addr learned my addr 172.21.15.19:0/3494815655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b5413640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b63240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:34660/0 (socket says 172.21.15.19:34660) 2024-05-17T09:27:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b5c14640 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817c31b0 0x55a381b750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 -- 172.21.15.19:0/3494815655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 msgr2=0x55a381b63240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b63240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 -- 172.21.15.19:0/3494815655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817c31b0 msgr2=0x55a381b750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817c31b0 0x55a381b750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 -- 172.21.15.19:0/3494815655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a381b6e8b0 con 0x55a3817a24d0 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f63b4c12640 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 0x55a381b63780 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55a381b6ed20 tx=0x7f63a001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ec003d6458ea09 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.234+0000 7f639affd640 1 -- 172.21.15.19:0/3494815655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a002f020 con 0x55a3817a24d0 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f639affd640 1 -- 172.21.15.19:0/3494815655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a0016040 con 0x55a3817a24d0 2024-05-17T09:27:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f639affd640 1 -- 172.21.15.19:0/3494815655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a002b880 con 0x55a3817a24d0 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f63be605740 1 -- 172.21.15.19:0/3494815655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 msgr2=0x55a381b63780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f63be605740 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 0x55a381b63780 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55a381b6ed20 tx=0x7f63a001b040 comp rx=0 tx=0).stop 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f63be605740 1 -- 172.21.15.19:0/3494815655 shutdown_connections 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f63be605740 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817c31b0 0x55a381b750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f63be605740 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817a24d0 0x55a381b63780 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.235+0000 7f63be605740 1 --2- 172.21.15.19:0/3494815655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b63240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- 172.21.15.19:0/3494815655 >> 172.21.15.19:0/3494815655 conn(0x55a3817b7300 msgr2=0x55a381b62b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- 172.21.15.19:0/3494815655 shutdown_connections 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- 172.21.15.19:0/3494815655 wait complete. 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- start start 2024-05-17T09:27:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 Processor -- start 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b72340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aafa0 con 0x55a3817c31b0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ac410 con 0x55a3817a24d0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ab670 con 0x55a38179a2f0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.236+0000 7f63be605740 10 client.0 _ll_register_callbacks cb 0x7f63ac00fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b4c12640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b72340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b72880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39174/0 (socket says 172.21.15.19:39174) 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 -- 172.21.15.19:0/388370206 learned_addr learned my addr 172.21.15.19:0/388370206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5413640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 -- 172.21.15.19:0/388370206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 msgr2=0x55a381b757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 -- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 msgr2=0x55a381b72340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b72340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 -- 172.21.15.19:0/388370206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a381be1f80 con 0x55a3817c31b0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b4c12640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b72340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63b5c14640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b72880 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f63a80049d0 tx=0x7f63a80ca4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d830c3d9727b109 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f6398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f6398ff9640 1 -- 172.21.15.19:0/388370206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80ca7d0 con 0x55a3817c31b0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f6398ff9640 1 -- 172.21.15.19:0/388370206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f63a80c6f50 con 0x55a3817c31b0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.237+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0016040 con 0x55a3817c31b0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f6398ff9640 1 -- 172.21.15.19:0/388370206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f63a80cb3c0 con 0x55a3817c31b0 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 10 client.16983 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a381be31a0 con 0x55a3817c31b0 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f6398ff9640 1 -- 172.21.15.19:0/388370206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a0016040 con 0x55a3817c31b0 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f6398ff9640 1 -- 172.21.15.19:0/388370206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f63a80e4020 con 0x55a3817c31b0 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 10 client.16983 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 20 client.16983 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a3817571c0 0x55a381be5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 4 client.16983 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a381ad2360 con 0x55a3817571c0 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.238+0000 7f63b5413640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a3817571c0 0x55a381be5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.239+0000 7f6398ff9640 10 client.16983 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.239+0000 7f63b5413640 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a3817571c0 0x55a381be5b60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5806d474f0b2a9b6 server_cookie=5261977c40bbda97 in_seq=0 out_seq=0 2024-05-17T09:27:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f6398ff9640 1 -- 172.21.15.19:0/388370206 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a381ad2360 con 0x55a3817571c0 2024-05-17T09:27:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f6398ff9640 10 client.16983 handle_command_reply: tid=0 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 client.16983 shutdown 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 msgr2=0x55a381b72880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b72880 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f63a80049d0 tx=0x7f63a80ca4f0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 shutdown_connections 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a3817571c0 0x55a381be5b60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3817c31b0 0x55a381b72880 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3817a24d0 0x55a381b72340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 --2- 172.21.15.19:0/388370206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a38179a2f0 0x55a381b757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 >> 172.21.15.19:0/388370206 conn(0x55a3817b7300 msgr2=0x55a381b63560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f6398ff9640 0 client.16983 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 shutdown_connections 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 1 -- 172.21.15.19:0/388370206 wait complete. 2024-05-17T09:27:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.259+0000 7f63be605740 20 client.16983 trim_cache size 0 max 16384 2024-05-17T09:27:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:27:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:27:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f639c0b28a0 msgr2=0x7f639c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f639c0b28a0 0x7f639c0b4db0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55a381b65780 tx=0x7f63ac005f90 comp rx=0 tx=0).stop 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 msgr2=0x7f63b01a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b01a1650 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005e30 tx=0x7f63a8003ba0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 shutdown_connections 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f639c0b28a0 0x7f639c0b4db0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0108a80 0x7f63b01a1650 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63b0103d80 0x7f63b019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 --2- 172.21.15.19:0/2912752343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b0102b40 0x7f63b019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 >> 172.21.15.19:0/2912752343 conn(0x7f63b00fdd30 msgr2=0x7f63b0109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 shutdown_connections 2024-05-17T09:27:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:10.260+0000 7f63b7677640 1 -- 172.21.15.19:0/2912752343 wait complete. 2024-05-17T09:27:10.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 106820 1.83 MB/sec execute 188 sec latency 19.890 ms 2024-05-17T09:27:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:10 smithi079 ceph-mon[35502]: pgmap v660: 129 pgs: 129 active+clean; 52 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-17T09:27:10.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:10 smithi069 ceph-mon[35081]: pgmap v660: 129 pgs: 129 active+clean; 52 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-17T09:27:10.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:10 smithi019 ceph-mon[31620]: pgmap v660: 129 pgs: 129 active+clean; 52 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-17T09:27:11.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 107067 1.82 MB/sec execute 189 sec latency 15.052 ms 2024-05-17T09:27:12.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 107581 1.82 MB/sec execute 190 sec latency 11.993 ms 2024-05-17T09:27:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:12 smithi079 ceph-mon[35502]: pgmap v661: 129 pgs: 129 active+clean; 53 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-17T09:27:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:12 smithi069 ceph-mon[35081]: pgmap v661: 129 pgs: 129 active+clean; 53 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-17T09:27:12.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:12 smithi019 ceph-mon[31620]: pgmap v661: 129 pgs: 129 active+clean; 53 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-17T09:27:13.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 108143 1.83 MB/sec execute 191 sec latency 12.616 ms 2024-05-17T09:27:14.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 108787 1.83 MB/sec execute 192 sec latency 21.626 ms 2024-05-17T09:27:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:14 smithi079 ceph-mon[35502]: pgmap v662: 129 pgs: 129 active+clean; 55 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:27:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:14 smithi069 ceph-mon[35081]: pgmap v662: 129 pgs: 129 active+clean; 55 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:27:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:14 smithi019 ceph-mon[31620]: pgmap v662: 129 pgs: 129 active+clean; 55 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:27:15.423 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.1 2024-05-17T09:27:15.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:27:15.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 109330 1.84 MB/sec execute 193 sec latency 17.897 ms 2024-05-17T09:27:15.711 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:27:16.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 109786 1.85 MB/sec execute 194 sec latency 15.750 ms 2024-05-17T09:27:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:16 smithi079 ceph-mon[35502]: pgmap v663: 129 pgs: 129 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:27:16.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:16 smithi019 ceph-mon[31620]: pgmap v663: 129 pgs: 129 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:27:16.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:16 smithi069 ceph-mon[35081]: pgmap v663: 129 pgs: 129 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:27:17.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 110249 1.86 MB/sec execute 195 sec latency 26.406 ms 2024-05-17T09:27:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 -- 172.21.15.19:0/1127587979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a01045e0 msgr2=0x7f39a01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/1127587979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a01045e0 0x7f39a01049f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f39900099c0 tx=0x7f3990009cd0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 -- 172.21.15.19:0/1127587979 shutdown_connections 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/1127587979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39a0109860 0x7f39a01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/1127587979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a01045e0 0x7f39a01049f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/1127587979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a0108f10 0x7f39a0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 -- 172.21.15.19:0/1127587979 >> 172.21.15.19:0/1127587979 conn(0x7f39a0100030 msgr2=0x7f39a0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 -- 172.21.15.19:0/1127587979 shutdown_connections 2024-05-17T09:27:17.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.604+0000 7f39a5cc8700 1 -- 172.21.15.19:0/1127587979 wait complete. 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.605+0000 7f39a5cc8700 1 Processor -- start 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.605+0000 7f39a5cc8700 1 -- start start 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f39a5cc8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a01045e0 0x7f39a019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f39a5cc8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 0x7f39a019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f39a5cc8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39a0109860 0x7f39a019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f39a5cc8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0114340 con 0x7f39a0108f10 2024-05-17T09:27:17.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f39a5cc8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01141c0 con 0x7f39a0109860 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f39a5cc8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01144c0 con 0x7f39a01045e0 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 0x7f39a019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 0x7f39a019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37188/0 (socket says 172.21.15.19:37188) 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399effd700 1 -- 172.21.15.19:0/3574123002 learned_addr learned my addr 172.21.15.19:0/3574123002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a01045e0 0x7f39a019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399ffff700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39a0109860 0x7f39a019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399f7fe700 1 -- 172.21.15.19:0/3574123002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39a0109860 msgr2=0x7f39a019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399f7fe700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39a0109860 0x7f39a019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399f7fe700 1 -- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 msgr2=0x7f39a019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399f7fe700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 0x7f39a019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399f7fe700 1 -- 172.21.15.19:0/3574123002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a019fc40 con 0x7f39a01045e0 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.606+0000 7f399effd700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 0x7f39a019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:17.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.607+0000 7f399f7fe700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a01045e0 0x7f39a019ad70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39900098e0 tx=0x7f39900045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.607+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990026070 con 0x7f39a01045e0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.607+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f399000f460 con 0x7f39a01045e0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.607+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990009e60 con 0x7f39a01045e0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.607+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a019fed0 con 0x7f39a01045e0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.607+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01a03b0 con 0x7f39a01045e0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.608+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a00512b0 con 0x7f39a01045e0 2024-05-17T09:27:17.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.610+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3990029070 con 0x7f39a01045e0 2024-05-17T09:27:17.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.610+0000 7f399cff9700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f39880b2880 0x7f39880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:17.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.611+0000 7f399effd700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f39880b2880 0x7f39880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:17.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.611+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f39900e3070 con 0x7f39a01045e0 2024-05-17T09:27:17.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.611+0000 7f399effd700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f39880b2880 0x7f39880b4d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f39a01075e0 tx=0x7f399400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:17.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.614+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f399005edf0 con 0x7f39a01045e0 2024-05-17T09:27:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.924+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a002e5e0 con 0x7f39a01045e0 2024-05-17T09:27:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.925+0000 7f399cff9700 1 -- 172.21.15.19:0/3574123002 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f3990013070 con 0x7f39a01045e0 2024-05-17T09:27:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:27:17.925 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f39880b2880 msgr2=0x7f39880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f39880b2880 0x7f39880b4d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f39a01075e0 tx=0x7f399400b480 comp rx=0 tx=0).stop 2024-05-17T09:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a01045e0 msgr2=0x7f39a019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a01045e0 0x7f39a019ad70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39900098e0 tx=0x7f39900045e0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 shutdown_connections 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f39880b2880 0x7f39880b4d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39a01045e0 0x7f39a019ad70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39a0109860 0x7f39a019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 --2- 172.21.15.19:0/3574123002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39a0108f10 0x7f39a019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.928+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 >> 172.21.15.19:0/3574123002 conn(0x7f39a0100030 msgr2=0x7f39a0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.929+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 shutdown_connections 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:17.929+0000 7f39a5cc8700 1 -- 172.21.15.19:0/3574123002 wait complete. 2024-05-17T09:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:27:18.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 110482 1.86 MB/sec execute 196 sec latency 13.350 ms 2024-05-17T09:27:18.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:18 smithi019 ceph-mon[31620]: pgmap v664: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-17T09:27:18.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:18 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3574123002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:27:18.653 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 143.1 --await-for 45.0' 2024-05-17T09:27:18.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 143.1 --await-for 45.0 2024-05-17T09:27:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:18 smithi079 ceph-mon[35502]: pgmap v664: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-17T09:27:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:18 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3574123002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:27:18.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:18 smithi069 ceph-mon[35081]: pgmap v664: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-17T09:27:18.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:18 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3574123002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:27:18.977 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:27:19.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 110870 1.85 MB/sec execute 197 sec latency 14.825 ms 2024-05-17T09:27:20.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111384 1.86 MB/sec execute 198 sec latency 15.715 ms 2024-05-17T09:27:20.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:20 smithi019 ceph-mon[31620]: pgmap v665: 129 pgs: 129 active+clean; 56 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:27:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:20 smithi079 ceph-mon[35502]: pgmap v665: 129 pgs: 129 active+clean; 56 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:27:20.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 -- 172.21.15.19:0/4242579802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc107e00 msgr2=0x7f19fc108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/4242579802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc107e00 0x7f19fc108210 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f19f00099c0 tx=0x7f19f0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 -- 172.21.15.19:0/4242579802 shutdown_connections 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/4242579802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19fc1034d0 0x7f19fc1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/4242579802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/4242579802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc107e00 0x7f19fc108210 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 -- 172.21.15.19:0/4242579802 >> 172.21.15.19:0/4242579802 conn(0x7f19fc0fef20 msgr2=0x7f19fc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 -- 172.21.15.19:0/4242579802 shutdown_connections 2024-05-17T09:27:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.808+0000 7f1a01eb1700 1 -- 172.21.15.19:0/4242579802 wait complete. 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 Processor -- start 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 -- start start 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc1034d0 0x7f19fc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19fc107e00 0x7f19fc19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc113210 con 0x7f19fc1034d0 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc113090 con 0x7f19fc108750 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.809+0000 7f1a01eb1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc113390 con 0x7f19fc107e00 2024-05-17T09:27:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc1034d0 0x7f19fc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc1034d0 0x7f19fc19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37210/0 (socket says 172.21.15.19:37210) 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fb7fe700 1 -- 172.21.15.19:0/751769577 learned_addr learned my addr 172.21.15.19:0/751769577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52852/0 (socket says 172.21.15.19:52852) 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19faffd700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19fc107e00 0x7f19fc19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 -- 172.21.15.19:0/751769577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19fc107e00 msgr2=0x7f19fc19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19fc107e00 0x7f19fc19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 -- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc1034d0 msgr2=0x7f19fc19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc1034d0 0x7f19fc19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 -- 172.21.15.19:0/751769577 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19fc19a140 con 0x7f19fc108750 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19fbfff700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc199a90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19f400e580 tx=0x7f19f400e890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19f8ff9700 1 -- 172.21.15.19:0/751769577 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f400ee20 con 0x7f19fc108750 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19f8ff9700 1 -- 172.21.15.19:0/751769577 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f4004bb0 con 0x7f19fc108750 2024-05-17T09:27:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19fc19a430 con 0x7f19fc108750 2024-05-17T09:27:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.810+0000 7f19f8ff9700 1 -- 172.21.15.19:0/751769577 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4004670 con 0x7f19fc108750 2024-05-17T09:27:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.811+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19fc06c030 con 0x7f19fc108750 2024-05-17T09:27:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.813+0000 7f19f8ff9700 1 -- 172.21.15.19:0/751769577 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19f4007500 con 0x7f19fc108750 2024-05-17T09:27:20.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.814+0000 7f19f8ff9700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19e40b2900 0x7f19e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19e40b2900 0x7f19e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19f8ff9700 1 -- 172.21.15.19:0/751769577 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f19f40e4430 con 0x7f19fc108750 2024-05-17T09:27:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 Processor -- start 2024-05-17T09:27:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 -- start start 2024-05-17T09:27:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e338d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e1b5fd0 0x56192e3392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e33d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19e40b2900 0x7f19e40b4e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19fc19abd0 tx=0x7f19ec007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56192e262e20 con 0x56192e33d510 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56192e2529c0 con 0x56192e233560 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56192e256a00 con 0x56192e1b5fd0 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e33d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e33d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37220/0 (socket says 172.21.15.19:37220) 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19fbfff700 1 -- 172.21.15.19:0/1237710900 learned_addr learned my addr 172.21.15.19:0/1237710900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19faffd700 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e1b5fd0 0x56192e3392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.815+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e338d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19fbfff700 1 -- 172.21.15.19:0/1237710900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e1b5fd0 msgr2=0x56192e3392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19fbfff700 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e1b5fd0 0x56192e3392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19fbfff700 1 -- 172.21.15.19:0/1237710900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 msgr2=0x56192e338d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19fbfff700 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e338d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19fbfff700 1 -- 172.21.15.19:0/1237710900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192e1a2a60 con 0x56192e33d510 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19fbfff700 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e33d920 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f19f40a7720 tx=0x7f19f40ab1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32bdda5fd1d6d15 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19e1ffb700 1 -- 172.21.15.19:0/1237710900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f401f040 con 0x56192e33d510 2024-05-17T09:27:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19e1ffb700 1 -- 172.21.15.19:0/1237710900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f40ab6a0 con 0x56192e33d510 2024-05-17T09:27:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.816+0000 7f19e1ffb700 1 -- 172.21.15.19:0/1237710900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40e4140 con 0x56192e33d510 2024-05-17T09:27:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1237710900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 msgr2=0x56192e33d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e33d920 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f19f40a7720 tx=0x7f19f40ab1b0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1237710900 shutdown_connections 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e33d920 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e338d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1237710900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e1b5fd0 0x56192e3392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1237710900 >> 172.21.15.19:0/1237710900 conn(0x56192e1f4e10 msgr2=0x56192e32f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1237710900 shutdown_connections 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.817+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1237710900 wait complete. 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 -- start start 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 Processor -- start 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e342790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e233560 0x56192e342cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e343210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56192e159130 con 0x56192e233560 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56192e1f4950 con 0x56192e1b5fd0 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56192e33e2c0 con 0x56192e33d510 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f1a108c6b80 10 client.0 _ll_register_callbacks cb 0x56192e1edab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f19fb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e342790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f19fb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e342790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52876/0 (socket says 172.21.15.19:52876) 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.818+0000 7f19fb7fe700 1 -- 172.21.15.19:0/1290160804 learned_addr learned my addr 172.21.15.19:0/1290160804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19faffd700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e233560 0x56192e342cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fbfff700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e343210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fb7fe700 1 -- 172.21.15.19:0/1290160804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 msgr2=0x56192e343210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e343210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fb7fe700 1 -- 172.21.15.19:0/1290160804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e233560 msgr2=0x56192e342cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e233560 0x56192e342cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fb7fe700 1 -- 172.21.15.19:0/1290160804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192e1f1820 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e342790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0116f0 tx=0x7f19ec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b57fb2ded191b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19daffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19daffd700 1 -- 172.21.15.19:0/1290160804 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec004830 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f19daffd700 1 -- 172.21.15.19:0/1290160804 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f19ec0049d0 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.819+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f40ab6a0 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f19daffd700 1 -- 172.21.15.19:0/1290160804 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19ec030050 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 10 client.26645 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56192e350110 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f19daffd700 1 -- 172.21.15.19:0/1290160804 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f40ab6a0 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f19daffd700 1 -- 172.21.15.19:0/1290160804 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f19ec01f430 con 0x56192e1b5fd0 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 10 client.26645 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 10 client.26645 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:27:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 20 client.26645 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e352750 0x56192e354bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 4 client.26645 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:27:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56192e264d70 con 0x56192e352750 2024-05-17T09:27:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.820+0000 7f19faffd700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e352750 0x56192e354bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.821+0000 7f19daffd700 10 client.26645 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.821+0000 7f19faffd700 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e352750 0x56192e354bf0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bede237a5860c1e2 server_cookie=202f509e03eecbfc in_seq=0 out_seq=0 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.821+0000 7f19daffd700 1 -- 172.21.15.19:0/1290160804 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56192e264d70 con 0x56192e352750 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.821+0000 7f19daffd700 10 client.26645 handle_command_reply: tid=0 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.821+0000 7f1a108c6b80 1 client.26645 shutdown 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 msgr2=0x56192e342790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e342790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0116f0 tx=0x7f19ec0047c0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 shutdown_connections 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e343210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e352750 0x56192e354bf0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e342790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1290160804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e233560 0x56192e342cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 >> 172.21.15.19:0/1290160804 conn(0x56192e1f4e10 msgr2=0x56192e329310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f19daffd700 0 client.26645 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 shutdown_connections 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1290160804 wait complete. 2024-05-17T09:27:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.822+0000 7f1a108c6b80 20 client.26645 trim_cache size 0 max 16384 2024-05-17T09:27:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 Processor -- start 2024-05-17T09:27:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 -- start start 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e343050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 0x56192e34e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e341790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1b0a00 con 0x56192e33d510 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1ae290 con 0x56192e1b5fd0 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.849+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1ac0e0 con 0x56192e233560 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 0x56192e34e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 0x56192e34e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:40068/0 (socket says 172.21.15.19:40068) 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 -- 172.21.15.19:0/2789599837 learned_addr learned my addr 172.21.15.19:0/2789599837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:20.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19fbfff700 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e341790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e343050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 -- 172.21.15.19:0/2789599837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 msgr2=0x56192e343050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e343050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 -- 172.21.15.19:0/2789599837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 msgr2=0x56192e341790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e341790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 -- 172.21.15.19:0/2789599837 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192e32fa50 con 0x56192e233560 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e343050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19faffd700 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 0x56192e34e6a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f19f0000c00 tx=0x7f19f0005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c87443d77059131 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.850+0000 7f19e17fa700 1 -- 172.21.15.19:0/2789599837 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0005610 con 0x56192e233560 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f19e17fa700 1 -- 172.21.15.19:0/2789599837 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f0005940 con 0x56192e233560 2024-05-17T09:27:20.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f19e17fa700 1 -- 172.21.15.19:0/2789599837 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f000bd70 con 0x56192e233560 2024-05-17T09:27:20.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f1a108c6b80 1 -- 172.21.15.19:0/2789599837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 msgr2=0x56192e34e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 0x56192e34e6a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f19f0000c00 tx=0x7f19f0005040 comp rx=0 tx=0).stop 2024-05-17T09:27:20.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f1a108c6b80 1 -- 172.21.15.19:0/2789599837 shutdown_connections 2024-05-17T09:27:20.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e33d510 0x56192e341790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e1b5fd0 0x56192e343050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.851+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/2789599837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e233560 0x56192e34e6a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.852+0000 7f1a108c6b80 1 -- 172.21.15.19:0/2789599837 >> 172.21.15.19:0/2789599837 conn(0x56192e1f4e10 msgr2=0x56192e341080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.852+0000 7f1a108c6b80 1 -- 172.21.15.19:0/2789599837 shutdown_connections 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.852+0000 7f1a108c6b80 1 -- 172.21.15.19:0/2789599837 wait complete. 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.852+0000 7f1a108c6b80 1 -- start start 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.852+0000 7f1a108c6b80 1 Processor -- start 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e1b5fd0 0x56192e34aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e34b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e3458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1ac750 con 0x56192e1b5fd0 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1adbc0 con 0x56192e233560 2024-05-17T09:27:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1ace20 con 0x56192e33d510 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f1a108c6b80 10 client.0 _ll_register_callbacks cb 0x56192e1edab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e34b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e34b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52896/0 (socket says 172.21.15.19:52896) 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 -- 172.21.15.19:0/1211896560 learned_addr learned my addr 172.21.15.19:0/1211896560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e1b5fd0 0x56192e34aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19fbfff700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e3458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 -- 172.21.15.19:0/1211896560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 msgr2=0x56192e3458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e3458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 -- 172.21.15.19:0/1211896560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e1b5fd0 msgr2=0x56192e34aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e1b5fd0 0x56192e34aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 -- 172.21.15.19:0/1211896560 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192e346070 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e1b5fd0 0x56192e34aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19faffd700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e34b030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098e0 tx=0x7f19f000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=217d3a76c31e2f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.853+0000 7f19daffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f19daffd700 1 -- 172.21.15.19:0/1211896560 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f000b9b0 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f0005940 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f19daffd700 1 -- 172.21.15.19:0/1211896560 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f19f000bb50 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f19daffd700 1 -- 172.21.15.19:0/1211896560 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19f0043050 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 10 client.26651 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56192e3bdce0 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f19daffd700 1 -- 172.21.15.19:0/1211896560 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f0005940 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f19daffd700 1 -- 172.21.15.19:0/1211896560 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f19f002e070 con 0x56192e233560 2024-05-17T09:27:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 10 client.26651 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 10 client.26651 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 20 client.26651 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e3be2f0 0x56192e3c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 4 client.26651 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.1, "await_for": 45.0}] 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.854+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.1, "await_for": 45.0}) -- 0x56192e25e550 con 0x56192e3be2f0 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.855+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e3be2f0 0x56192e3c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.855+0000 7f19daffd700 10 client.26651 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.855+0000 7f19fb7fe700 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e3be2f0 0x56192e3c0710 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e737684eb068117f server_cookie=ac40d654786defce in_seq=0 out_seq=0 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.914+0000 7f19daffd700 1 -- 172.21.15.19:0/1211896560 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56192e25e550 con 0x56192e3be2f0 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.914+0000 7f19daffd700 10 client.26651 handle_command_reply: tid=0 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.914+0000 7f1a108c6b80 1 client.26651 shutdown 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.914+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 msgr2=0x56192e34b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.914+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e34b030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098e0 tx=0x7f19f000b8b0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 shutdown_connections 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56192e33d510 0x56192e3458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192e3be2f0 0x56192e3c0710 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192e1b5fd0 0x56192e34aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 --2- 172.21.15.19:0/1211896560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56192e233560 0x56192e34b030 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f19daffd700 0 client.26651 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 >> 172.21.15.19:0/1211896560 conn(0x56192e1f4e10 msgr2=0x56192e3336a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 shutdown_connections 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 1 -- 172.21.15.19:0/1211896560 wait complete. 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.915+0000 7f1a108c6b80 20 client.26651 trim_cache size 0 max 16384 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:27:20.916 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 26, 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "de0eed95": { 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 26, 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 143.1, 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:27:20.917 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:20.918 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19e40b2900 msgr2=0x7f19e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19e40b2900 0x7f19e40b4e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19fc19abd0 tx=0x7f19ec007480 comp rx=0 tx=0).stop 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 msgr2=0x7f19fc199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc199a90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19f400e580 tx=0x7f19f400e890 comp rx=0 tx=0).stop 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 shutdown_connections 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc108750 0x7f19fc199a90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19fc1034d0 0x7f19fc19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19e40b2900 0x7f19e40b4e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 --2- 172.21.15.19:0/751769577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19fc107e00 0x7f19fc19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:20.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 >> 172.21.15.19:0/751769577 conn(0x7f19fc0fef20 msgr2=0x7f19fc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:20.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 shutdown_connections 2024-05-17T09:27:20.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:20.916+0000 7f1a01eb1700 1 -- 172.21.15.19:0/751769577 wait complete. 2024-05-17T09:27:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:20 smithi069 ceph-mon[35081]: pgmap v665: 129 pgs: 129 active+clean; 56 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:27:21.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.85 MB/sec execute 199 sec latency 434.719 ms 2024-05-17T09:27:21.724 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'de0eed95', quiesce took 6.3 seconds. Will release after: 16.8 2024-05-17T09:27:22.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.84 MB/sec execute 200 sec latency 1434.820 ms 2024-05-17T09:27:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:22 smithi079 ceph-mon[35502]: pgmap v666: 129 pgs: 129 active+clean; 56 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-17T09:27:22.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:22 smithi069 ceph-mon[35081]: pgmap v666: 129 pgs: 129 active+clean; 56 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-17T09:27:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:22 smithi019 ceph-mon[31620]: pgmap v666: 129 pgs: 129 active+clean; 56 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-17T09:27:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 -- 172.21.15.19:0/2795714689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 msgr2=0x7fa99810be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 --2- 172.21.15.19:0/2795714689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99810be00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa990009ca0 tx=0x7fa99002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 -- 172.21.15.19:0/2795714689 shutdown_connections 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 --2- 172.21.15.19:0/2795714689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99810be00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 --2- 172.21.15.19:0/2795714689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 0x7fa998101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 --2- 172.21.15.19:0/2795714689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9981051f0 0x7fa998105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 -- 172.21.15.19:0/2795714689 >> 172.21.15.19:0/2795714689 conn(0x7fa9980fd4e0 msgr2=0x7fa9980ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 -- 172.21.15.19:0/2795714689 shutdown_connections 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.374+0000 7fa99f01e640 1 -- 172.21.15.19:0/2795714689 wait complete. 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 Processor -- start 2024-05-17T09:27:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 -- start start 2024-05-17T09:27:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 0x7fa99819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9981051f0 0x7fa9981a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa99810f660 con 0x7fa9981023a0 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa99810f4e0 con 0x7fa9981018a0 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99f01e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa99810f7e0 con 0x7fa9981051f0 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa997fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa997fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37264/0 (socket says 172.21.15.19:37264) 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa997fff640 1 -- 172.21.15.19:0/3894676742 learned_addr learned my addr 172.21.15.19:0/3894676742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99cdba640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 0x7fa99819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.375+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9981051f0 0x7fa9981a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa997fff640 1 -- 172.21.15.19:0/3894676742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9981051f0 msgr2=0x7fa9981a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa997fff640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9981051f0 0x7fa9981a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa997fff640 1 -- 172.21.15.19:0/3894676742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 msgr2=0x7fa99819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa997fff640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 0x7fa99819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa997fff640 1 -- 172.21.15.19:0/3894676742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9981a1c50 con 0x7fa9981023a0 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa99cdba640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 0x7fa99819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa997fff640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99819a650 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00e9f0 tx=0x7fa98c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa995ffb640 1 -- 172.21.15.19:0/3894676742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00cd80 con 0x7fa9981023a0 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa995ffb640 1 -- 172.21.15.19:0/3894676742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa98c0108c0 con 0x7fa9981023a0 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa995ffb640 1 -- 172.21.15.19:0/3894676742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0187c0 con 0x7fa9981023a0 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9981a1f40 con 0x7fa9981023a0 2024-05-17T09:27:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.376+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9981a23b0 con 0x7fa9981023a0 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 Processor -- start 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 -- start start 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41f41b0 0x55d1e45953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 0x55d1e4595920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa995ffb640 1 -- 172.21.15.19:0/3894676742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa98c010430 con 0x7fa9981023a0 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41d34d0 0x55d1e459c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4507c30 con 0x55d1e41cb2f0 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1e44f77d0 con 0x55d1e41f41b0 2024-05-17T09:27:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d1e44fb810 con 0x55d1e41d34d0 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa99cdba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 0x55d1e4595920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa99cdba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 0x55d1e4595920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37268/0 (socket says 172.21.15.19:37268) 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.379+0000 7fa99cdba640 1 -- 172.21.15.19:0/757917604 learned_addr learned my addr 172.21.15.19:0/757917604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41f41b0 0x55d1e45953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa997fff640 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41d34d0 0x55d1e459c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99cdba640 1 -- 172.21.15.19:0/757917604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41d34d0 msgr2=0x55d1e459c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99cdba640 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41d34d0 0x55d1e459c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99cdba640 1 -- 172.21.15.19:0/757917604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41f41b0 msgr2=0x55d1e45953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99cdba640 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41f41b0 0x55d1e45953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99cdba640 1 -- 172.21.15.19:0/757917604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e420c210 con 0x55d1e41cb2f0 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41f41b0 0x55d1e45953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa99cdba640 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 0x55d1e4595920 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa988010480 tx=0x7fa988010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318d1506848b1d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa995ffb640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9800b28a0 0x7fa9800b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa97effd640 1 -- 172.21.15.19:0/757917604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988011820 con 0x55d1e41cb2f0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.380+0000 7fa97effd640 1 -- 172.21.15.19:0/757917604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa988011e50 con 0x55d1e41cb2f0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa97effd640 1 -- 172.21.15.19:0/757917604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98801cdc0 con 0x55d1e41cb2f0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9800b28a0 0x7fa9800b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa995ffb640 1 -- 172.21.15.19:0/3894676742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa98c0e2930 con 0x7fa9981023a0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/757917604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 msgr2=0x55d1e4595920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9800b28a0 0x7fa9800b4db0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55d1e4596570 tx=0x7fa990031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 0x55d1e4595920 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa988010480 tx=0x7fa988010940 comp rx=0 tx=0).stop 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/757917604 shutdown_connections 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41d34d0 0x55d1e459c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41cb2f0 0x55d1e4595920 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/757917604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41f41b0 0x55d1e45953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/757917604 >> 172.21.15.19:0/757917604 conn(0x55d1e41e8300 msgr2=0x55d1e4590d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.381+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/757917604 shutdown_connections 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/757917604 wait complete. 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 -- start start 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 Processor -- start 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41cb2f0 0x55d1e459ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 0x55d1e459f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41f41b0 0x55d1e45a6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1e41dd790 con 0x55d1e41d34d0 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1e44b3890 con 0x55d1e41cb2f0 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d1e459d270 con 0x55d1e41f41b0 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.382+0000 7fa9a5fa8740 10 client.0 _ll_register_callbacks cb 0x55d1e44b6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 0x55d1e459f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 0x55d1e459f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37272/0 (socket says 172.21.15.19:37272) 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 -- 172.21.15.19:0/2315997444 learned_addr learned my addr 172.21.15.19:0/2315997444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa99cdba640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41cb2f0 0x55d1e459ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41f41b0 0x55d1e45a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 -- 172.21.15.19:0/2315997444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41f41b0 msgr2=0x55d1e45a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41f41b0 0x55d1e45a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 -- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41cb2f0 msgr2=0x55d1e459ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41cb2f0 0x55d1e459ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 -- 172.21.15.19:0/2315997444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e42d8620 con 0x55d1e41d34d0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa997fff640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 0x55d1e459f230 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0a9be0 tx=0x7fa98c010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ede32c62038af62 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa977fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa977fff640 1 -- 172.21.15.19:0/2315997444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0a9c50 con 0x55d1e41d34d0 2024-05-17T09:27:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa977fff640 1 -- 172.21.15.19:0/2315997444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa98c0a8eb0 con 0x55d1e41d34d0 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa988011e50 con 0x55d1e41d34d0 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.383+0000 7fa977fff640 1 -- 172.21.15.19:0/2315997444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa98c0adf50 con 0x55d1e41d34d0 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 10 client.17019 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1e45a9130 con 0x55d1e41d34d0 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa977fff640 1 -- 172.21.15.19:0/2315997444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa988011e50 con 0x55d1e41d34d0 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa977fff640 1 -- 172.21.15.19:0/2315997444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa98c0a8920 con 0x55d1e41d34d0 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 20 client.17019 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41b2d80 0x55d1e45ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 4 client.17019 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e4509b80 con 0x55d1e41b2d80 2024-05-17T09:27:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.384+0000 7fa99cdba640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41b2d80 0x55d1e45ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.385+0000 7fa977fff640 10 client.17019 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.385+0000 7fa99cdba640 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41b2d80 0x55d1e45ac830 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52754befdbf0338 server_cookie=be64750ff0f3d71c in_seq=0 out_seq=0 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.385+0000 7fa977fff640 1 -- 172.21.15.19:0/2315997444 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d1e4509b80 con 0x55d1e41b2d80 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.385+0000 7fa977fff640 10 client.17019 handle_command_reply: tid=0 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 client.17019 shutdown 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 msgr2=0x55d1e459f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 0x55d1e459f230 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0a9be0 tx=0x7fa98c010720 comp rx=0 tx=0).stop 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 shutdown_connections 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41b2d80 0x55d1e45ac830 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41f41b0 0x55d1e45a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41d34d0 0x55d1e459f230 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/2315997444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41cb2f0 0x55d1e459ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa977fff640 0 client.17019 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 >> 172.21.15.19:0/2315997444 conn(0x55d1e41e8300 msgr2=0x55d1e4593c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 shutdown_connections 2024-05-17T09:27:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/2315997444 wait complete. 2024-05-17T09:27:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.386+0000 7fa9a5fa8740 20 client.17019 trim_cache size 0 max 16384 2024-05-17T09:27:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 Processor -- start 2024-05-17T09:27:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 -- start start 2024-05-17T09:27:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981af2a0 con 0x55d1e41f41b0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981acb70 con 0x55d1e41d34d0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981aa9c0 con 0x55d1e41cb2f0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37276/0 (socket says 172.21.15.19:37276) 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 -- 172.21.15.19:0/736289450 learned_addr learned my addr 172.21.15.19:0/736289450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99cdba640 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa997fff640 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 -- 172.21.15.19:0/736289450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 msgr2=0x55d1e45942f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45942f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 -- 172.21.15.19:0/736289450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 msgr2=0x55d1e45a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.402+0000 7fa99d5bb640 1 -- 172.21.15.19:0/736289450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e45965a0 con 0x55d1e41f41b0 2024-05-17T09:27:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.403+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45987c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa990009ca0 tx=0x7fa990044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e719a03b6a6a82e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.403+0000 7fa97e7fc640 1 -- 172.21.15.19:0/736289450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990003c70 con 0x55d1e41f41b0 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.403+0000 7fa97e7fc640 1 -- 172.21.15.19:0/736289450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa990036050 con 0x55d1e41f41b0 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.403+0000 7fa97e7fc640 1 -- 172.21.15.19:0/736289450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9900309c0 con 0x55d1e41f41b0 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/736289450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 msgr2=0x55d1e45987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45987c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa990009ca0 tx=0x7fa990044000 comp rx=0 tx=0).stop 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/736289450 shutdown_connections 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45987c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/736289450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/736289450 >> 172.21.15.19:0/736289450 conn(0x55d1e41e8300 msgr2=0x55d1e4593be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/736289450 shutdown_connections 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/736289450 wait complete. 2024-05-17T09:27:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 -- start start 2024-05-17T09:27:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.404+0000 7fa9a5fa8740 1 Processor -- start 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ab030 con 0x55d1e41f41b0 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ac4a0 con 0x55d1e41d34d0 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ab700 con 0x55d1e41cb2f0 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa9a5fa8740 10 client.0 _ll_register_callbacks cb 0x7fa98c0a95a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37282/0 (socket says 172.21.15.19:37282) 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 -- 172.21.15.19:0/3355275432 learned_addr learned my addr 172.21.15.19:0/3355275432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa997fff640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99cdba640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 -- 172.21.15.19:0/3355275432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 msgr2=0x55d1e45a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 -- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 msgr2=0x55d1e45a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 -- 172.21.15.19:0/3355275432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e45a52a0 con 0x55d1e41f41b0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa997fff640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99cdba640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa99d5bb640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45a4ce0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa9900049d0 tx=0x7fa990031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c880792dce25f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.405+0000 7fa977fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa977fff640 1 -- 172.21.15.19:0/3355275432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99005b030 con 0x55d1e41f41b0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa977fff640 1 -- 172.21.15.19:0/3355275432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa9900306c0 con 0x55d1e41f41b0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa990036050 con 0x55d1e41f41b0 2024-05-17T09:27:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa977fff640 1 -- 172.21.15.19:0/3355275432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa990004b40 con 0x55d1e41f41b0 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 10 client.17025 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1e4614f00 con 0x55d1e41f41b0 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa977fff640 1 -- 172.21.15.19:0/3355275432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa990036050 con 0x55d1e41f41b0 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa977fff640 1 -- 172.21.15.19:0/3355275432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa990004250 con 0x55d1e41f41b0 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 20 client.17025 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41881c0 0x55d1e46177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 4 client.17025 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.406+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1e4503360 con 0x55d1e41881c0 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.407+0000 7fa99cdba640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41881c0 0x55d1e46177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.407+0000 7fa977fff640 10 client.17025 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.407+0000 7fa99cdba640 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41881c0 0x55d1e46177b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d24945b647e76f7 server_cookie=5cf811c250cd0faf in_seq=0 out_seq=0 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa977fff640 1 -- 172.21.15.19:0/3355275432 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1e4503360 con 0x55d1e41881c0 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa977fff640 10 client.17025 handle_command_reply: tid=0 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 client.17025 shutdown 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 msgr2=0x55d1e45a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45a4ce0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa9900049d0 tx=0x7fa990031750 comp rx=0 tx=0).stop 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 shutdown_connections 2024-05-17T09:27:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d1e41881c0 0x55d1e46177b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1e41f41b0 0x55d1e45a4ce0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1e41d34d0 0x55d1e45a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa977fff640 0 client.17025 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 --2- 172.21.15.19:0/3355275432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1e41cb2f0 0x55d1e45a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 >> 172.21.15.19:0/3355275432 conn(0x55d1e41e8300 msgr2=0x55d1e458f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 shutdown_connections 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 1 -- 172.21.15.19:0/3355275432 wait complete. 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.429+0000 7fa9a5fa8740 20 client.17025 trim_cache size 0 max 16384 2024-05-17T09:27:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:27:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:27:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:27:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9800b28a0 msgr2=0x7fa9800b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9800b28a0 0x7fa9800b4db0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55d1e4596570 tx=0x7fa990031000 comp rx=0 tx=0).stop 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 msgr2=0x7fa99819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99819a650 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00e9f0 tx=0x7fa98c00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 shutdown_connections 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9981051f0 0x7fa9981a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9800b28a0 0x7fa9800b4db0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9981023a0 0x7fa99819a650 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 --2- 172.21.15.19:0/3894676742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9981018a0 0x7fa99819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 >> 172.21.15.19:0/3894676742 conn(0x7fa9980fd4e0 msgr2=0x7fa998107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 shutdown_connections 2024-05-17T09:27:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:23.430+0000 7fa99f01e640 1 -- 172.21.15.19:0/3894676742 wait complete. 2024-05-17T09:27:23.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.84 MB/sec execute 201 sec latency 2434.878 ms 2024-05-17T09:27:24.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.83 MB/sec execute 202 sec latency 3434.906 ms 2024-05-17T09:27:24.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:24 smithi019 ceph-mon[31620]: pgmap v667: 129 pgs: 129 active+clean; 55 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:27:24.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:24 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:27:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:24 smithi079 ceph-mon[35502]: pgmap v667: 129 pgs: 129 active+clean; 55 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:27:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:24 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:27:24.793 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:24 smithi069 ceph-mon[35081]: pgmap v667: 129 pgs: 129 active+clean; 55 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:27:24.793 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:24 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:27:25.433 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.82 MB/sec execute 203 sec latency 4434.937 ms 2024-05-17T09:27:26.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.81 MB/sec execute 204 sec latency 5435.017 ms 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: pgmap v668: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:27:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:26 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: pgmap v668: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:27:26.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:27:26.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:26 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: pgmap v668: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:26.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:27:26.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:27:26.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:26 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:27:27.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.80 MB/sec execute 205 sec latency 6435.122 ms 2024-05-17T09:27:28.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.79 MB/sec execute 206 sec latency 7435.226 ms 2024-05-17T09:27:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:28 smithi079 ceph-mon[35502]: pgmap v669: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 921 KiB/s wr, 126 op/s 2024-05-17T09:27:28.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:28 smithi069 ceph-mon[35081]: pgmap v669: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 921 KiB/s wr, 126 op/s 2024-05-17T09:27:28.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:28 smithi019 ceph-mon[31620]: pgmap v669: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 921 KiB/s wr, 126 op/s 2024-05-17T09:27:29.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.78 MB/sec execute 207 sec latency 8435.344 ms 2024-05-17T09:27:30.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.77 MB/sec execute 208 sec latency 9435.463 ms 2024-05-17T09:27:30.732 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:30 smithi069 ceph-mon[35081]: pgmap v670: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 551 KiB/s wr, 81 op/s 2024-05-17T09:27:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:30 smithi079 ceph-mon[35502]: pgmap v670: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 551 KiB/s wr, 81 op/s 2024-05-17T09:27:30.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:30 smithi019 ceph-mon[31620]: pgmap v670: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 551 KiB/s wr, 81 op/s 2024-05-17T09:27:31.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.76 MB/sec execute 209 sec latency 10435.555 ms 2024-05-17T09:27:32.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.76 MB/sec execute 210 sec latency 11435.667 ms 2024-05-17T09:27:32.944 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:32 smithi069 ceph-mon[35081]: pgmap v671: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 353 KiB/s wr, 75 op/s 2024-05-17T09:27:32.944 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:32 smithi079 ceph-mon[35502]: pgmap v671: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 353 KiB/s wr, 75 op/s 2024-05-17T09:27:32.944 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:32 smithi019 ceph-mon[31620]: pgmap v671: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 353 KiB/s wr, 75 op/s 2024-05-17T09:27:33.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.75 MB/sec execute 211 sec latency 12435.759 ms 2024-05-17T09:27:34.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.74 MB/sec execute 212 sec latency 13435.880 ms 2024-05-17T09:27:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:34 smithi079 ceph-mon[35502]: pgmap v672: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 270 KiB/s wr, 49 op/s 2024-05-17T09:27:34.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:34 smithi069 ceph-mon[35081]: pgmap v672: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 270 KiB/s wr, 49 op/s 2024-05-17T09:27:34.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:34 smithi019 ceph-mon[31620]: pgmap v672: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 270 KiB/s wr, 49 op/s 2024-05-17T09:27:35.434 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.73 MB/sec execute 213 sec latency 14435.954 ms 2024-05-17T09:27:36.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.72 MB/sec execute 214 sec latency 15436.079 ms 2024-05-17T09:27:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 -- 172.21.15.19:0/3324931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 msgr2=0x7f22a4104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3324931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a4104960 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2294009900 tx=0x7f229402a410 comp rx=0 tx=0).stop 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 -- 172.21.15.19:0/3324931741 shutdown_connections 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3324931741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4109ba0 0x7f22a410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3324931741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4104ea0 0x7f22a41052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3324931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a4104960 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 -- 172.21.15.19:0/3324931741 >> 172.21.15.19:0/3324931741 conn(0x7f22a40fee50 msgr2=0x7f22a4101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 -- 172.21.15.19:0/3324931741 shutdown_connections 2024-05-17T09:27:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.544+0000 7f22a96c6640 1 -- 172.21.15.19:0/3324931741 wait complete. 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 Processor -- start 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 -- start start 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4104ea0 0x7f22a419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4109ba0 0x7f22a41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410f610 con 0x7f22a4103c60 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410f490 con 0x7f22a4104ea0 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.545+0000 7f22a96c6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410f790 con 0x7f22a4109ba0 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52204/0 (socket says 172.21.15.19:52204) 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 -- 172.21.15.19:0/3033055399 learned_addr learned my addr 172.21.15.19:0/3033055399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a27fc640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4104ea0 0x7f22a419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a37fe640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4109ba0 0x7f22a41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 -- 172.21.15.19:0/3033055399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4109ba0 msgr2=0x7f22a41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4109ba0 0x7f22a41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 -- 172.21.15.19:0/3033055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4104ea0 msgr2=0x7f22a419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4104ea0 0x7f22a419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 -- 172.21.15.19:0/3033055399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a41a1d00 con 0x7f22a4103c60 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a37fe640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4109ba0 0x7f22a41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a419a1c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2294005760 tx=0x7f2294005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f228bfff640 1 -- 172.21.15.19:0/3033055399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229403f030 con 0x7f22a4103c60 2024-05-17T09:27:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41a1f90 con 0x7f22a4103c60 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f228bfff640 1 -- 172.21.15.19:0/3033055399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22940043d0 con 0x7f22a4103c60 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.546+0000 7f228bfff640 1 -- 172.21.15.19:0/3033055399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294035070 con 0x7f22a4103c60 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.547+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a23f0 con 0x7f22a4103c60 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 Processor -- start 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 -- start start 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 0x55f72e6223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e622920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e629820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f72e594c30 con 0x55f72e2811b0 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f72e5847d0 con 0x55f72e2582f0 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22b0577740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f72e588810 con 0x55f72e2604d0 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 0x55f72e6223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f228bfff640 1 -- 172.21.15.19:0/3033055399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2294042030 con 0x7f22a4103c60 2024-05-17T09:27:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 0x55f72e6223a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52206/0 (socket says 172.21.15.19:52206) 2024-05-17T09:27:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.549+0000 7f22a37fe640 1 -- 172.21.15.19:0/3903106507 learned_addr learned my addr 172.21.15.19:0/3903106507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a27fc640 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e629820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a37fe640 1 -- 172.21.15.19:0/3903106507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 msgr2=0x55f72e629820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a37fe640 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e629820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e622920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a37fe640 1 -- 172.21.15.19:0/3903106507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 msgr2=0x55f72e622920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a37fe640 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e622920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a37fe640 1 -- 172.21.15.19:0/3903106507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72e299210 con 0x55f72e2811b0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e622920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22a37fe640 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 0x55f72e6223a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f229c009fd0 tx=0x7f229c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacf3b85cb6037ab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22897fa640 1 -- 172.21.15.19:0/3903106507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c011760 con 0x55f72e2811b0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22897fa640 1 -- 172.21.15.19:0/3903106507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f229c011d90 con 0x55f72e2811b0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.550+0000 7f22897fa640 1 -- 172.21.15.19:0/3903106507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c01be50 con 0x55f72e2811b0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f228bfff640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f228c0b28a0 0x7f228c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f228c0b28a0 0x7f228c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f228bfff640 1 -- 172.21.15.19:0/3033055399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2294102960 con 0x7f22a4103c60 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 -- 172.21.15.19:0/3903106507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 msgr2=0x55f72e6223a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 0x55f72e6223a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f229c009fd0 tx=0x7f229c0108e0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22a2ffd640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f228c0b28a0 0x7f228c0b4db0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22940824c0 tx=0x7f229402a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 -- 172.21.15.19:0/3903106507 shutdown_connections 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e629820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e622920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 --2- 172.21.15.19:0/3903106507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2811b0 0x55f72e6223a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 -- 172.21.15.19:0/3903106507 >> 172.21.15.19:0/3903106507 conn(0x55f72e275300 msgr2=0x55f72e61dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 -- 172.21.15.19:0/3903106507 shutdown_connections 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.551+0000 7f22b0577740 1 -- 172.21.15.19:0/3903106507 wait complete. 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 -- start start 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 Processor -- start 2024-05-17T09:27:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 0x55f72e62bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e62c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e633190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f72e26a790 con 0x55f72e2582f0 2024-05-17T09:27:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f72e540890 con 0x55f72e2811b0 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f72e62a270 con 0x55f72e2604d0 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22b0577740 10 client.0 _ll_register_callbacks cb 0x55f72e543d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 0x55f72e62bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 0x55f72e62bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52222/0 (socket says 172.21.15.19:52222) 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.552+0000 7f22a37fe640 1 -- 172.21.15.19:0/2062463192 learned_addr learned my addr 172.21.15.19:0/2062463192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e62c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e633190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 -- 172.21.15.19:0/2062463192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 msgr2=0x55f72e62c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e62c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 -- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 msgr2=0x55f72e633190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e633190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 -- 172.21.15.19:0/2062463192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72e365620 con 0x55f72e2582f0 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 0x55f72e62bcf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f229c011fd0 tx=0x7f229c011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c5ef52c0af5f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f2286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f2286ff5640 1 -- 172.21.15.19:0/2062463192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c011970 con 0x55f72e2582f0 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f2286ff5640 1 -- 172.21.15.19:0/2062463192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f229c011b10 con 0x55f72e2582f0 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c011d90 con 0x55f72e2582f0 2024-05-17T09:27:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f2286ff5640 1 -- 172.21.15.19:0/2062463192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f229c033050 con 0x55f72e2582f0 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22b0577740 10 client.17037 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.553+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f72e636030 con 0x55f72e2582f0 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f2286ff5640 1 -- 172.21.15.19:0/2062463192 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f229c011d90 con 0x55f72e2582f0 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f2286ff5640 1 -- 172.21.15.19:0/2062463192 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f229c0283f0 con 0x55f72e2582f0 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22b0577740 10 client.17037 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22b0577740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22b0577740 20 client.17037 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22b0577740 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e23fd80 0x55f72e6394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22b0577740 4 client.17037 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72e596b80 con 0x55f72e23fd80 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.554+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e23fd80 0x55f72e6394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.555+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e23fd80 0x55f72e6394a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570fede0d15645f1 server_cookie=e3ee0b11b27ce37e in_seq=0 out_seq=0 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.555+0000 7f2286ff5640 10 client.17037 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.555+0000 7f2286ff5640 1 -- 172.21.15.19:0/2062463192 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f72e596b80 con 0x55f72e23fd80 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.555+0000 7f2286ff5640 10 client.17037 handle_command_reply: tid=0 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.555+0000 7f22b0577740 1 client.17037 shutdown 2024-05-17T09:27:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 msgr2=0x55f72e62bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 0x55f72e62bcf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f229c011fd0 tx=0x7f229c011f40 comp rx=0 tx=0).stop 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 shutdown_connections 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e23fd80 0x55f72e6394a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e633190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2604d0 0x55f72e62c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 --2- 172.21.15.19:0/2062463192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2582f0 0x55f72e62bcf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f2286ff5640 0 client.17037 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 >> 172.21.15.19:0/2062463192 conn(0x55f72e275300 msgr2=0x55f72e61cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 shutdown_connections 2024-05-17T09:27:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 1 -- 172.21.15.19:0/2062463192 wait complete. 2024-05-17T09:27:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.556+0000 7f22b0577740 20 client.17037 trim_cache size 0 max 16384 2024-05-17T09:27:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 Processor -- start 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 -- start start 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2582f0 0x55f72e626890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e621240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e621780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41af2d0 con 0x55f72e2604d0 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41acba0 con 0x55f72e2811b0 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.571+0000 7f22b0577740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41aa9f0 con 0x55f72e2582f0 2024-05-17T09:27:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e621240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e621240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52238/0 (socket says 172.21.15.19:52238) 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 -- 172.21.15.19:0/2518836711 learned_addr learned my addr 172.21.15.19:0/2518836711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e621780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2582f0 0x55f72e626890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 -- 172.21.15.19:0/2518836711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2582f0 msgr2=0x55f72e626890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2582f0 0x55f72e626890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 -- 172.21.15.19:0/2518836711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 msgr2=0x55f72e621780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e621780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 -- 172.21.15.19:0/2518836711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72e6191b0 con 0x55f72e2604d0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e621240 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f22980058c0 tx=0x7f2298005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f674ceb503b1d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f2288ff9640 1 -- 172.21.15.19:0/2518836711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298003cd0 con 0x55f72e2604d0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f2288ff9640 1 -- 172.21.15.19:0/2518836711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22980108c0 con 0x55f72e2604d0 2024-05-17T09:27:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.572+0000 7f2288ff9640 1 -- 172.21.15.19:0/2518836711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229801f760 con 0x55f72e2604d0 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 -- 172.21.15.19:0/2518836711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 msgr2=0x55f72e621240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e621240 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f22980058c0 tx=0x7f2298005f90 comp rx=0 tx=0).stop 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 -- 172.21.15.19:0/2518836711 shutdown_connections 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2811b0 0x55f72e621780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e621240 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 --2- 172.21.15.19:0/2518836711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2582f0 0x55f72e626890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 -- 172.21.15.19:0/2518836711 >> 172.21.15.19:0/2518836711 conn(0x55f72e275300 msgr2=0x55f72e620b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 -- 172.21.15.19:0/2518836711 shutdown_connections 2024-05-17T09:27:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 -- 172.21.15.19:0/2518836711 wait complete. 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.573+0000 7f22b0577740 1 -- start start 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 Processor -- start 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e69cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e69d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2811b0 0x55f72e6a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ab060 con 0x55f72e2604d0 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ac4d0 con 0x55f72e2582f0 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ab730 con 0x55f72e2811b0 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22b0577740 10 client.0 _ll_register_callbacks cb 0x7f229c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e69d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e69d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52246/0 (socket says 172.21.15.19:52246) 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 -- 172.21.15.19:0/2229519378 learned_addr learned my addr 172.21.15.19:0/2229519378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2811b0 0x55f72e6a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e69cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 -- 172.21.15.19:0/2229519378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2811b0 msgr2=0x55f72e6a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2811b0 0x55f72e6a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 -- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 msgr2=0x55f72e69cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e69cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.574+0000 7f22a27fc640 1 -- 172.21.15.19:0/2229519378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72e6a4630 con 0x55f72e2604d0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22a27fc640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e69d180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2298005bc0 tx=0x7f2298010600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e757bdcdc580c556 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f2286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f2286ff5640 1 -- 172.21.15.19:0/2229519378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298003ce0 con 0x55f72e2604d0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f2286ff5640 1 -- 172.21.15.19:0/2229519378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2298010a70 con 0x55f72e2604d0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22980108c0 con 0x55f72e2604d0 2024-05-17T09:27:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f2286ff5640 1 -- 172.21.15.19:0/2229519378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f229803c050 con 0x55f72e2604d0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 10 client.17043 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f72e6a5860 con 0x55f72e2604d0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f2286ff5640 1 -- 172.21.15.19:0/2229519378 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22980108c0 con 0x55f72e2604d0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f2286ff5640 1 -- 172.21.15.19:0/2229519378 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2298028440 con 0x55f72e2604d0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 20 client.17043 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e2151c0 0x55f72e6a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 4 client.17043 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.575+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f72e590360 con 0x55f72e2151c0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.576+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e2151c0 0x55f72e6a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.576+0000 7f22a37fe640 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e2151c0 0x55f72e6a81d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b45b8fe8ff88a0 server_cookie=5de574ec8f2a7a6c in_seq=0 out_seq=0 2024-05-17T09:27:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.576+0000 7f2286ff5640 10 client.17043 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.597+0000 7f2286ff5640 1 -- 172.21.15.19:0/2229519378 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f72e590360 con 0x55f72e2151c0 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.597+0000 7f2286ff5640 10 client.17043 handle_command_reply: tid=0 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.597+0000 7f22b0577740 1 client.17043 shutdown 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 msgr2=0x55f72e69d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e69d180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2298005bc0 tx=0x7f2298010600 comp rx=0 tx=0).stop 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 shutdown_connections 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f72e2151c0 0x55f72e6a81d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f72e2811b0 0x55f72e6a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f72e2604d0 0x55f72e69d180 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f2286ff5640 0 client.17043 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 --2- 172.21.15.19:0/2229519378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f72e2582f0 0x55f72e69cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 >> 172.21.15.19:0/2229519378 conn(0x55f72e275300 msgr2=0x55f72e621560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 shutdown_connections 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 1 -- 172.21.15.19:0/2229519378 wait complete. 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22b0577740 20 client.17043 trim_cache size 0 max 16384 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:27:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:27:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:27:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f228c0b28a0 msgr2=0x7f228c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f228c0b28a0 0x7f228c0b4db0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22940824c0 tx=0x7f229402a410 comp rx=0 tx=0).stop 2024-05-17T09:27:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 msgr2=0x7f22a419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.598+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a419a1c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2294005760 tx=0x7f2294005e20 comp rx=0 tx=0).stop 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 shutdown_connections 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22a4109ba0 0x7f22a41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22a4104ea0 0x7f22a419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f228c0b28a0 0x7f228c0b4db0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 --2- 172.21.15.19:0/3033055399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a4103c60 0x7f22a419a1c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 >> 172.21.15.19:0/3033055399 conn(0x7f22a40fee50 msgr2=0x7f22a4105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 shutdown_connections 2024-05-17T09:27:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:36.599+0000 7f22a96c6640 1 -- 172.21.15.19:0/3033055399 wait complete. 2024-05-17T09:27:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:36 smithi079 ceph-mon[35502]: pgmap v673: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 83 KiB/s wr, 20 op/s 2024-05-17T09:27:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:36 smithi069 ceph-mon[35081]: pgmap v673: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 83 KiB/s wr, 20 op/s 2024-05-17T09:27:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:36 smithi019 ceph-mon[31620]: pgmap v673: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 83 KiB/s wr, 20 op/s 2024-05-17T09:27:37.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.72 MB/sec execute 215 sec latency 16436.189 ms 2024-05-17T09:27:37.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:37 smithi079 ceph-mon[35502]: pgmap v674: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-05-17T09:27:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:37 smithi069 ceph-mon[35081]: pgmap v674: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-05-17T09:27:37.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:37 smithi019 ceph-mon[31620]: pgmap v674: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-05-17T09:27:38.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.71 MB/sec execute 216 sec latency 17436.251 ms 2024-05-17T09:27:38.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:27:38.792 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:27:39.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.70 MB/sec execute 217 sec latency 18436.346 ms 2024-05-17T09:27:40.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:40 smithi019 ceph-mon[31620]: pgmap v675: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-17T09:27:40.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:40 smithi069 ceph-mon[35081]: pgmap v675: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-17T09:27:40.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.69 MB/sec execute 218 sec latency 19436.389 ms 2024-05-17T09:27:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:40 smithi079 ceph-mon[35502]: pgmap v675: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3589801552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 msgr2=0x7f2a94110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3589801552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a94110090 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b0d0 tx=0x7f2a8c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3589801552 shutdown_connections 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3589801552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a941034b0 0x7f2a941038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3589801552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a94110090 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3589801552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a94107de0 0x7f2a941081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3589801552 >> 172.21.15.19:0/3589801552 conn(0x7f2a940fede0 msgr2=0x7f2a94101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3589801552 shutdown_connections 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.648+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3589801552 wait complete. 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.649+0000 7f2a99a2c700 1 Processor -- start 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.649+0000 7f2a99a2c700 1 -- start start 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a99a2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a941034b0 0x7f2a940ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a99a2c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a94107de0 0x7f2a940ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a99a2c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a940ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a99a2c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94113230 con 0x7f2a94108730 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a99a2c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941130b0 con 0x7f2a94107de0 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a99a2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941133b0 con 0x7f2a941034b0 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a940ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a940ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52268/0 (socket says 172.21.15.19:52268) 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 -- 172.21.15.19:0/3827923713 learned_addr learned my addr 172.21.15.19:0/3827923713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a92ffd700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a941034b0 0x7f2a940ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a927fc700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a94107de0 0x7f2a940ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 -- 172.21.15.19:0/3827923713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a941034b0 msgr2=0x7f2a940ff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a941034b0 0x7f2a940ff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 -- 172.21.15.19:0/3827923713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a94107de0 msgr2=0x7f2a940ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a94107de0 0x7f2a940ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 -- 172.21.15.19:0/3827923713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a941a90b0 con 0x7f2a94108730 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a937fe700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a940ffd40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009850 tx=0x7f2a8c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.650+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c01f070 con 0x7f2a94108730 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.651+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a8c005500 con 0x7f2a94108730 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.651+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a941a9340 con 0x7f2a94108730 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.651+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c01b8f0 con 0x7f2a94108730 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.651+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941a9790 con 0x7f2a94108730 2024-05-17T09:27:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.652+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a940512b0 con 0x7f2a94108730 2024-05-17T09:27:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.653+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2a8c022070 con 0x7f2a94108730 2024-05-17T09:27:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.654+0000 7f2a98a2a700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a7c0b2880 0x7f2a7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.654+0000 7f2a92ffd700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a7c0b2880 0x7f2a7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.654+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2a8c00f020 con 0x7f2a94108730 2024-05-17T09:27:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.655+0000 7f2a92ffd700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a7c0b2880 0x7f2a7c0b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a84004630 tx=0x7f2a8400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:40.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.659+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a8c029020 con 0x7f2a94108730 2024-05-17T09:27:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.932+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a9402e5e0 con 0x7f2a94108730 2024-05-17T09:27:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.932+0000 7f2a98a2a700 1 -- 172.21.15.19:0/3827923713 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f2a8c0ab370 con 0x7f2a94108730 2024-05-17T09:27:40.933 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:27:40.933 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:27:40.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a7c0b2880 msgr2=0x7f2a7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:40.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a7c0b2880 0x7f2a7c0b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a84004630 tx=0x7f2a8400b3c0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 msgr2=0x7f2a940ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a940ffd40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009850 tx=0x7f2a8c007d90 comp rx=0 tx=0).stop 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 shutdown_connections 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a7c0b2880 0x7f2a7c0b4d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a941034b0 0x7f2a940ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a94108730 0x7f2a940ffd40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 --2- 172.21.15.19:0/3827923713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a94107de0 0x7f2a940ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 >> 172.21.15.19:0/3827923713 conn(0x7f2a940fede0 msgr2=0x7f2a94075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 shutdown_connections 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:40.935+0000 7f2a99a2c700 1 -- 172.21.15.19:0/3827923713 wait complete. 2024-05-17T09:27:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:27:41.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:41 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3827923713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:27:41.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:41 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3827923713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:27:41.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.68 MB/sec execute 219 sec latency 20436.471 ms 2024-05-17T09:27:41.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:41 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3827923713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:27:41.698 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id de0eed95 --release --await' 2024-05-17T09:27:41.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id de0eed95 --release --await 2024-05-17T09:27:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:27:42.348 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:42 smithi019 ceph-mon[31620]: pgmap v676: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-05-17T09:27:42.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:42 smithi069 ceph-mon[35081]: pgmap v676: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-05-17T09:27:42.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.68 MB/sec execute 220 sec latency 21436.537 ms 2024-05-17T09:27:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:42 smithi079 ceph-mon[35502]: pgmap v676: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-05-17T09:27:43.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111599 1.67 MB/sec execute 221 sec latency 22436.622 ms 2024-05-17T09:27:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 -- 172.21.15.19:0/2483286091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 msgr2=0x7f9324103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 --2- 172.21.15.19:0/2483286091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f9324103920 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f93180099c0 tx=0x7f9318009cd0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 -- 172.21.15.19:0/2483286091 shutdown_connections 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 --2- 172.21.15.19:0/2483286091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9324108740 0x7f93241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 --2- 172.21.15.19:0/2483286091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f9324103920 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 --2- 172.21.15.19:0/2483286091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9324107df0 0x7f9324108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 -- 172.21.15.19:0/2483286091 >> 172.21.15.19:0/2483286091 conn(0x7f93240fef80 msgr2=0x7f93241013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 -- 172.21.15.19:0/2483286091 shutdown_connections 2024-05-17T09:27:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.857+0000 7f932bdad700 1 -- 172.21.15.19:0/2483286091 wait complete. 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 Processor -- start 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 -- start start 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f932419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9324107df0 0x7f932419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9324108740 0x7f932419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9324113220 con 0x7f9324103510 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93241130a0 con 0x7f9324107df0 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f932bdad700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93241133a0 con 0x7f9324108740 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f9329b49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f932419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f9329b49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f932419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36384/0 (socket says 172.21.15.19:36384) 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.858+0000 7f9329b49700 1 -- 172.21.15.19:0/1254937250 learned_addr learned my addr 172.21.15.19:0/1254937250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329348700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9324107df0 0x7f932419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f932a34a700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9324108740 0x7f932419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329b49700 1 -- 172.21.15.19:0/1254937250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9324108740 msgr2=0x7f932419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329b49700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9324108740 0x7f932419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329b49700 1 -- 172.21.15.19:0/1254937250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9324107df0 msgr2=0x7f932419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329b49700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9324107df0 0x7f932419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329b49700 1 -- 172.21.15.19:0/1254937250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932419fb80 con 0x7f9324103510 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9329b49700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f932419ad60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f93180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9316ffd700 1 -- 172.21.15.19:0/1254937250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318004830 con 0x7f9324103510 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9316ffd700 1 -- 172.21.15.19:0/1254937250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f931800f7f0 con 0x7f9324103510 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932419fe10 con 0x7f9324103510 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f9316ffd700 1 -- 172.21.15.19:0/1254937250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93180053a0 con 0x7f9324103510 2024-05-17T09:27:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.859+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241a02e0 con 0x7f9324103510 2024-05-17T09:27:43.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.863+0000 7f9316ffd700 1 -- 172.21.15.19:0/1254937250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9318029060 con 0x7f9324103510 2024-05-17T09:27:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f9316ffd700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f93100b2900 0x7f93100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f933a7c2b80 1 Processor -- start 2024-05-17T09:27:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f9329348700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f93100b2900 0x7f93100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f9316ffd700 1 -- 172.21.15.19:0/1254937250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f93180e3180 con 0x7f9324103510 2024-05-17T09:27:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f933a7c2b80 1 -- start start 2024-05-17T09:27:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f9329348700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f93100b2900 0x7f93100b4e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9324107a30 tx=0x7f932000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b536d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.864+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b51e8f40 0x5611b536daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b5367620 0x5611b5367a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611b5295d90 con 0x5611b51e8f40 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5611b5285930 con 0x5611b52664d0 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5611b5289970 con 0x5611b5367620 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329348700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b51e8f40 0x5611b536daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b536d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329348700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b51e8f40 0x5611b536daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36394/0 (socket says 172.21.15.19:36394) 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329348700 1 -- 172.21.15.19:0/2188328089 learned_addr learned my addr 172.21.15.19:0/2188328089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f932a34a700 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b5367620 0x5611b5367a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 -- 172.21.15.19:0/2188328089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b5367620 msgr2=0x5611b5367a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b5367620 0x5611b5367a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 -- 172.21.15.19:0/2188328089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b51e8f40 msgr2=0x5611b536daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b51e8f40 0x5611b536daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 -- 172.21.15.19:0/2188328089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b51d59d0 con 0x5611b52664d0 2024-05-17T09:27:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f9329b49700 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b536d560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93180a7690 tx=0x7f93180a9fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69114bfdb8e463cb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f930ffff700 1 -- 172.21.15.19:0/2188328089 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93180ae7d0 con 0x5611b52664d0 2024-05-17T09:27:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.865+0000 7f930ffff700 1 -- 172.21.15.19:0/2188328089 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93180b31f0 con 0x5611b52664d0 2024-05-17T09:27:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f930ffff700 1 -- 172.21.15.19:0/2188328089 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93180aa8e0 con 0x5611b52664d0 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2188328089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 msgr2=0x5611b536d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b536d560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93180a7690 tx=0x7f93180a9fb0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2188328089 shutdown_connections 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b5367620 0x5611b5367a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b536d560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2188328089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b51e8f40 0x5611b536daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.866+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2188328089 >> 172.21.15.19:0/2188328089 conn(0x5611b5227d80 msgr2=0x5611b5362520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2188328089 shutdown_connections 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2188328089 wait complete. 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 -- start start 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 Processor -- start 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b5376c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 0x5611b5377140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.867+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b5367620 0x5611b5371000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611b518c0a0 con 0x5611b52664d0 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5611b52278c0 con 0x5611b5367620 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5611b5285930 con 0x5611b51e8f40 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f933a7c2b80 10 client.0 _ll_register_callbacks cb 0x5611b5220a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 0x5611b5377140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 0x5611b5377140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36410/0 (socket says 172.21.15.19:36410) 2024-05-17T09:27:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 -- 172.21.15.19:0/2675927202 learned_addr learned my addr 172.21.15.19:0/2675927202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f932a34a700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b5367620 0x5611b5371000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329b49700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b5376c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 -- 172.21.15.19:0/2675927202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 msgr2=0x5611b5376c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b5376c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 -- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b5367620 msgr2=0x5611b5371000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b5367620 0x5611b5371000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 -- 172.21.15.19:0/2675927202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611b5224790 con 0x5611b52664d0 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f9329348700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 0x5611b5377140 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f93200116f0 tx=0x7f93200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744113e467ff6d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f930d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f930d7fa700 1 -- 172.21.15.19:0/2675927202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320004a30 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.868+0000 7f930d7fa700 1 -- 172.21.15.19:0/2675927202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9320004bd0 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93180b31f0 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f930d7fa700 1 -- 172.21.15.19:0/2675927202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9320030050 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 10 client.17061 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5611b537e880 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f930d7fa700 1 -- 172.21.15.19:0/2675927202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93180b31f0 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f930d7fa700 1 -- 172.21.15.19:0/2675927202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f932001f4c0 con 0x5611b52664d0 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 10 client.17061 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 20 client.17061 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b5380ec0 0x5611b53832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 4 client.17061 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.869+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611b5297ce0 con 0x5611b5380ec0 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.870+0000 7f9329b49700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b5380ec0 0x5611b53832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.870+0000 7f9329b49700 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b5380ec0 0x5611b53832e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21d299628b92f5f server_cookie=bfdc871dcee63aaa in_seq=0 out_seq=0 2024-05-17T09:27:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.870+0000 7f930d7fa700 10 client.17061 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:43.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f930d7fa700 1 -- 172.21.15.19:0/2675927202 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5611b5297ce0 con 0x5611b5380ec0 2024-05-17T09:27:43.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f930d7fa700 10 client.17061 handle_command_reply: tid=0 2024-05-17T09:27:43.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 client.17061 shutdown 2024-05-17T09:27:43.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 msgr2=0x5611b5377140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 0x5611b5377140 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f93200116f0 tx=0x7f93200047c0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 shutdown_connections 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b5376c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b5380ec0 0x5611b53832e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b5367620 0x5611b5371000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2675927202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b52664d0 0x5611b5377140 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f930d7fa700 0 client.17061 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 >> 172.21.15.19:0/2675927202 conn(0x5611b5227d80 msgr2=0x5611b535c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 shutdown_connections 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2675927202 wait complete. 2024-05-17T09:27:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.871+0000 7f933a7c2b80 20 client.17061 trim_cache size 0 max 16384 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 Processor -- start 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 -- start start 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b535c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b5388a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b5388f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93241b2ac0 con 0x5611b5367620 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93241b0350 con 0x5611b52664d0 2024-05-17T09:27:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93241ae1a0 con 0x5611b51e8f40 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b5388f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b5388f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36426/0 (socket says 172.21.15.19:36426) 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 -- 172.21.15.19:0/748240239 learned_addr learned my addr 172.21.15.19:0/748240239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f9329348700 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b5388a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f9329b49700 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b535c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 -- 172.21.15.19:0/748240239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 msgr2=0x5611b535c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b535c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 -- 172.21.15.19:0/748240239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 msgr2=0x5611b5388a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b5388a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f932a34a700 1 -- 172.21.15.19:0/748240239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b5377490 con 0x5611b5367620 2024-05-17T09:27:43.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.892+0000 7f9329348700 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b5388a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:27:43.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.893+0000 7f932a34a700 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b5388f80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f931c013400 tx=0x7f931c007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcf1274344f7101 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:43.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.893+0000 7f930f7fe700 1 -- 172.21.15.19:0/748240239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c01a030 con 0x5611b5367620 2024-05-17T09:27:43.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.893+0000 7f930f7fe700 1 -- 172.21.15.19:0/748240239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f931c013d50 con 0x5611b5367620 2024-05-17T09:27:43.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.893+0000 7f930f7fe700 1 -- 172.21.15.19:0/748240239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c01e4b0 con 0x5611b5367620 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 -- 172.21.15.19:0/748240239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 msgr2=0x5611b5388f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b5388f80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f931c013400 tx=0x7f931c007950 comp rx=0 tx=0).stop 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 -- 172.21.15.19:0/748240239 shutdown_connections 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b51e8f40 0x5611b535c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b5388f80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/748240239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b52664d0 0x5611b5388a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 -- 172.21.15.19:0/748240239 >> 172.21.15.19:0/748240239 conn(0x5611b5227d80 msgr2=0x5611b535c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 -- 172.21.15.19:0/748240239 shutdown_connections 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 -- 172.21.15.19:0/748240239 wait complete. 2024-05-17T09:27:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 -- start start 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.894+0000 7f933a7c2b80 1 Processor -- start 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b51e8f40 0x5611b537d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b52664d0 0x5611b537b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b537bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93241ae810 con 0x5611b5367620 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93241afc80 con 0x5611b51e8f40 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93241aeee0 con 0x5611b52664d0 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f933a7c2b80 10 client.0 _ll_register_callbacks cb 0x5611b5220a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b537bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b537bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36436/0 (socket says 172.21.15.19:36436) 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 -- 172.21.15.19:0/2771132570 learned_addr learned my addr 172.21.15.19:0/2771132570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f9329348700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b52664d0 0x5611b537b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f9329b49700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b51e8f40 0x5611b537d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 -- 172.21.15.19:0/2771132570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b52664d0 msgr2=0x5611b537b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b52664d0 0x5611b537b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 -- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b51e8f40 msgr2=0x5611b537d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b51e8f40 0x5611b537d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 -- 172.21.15.19:0/2771132570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611b537c090 con 0x5611b5367620 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f932a34a700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b537bb50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f931c016eb0 tx=0x7f931c016680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545f38911d79ea0e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.895+0000 7f930d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f930d7fa700 1 -- 172.21.15.19:0/2771132570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c01a070 con 0x5611b5367620 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f930d7fa700 1 -- 172.21.15.19:0/2771132570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f931c0168a0 con 0x5611b5367620 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931c013d50 con 0x5611b5367620 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f930d7fa700 1 -- 172.21.15.19:0/2771132570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f931c016aa0 con 0x5611b5367620 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 10 client.17067 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5611b53f0900 con 0x5611b5367620 2024-05-17T09:27:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f930d7fa700 1 -- 172.21.15.19:0/2771132570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f931c013d50 con 0x5611b5367620 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f930d7fa700 1 -- 172.21.15.19:0/2771132570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f931c028430 con 0x5611b5367620 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 10 client.17067 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 10 client.17067 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 20 client.17067 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b53f0f80 0x5611b53f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 4 client.17067 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "de0eed95", "release": true, "await": true}] 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.896+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "de0eed95", "release": true, "await": true}) -- 0x5611b52914c0 con 0x5611b53f0f80 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.897+0000 7f9329b49700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b53f0f80 0x5611b53f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.897+0000 7f9329b49700 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b53f0f80 0x5611b53f33a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a61d304ef6b4ed server_cookie=37001989f319bb9b in_seq=0 out_seq=0 2024-05-17T09:27:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.897+0000 7f930d7fa700 10 client.17067 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:43.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.901+0000 7f930d7fa700 1 -- 172.21.15.19:0/2771132570 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5611b52914c0 con 0x5611b53f0f80 2024-05-17T09:27:43.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.901+0000 7f930d7fa700 10 client.17067 handle_command_reply: tid=0 2024-05-17T09:27:43.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.901+0000 7f933a7c2b80 1 client.17067 shutdown 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.901+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 msgr2=0x5611b537bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.901+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b537bb50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f931c016eb0 tx=0x7f931c016680 comp rx=0 tx=0).stop 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 shutdown_connections 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5611b53f0f80 0x5611b53f33a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611b5367620 0x5611b537bb50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5611b51e8f40 0x5611b537d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 --2- 172.21.15.19:0/2771132570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5611b52664d0 0x5611b537b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f930d7fa700 0 client.17067 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 >> 172.21.15.19:0/2771132570 conn(0x5611b5227d80 msgr2=0x5611b5366610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 shutdown_connections 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 1 -- 172.21.15.19:0/2771132570 wait complete. 2024-05-17T09:27:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f933a7c2b80 20 client.17067 trim_cache size 0 max 16384 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 28, 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "de0eed95": { 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 28, 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:27:43.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 143.1, 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:27:43.905 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:27:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f93100b2900 msgr2=0x7f93100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f932bdad700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f93100b2900 0x7f93100b4e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9324107a30 tx=0x7f932000b410 comp rx=0 tx=0).stop 2024-05-17T09:27:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 msgr2=0x7f932419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.902+0000 7f932bdad700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f932419ad60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f93180047b0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 shutdown_connections 2024-05-17T09:27:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f93100b2900 0x7f93100b4e10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9324108740 0x7f932419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9324103510 0x7f932419ad60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 --2- 172.21.15.19:0/1254937250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9324107df0 0x7f932419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 >> 172.21.15.19:0/1254937250 conn(0x7f93240fef80 msgr2=0x7f93241063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 shutdown_connections 2024-05-17T09:27:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:27:43.903+0000 7f932bdad700 1 -- 172.21.15.19:0/1254937250 wait complete. 2024-05-17T09:27:44.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:44 smithi069 ceph-mon[35081]: pgmap v677: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-05-17T09:27:44.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111888 1.66 MB/sec execute 222 sec latency 22904.356 ms 2024-05-17T09:27:44.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:44 smithi019 ceph-mon[31620]: pgmap v677: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-05-17T09:27:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:44 smithi079 ceph-mon[35502]: pgmap v677: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-05-17T09:27:44.628 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'de0eed95', total seconds elapsed: 29.2 2024-05-17T09:27:45.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 112441 1.66 MB/sec execute 223 sec latency 12.626 ms 2024-05-17T09:27:46.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:46 smithi069 ceph-mon[35081]: pgmap v678: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-17T09:27:46.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 113172 1.68 MB/sec execute 224 sec latency 15.764 ms 2024-05-17T09:27:46.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:46 smithi019 ceph-mon[31620]: pgmap v678: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-17T09:27:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:46 smithi079 ceph-mon[35502]: pgmap v678: 129 pgs: 129 active+clean; 55 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-17T09:27:47.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 113614 1.69 MB/sec execute 225 sec latency 14.072 ms 2024-05-17T09:27:48.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:48 smithi069 ceph-mon[35081]: pgmap v679: 129 pgs: 129 active+clean; 56 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 148 KiB/s wr, 28 op/s 2024-05-17T09:27:48.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 113911 1.69 MB/sec execute 226 sec latency 16.046 ms 2024-05-17T09:27:48.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:48 smithi019 ceph-mon[31620]: pgmap v679: 129 pgs: 129 active+clean; 56 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 148 KiB/s wr, 28 op/s 2024-05-17T09:27:48.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:48 smithi079 ceph-mon[35502]: pgmap v679: 129 pgs: 129 active+clean; 56 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 148 KiB/s wr, 28 op/s 2024-05-17T09:27:49.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 114169 1.69 MB/sec execute 227 sec latency 14.338 ms 2024-05-17T09:27:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 -- 172.21.15.19:0/1781365988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20102b40 msgr2=0x7f6e20103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/1781365988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20102b40 0x7f6e20103840 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009900 tx=0x7f6e1002a410 comp rx=0 tx=0).stop 2024-05-17T09:27:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 -- 172.21.15.19:0/1781365988 shutdown_connections 2024-05-17T09:27:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/1781365988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 0x7f6e2010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/1781365988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20103d80 0x7f6e20104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/1781365988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20102b40 0x7f6e20103840 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 -- 172.21.15.19:0/1781365988 >> 172.21.15.19:0/1781365988 conn(0x7f6e200fdd30 msgr2=0x7f6e201001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 -- 172.21.15.19:0/1781365988 shutdown_connections 2024-05-17T09:27:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.711+0000 7f6e25aa8640 1 -- 172.21.15.19:0/1781365988 wait complete. 2024-05-17T09:27:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 Processor -- start 2024-05-17T09:27:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 -- start start 2024-05-17T09:27:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20102b40 0x7f6e2006b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 0x7f6e2019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 0x7f6e201a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010e4f0 con 0x7f6e20103d80 2024-05-17T09:27:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010e370 con 0x7f6e20108a80 2024-05-17T09:27:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.712+0000 7f6e25aa8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010e670 con 0x7f6e20102b40 2024-05-17T09:27:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 0x7f6e2019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20102b40 0x7f6e2006b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 0x7f6e2019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36460/0 (socket says 172.21.15.19:36460) 2024-05-17T09:27:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 0x7f6e201a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 -- 172.21.15.19:0/2143951367 learned_addr learned my addr 172.21.15.19:0/2143951367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 -- 172.21.15.19:0/2143951367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20102b40 msgr2=0x7f6e2006b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20102b40 0x7f6e2006b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 -- 172.21.15.19:0/2143951367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 msgr2=0x7f6e201a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 0x7f6e201a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 -- 172.21.15.19:0/2143951367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e201a1d50 con 0x7f6e20103d80 2024-05-17T09:27:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20102b40 0x7f6e2006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1ffff640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 0x7f6e201a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 0x7f6e2019efc0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9f0 tx=0x7f6e0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1d7fa640 1 -- 172.21.15.19:0/2143951367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800cd80 con 0x7f6e20103d80 2024-05-17T09:27:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201a1fe0 con 0x7f6e20103d80 2024-05-17T09:27:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1d7fa640 1 -- 172.21.15.19:0/2143951367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e080054b0 con 0x7f6e20103d80 2024-05-17T09:27:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e1d7fa640 1 -- 172.21.15.19:0/2143951367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08010430 con 0x7f6e20103d80 2024-05-17T09:27:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.713+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a24a0 con 0x7f6e20103d80 2024-05-17T09:27:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 Processor -- start 2024-05-17T09:27:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 -- start start 2024-05-17T09:27:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bc63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bcd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e1d7fa640 1 -- 172.21.15.19:0/2143951367 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6e080105f0 con 0x7f6e20103d80 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d47b38c30 con 0x555d477fc2f0 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555d47b287d0 con 0x555d478044d0 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.716+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555d47b2c810 con 0x555d478251b0 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bc6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36474/0 (socket says 172.21.15.19:36474) 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/4085045593 learned_addr learned my addr 172.21.15.19:0/4085045593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bcd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1ffff640 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bc63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/4085045593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 msgr2=0x555d47bc63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bc63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/4085045593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 msgr2=0x555d47bcd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bcd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/4085045593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4783d210 con 0x555d477fc2f0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bc6920 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6e10005ab0 tx=0x7f6e10005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf41d365f85b05b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e15ffb640 1 -- 172.21.15.19:0/4085045593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10035030 con 0x555d477fc2f0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e15ffb640 1 -- 172.21.15.19:0/4085045593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e10040070 con 0x555d477fc2f0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e1d7fa640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e0c0b28a0 0x7f6e0c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.717+0000 7f6e15ffb640 1 -- 172.21.15.19:0/4085045593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10003af0 con 0x555d477fc2f0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e1ffff640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e0c0b28a0 0x7f6e0c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e1d7fa640 1 -- 172.21.15.19:0/2143951367 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6e080e2930 con 0x7f6e20103d80 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 -- 172.21.15.19:0/4085045593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 msgr2=0x555d47bc6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bc6920 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6e10005ab0 tx=0x7f6e10005d50 comp rx=0 tx=0).stop 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e1ffff640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e0c0b28a0 0x7f6e0c0b4db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x555d47bc7570 tx=0x7f6e18009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 -- 172.21.15.19:0/4085045593 shutdown_connections 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bcd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bc6920 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/4085045593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bc63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 -- 172.21.15.19:0/4085045593 >> 172.21.15.19:0/4085045593 conn(0x555d47819300 msgr2=0x555d47bc1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.718+0000 7f6e2ca37740 1 -- 172.21.15.19:0/4085045593 shutdown_connections 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 -- 172.21.15.19:0/4085045593 wait complete. 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 -- start start 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 Processor -- start 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bcfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bd01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bd7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d4780e790 con 0x555d477fc2f0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555d47ae4890 con 0x555d478044d0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.719+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555d47bce210 con 0x555d478251b0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e2ca37740 10 client.0 _ll_register_callbacks cb 0x555d47ae7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bcfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bcfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36488/0 (socket says 172.21.15.19:36488) 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/3082332673 learned_addr learned my addr 172.21.15.19:0/3082332673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bd01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1ffff640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bd7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/3082332673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 msgr2=0x555d47bd7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bd7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 msgr2=0x555d47bd01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bd01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/3082332673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d47909620 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bd01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bcfc60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6e100057c0 tx=0x7f6e10031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9b1e57ef603aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6dfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6dfeffd640 1 -- 172.21.15.19:0/3082332673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10052060 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e10040070 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6dfeffd640 1 -- 172.21.15.19:0/3082332673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6e10004240 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6dfeffd640 1 -- 172.21.15.19:0/3082332673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e10035040 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e2ca37740 10 client.17079 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.720+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d47bda030 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6dfeffd640 1 -- 172.21.15.19:0/3082332673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e10040070 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6dfeffd640 1 -- 172.21.15.19:0/3082332673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6e1003c560 con 0x555d477fc2f0 2024-05-17T09:27:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e2ca37740 10 client.17079 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e2ca37740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e2ca37740 20 client.17079 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477e3d80 0x555d47bdd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e2ca37740 4 client.17079 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d47b3ab80 con 0x555d477e3d80 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477e3d80 0x555d47bdd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6dfeffd640 10 client.17079 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.721+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477e3d80 0x555d47bdd390 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b30c40ff06e92d2 server_cookie=bc2b0a5feac2edf4 in_seq=0 out_seq=0 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6dfeffd640 1 -- 172.21.15.19:0/3082332673 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555d47b3ab80 con 0x555d477e3d80 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6dfeffd640 10 client.17079 handle_command_reply: tid=0 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 client.17079 shutdown 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 msgr2=0x555d47bcfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bcfc60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6e100057c0 tx=0x7f6e10031ca0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 shutdown_connections 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477e3d80 0x555d47bdd390 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47bd7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478044d0 0x555d47bd01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.722+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/3082332673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bcfc60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.723+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 >> 172.21.15.19:0/3082332673 conn(0x555d47819300 msgr2=0x555d47bc4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.723+0000 7f6dfeffd640 0 client.17079 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.723+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 shutdown_connections 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.723+0000 7f6e2ca37740 1 -- 172.21.15.19:0/3082332673 wait complete. 2024-05-17T09:27:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.723+0000 7f6e2ca37740 20 client.17079 trim_cache size 0 max 16384 2024-05-17T09:27:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 Processor -- start 2024-05-17T09:27:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 -- start start 2024-05-17T09:27:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d477fc2f0 0x555d47bd0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 0x555d47bd7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47be1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201b14d0 con 0x555d478044d0 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201aeda0 con 0x555d477fc2f0 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201acbf0 con 0x555d478251b0 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 0x555d47bd7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 0x555d47bd7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36500/0 (socket says 172.21.15.19:36500) 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e16ffd640 1 -- 172.21.15.19:0/869210842 learned_addr learned my addr 172.21.15.19:0/869210842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.739+0000 7f6e1ffff640 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47be1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d477fc2f0 0x555d47bd0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e16ffd640 1 -- 172.21.15.19:0/869210842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 msgr2=0x555d47be1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47be1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e16ffd640 1 -- 172.21.15.19:0/869210842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d477fc2f0 msgr2=0x555d47bd0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d477fc2f0 0x555d47bd0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e16ffd640 1 -- 172.21.15.19:0/869210842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d47bcdbd0 con 0x555d478044d0 2024-05-17T09:27:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 0x555d47bd7020 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9c0 tx=0x7f6e080e8550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3598423367e437a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e157fa640 1 -- 172.21.15.19:0/869210842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080a94f0 con 0x555d478044d0 2024-05-17T09:27:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e157fa640 1 -- 172.21.15.19:0/869210842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e080cae50 con 0x555d478044d0 2024-05-17T09:27:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.740+0000 7f6e157fa640 1 -- 172.21.15.19:0/869210842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080a5330 con 0x555d478044d0 2024-05-17T09:27:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 -- 172.21.15.19:0/869210842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 msgr2=0x555d47bd7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 0x555d47bd7020 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9c0 tx=0x7f6e080e8550 comp rx=0 tx=0).stop 2024-05-17T09:27:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 -- 172.21.15.19:0/869210842 shutdown_connections 2024-05-17T09:27:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478251b0 0x555d47be1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d478044d0 0x555d47bd7020 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/869210842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d477fc2f0 0x555d47bd0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 -- 172.21.15.19:0/869210842 >> 172.21.15.19:0/869210842 conn(0x555d47819300 msgr2=0x555d47bc95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 -- 172.21.15.19:0/869210842 shutdown_connections 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 -- 172.21.15.19:0/869210842 wait complete. 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.741+0000 7f6e2ca37740 1 -- start start 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 Processor -- start 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bd5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 0x555d47bd6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478251b0 0x555d47c438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201ad260 con 0x555d477fc2f0 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201ae6d0 con 0x555d478251b0 2024-05-17T09:27:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201ad930 con 0x555d478044d0 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e2ca37740 10 client.0 _ll_register_callbacks cb 0x7f6e10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bd5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 0x555d47bd6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e1ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478251b0 0x555d47c438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bd5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36516/0 (socket says 172.21.15.19:36516) 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e1f7fe640 1 -- 172.21.15.19:0/1222392359 learned_addr learned my addr 172.21.15.19:0/1222392359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 0x555d47bd6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:43574/0 (socket says 172.21.15.19:43574) 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 -- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478251b0 msgr2=0x555d47c438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478251b0 0x555d47c438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 -- 172.21.15.19:0/1222392359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 msgr2=0x555d47bd5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bd5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e16ffd640 1 -- 172.21.15.19:0/1222392359 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d47bd6880 con 0x555d478044d0 2024-05-17T09:27:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.742+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bd5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e16ffd640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 0x555d47bd6090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9c0 tx=0x7f6e08005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4b9b3cf24b775c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6dfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6dfeffd640 1 -- 172.21.15.19:0/1222392359 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080c9060 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6dfeffd640 1 -- 172.21.15.19:0/1222392359 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6e080a5e30 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e080cae50 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6dfeffd640 1 -- 172.21.15.19:0/1222392359 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e080a9310 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 10 client.26701 fetch_fsmap learned FSMap version 32 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d47c44e60 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6dfeffd640 1 -- 172.21.15.19:0/1222392359 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e080cae50 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6dfeffd640 1 -- 172.21.15.19:0/1222392359 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6e080b2570 con 0x555d478044d0 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 10 client.26701 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:27:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 10 client.26701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 20 client.26701 populate_metadata read hostname 'smithi019' 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477b91c0 0x555d47c47800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 4 client.26701 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.743+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d47b34360 con 0x555d477b91c0 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.744+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477b91c0 0x555d47c47800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.744+0000 7f6dfeffd640 10 client.26701 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.744+0000 7f6e1f7fe640 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477b91c0 0x555d47c47800 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b101c34513f00d server_cookie=e7214a2ad896ef00 in_seq=0 out_seq=0 2024-05-17T09:27:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.759+0000 7f6dfeffd640 1 -- 172.21.15.19:0/1222392359 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d47b34360 con 0x555d477b91c0 2024-05-17T09:27:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.759+0000 7f6dfeffd640 10 client.26701 handle_command_reply: tid=0 2024-05-17T09:27:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.759+0000 7f6e2ca37740 1 client.26701 shutdown 2024-05-17T09:27:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.759+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 msgr2=0x555d47bd6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.759+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 0x555d47bd6090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9c0 tx=0x7f6e08005ab0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 shutdown_connections 2024-05-17T09:27:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555d477b91c0 0x555d47c47800 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d478251b0 0x555d47c438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d478044d0 0x555d47bd6090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6dfeffd640 0 client.26701 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:27:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 --2- 172.21.15.19:0/1222392359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d477fc2f0 0x555d47bd5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 >> 172.21.15.19:0/1222392359 conn(0x555d47819300 msgr2=0x555d47bcffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 shutdown_connections 2024-05-17T09:27:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 1 -- 172.21.15.19:0/1222392359 wait complete. 2024-05-17T09:27:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e2ca37740 20 client.26701 trim_cache size 0 max 16384 2024-05-17T09:27:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:27:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:27:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:27:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e0c0b28a0 msgr2=0x7f6e0c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e0c0b28a0 0x7f6e0c0b4db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x555d47bc7570 tx=0x7f6e18009500 comp rx=0 tx=0).stop 2024-05-17T09:27:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 msgr2=0x7f6e2019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:27:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.760+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 0x7f6e2019efc0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800e9f0 tx=0x7f6e0800edf0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 shutdown_connections 2024-05-17T09:27:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e20108a80 0x7f6e201a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e0c0b28a0 0x7f6e0c0b4db0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e20103d80 0x7f6e2019efc0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 --2- 172.21.15.19:0/2143951367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e20102b40 0x7f6e2006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:27:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 >> 172.21.15.19:0/2143951367 conn(0x7f6e200fdd30 msgr2=0x7f6e20104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:27:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 shutdown_connections 2024-05-17T09:27:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:27:49.761+0000 7f6e25aa8640 1 -- 172.21.15.19:0/2143951367 wait complete. 2024-05-17T09:27:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:50 smithi069 ceph-mon[35081]: pgmap v680: 129 pgs: 129 active+clean; 57 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 360 KiB/s wr, 29 op/s 2024-05-17T09:27:50.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 114652 1.69 MB/sec execute 228 sec latency 15.511 ms 2024-05-17T09:27:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:50 smithi019 ceph-mon[31620]: pgmap v680: 129 pgs: 129 active+clean; 57 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 360 KiB/s wr, 29 op/s 2024-05-17T09:27:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:50 smithi079 ceph-mon[35502]: pgmap v680: 129 pgs: 129 active+clean; 57 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 360 KiB/s wr, 29 op/s 2024-05-17T09:27:51.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 115195 1.69 MB/sec execute 229 sec latency 16.115 ms 2024-05-17T09:27:52.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:52 smithi069 ceph-mon[35081]: pgmap v681: 129 pgs: 129 active+clean; 54 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 496 KiB/s wr, 46 op/s 2024-05-17T09:27:52.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 115851 1.69 MB/sec execute 230 sec latency 18.675 ms 2024-05-17T09:27:52.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:52 smithi019 ceph-mon[31620]: pgmap v681: 129 pgs: 129 active+clean; 54 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 496 KiB/s wr, 46 op/s 2024-05-17T09:27:52.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:52 smithi079 ceph-mon[35502]: pgmap v681: 129 pgs: 129 active+clean; 54 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 496 KiB/s wr, 46 op/s 2024-05-17T09:27:53.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 116420 1.71 MB/sec execute 231 sec latency 14.204 ms 2024-05-17T09:27:54.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:54 smithi069 ceph-mon[35081]: pgmap v682: 129 pgs: 129 active+clean; 55 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 724 KiB/s wr, 76 op/s 2024-05-17T09:27:54.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 116898 1.71 MB/sec execute 232 sec latency 11.910 ms 2024-05-17T09:27:54.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:54 smithi019 ceph-mon[31620]: pgmap v682: 129 pgs: 129 active+clean; 55 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 724 KiB/s wr, 76 op/s 2024-05-17T09:27:54.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:54 smithi079 ceph-mon[35502]: pgmap v682: 129 pgs: 129 active+clean; 55 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 724 KiB/s wr, 76 op/s 2024-05-17T09:27:55.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 117365 1.72 MB/sec execute 233 sec latency 15.286 ms 2024-05-17T09:27:56.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:56 smithi069 ceph-mon[35081]: pgmap v683: 129 pgs: 129 active+clean; 54 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 875 KiB/s wr, 97 op/s 2024-05-17T09:27:56.435 INFO:tasks.workunit.client.0.smithi019.stdout: 1 117615 1.72 MB/sec execute 234 sec latency 16.060 ms 2024-05-17T09:27:56.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:56 smithi019 ceph-mon[31620]: pgmap v683: 129 pgs: 129 active+clean; 54 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 875 KiB/s wr, 97 op/s 2024-05-17T09:27:56.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:56 smithi079 ceph-mon[35502]: pgmap v683: 129 pgs: 129 active+clean; 54 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 875 KiB/s wr, 97 op/s 2024-05-17T09:27:57.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 118013 1.71 MB/sec execute 235 sec latency 15.692 ms 2024-05-17T09:27:58.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:27:58 smithi069 ceph-mon[35081]: pgmap v684: 129 pgs: 129 active+clean; 55 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:27:58.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 118545 1.72 MB/sec execute 236 sec latency 33.254 ms 2024-05-17T09:27:58.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:27:58 smithi019 ceph-mon[31620]: pgmap v684: 129 pgs: 129 active+clean; 55 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:27:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:27:58 smithi079 ceph-mon[35502]: pgmap v684: 129 pgs: 129 active+clean; 55 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-17T09:27:59.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 119093 1.72 MB/sec execute 237 sec latency 23.560 ms 2024-05-17T09:28:00.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:00 smithi069 ceph-mon[35081]: pgmap v685: 129 pgs: 129 active+clean; 56 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-17T09:28:00.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 119558 1.72 MB/sec execute 238 sec latency 96.203 ms 2024-05-17T09:28:00.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:00 smithi019 ceph-mon[31620]: pgmap v685: 129 pgs: 129 active+clean; 56 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-17T09:28:00.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:00 smithi079 ceph-mon[35502]: pgmap v685: 129 pgs: 129 active+clean; 56 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-17T09:28:01.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 120277 1.74 MB/sec execute 239 sec latency 15.739 ms 2024-05-17T09:28:02.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 120663 1.75 MB/sec execute 240 sec latency 16.441 ms 2024-05-17T09:28:02.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:02 smithi019 ceph-mon[31620]: pgmap v686: 129 pgs: 129 active+clean; 57 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:28:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:02 smithi079 ceph-mon[35502]: pgmap v686: 129 pgs: 129 active+clean; 57 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:28:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:02 smithi069 ceph-mon[35081]: pgmap v686: 129 pgs: 129 active+clean; 57 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:28:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 -- 172.21.15.19:0/1591383675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 msgr2=0x7f1688103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1591383675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f1688103480 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1670009900 tx=0x7f167002a410 comp rx=0 tx=0).stop 2024-05-17T09:28:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 -- 172.21.15.19:0/1591383675 shutdown_connections 2024-05-17T09:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1591383675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688104590 0x7f168810acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1591383675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1688103a50 0x7f1688103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1591383675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f1688103480 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 -- 172.21.15.19:0/1591383675 >> 172.21.15.19:0/1591383675 conn(0x7f16880fd470 msgr2=0x7f16880ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 -- 172.21.15.19:0/1591383675 shutdown_connections 2024-05-17T09:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.868+0000 7f168cc3b640 1 -- 172.21.15.19:0/1591383675 wait complete. 2024-05-17T09:28:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 Processor -- start 2024-05-17T09:28:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 -- start start 2024-05-17T09:28:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f168819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688103a50 0x7f168819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1688104590 0x7f16881a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f168810e520 con 0x7f1688103070 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f168810e3a0 con 0x7f1688103a50 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168cc3b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f168810e6a0 con 0x7f1688104590 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f168819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f168819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35132/0 (socket says 172.21.15.19:35132) 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f168659c640 1 -- 172.21.15.19:0/1993375558 learned_addr learned my addr 172.21.15.19:0/1993375558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f1685d9b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688103a50 0x7f168819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.869+0000 7f1686d9d640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1688104590 0x7f16881a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168659c640 1 -- 172.21.15.19:0/1993375558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1688104590 msgr2=0x7f16881a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168659c640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1688104590 0x7f16881a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168659c640 1 -- 172.21.15.19:0/1993375558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688103a50 msgr2=0x7f168819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168659c640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688103a50 0x7f168819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168659c640 1 -- 172.21.15.19:0/1993375558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16881a1cf0 con 0x7f1688103070 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f1685d9b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688103a50 0x7f168819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168659c640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f168819a1b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1670005760 tx=0x7f1670003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f167f7fe640 1 -- 172.21.15.19:0/1993375558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167003f070 con 0x7f1688103070 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f167f7fe640 1 -- 172.21.15.19:0/1993375558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1670031ba0 con 0x7f1688103070 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16881a1f80 con 0x7f1688103070 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f167f7fe640 1 -- 172.21.15.19:0/1993375558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670035030 con 0x7f1688103070 2024-05-17T09:28:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.870+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881a2380 con 0x7f1688103070 2024-05-17T09:28:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 Processor -- start 2024-05-17T09:28:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 -- start start 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 0x55a58234c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58234c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 0x55a582353820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5822bec30 con 0x55a581f8a4d0 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a5822ae7d0 con 0x55a581f822f0 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.873+0000 7f1693bc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a5822b2810 con 0x55a581fab1b0 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1686d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 0x55a58234c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 0x55a582353820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f167f7fe640 1 -- 172.21.15.19:0/1993375558 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1670042070 con 0x7f1688103070 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 0x55a582353820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35134/0 (socket says 172.21.15.19:35134) 2024-05-17T09:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1686d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 0x55a58234c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52360/0 (socket says 172.21.15.19:52360) 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 -- 172.21.15.19:0/4220861378 learned_addr learned my addr 172.21.15.19:0/4220861378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f168659c640 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58234c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 -- 172.21.15.19:0/4220861378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 msgr2=0x55a58234c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 0x55a58234c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 -- 172.21.15.19:0/4220861378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 msgr2=0x55a58234c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58234c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 -- 172.21.15.19:0/4220861378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a581fc3210 con 0x55a581f8a4d0 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1686d9d640 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 0x55a58234c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f168659c640 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58234c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.874+0000 7f1685d9b640 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 0x55a582353820 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f16780114f0 tx=0x7f16780119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfafb057d008d340 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.875+0000 7f167cff9640 1 -- 172.21.15.19:0/4220861378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678004020 con 0x55a581f8a4d0 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.875+0000 7f167cff9640 1 -- 172.21.15.19:0/4220861378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16780041c0 con 0x55a581f8a4d0 2024-05-17T09:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.875+0000 7f167cff9640 1 -- 172.21.15.19:0/4220861378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167800f820 con 0x55a581f8a4d0 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.875+0000 7f167f7fe640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f16740b28a0 0x7f16740b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.875+0000 7f1686d9d640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f16740b28a0 0x7f16740b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.875+0000 7f167f7fe640 1 -- 172.21.15.19:0/1993375558 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1670103080 con 0x7f1688103070 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 -- 172.21.15.19:0/4220861378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 msgr2=0x55a582353820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 0x55a582353820 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f16780114f0 tx=0x7f16780119b0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1686d9d640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f16740b28a0 0x7f16740b4db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55a58234d570 tx=0x7f1680009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 -- 172.21.15.19:0/4220861378 shutdown_connections 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f8a4d0 0x55a582353820 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58234c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4220861378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581fab1b0 0x55a58234c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 -- 172.21.15.19:0/4220861378 >> 172.21.15.19:0/4220861378 conn(0x55a581f9f300 msgr2=0x55a582347d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 -- 172.21.15.19:0/4220861378 shutdown_connections 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 -- 172.21.15.19:0/4220861378 wait complete. 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 -- start start 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.876+0000 7f1693bc8740 1 Processor -- start 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 0x55a582355d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f8a4d0 0x55a582356250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a581f94790 con 0x55a581fab1b0 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a58226a890 con 0x55a581f8a4d0 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a582354270 con 0x55a581f822f0 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1693bc8740 10 client.0 _ll_register_callbacks cb 0x55a58226dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1686d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 0x55a582355d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1685d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f8a4d0 0x55a582356250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 0x55a582355d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52370/0 (socket says 172.21.15.19:52370) 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f1686d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35146/0 (socket says 172.21.15.19:35146) 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 -- 172.21.15.19:0/3096329206 learned_addr learned my addr 172.21.15.19:0/3096329206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 -- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f8a4d0 msgr2=0x55a582356250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f8a4d0 0x55a582356250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 -- 172.21.15.19:0/3096329206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 msgr2=0x55a58235d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 -- 172.21.15.19:0/3096329206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58208f620 con 0x55a581f822f0 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f168659c640 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 0x55a582355d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16700c7610 tx=0x7f16700c9b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac7fc11b3cd8755 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f166dffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.877+0000 7f166dffb640 1 -- 172.21.15.19:0/3096329206 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670049030 con 0x55a581f822f0 2024-05-17T09:28:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16780041c0 con 0x55a581f822f0 2024-05-17T09:28:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f166dffb640 1 -- 172.21.15.19:0/3096329206 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f167003b460 con 0x55a581f822f0 2024-05-17T09:28:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f166dffb640 1 -- 172.21.15.19:0/3096329206 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f16700ca3a0 con 0x55a581f822f0 2024-05-17T09:28:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 10 client.26713 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a582360050 con 0x55a581f822f0 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f166dffb640 1 -- 172.21.15.19:0/3096329206 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16780041c0 con 0x55a581f822f0 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f166dffb640 1 -- 172.21.15.19:0/3096329206 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f16700e4c70 con 0x55a581f822f0 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 10 client.26713 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 10 client.26713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 20 client.26713 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f69d80 0x55a582363790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 4 client.26713 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5822c0b80 con 0x55a581f69d80 2024-05-17T09:28:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.878+0000 7f1685d9b640 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f69d80 0x55a582363790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.879+0000 7f1685d9b640 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f69d80 0x55a582363790 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3990d251bf7d800d server_cookie=4c409f1b1c8ea in_seq=0 out_seq=0 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.879+0000 7f166dffb640 10 client.26713 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.879+0000 7f166dffb640 1 -- 172.21.15.19:0/3096329206 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a5822c0b80 con 0x55a581f69d80 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f166dffb640 10 client.26713 handle_command_reply: tid=0 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 client.26713 shutdown 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 msgr2=0x55a582355d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 0x55a582355d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16700c7610 tx=0x7f16700c9b10 comp rx=0 tx=0).stop 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 shutdown_connections 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f69d80 0x55a582363790 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f8a4d0 0x55a582356250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f166dffb640 0 client.26713 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 --2- 172.21.15.19:0/3096329206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f822f0 0x55a582355d10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 >> 172.21.15.19:0/3096329206 conn(0x55a581f9f300 msgr2=0x55a582346d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 shutdown_connections 2024-05-17T09:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 1 -- 172.21.15.19:0/3096329206 wait complete. 2024-05-17T09:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.880+0000 7f1693bc8740 20 client.26713 trim_cache size 0 max 16384 2024-05-17T09:28:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 Processor -- start 2024-05-17T09:28:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 -- start start 2024-05-17T09:28:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 0x55a58234b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581fab1b0 0x55a58234f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16881af200 con 0x55a581f822f0 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16881acad0 con 0x55a581fab1b0 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1693bc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16881aa920 con 0x55a581f8a4d0 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 0x55a58234b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 0x55a58234b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35160/0 (socket says 172.21.15.19:35160) 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 -- 172.21.15.19:0/2264962697 learned_addr learned my addr 172.21.15.19:0/2264962697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1685d9b640 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1686d9d640 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581fab1b0 0x55a58234f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 -- 172.21.15.19:0/2264962697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 msgr2=0x55a58235d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 -- 172.21.15.19:0/2264962697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581fab1b0 msgr2=0x55a58234f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581fab1b0 0x55a58234f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f168659c640 1 -- 172.21.15.19:0/2264962697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a582353660 con 0x55a581f822f0 2024-05-17T09:28:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.898+0000 7f1685d9b640 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.899+0000 7f168659c640 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 0x55a58234b2c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1670000c00 tx=0x7f16700b20f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8aa3f4d7f9ec61b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.899+0000 7f166ffff640 1 -- 172.21.15.19:0/2264962697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670049070 con 0x55a581f822f0 2024-05-17T09:28:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.899+0000 7f166ffff640 1 -- 172.21.15.19:0/2264962697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16700ce6a0 con 0x55a581f822f0 2024-05-17T09:28:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.899+0000 7f166ffff640 1 -- 172.21.15.19:0/2264962697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16700e4be0 con 0x55a581f822f0 2024-05-17T09:28:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 -- 172.21.15.19:0/2264962697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 msgr2=0x55a58234b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 0x55a58234b2c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1670000c00 tx=0x7f16700b20f0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 -- 172.21.15.19:0/2264962697 shutdown_connections 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581fab1b0 0x55a58234f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- 172.21.15.19:0/2264962697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581f822f0 0x55a58234b2c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 -- 172.21.15.19:0/2264962697 >> 172.21.15.19:0/2264962697 conn(0x55a581f9f300 msgr2=0x55a58234abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 -- 172.21.15.19:0/2264962697 shutdown_connections 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 -- 172.21.15.19:0/2264962697 wait complete. 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 -- start start 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 Processor -- start 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58235a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.900+0000 7f1693bc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1693bc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16881aaf90 con 0x55a581fab1b0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1693bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16881ac400 con 0x55a581f822f0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1693bc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16881ab660 con 0x55a581f8a4d0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1693bc8740 10 client.0 _ll_register_callbacks cb 0x7f16700c62d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35170/0 (socket says 172.21.15.19:35170) 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 -- 172.21.15.19:0/4064723744 learned_addr learned my addr 172.21.15.19:0/4064723744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1685d9b640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f168659c640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58235a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 -- 172.21.15.19:0/4064723744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 msgr2=0x55a58235a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 -- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 msgr2=0x55a58235a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58235a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 -- 172.21.15.19:0/4064723744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58235bee0 con 0x55a581fab1b0 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1685d9b640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1686d9d640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235b920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1680003f50 tx=0x7f1680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0f060a5672f43f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f166dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f166dffb640 1 -- 172.21.15.19:0/4064723744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168000fe60 con 0x55a581fab1b0 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f166dffb640 1 -- 172.21.15.19:0/4064723744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1680004750 con 0x55a581fab1b0 2024-05-17T09:28:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.901+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16700ce6a0 con 0x55a581fab1b0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f166dffb640 1 -- 172.21.15.19:0/4064723744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1680034050 con 0x55a581fab1b0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 10 client.17103 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a58235d240 con 0x55a581fab1b0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f166dffb640 1 -- 172.21.15.19:0/4064723744 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16700ce6a0 con 0x55a581fab1b0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f166dffb640 1 -- 172.21.15.19:0/4064723744 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1680005740 con 0x55a581fab1b0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 10 client.17103 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 20 client.17103 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f3f1c0 0x55a5823ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 4 client.17103 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5822ba360 con 0x55a581f3f1c0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.902+0000 7f168659c640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f3f1c0 0x55a5823ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.903+0000 7f168659c640 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f3f1c0 0x55a5823ce4c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c552b5addce3a78b server_cookie=5aea7e55466fa912 in_seq=0 out_seq=0 2024-05-17T09:28:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.903+0000 7f166dffb640 10 client.17103 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.919+0000 7f166dffb640 1 -- 172.21.15.19:0/4064723744 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a5822ba360 con 0x55a581f3f1c0 2024-05-17T09:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.919+0000 7f166dffb640 10 client.17103 handle_command_reply: tid=0 2024-05-17T09:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.919+0000 7f1693bc8740 1 client.17103 shutdown 2024-05-17T09:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 msgr2=0x55a58235b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235b920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1680003f50 tx=0x7f1680004a40 comp rx=0 tx=0).stop 2024-05-17T09:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 shutdown_connections 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a581f3f1c0 0x55a5823ce4c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a581fab1b0 0x55a58235b920 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a581f8a4d0 0x55a58235a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f166dffb640 0 client.17103 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 --2- 172.21.15.19:0/4064723744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a581f822f0 0x55a58235a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 >> 172.21.15.19:0/4064723744 conn(0x55a581f9f300 msgr2=0x55a582356050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 shutdown_connections 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 1 -- 172.21.15.19:0/4064723744 wait complete. 2024-05-17T09:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f1693bc8740 20 client.17103 trim_cache size 0 max 16384 2024-05-17T09:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f16740b28a0 msgr2=0x7f16740b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f16740b28a0 0x7f16740b4db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55a58234d570 tx=0x7f1680009500 comp rx=0 tx=0).stop 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 msgr2=0x7f168819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.920+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f168819a1b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1670005760 tx=0x7f1670003c10 comp rx=0 tx=0).stop 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 shutdown_connections 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1688104590 0x7f16881a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1688103a50 0x7f168819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f16740b28a0 0x7f16740b4db0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 --2- 172.21.15.19:0/1993375558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1688103070 0x7f168819a1b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 >> 172.21.15.19:0/1993375558 conn(0x7f16880fd470 msgr2=0x7f1688109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 shutdown_connections 2024-05-17T09:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:02.921+0000 7f168cc3b640 1 -- 172.21.15.19:0/1993375558 wait complete. 2024-05-17T09:28:03.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 121000 1.75 MB/sec execute 241 sec latency 21.243 ms 2024-05-17T09:28:04.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 121255 1.74 MB/sec execute 242 sec latency 13.013 ms 2024-05-17T09:28:04.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:04 smithi019 ceph-mon[31620]: pgmap v687: 129 pgs: 129 active+clean; 57 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:28:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:04 smithi079 ceph-mon[35502]: pgmap v687: 129 pgs: 129 active+clean; 57 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:28:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:04 smithi069 ceph-mon[35081]: pgmap v687: 129 pgs: 129 active+clean; 57 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-17T09:28:05.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 121764 1.74 MB/sec execute 243 sec latency 14.511 ms 2024-05-17T09:28:06.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 122299 1.75 MB/sec execute 244 sec latency 17.971 ms 2024-05-17T09:28:06.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:06 smithi019 ceph-mon[31620]: pgmap v688: 129 pgs: 129 active+clean; 57 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:28:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:06 smithi079 ceph-mon[35502]: pgmap v688: 129 pgs: 129 active+clean; 57 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:28:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:06 smithi069 ceph-mon[35081]: pgmap v688: 129 pgs: 129 active+clean; 57 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:28:07.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 122933 1.75 MB/sec execute 245 sec latency 16.110 ms 2024-05-17T09:28:08.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 123405 1.76 MB/sec execute 246 sec latency 16.173 ms 2024-05-17T09:28:08.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:08 smithi019 ceph-mon[31620]: pgmap v689: 129 pgs: 129 active+clean; 57 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-17T09:28:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:08 smithi079 ceph-mon[35502]: pgmap v689: 129 pgs: 129 active+clean; 57 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-17T09:28:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:08 smithi069 ceph-mon[35081]: pgmap v689: 129 pgs: 129 active+clean; 57 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-17T09:28:09.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 123959 1.76 MB/sec execute 247 sec latency 15.571 ms 2024-05-17T09:28:10.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 124420 1.77 MB/sec execute 248 sec latency 16.053 ms 2024-05-17T09:28:10.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:10 smithi019 ceph-mon[31620]: pgmap v690: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:28:10.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:10 smithi079 ceph-mon[35502]: pgmap v690: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:28:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:10 smithi069 ceph-mon[35081]: pgmap v690: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:28:11.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 124656 1.77 MB/sec execute 249 sec latency 15.980 ms 2024-05-17T09:28:12.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 125055 1.76 MB/sec execute 250 sec latency 14.797 ms 2024-05-17T09:28:12.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:12 smithi019 ceph-mon[31620]: pgmap v691: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-05-17T09:28:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:12 smithi079 ceph-mon[35502]: pgmap v691: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-05-17T09:28:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:12 smithi069 ceph-mon[35081]: pgmap v691: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-05-17T09:28:13.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 125571 1.77 MB/sec execute 251 sec latency 25.163 ms 2024-05-17T09:28:14.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 126105 1.77 MB/sec execute 252 sec latency 17.859 ms 2024-05-17T09:28:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:14 smithi079 ceph-mon[35502]: pgmap v692: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-17T09:28:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:14 smithi069 ceph-mon[35081]: pgmap v692: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-17T09:28:14.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:14 smithi019 ceph-mon[31620]: pgmap v692: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-17T09:28:15.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 126641 1.77 MB/sec execute 253 sec latency 14.906 ms 2024-05-17T09:28:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 -- 172.21.15.19:0/4151898518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 msgr2=0x7f2a78104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/4151898518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 0x7f2a78104190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009900 tx=0x7f2a6c02a410 comp rx=0 tx=0).stop 2024-05-17T09:28:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 -- 172.21.15.19:0/4151898518 shutdown_connections 2024-05-17T09:28:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/4151898518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a78108a80 0x7f2a7810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/4151898518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 0x7f2a78104190 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/4151898518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a78103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 -- 172.21.15.19:0/4151898518 >> 172.21.15.19:0/4151898518 conn(0x7f2a780fdd30 msgr2=0x7f2a781001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 -- 172.21.15.19:0/4151898518 shutdown_connections 2024-05-17T09:28:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.030+0000 7f2a7dbba640 1 -- 172.21.15.19:0/4151898518 wait complete. 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 Processor -- start 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 -- start start 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a7819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 0x7f2a7819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a78108a80 0x7f2a781a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7810e510 con 0x7f2a78103d80 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7810e390 con 0x7f2a78108a80 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.031+0000 7f2a7dbba640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7810e690 con 0x7f2a78102b40 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a7819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a7819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:48012/0 (socket says 172.21.15.19:48012) 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 -- 172.21.15.19:0/745621571 learned_addr learned my addr 172.21.15.19:0/745621571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 0x7f2a7819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a77fff640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a78108a80 0x7f2a781a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 -- 172.21.15.19:0/745621571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a78108a80 msgr2=0x7f2a781a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a78108a80 0x7f2a781a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 -- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 msgr2=0x7f2a7819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 0x7f2a7819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 -- 172.21.15.19:0/745621571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a780fe1c0 con 0x7f2a78102b40 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.032+0000 7f2a777fe640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a7819a1d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.033+0000 7f2a74ff9640 1 -- 172.21.15.19:0/745621571 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800ccc0 con 0x7f2a78102b40 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.033+0000 7f2a74ff9640 1 -- 172.21.15.19:0/745621571 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a680035f0 con 0x7f2a78102b40 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.033+0000 7f2a74ff9640 1 -- 172.21.15.19:0/745621571 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800ce60 con 0x7f2a78102b40 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.033+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a780fe4b0 con 0x7f2a78102b40 2024-05-17T09:28:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.033+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a780fe9b0 con 0x7f2a78102b40 2024-05-17T09:28:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 Processor -- start 2024-05-17T09:28:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a74ff9640 1 -- 172.21.15.19:0/745621571 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2a68004000 con 0x7f2a78102b40 2024-05-17T09:28:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 -- start start 2024-05-17T09:28:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 0x55f091df13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091df1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091df8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f091d63c30 con 0x55f091a272f0 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f091d537d0 con 0x55f091a2f4d0 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f091d57810 con 0x55f091a501b0 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091df1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 0x55f091df13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a76ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091df8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091df1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59074/0 (socket says 172.21.15.19:59074) 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a777fe640 1 -- 172.21.15.19:0/3851805406 learned_addr learned my addr 172.21.15.19:0/3851805406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.036+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 0x55f091df13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:48016/0 (socket says 172.21.15.19:48016) 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a77fff640 1 -- 172.21.15.19:0/3851805406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 msgr2=0x55f091df8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091df8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a77fff640 1 -- 172.21.15.19:0/3851805406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 msgr2=0x55f091df1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091df1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a77fff640 1 -- 172.21.15.19:0/3851805406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f091a68210 con 0x55f091a501b0 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a74ff9640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a600b28a0 0x7f2a600b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 0x55f091df13a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009f00 tx=0x7f2a70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88929e920188e3d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a777fe640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a600b28a0 0x7f2a600b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a5dffb640 1 -- 172.21.15.19:0/3851805406 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70004d20 con 0x55f091a501b0 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a5dffb640 1 -- 172.21.15.19:0/3851805406 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a70004030 con 0x55f091a501b0 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a5dffb640 1 -- 172.21.15.19:0/3851805406 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005280 con 0x55f091a501b0 2024-05-17T09:28:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.037+0000 7f2a74ff9640 1 -- 172.21.15.19:0/745621571 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2a680e2970 con 0x7f2a78102b40 2024-05-17T09:28:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3851805406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 msgr2=0x55f091df13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a777fe640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a600b28a0 0x7f2a600b4db0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2a68003fd0 tx=0x7f2a680e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 0x55f091df13a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009f00 tx=0x7f2a70011810 comp rx=0 tx=0).stop 2024-05-17T09:28:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3851805406 shutdown_connections 2024-05-17T09:28:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091df8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091df1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3851805406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a501b0 0x55f091df13a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3851805406 >> 172.21.15.19:0/3851805406 conn(0x55f091a44300 msgr2=0x55f091decd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3851805406 shutdown_connections 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3851805406 wait complete. 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.038+0000 7f2a84a6a740 1 -- start start 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 Processor -- start 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091dfac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091dfb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a501b0 0x55f091e020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f091a39790 con 0x55f091a501b0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f091d0f890 con 0x55f091a272f0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f091df9270 con 0x55f091a2f4d0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a84a6a740 10 client.0 _ll_register_callbacks cb 0x55f091d12d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091dfac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091dfac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54170/0 (socket says 172.21.15.19:54170) 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a77fff640 1 -- 172.21.15.19:0/3888619330 learned_addr learned my addr 172.21.15.19:0/3888619330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091dfb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.039+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a501b0 0x55f091e020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a76ffd640 1 -- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 msgr2=0x55f091dfac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091dfac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a76ffd640 1 -- 172.21.15.19:0/3888619330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a501b0 msgr2=0x55f091e020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a501b0 0x55f091e020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a76ffd640 1 -- 172.21.15.19:0/3888619330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f091b34620 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091dfac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091dfb160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005760 tx=0x7f2a6c031870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239b4a20c1d2b6b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3888619330 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c035040 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a70004030 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3888619330 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2a6c004240 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3888619330 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c04e050 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a84a6a740 10 client.26731 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f091e05060 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.040+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3888619330 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a70004030 con 0x55f091a2f4d0 2024-05-17T09:28:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3888619330 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2a6c03c440 con 0x55f091a2f4d0 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a84a6a740 10 client.26731 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a84a6a740 10 client.26731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a84a6a740 20 client.26731 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a0ed80 0x55f091e087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a84a6a740 4 client.26731 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f091d65b80 con 0x55f091a0ed80 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a0ed80 0x55f091e087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a56ffd640 10 client.26731 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.041+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a0ed80 0x55f091e087b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872b6eefcc54f5ec server_cookie=42e6b6f1664f7fe2 in_seq=0 out_seq=0 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.042+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3888619330 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f091d65b80 con 0x55f091a0ed80 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.042+0000 7f2a56ffd640 10 client.26731 handle_command_reply: tid=0 2024-05-17T09:28:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.042+0000 7f2a84a6a740 1 client.26731 shutdown 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 msgr2=0x55f091dfb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091dfb160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005760 tx=0x7f2a6c031870 comp rx=0 tx=0).stop 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 shutdown_connections 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a0ed80 0x55f091e087b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a501b0 0x55f091e020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091dfb160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a56ffd640 0 client.26731 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3888619330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091dfac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 >> 172.21.15.19:0/3888619330 conn(0x55f091a44300 msgr2=0x55f091debd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 shutdown_connections 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3888619330 wait complete. 2024-05-17T09:28:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.043+0000 7f2a84a6a740 20 client.26731 trim_cache size 0 max 16384 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 Processor -- start 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 -- start start 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 0x55f091deba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091e01fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091e02520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781b14d0 con 0x55f091a272f0 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781aeda0 con 0x55f091a2f4d0 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781acbf0 con 0x55f091a0ed80 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 0x55f091deba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a76ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091e01fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 0x55f091deba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 -- 172.21.15.19:0/3685217126 learned_addr learned my addr 172.21.15.19:0/3685217126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091e02520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 -- 172.21.15.19:0/3685217126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 msgr2=0x55f091e02520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091e02520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 -- 172.21.15.19:0/3685217126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 msgr2=0x55f091e01fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091e01fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 -- 172.21.15.19:0/3685217126 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f091df93b0 con 0x55f091a0ed80 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091e01fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.060+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 0x55f091deba40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009ca0 tx=0x7f2a70009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a837a37d18c0f2df server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.061+0000 7f2a5d7fa640 1 -- 172.21.15.19:0/3685217126 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7001e040 con 0x55f091a0ed80 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.061+0000 7f2a5d7fa640 1 -- 172.21.15.19:0/3685217126 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a700054b0 con 0x55f091a0ed80 2024-05-17T09:28:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.061+0000 7f2a5d7fa640 1 -- 172.21.15.19:0/3685217126 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70004a60 con 0x55f091a0ed80 2024-05-17T09:28:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3685217126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 msgr2=0x55f091deba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 0x55f091deba40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009ca0 tx=0x7f2a70009500 comp rx=0 tx=0).stop 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3685217126 shutdown_connections 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a2f4d0 0x55f091e02520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a272f0 0x55f091e01fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3685217126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a0ed80 0x55f091deba40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3685217126 >> 172.21.15.19:0/3685217126 conn(0x55f091a44300 msgr2=0x55f091deb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3685217126 shutdown_connections 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3685217126 wait complete. 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 -- start start 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.062+0000 7f2a84a6a740 1 Processor -- start 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 0x55f091dffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091e00190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 0x55f091dffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091e010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781ad260 con 0x55f091a0ed80 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781ae6d0 con 0x55f091a272f0 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781ad930 con 0x55f091a2f4d0 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a84a6a740 10 client.0 _ll_register_callbacks cb 0x7f2a6c031500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 0x55f091dffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59104/0 (socket says 172.21.15.19:59104) 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a77fff640 1 -- 172.21.15.19:0/3351464150 learned_addr learned my addr 172.21.15.19:0/3351464150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091e00190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.063+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091e010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a77fff640 1 -- 172.21.15.19:0/3351464150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 msgr2=0x55f091e010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091e010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a77fff640 1 -- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 msgr2=0x55f091e00190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091e00190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a77fff640 1 -- 172.21.15.19:0/3351464150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f091e01680 con 0x55f091a0ed80 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a77fff640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 0x55f091dffc50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2a70000f80 tx=0x7f2a70009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28ec4a88be9c624 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3351464150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70041060 con 0x55f091a0ed80 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.064+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3351464150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2a70003f30 con 0x55f091a0ed80 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a700054b0 con 0x55f091a0ed80 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3351464150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a70036460 con 0x55f091a0ed80 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 10 client.17121 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f091e02910 con 0x55f091a0ed80 2024-05-17T09:28:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3351464150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a700054b0 con 0x55f091a0ed80 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3351464150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2a70034020 con 0x55f091a0ed80 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 10 client.17121 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 20 client.17121 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a501b0 0x55f091e73140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 4 client.17121 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.065+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f091d5f360 con 0x55f091a501b0 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.066+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a501b0 0x55f091e73140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.066+0000 7f2a56ffd640 10 client.17121 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.066+0000 7f2a76ffd640 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a501b0 0x55f091e73140 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3d0900fb437e40 server_cookie=d2be615bd6dda141 in_seq=0 out_seq=0 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.087+0000 7f2a56ffd640 1 -- 172.21.15.19:0/3351464150 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f091d5f360 con 0x55f091a501b0 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.087+0000 7f2a56ffd640 10 client.17121 handle_command_reply: tid=0 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.087+0000 7f2a84a6a740 1 client.17121 shutdown 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.087+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 msgr2=0x55f091dffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.087+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 0x55f091dffc50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2a70000f80 tx=0x7f2a70009cd0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 shutdown_connections 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f091a501b0 0x55f091e73140 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f091a2f4d0 0x55f091e010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a56ffd640 0 client.17121 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f091a272f0 0x55f091e00190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 --2- 172.21.15.19:0/3351464150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a0ed80 0x55f091dffc50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 >> 172.21.15.19:0/3351464150 conn(0x55f091a44300 msgr2=0x55f091deb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 shutdown_connections 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 1 -- 172.21.15.19:0/3351464150 wait complete. 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a84a6a740 20 client.17121 trim_cache size 0 max 16384 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:28:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:28:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a600b28a0 msgr2=0x7f2a600b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a600b28a0 0x7f2a600b4db0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2a68003fd0 tx=0x7f2a680e8530 comp rx=0 tx=0).stop 2024-05-17T09:28:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 msgr2=0x7f2a7819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.088+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a7819a1d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 shutdown_connections 2024-05-17T09:28:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a78108a80 0x7f2a781a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2a600b28a0 0x7f2a600b4db0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a78103d80 0x7f2a7819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 --2- 172.21.15.19:0/745621571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78102b40 0x7f2a7819a1d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 >> 172.21.15.19:0/745621571 conn(0x7f2a780fdd30 msgr2=0x7f2a78109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 shutdown_connections 2024-05-17T09:28:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:16.089+0000 7f2a7dbba640 1 -- 172.21.15.19:0/745621571 wait complete. 2024-05-17T09:28:16.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 127359 1.78 MB/sec execute 254 sec latency 20.269 ms 2024-05-17T09:28:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:16 smithi079 ceph-mon[35502]: pgmap v693: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-17T09:28:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:16 smithi069 ceph-mon[35081]: pgmap v693: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-17T09:28:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:16 smithi019 ceph-mon[31620]: pgmap v693: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-17T09:28:17.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 127762 1.79 MB/sec execute 255 sec latency 14.863 ms 2024-05-17T09:28:18.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 128098 1.79 MB/sec execute 256 sec latency 15.145 ms 2024-05-17T09:28:18.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:18 smithi079 ceph-mon[35502]: pgmap v694: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-17T09:28:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:18 smithi069 ceph-mon[35081]: pgmap v694: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-17T09:28:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:18 smithi019 ceph-mon[31620]: pgmap v694: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-17T09:28:19.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 128343 1.79 MB/sec execute 257 sec latency 13.734 ms 2024-05-17T09:28:20.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 128824 1.79 MB/sec execute 258 sec latency 15.703 ms 2024-05-17T09:28:20.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:20 smithi079 ceph-mon[35502]: pgmap v695: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-17T09:28:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:20 smithi069 ceph-mon[35081]: pgmap v695: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-17T09:28:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:20 smithi019 ceph-mon[31620]: pgmap v695: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-17T09:28:21.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 129353 1.79 MB/sec execute 259 sec latency 15.592 ms 2024-05-17T09:28:22.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 130029 1.79 MB/sec execute 260 sec latency 18.372 ms 2024-05-17T09:28:22.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:22 smithi079 ceph-mon[35502]: pgmap v696: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-17T09:28:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:22 smithi069 ceph-mon[35081]: pgmap v696: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-17T09:28:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:22 smithi019 ceph-mon[31620]: pgmap v696: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-17T09:28:23.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 130510 1.80 MB/sec execute 261 sec latency 11.925 ms 2024-05-17T09:28:24.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 131051 1.81 MB/sec execute 262 sec latency 16.437 ms 2024-05-17T09:28:24.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:24 smithi079 ceph-mon[35502]: pgmap v697: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-17T09:28:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:24 smithi069 ceph-mon[35081]: pgmap v697: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-17T09:28:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:24 smithi019 ceph-mon[31620]: pgmap v697: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-17T09:28:25.436 INFO:tasks.workunit.client.0.smithi019.stdout: 1 131505 1.82 MB/sec execute 263 sec latency 16.805 ms 2024-05-17T09:28:26.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 131727 1.81 MB/sec execute 264 sec latency 15.453 ms 2024-05-17T09:28:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:26 smithi079 ceph-mon[35502]: pgmap v698: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-17T09:28:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:26 smithi069 ceph-mon[35081]: pgmap v698: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-17T09:28:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:26 smithi019 ceph-mon[31620]: pgmap v698: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-17T09:28:27.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 132034 1.81 MB/sec execute 265 sec latency 17.282 ms 2024-05-17T09:28:27.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:27 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:28:27.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:27 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:27 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:27 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:27 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.530 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:28:27.530 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.6 2024-05-17T09:28:27.530 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:28:27.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:28:27.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:27 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:27 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:28:27.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:27 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:27 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:27 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:27 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:28:28.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 132484 1.81 MB/sec execute 266 sec latency 18.540 ms 2024-05-17T09:28:28.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:28 smithi019 ceph-mon[31620]: pgmap v699: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-05-17T09:28:28.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:28:28.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:28:28.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:28.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:28 smithi079 ceph-mon[35502]: pgmap v699: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-05-17T09:28:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:28:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:28:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:28 smithi069 ceph-mon[35081]: pgmap v699: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-05-17T09:28:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:28:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:28:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.202+0000 7fa7a58b8640 1 -- 172.21.15.19:0/1745096408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 msgr2=0x7fa7a0104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.202+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/1745096408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a0104190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa788009900 tx=0x7fa78802a410 comp rx=0 tx=0).stop 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 -- 172.21.15.19:0/1745096408 shutdown_connections 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/1745096408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0108a80 0x7fa7a010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/1745096408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a0104190 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/1745096408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0102b40 0x7fa7a0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 -- 172.21.15.19:0/1745096408 >> 172.21.15.19:0/1745096408 conn(0x7fa7a00fdd30 msgr2=0x7fa7a01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 -- 172.21.15.19:0/1745096408 shutdown_connections 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 -- 172.21.15.19:0/1745096408 wait complete. 2024-05-17T09:28:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 Processor -- start 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.203+0000 7fa7a58b8640 1 -- start start 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa7a58b8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102b40 0x7fa7a019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa7a58b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa7a58b8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 0x7fa7a01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa7a58b8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a010e4f0 con 0x7fa7a0103d80 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa7a58b8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a010e370 con 0x7fa7a0108a80 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa7a58b8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a010e670 con 0x7fa7a0102b40 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 0x7fa7a01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102b40 0x7fa7a019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 0x7fa7a01a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51346/0 (socket says 172.21.15.19:51346) 2024-05-17T09:28:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79f7fe640 1 -- 172.21.15.19:0/2923625457 learned_addr learned my addr 172.21.15.19:0/2923625457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa797fff640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79effd640 1 -- 172.21.15.19:0/2923625457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 msgr2=0x7fa7a01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79effd640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 0x7fa7a01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79effd640 1 -- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 msgr2=0x7fa7a019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79effd640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79effd640 1 -- 172.21.15.19:0/2923625457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a01a1cf0 con 0x7fa7a0102b40 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 0x7fa7a01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.204+0000 7fa797fff640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.205+0000 7fa79effd640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102b40 0x7fa7a019a1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0102410 tx=0x7fa79000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.205+0000 7fa7a48b6640 1 -- 172.21.15.19:0/2923625457 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000cd00 con 0x7fa7a0102b40 2024-05-17T09:28:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.205+0000 7fa7a48b6640 1 -- 172.21.15.19:0/2923625457 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7900108c0 con 0x7fa7a0102b40 2024-05-17T09:28:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.205+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a01a1fe0 con 0x7fa7a0102b40 2024-05-17T09:28:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.205+0000 7fa7a48b6640 1 -- 172.21.15.19:0/2923625457 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790018760 con 0x7fa7a0102b40 2024-05-17T09:28:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.205+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a01a2440 con 0x7fa7a0102b40 2024-05-17T09:28:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.207+0000 7fa7ac768740 1 Processor -- start 2024-05-17T09:28:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 -- start start 2024-05-17T09:28:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 0x558fee3017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7a48b6640 1 -- 172.21.15.19:0/2923625457 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa79001f020 con 0x7fa7a0102b40 2024-05-17T09:28:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf332f0 0x558fee301d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf3b4d0 0x558fee308c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558fee26fc30 con 0x558fedf3b4d0 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558fee25f7d0 con 0x558fedf332f0 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa7ac768740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558fee263810 con 0x558fedf5c1b0 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa797fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf3b4d0 0x558fee308c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 0x558fee3017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf332f0 0x558fee301d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 0x558fee3017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58380/0 (socket says 172.21.15.19:58380) 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 -- 172.21.15.19:0/1959951219 learned_addr learned my addr 172.21.15.19:0/1959951219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa797fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf3b4d0 0x558fee308c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49616/0 (socket says 172.21.15.19:49616) 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 -- 172.21.15.19:0/1959951219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf332f0 msgr2=0x558fee301d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf332f0 0x558fee301d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 -- 172.21.15.19:0/1959951219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf3b4d0 msgr2=0x558fee308c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf3b4d0 0x558fee308c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.208+0000 7fa79f7fe640 1 -- 172.21.15.19:0/1959951219 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fedf74210 con 0x558fedf5c1b0 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 0x558fee3017b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7980118b0 tx=0x7fa798011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9665067d2b887e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa795ffb640 1 -- 172.21.15.19:0/1959951219 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79801e070 con 0x558fedf5c1b0 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa795ffb640 1 -- 172.21.15.19:0/1959951219 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa79800fe50 con 0x558fedf5c1b0 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa795ffb640 1 -- 172.21.15.19:0/1959951219 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798019d00 con 0x558fedf5c1b0 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa7a48b6640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa78c0b28a0 0x7fa78c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa79effd640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa78c0b28a0 0x7fa78c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.209+0000 7fa7a48b6640 1 -- 172.21.15.19:0/2923625457 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa7900e2550 con 0x7fa7a0102b40 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 -- 172.21.15.19:0/1959951219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 msgr2=0x558fee3017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 0x558fee3017b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7980118b0 tx=0x7fa798011d70 comp rx=0 tx=0).stop 2024-05-17T09:28:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa79effd640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa78c0b28a0 0x7fa78c0b4db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa7900e5fd0 tx=0x7fa7900b2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 -- 172.21.15.19:0/1959951219 shutdown_connections 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf3b4d0 0x558fee308c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf332f0 0x558fee301d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1959951219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf5c1b0 0x558fee3017b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 -- 172.21.15.19:0/1959951219 >> 172.21.15.19:0/1959951219 conn(0x558fedf50300 msgr2=0x558fee2f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 -- 172.21.15.19:0/1959951219 shutdown_connections 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 -- 172.21.15.19:0/1959951219 wait complete. 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.210+0000 7fa7ac768740 1 -- start start 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 Processor -- start 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee30b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee30b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee3124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558fedf45790 con 0x558fedf5c1b0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558fee21b890 con 0x558fedf3b4d0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558fee309680 con 0x558fedf332f0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa7ac768740 10 client.0 _ll_register_callbacks cb 0x558fee21ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee3124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee30b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee3124a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49628/0 (socket says 172.21.15.19:49628) 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee30b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51360/0 (socket says 172.21.15.19:51360) 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa79f7fe640 1 -- 172.21.15.19:0/1734807065 learned_addr learned my addr 172.21.15.19:0/1734807065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee30b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 -- 172.21.15.19:0/1734807065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 msgr2=0x558fee30b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee30b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 -- 172.21.15.19:0/1734807065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 msgr2=0x558fee3124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee3124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.211+0000 7fa797fff640 1 -- 172.21.15.19:0/1734807065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fee040620 con 0x558fedf3b4d0 2024-05-17T09:28:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.212+0000 7fa797fff640 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee30b540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa78802a930 tx=0x7fa788031a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=942639d562a75bb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.212+0000 7fa77effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.214+0000 7fa77effd640 1 -- 172.21.15.19:0/1734807065 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788035040 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.214+0000 7fa77effd640 1 -- 172.21.15.19:0/1734807065 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa788004240 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.214+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79800fe50 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.214+0000 7fa77effd640 1 -- 172.21.15.19:0/1734807065 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa78804d050 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.214+0000 7fa7ac768740 10 client.26762 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.214+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558fee315490 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa77effd640 1 -- 172.21.15.19:0/1734807065 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa79800fe50 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa77effd640 1 -- 172.21.15.19:0/1734807065 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa78803a040 con 0x558fedf3b4d0 2024-05-17T09:28:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa7ac768740 10 client.26762 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa7ac768740 10 client.26762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa7ac768740 20 client.26762 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedf1ad80 0x558fee318c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa7ac768740 4 client.26762 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fee271b80 con 0x558fedf1ad80 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedf1ad80 0x558fee318c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.215+0000 7fa77effd640 10 client.26762 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.216+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedf1ad80 0x558fee318c00 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513efe72386b3ce3 server_cookie=a861e902468c1a7c in_seq=0 out_seq=0 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa77effd640 1 -- 172.21.15.19:0/1734807065 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558fee271b80 con 0x558fedf1ad80 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa77effd640 10 client.26762 handle_command_reply: tid=0 2024-05-17T09:28:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 client.26762 shutdown 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 msgr2=0x558fee30b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee30b540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa78802a930 tx=0x7fa788031a70 comp rx=0 tx=0).stop 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 shutdown_connections 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedf1ad80 0x558fee318c00 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee3124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee30b540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa77effd640 0 client.26762 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1734807065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee30b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 >> 172.21.15.19:0/1734807065 conn(0x558fedf50300 msgr2=0x558fee2fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 shutdown_connections 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 1 -- 172.21.15.19:0/1734807065 wait complete. 2024-05-17T09:28:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.217+0000 7fa7ac768740 20 client.26762 trim_cache size 0 max 16384 2024-05-17T09:28:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 Processor -- start 2024-05-17T09:28:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 -- start start 2024-05-17T09:28:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee2fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee305830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01ad0f0 con 0x558fedf5c1b0 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01aa9c0 con 0x558fedf3b4d0 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa7ac768740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01a8810 con 0x558fedf332f0 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee305830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee305830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49636/0 (socket says 172.21.15.19:49636) 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa79f7fe640 1 -- 172.21.15.19:0/2156980171 learned_addr learned my addr 172.21.15.19:0/2156980171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa797fff640 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee2fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa797fff640 1 -- 172.21.15.19:0/2156980171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 msgr2=0x558fee2fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa797fff640 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee2fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa797fff640 1 -- 172.21.15.19:0/2156980171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 msgr2=0x558fee305830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa797fff640 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee305830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa797fff640 1 -- 172.21.15.19:0/2156980171 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fee30ae00 con 0x558fedf3b4d0 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.236+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee305830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.237+0000 7fa797fff640 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3123c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa788009900 tx=0x7fa78802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a64730da7fb07b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.237+0000 7fa7957fa640 1 -- 172.21.15.19:0/2156980171 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788035040 con 0x558fedf3b4d0 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.237+0000 7fa7957fa640 1 -- 172.21.15.19:0/2156980171 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7880044c0 con 0x558fedf3b4d0 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.237+0000 7fa7957fa640 1 -- 172.21.15.19:0/2156980171 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788003a30 con 0x558fedf3b4d0 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 -- 172.21.15.19:0/2156980171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 msgr2=0x558fee3123c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3123c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa788009900 tx=0x7fa78802a410 comp rx=0 tx=0).stop 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 -- 172.21.15.19:0/2156980171 shutdown_connections 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee305830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3123c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 --2- 172.21.15.19:0/2156980171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee2fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 -- 172.21.15.19:0/2156980171 >> 172.21.15.19:0/2156980171 conn(0x558fedf50300 msgr2=0x558fee30a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 -- 172.21.15.19:0/2156980171 shutdown_connections 2024-05-17T09:28:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 -- 172.21.15.19:0/2156980171 wait complete. 2024-05-17T09:28:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 -- start start 2024-05-17T09:28:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.238+0000 7fa7ac768740 1 Processor -- start 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee310c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee311700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01a8e80 con 0x558fedf5c1b0 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01aa2f0 con 0x558fedf3b4d0 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01a9550 con 0x558fedf332f0 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa7ac768740 10 client.0 _ll_register_callbacks cb 0x558fee21ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee311700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa797fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee311700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49648/0 (socket says 172.21.15.19:49648) 2024-05-17T09:28:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa79f7fe640 1 -- 172.21.15.19:0/1070044625 learned_addr learned my addr 172.21.15.19:0/1070044625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee310c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa797fff640 1 -- 172.21.15.19:0/1070044625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 msgr2=0x558fee310c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa797fff640 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee310c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa797fff640 1 -- 172.21.15.19:0/1070044625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 msgr2=0x558fee311700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa797fff640 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee311700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.239+0000 7fa797fff640 1 -- 172.21.15.19:0/1070044625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fee37f310 con 0x558fedf3b4d0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa797fff640 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3111c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa788005760 tx=0x7fa78803ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4f91b8729444b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa77effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa77effd640 1 -- 172.21.15.19:0/1070044625 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788035070 con 0x558fedf3b4d0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa77effd640 1 -- 172.21.15.19:0/1070044625 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa78803c860 con 0x558fedf3b4d0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fee26b360 con 0x558fedf3b4d0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa77effd640 1 -- 172.21.15.19:0/1070044625 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa788062050 con 0x558fedf3b4d0 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 10 client.26768 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558fee3805a0 con 0x558fedf3b4d0 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa77effd640 1 -- 172.21.15.19:0/1070044625 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558fee26b360 con 0x558fedf3b4d0 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa77effd640 1 -- 172.21.15.19:0/1070044625 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa7880514c0 con 0x558fedf3b4d0 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 10 client.26768 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 10 client.26768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 20 client.26768 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.240+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedef01c0 0x558fee382f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.241+0000 7fa7ac768740 4 client.26768 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.241+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558fee1e55f0 con 0x558fedef01c0 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.241+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedef01c0 0x558fee382f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.241+0000 7fa79f7fe640 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedef01c0 0x558fee382f20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f7ebca21553ed9 server_cookie=8b2709b5b343093e in_seq=0 out_seq=0 2024-05-17T09:28:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.241+0000 7fa77effd640 10 client.26768 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.262+0000 7fa77effd640 1 -- 172.21.15.19:0/1070044625 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558fee1e55f0 con 0x558fedef01c0 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.262+0000 7fa77effd640 10 client.26768 handle_command_reply: tid=0 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.262+0000 7fa7ac768740 1 client.26768 shutdown 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 msgr2=0x558fee3111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3111c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa788005760 tx=0x7fa78803ddf0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 shutdown_connections 2024-05-17T09:28:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x558fedef01c0 0x558fee382f20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fedf5c1b0 0x558fee311700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa77effd640 0 client.26768 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fedf3b4d0 0x558fee3111c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 --2- 172.21.15.19:0/1070044625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558fedf332f0 0x558fee310c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 >> 172.21.15.19:0/1070044625 conn(0x558fedf50300 msgr2=0x558fee30a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 shutdown_connections 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 1 -- 172.21.15.19:0/1070044625 wait complete. 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.263+0000 7fa7ac768740 20 client.26768 trim_cache size 0 max 16384 2024-05-17T09:28:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:28:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:28:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa78c0b28a0 msgr2=0x7fa78c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa78c0b28a0 0x7fa78c0b4db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa7900e5fd0 tx=0x7fa7900b2040 comp rx=0 tx=0).stop 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102b40 msgr2=0x7fa7a019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102b40 0x7fa7a019a1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0102410 tx=0x7fa79000ee50 comp rx=0 tx=0).stop 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 shutdown_connections 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7a0108a80 0x7fa7a01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa78c0b28a0 0x7fa78c0b4db0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a0103d80 0x7fa7a019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 --2- 172.21.15.19:0/2923625457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7a0102b40 0x7fa7a019a1b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 >> 172.21.15.19:0/2923625457 conn(0x7fa7a00fdd30 msgr2=0x7fa7a0109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 shutdown_connections 2024-05-17T09:28:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:29.264+0000 7fa7a58b8640 1 -- 172.21.15.19:0/2923625457 wait complete. 2024-05-17T09:28:29.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 133052 1.81 MB/sec execute 267 sec latency 19.367 ms 2024-05-17T09:28:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 -- 172.21.15.19:0/376006348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c108760 msgr2=0x7f488c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 --2- 172.21.15.19:0/376006348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c108760 0x7f488c1100c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f488400b0d0 tx=0x7f488400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 -- 172.21.15.19:0/376006348 shutdown_connections 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 --2- 172.21.15.19:0/376006348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c108760 0x7f488c1100c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 --2- 172.21.15.19:0/376006348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c103c10 0x7f488c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 --2- 172.21.15.19:0/376006348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488c107e10 0x7f488c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 -- 172.21.15.19:0/376006348 >> 172.21.15.19:0/376006348 conn(0x7f488c0fe6e0 msgr2=0x7f488c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 -- 172.21.15.19:0/376006348 shutdown_connections 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.951+0000 7f489307c700 1 -- 172.21.15.19:0/376006348 wait complete. 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 Processor -- start 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 -- start start 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 0x7f488c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488c107e10 0x7f488c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c108760 0x7f488c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f488c113260 con 0x7f488c103c10 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f488c1130e0 con 0x7f488c108760 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.952+0000 7f489307c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f488c1133e0 con 0x7f488c107e10 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 0x7f488c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 0x7f488c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49664/0 (socket says 172.21.15.19:49664) 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 -- 172.21.15.19:0/4091060717 learned_addr learned my addr 172.21.15.19:0/4091060717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f488bfff700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488c107e10 0x7f488c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4891619700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c108760 0x7f488c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 -- 172.21.15.19:0/4091060717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488c107e10 msgr2=0x7f488c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488c107e10 0x7f488c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 -- 172.21.15.19:0/4091060717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c108760 msgr2=0x7f488c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c108760 0x7f488c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 -- 172.21.15.19:0/4091060717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c19a0d0 con 0x7f488c103c10 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4891619700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c108760 0x7f488c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4890e18700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 0x7f488c19f800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f488000e970 tx=0x7f488000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880004a30 con 0x7f488c103c10 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f488000cc70 con 0x7f488c103c10 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880005520 con 0x7f488c103c10 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.953+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c19a3c0 con 0x7f488c103c10 2024-05-17T09:28:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.954+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c1a6690 con 0x7f488c103c10 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.955+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c0512b0 con 0x7f488c103c10 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.957+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f488000c7e0 con 0x7f488c103c10 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.957+0000 7f4889ffb700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f487c0b2900 0x7f487c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.957+0000 7f488bfff700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f487c0b2900 0x7f487c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.958+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f48800e2690 con 0x7f488c103c10 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.958+0000 7f488bfff700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f487c0b2900 0x7f487c0b4e10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f488c106750 tx=0x7f487800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:29.963+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f488000ce20 con 0x7f488c103c10 2024-05-17T09:28:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.279+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f488c19af50 con 0x7f488c103c10 2024-05-17T09:28:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.280+0000 7f4889ffb700 1 -- 172.21.15.19:0/4091060717 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f48800aa000 con 0x7f488c103c10 2024-05-17T09:28:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:28:30.280 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:28:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f487c0b2900 msgr2=0x7f487c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f487c0b2900 0x7f487c0b4e10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f488c106750 tx=0x7f487800b410 comp rx=0 tx=0).stop 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 msgr2=0x7f488c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 0x7f488c19f800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f488000e970 tx=0x7f488000ec80 comp rx=0 tx=0).stop 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 shutdown_connections 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f488c108760 0x7f488c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f488c103c10 0x7f488c19f800 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f487c0b2900 0x7f487c0b4e10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 --2- 172.21.15.19:0/4091060717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488c107e10 0x7f488c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 >> 172.21.15.19:0/4091060717 conn(0x7f488c0fe6e0 msgr2=0x7f488c109670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 shutdown_connections 2024-05-17T09:28:30.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:30.282+0000 7f489307c700 1 -- 172.21.15.19:0/4091060717 wait complete. 2024-05-17T09:28:30.283 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:28:30.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 133635 1.81 MB/sec execute 268 sec latency 67.916 ms 2024-05-17T09:28:30.482 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:30 smithi069 ceph-mon[35081]: pgmap v700: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-17T09:28:30.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:30 smithi079 ceph-mon[35502]: pgmap v700: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-17T09:28:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:30 smithi019 ceph-mon[31620]: pgmap v700: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-17T09:28:31.424 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 141.6 --await-for 45.0' 2024-05-17T09:28:31.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 141.6 --await-for 45.0 2024-05-17T09:28:31.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 134183 1.82 MB/sec execute 269 sec latency 16.366 ms 2024-05-17T09:28:31.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4091060717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:31.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4091060717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:31.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4091060717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:31.710 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:28:32.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:32 smithi019 ceph-mon[31620]: pgmap v701: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-05-17T09:28:32.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 134610 1.83 MB/sec execute 270 sec latency 16.041 ms 2024-05-17T09:28:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:32 smithi079 ceph-mon[35502]: pgmap v701: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-05-17T09:28:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:32 smithi069 ceph-mon[35081]: pgmap v701: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-05-17T09:28:33.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135046 1.83 MB/sec execute 271 sec latency 78.171 ms 2024-05-17T09:28:34.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135237 1.83 MB/sec execute 272 sec latency 81.899 ms 2024-05-17T09:28:34.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:34 smithi079 ceph-mon[35502]: pgmap v702: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-17T09:28:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:34 smithi069 ceph-mon[35081]: pgmap v702: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-17T09:28:34.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:34 smithi019 ceph-mon[31620]: pgmap v702: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-17T09:28:35.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135521 1.82 MB/sec execute 273 sec latency 89.362 ms 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 -- 172.21.15.19:0/3227986858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 msgr2=0x7fb134108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 --2- 172.21.15.19:0/3227986858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 0x7fb134108600 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb130009990 tx=0x7fb130009ca0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 -- 172.21.15.19:0/3227986858 shutdown_connections 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 --2- 172.21.15.19:0/3227986858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb134104230 0x7fb134110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 --2- 172.21.15.19:0/3227986858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 0x7fb134108600 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 --2- 172.21.15.19:0/3227986858 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1341038e0 0x7fb134103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 -- 172.21.15.19:0/3227986858 >> 172.21.15.19:0/3227986858 conn(0x7fb1340fe6e0 msgr2=0x7fb134100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 -- 172.21.15.19:0/3227986858 shutdown_connections 2024-05-17T09:28:36.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.276+0000 7fb13b595700 1 -- 172.21.15.19:0/3227986858 wait complete. 2024-05-17T09:28:36.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.277+0000 7fb13b595700 1 Processor -- start 2024-05-17T09:28:36.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.277+0000 7fb13b595700 1 -- start start 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb13b595700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 0x7fb1340febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb13b595700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb134104230 0x7fb1340ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb13b595700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 0x7fb1340ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb13b595700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb134113210 con 0x7fb1341081f0 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb13b595700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb134113090 con 0x7fb1341038e0 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb13b595700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb134113390 con 0x7fb134104230 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139b32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 0x7fb1340ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 0x7fb1340febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 0x7fb1340febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55188/0 (socket says 172.21.15.19:55188) 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 -- 172.21.15.19:0/1173262968 learned_addr learned my addr 172.21.15.19:0/1173262968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb138b30700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb134104230 0x7fb1340ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 -- 172.21.15.19:0/1173262968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb134104230 msgr2=0x7fb1340ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb134104230 0x7fb1340ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 -- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 msgr2=0x7fb1340ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 0x7fb1340ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 -- 172.21.15.19:0/1173262968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1341a90f0 con 0x7fb1341038e0 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb138b30700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb134104230 0x7fb1340ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.278+0000 7fb139331700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 0x7fb1340febc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1300057c0 tx=0x7fb1300057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.279+0000 7fb12a7fc700 1 -- 172.21.15.19:0/1173262968 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130026070 con 0x7fb1341038e0 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.279+0000 7fb12a7fc700 1 -- 172.21.15.19:0/1173262968 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb130021410 con 0x7fb1341038e0 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.279+0000 7fb12a7fc700 1 -- 172.21.15.19:0/1173262968 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130017650 con 0x7fb1341038e0 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.279+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1341a9380 con 0x7fb1341038e0 2024-05-17T09:28:36.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.279+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1341a9780 con 0x7fb1341038e0 2024-05-17T09:28:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.282+0000 7fb12a7fc700 1 -- 172.21.15.19:0/1173262968 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb130029070 con 0x7fb1341038e0 2024-05-17T09:28:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.282+0000 7fb12a7fc700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb1200b2880 0x7fb1200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.283+0000 7fb138b30700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb1200b2880 0x7fb1200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.283+0000 7fb12a7fc700 1 -- 172.21.15.19:0/1173262968 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb1300e33b0 con 0x7fb1341038e0 2024-05-17T09:28:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.283+0000 7fb149faab80 1 Processor -- start 2024-05-17T09:28:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.283+0000 7fb149faab80 1 -- start start 2024-05-17T09:28:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.283+0000 7fb138b30700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb1200b2880 0x7fb1200b4d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb134107d90 tx=0x7fb124009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb149faab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb149faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd9230fd0 0x563bd93b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb149faab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd93af6b0 0x563bd93afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb149faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563bd92dde20 con 0x563bd9230fd0 2024-05-17T09:28:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb149faab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bd92cd9c0 con 0x563bd92ae560 2024-05-17T09:28:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb149faab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563bd92d1a00 con 0x563bd93af6b0 2024-05-17T09:28:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb138b30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd9230fd0 0x563bd93b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55190/0 (socket says 172.21.15.19:55190) 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 -- 172.21.15.19:0/1991405189 learned_addr learned my addr 172.21.15.19:0/1991405189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb138b30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd9230fd0 0x563bd93b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52098/0 (socket says 172.21.15.19:52098) 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139b32700 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd93af6b0 0x563bd93afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 -- 172.21.15.19:0/1991405189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd93af6b0 msgr2=0x563bd93afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd93af6b0 0x563bd93afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 -- 172.21.15.19:0/1991405189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd9230fd0 msgr2=0x563bd93b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd9230fd0 0x563bd93b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 -- 172.21.15.19:0/1991405189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd921da60 con 0x563bd92ae560 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb139331700 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93b55f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1300a6770 tx=0x7fb1300aa0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a0608c49c43242 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.284+0000 7fb11f7fe700 1 -- 172.21.15.19:0/1991405189 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300aadf0 con 0x563bd92ae560 2024-05-17T09:28:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb11f7fe700 1 -- 172.21.15.19:0/1991405189 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1300aa6f0 con 0x563bd92ae560 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb11f7fe700 1 -- 172.21.15.19:0/1991405189 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300920d0 con 0x563bd92ae560 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb149faab80 1 -- 172.21.15.19:0/1991405189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 msgr2=0x563bd93b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb149faab80 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93b55f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1300a6770 tx=0x7fb1300aa0c0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb149faab80 1 -- 172.21.15.19:0/1991405189 shutdown_connections 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb149faab80 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd93af6b0 0x563bd93afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.285+0000 7fb149faab80 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93b55f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.286+0000 7fb149faab80 1 --2- 172.21.15.19:0/1991405189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd9230fd0 0x563bd93b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.286+0000 7fb149faab80 1 -- 172.21.15.19:0/1991405189 >> 172.21.15.19:0/1991405189 conn(0x563bd926fe10 msgr2=0x563bd93aa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.286+0000 7fb149faab80 1 -- 172.21.15.19:0/1991405189 shutdown_connections 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.286+0000 7fb149faab80 1 -- 172.21.15.19:0/1991405189 wait complete. 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.286+0000 7fb149faab80 1 -- start start 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.286+0000 7fb149faab80 1 Processor -- start 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd92ae560 0x563bd93bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd93af6b0 0x563bd93b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563bd91d4130 con 0x563bd92ae560 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563bd926f950 con 0x563bd93af6b0 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563bd93b03b0 con 0x563bd9230fd0 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb149faab80 10 client.0 _ll_register_callbacks cb 0x563bd9268ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb138b30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd92ae560 0x563bd93bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb138b30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd92ae560 0x563bd93bf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52106/0 (socket says 172.21.15.19:52106) 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb138b30700 1 -- 172.21.15.19:0/1930915973 learned_addr learned my addr 172.21.15.19:0/1930915973 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139b32700 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd93af6b0 0x563bd93b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 -- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd93af6b0 msgr2=0x563bd93b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd93af6b0 0x563bd93b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 -- 172.21.15.19:0/1930915973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd92ae560 msgr2=0x563bd93bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd92ae560 0x563bd93bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 -- 172.21.15.19:0/1930915973 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd926c820 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb139331700 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93bee00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x563bd93b0180 tx=0x7fb1300a6620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1379ad50e4dce20b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb11cff9700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.287+0000 7fb11cff9700 1 -- 172.21.15.19:0/1930915973 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300c4060 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb11cff9700 1 -- 172.21.15.19:0/1930915973 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb1300aa950 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1300aa6f0 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb11cff9700 1 -- 172.21.15.19:0/1930915973 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb1300aeaa0 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb149faab80 10 client.26773 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bd93c6a20 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb11cff9700 1 -- 172.21.15.19:0/1930915973 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1300aa6f0 con 0x563bd9230fd0 2024-05-17T09:28:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.288+0000 7fb11cff9700 1 -- 172.21.15.19:0/1930915973 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb130005110 con 0x563bd9230fd0 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb149faab80 10 client.26773 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb149faab80 10 client.26773 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb149faab80 20 client.26773 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb149faab80 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd93c8fe0 0x563bd93cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb149faab80 4 client.26773 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bd92dfd70 con 0x563bd93c8fe0 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.289+0000 7fb138b30700 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd93c8fe0 0x563bd93cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.290+0000 7fb11cff9700 10 client.26773 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.290+0000 7fb138b30700 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd93c8fe0 0x563bd93cb440 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b48d162c633a443 server_cookie=ab037ecfb8e99316 in_seq=0 out_seq=0 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.290+0000 7fb11cff9700 1 -- 172.21.15.19:0/1930915973 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563bd92dfd70 con 0x563bd93c8fe0 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.290+0000 7fb11cff9700 10 client.26773 handle_command_reply: tid=0 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.290+0000 7fb149faab80 1 client.26773 shutdown 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 msgr2=0x563bd93bee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93bee00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x563bd93b0180 tx=0x7fb1300a6620 comp rx=0 tx=0).stop 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 shutdown_connections 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93bee00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd93c8fe0 0x563bd93cb440 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd93af6b0 0x563bd93b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 --2- 172.21.15.19:0/1930915973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd92ae560 0x563bd93bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 >> 172.21.15.19:0/1930915973 conn(0x563bd926fe10 msgr2=0x563bd93a4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb11cff9700 0 client.26773 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 shutdown_connections 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 1 -- 172.21.15.19:0/1930915973 wait complete. 2024-05-17T09:28:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.291+0000 7fb149faab80 20 client.26773 trim_cache size 0 max 16384 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.315+0000 7fb149faab80 1 Processor -- start 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 -- start start 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341b4dc0 con 0x563bd93af6b0 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341b2650 con 0x563bd92ae560 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb149faab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341b04a0 con 0x563bd9230fd0 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:45224/0 (socket says 172.21.15.19:45224) 2024-05-17T09:28:36.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 -- 172.21.15.19:0/3529142668 learned_addr learned my addr 172.21.15.19:0/3529142668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139b32700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb138b30700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 -- 172.21.15.19:0/3529142668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 msgr2=0x563bd93d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 -- 172.21.15.19:0/3529142668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 msgr2=0x563bd93a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139331700 1 -- 172.21.15.19:0/3529142668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd93b0600 con 0x563bd9230fd0 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb139b32700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.316+0000 7fb138b30700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.317+0000 7fb139331700 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93d0bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb1300a6770 tx=0x7fb1300a65b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf387b2d2d5eb4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.317+0000 7fb11effd700 1 -- 172.21.15.19:0/3529142668 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300c4070 con 0x563bd9230fd0 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.317+0000 7fb11effd700 1 -- 172.21.15.19:0/3529142668 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1300aadf0 con 0x563bd9230fd0 2024-05-17T09:28:36.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.317+0000 7fb11effd700 1 -- 172.21.15.19:0/3529142668 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300c9080 con 0x563bd9230fd0 2024-05-17T09:28:36.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 -- 172.21.15.19:0/3529142668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 msgr2=0x563bd93d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93d0bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb1300a6770 tx=0x7fb1300a65b0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 -- 172.21.15.19:0/3529142668 shutdown_connections 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd9230fd0 0x563bd93d0bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 --2- 172.21.15.19:0/3529142668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd92ae560 0x563bd93d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 -- 172.21.15.19:0/3529142668 >> 172.21.15.19:0/3529142668 conn(0x563bd926fe10 msgr2=0x563bd93b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 -- 172.21.15.19:0/3529142668 shutdown_connections 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 -- 172.21.15.19:0/3529142668 wait complete. 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 -- start start 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.318+0000 7fb149faab80 1 Processor -- start 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 0x563bd93c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd92ae560 0x563bd93c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341b0b10 con 0x563bd93af6b0 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341b1f80 con 0x563bd9230fd0 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341b11e0 con 0x563bd92ae560 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb139b32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb139331700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 0x563bd93c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb139331700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 0x563bd93c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55234/0 (socket says 172.21.15.19:55234) 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb139b32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52116/0 (socket says 172.21.15.19:52116) 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb139331700 1 -- 172.21.15.19:0/1415439364 learned_addr learned my addr 172.21.15.19:0/1415439364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb138b30700 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd92ae560 0x563bd93c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.319+0000 7fb149faab80 10 client.0 _ll_register_callbacks cb 0x563bd9268ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb139331700 1 -- 172.21.15.19:0/1415439364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd92ae560 msgr2=0x563bd93c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb139331700 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd92ae560 0x563bd93c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb139331700 1 -- 172.21.15.19:0/1415439364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 msgr2=0x563bd93c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb139331700 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb139331700 1 -- 172.21.15.19:0/1415439364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd93a3f70 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb139331700 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 0x563bd93c4d70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb13000fb20 tx=0x7fb13000ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76955cc8b5d0ac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb11cff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb11cff9700 1 -- 172.21.15.19:0/1415439364 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300c9070 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb11cff9700 1 -- 172.21.15.19:0/1415439364 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb1300a6ce0 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1300aadf0 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb11cff9700 1 -- 172.21.15.19:0/1415439364 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb13008e050 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb149faab80 10 client.26792 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.320+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bd93a3f70 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb11cff9700 1 -- 172.21.15.19:0/1415439364 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1300aadf0 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb11cff9700 1 -- 172.21.15.19:0/1415439364 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb1300a6060 con 0x563bd9230fd0 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb149faab80 10 client.26792 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb149faab80 10 client.26792 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb149faab80 20 client.26792 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb149faab80 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd9434540 0x563bd9436960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb149faab80 4 client.26792 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.6, "await_for": 45.0}] 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.6, "await_for": 45.0}) -- 0x563bd92d9550 con 0x563bd9434540 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb138b30700 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd9434540 0x563bd9436960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.321+0000 7fb11cff9700 10 client.26792 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.322+0000 7fb138b30700 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd9434540 0x563bd9436960 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0f4578cafa7545 server_cookie=8995205094a6e170 in_seq=0 out_seq=0 2024-05-17T09:28:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.375+0000 7fb11cff9700 1 -- 172.21.15.19:0/1415439364 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563bd92d9550 con 0x563bd9434540 2024-05-17T09:28:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb11cff9700 10 client.26792 handle_command_reply: tid=0 2024-05-17T09:28:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 client.26792 shutdown 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 msgr2=0x563bd93c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 0x563bd93c4d70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb13000fb20 tx=0x7fb13000ff40 comp rx=0 tx=0).stop 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 shutdown_connections 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563bd9434540 0x563bd9436960 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563bd93af6b0 0x563bd93c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563bd9230fd0 0x563bd93c4d70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb11cff9700 0 client.26792 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 --2- 172.21.15.19:0/1415439364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563bd92ae560 0x563bd93c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 >> 172.21.15.19:0/1415439364 conn(0x563bd926fe10 msgr2=0x563bd93b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 shutdown_connections 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 1 -- 172.21.15.19:0/1415439364 wait complete. 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.376+0000 7fb149faab80 20 client.26792 trim_cache size 0 max 16384 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 30, 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:28:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "28f4e22": { 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 30, 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 141.6, 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:28:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:36.379 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb1200b2880 msgr2=0x7fb1200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb1200b2880 0x7fb1200b4d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb134107d90 tx=0x7fb124009260 comp rx=0 tx=0).stop 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 msgr2=0x7fb1340febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 0x7fb1340febc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1300057c0 tx=0x7fb1300057f0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 shutdown_connections 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1341081f0 0x7fb1340ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1341038e0 0x7fb1340febc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb1200b2880 0x7fb1200b4d90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 --2- 172.21.15.19:0/1173262968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb134104230 0x7fb1340ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 >> 172.21.15.19:0/1173262968 conn(0x7fb1340fe6e0 msgr2=0x7fb134102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 shutdown_connections 2024-05-17T09:28:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:36.377+0000 7fb13b595700 1 -- 172.21.15.19:0/1173262968 wait complete. 2024-05-17T09:28:36.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:36 smithi019 ceph-mon[31620]: pgmap v703: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-17T09:28:36.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.82 MB/sec execute 274 sec latency 59.752 ms 2024-05-17T09:28:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:36 smithi079 ceph-mon[35502]: pgmap v703: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-17T09:28:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:36 smithi069 ceph-mon[35081]: pgmap v703: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-17T09:28:37.187 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '28f4e22', quiesce took 9.7 seconds. Will release after: 11.900000000000002 2024-05-17T09:28:37.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.82 MB/sec execute 275 sec latency 1059.835 ms 2024-05-17T09:28:38.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.81 MB/sec execute 276 sec latency 2059.956 ms 2024-05-17T09:28:38.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:38 smithi079 ceph-mon[35502]: pgmap v704: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:28:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:38 smithi069 ceph-mon[35081]: pgmap v704: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:28:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:38 smithi019 ceph-mon[31620]: pgmap v704: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:28:39.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.80 MB/sec execute 277 sec latency 3060.065 ms 2024-05-17T09:28:40.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.80 MB/sec execute 278 sec latency 4060.115 ms 2024-05-17T09:28:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:40 smithi079 ceph-mon[35502]: pgmap v705: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:28:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:40 smithi069 ceph-mon[35081]: pgmap v705: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:28:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:40 smithi019 ceph-mon[31620]: pgmap v705: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:28:41.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.79 MB/sec execute 279 sec latency 5060.170 ms 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 -- 172.21.15.19:0/2226722346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8109b80 msgr2=0x7fb4c810bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 --2- 172.21.15.19:0/2226722346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8109b80 0x7fb4c810bfa0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ca0 tx=0x7fb4c002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 -- 172.21.15.19:0/2226722346 shutdown_connections 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 --2- 172.21.15.19:0/2226722346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8109b80 0x7fb4c810bfa0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 --2- 172.21.15.19:0/2226722346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4c8104e80 0x7fb4c8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 --2- 172.21.15.19:0/2226722346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4c8103c40 0x7fb4c8104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 -- 172.21.15.19:0/2226722346 >> 172.21.15.19:0/2226722346 conn(0x7fb4c80fee30 msgr2=0x7fb4c81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 -- 172.21.15.19:0/2226722346 shutdown_connections 2024-05-17T09:28:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.377+0000 7fb4ced72640 1 -- 172.21.15.19:0/2226722346 wait complete. 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 Processor -- start 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 -- start start 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4c8103c40 0x7fb4c819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 0x7fb4c819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4c8109b80 0x7fb4c81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810f670 con 0x7fb4c8104e80 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810f4f0 con 0x7fb4c8103c40 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.378+0000 7fb4ced72640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810f7f0 con 0x7fb4c8109b80 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 0x7fb4c819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 0x7fb4c819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52142/0 (socket says 172.21.15.19:52142) 2024-05-17T09:28:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 -- 172.21.15.19:0/3052421002 learned_addr learned my addr 172.21.15.19:0/3052421002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4c8103c40 0x7fb4c819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4c8109b80 0x7fb4c81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 -- 172.21.15.19:0/3052421002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4c8109b80 msgr2=0x7fb4c81a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4c8109b80 0x7fb4c81a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 -- 172.21.15.19:0/3052421002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4c8103c40 msgr2=0x7fb4c819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4c8103c40 0x7fb4c819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 -- 172.21.15.19:0/3052421002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c81a1c90 con 0x7fb4c8104e80 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 0x7fb4c819a690 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d950 tx=0x7fb4bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c5ffb640 1 -- 172.21.15.19:0/3052421002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc017070 con 0x7fb4c8104e80 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c5ffb640 1 -- 172.21.15.19:0/3052421002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4bc009c30 con 0x7fb4c8104e80 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a1f80 con 0x7fb4c8104e80 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.379+0000 7fb4c5ffb640 1 -- 172.21.15.19:0/3052421002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00f620 con 0x7fb4c8104e80 2024-05-17T09:28:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.380+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a2440 con 0x7fb4c8104e80 2024-05-17T09:28:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.382+0000 7fb4c5ffb640 1 -- 172.21.15.19:0/3052421002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb4bc010460 con 0x7fb4c8104e80 2024-05-17T09:28:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.382+0000 7fb4d5c24740 1 Processor -- start 2024-05-17T09:28:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.382+0000 7fb4d5c24740 1 -- start start 2024-05-17T09:28:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b7372f0 0x55559bb01920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 0x55559bb08820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55559ba73c30 con 0x55559b73f4d0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55559ba637d0 con 0x55559b7372f0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55559ba67810 con 0x55559b7601b0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 0x55559bb08820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 0x55559bb08820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52154/0 (socket says 172.21.15.19:52154) 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 -- 172.21.15.19:0/1980046235 learned_addr learned my addr 172.21.15.19:0/1980046235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c5ffb640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4b00b2920 0x7fb4b00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b7372f0 0x55559bb01920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4b00b2920 0x7fb4b00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c5ffb640 1 -- 172.21.15.19:0/3052421002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4bc0e2170 con 0x7fb4c8104e80 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 -- 172.21.15.19:0/1980046235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 msgr2=0x55559bb013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 -- 172.21.15.19:0/1980046235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b7372f0 msgr2=0x55559bb01920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b7372f0 0x55559bb01920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4c7fff640 1 -- 172.21.15.19:0/1980046235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559b778210 con 0x55559b73f4d0 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.383+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.384+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 0x55559bb08820 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0a6650 tx=0x7fb4bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c283ee3e3d0fefef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.384+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4b00b2920 0x7fb4b00b4e30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55559bb02570 tx=0x7fb4c002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.384+0000 7fb4aeffd640 1 -- 172.21.15.19:0/1980046235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc020070 con 0x55559b73f4d0 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.384+0000 7fb4aeffd640 1 -- 172.21.15.19:0/1980046235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4bc0a8ef0 con 0x55559b73f4d0 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.384+0000 7fb4aeffd640 1 -- 172.21.15.19:0/1980046235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0a98a0 con 0x55559b73f4d0 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 -- 172.21.15.19:0/1980046235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 msgr2=0x55559bb08820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 0x55559bb08820 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0a6650 tx=0x7fb4bc005880 comp rx=0 tx=0).stop 2024-05-17T09:28:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 -- 172.21.15.19:0/1980046235 shutdown_connections 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b73f4d0 0x55559bb08820 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b7372f0 0x55559bb01920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/1980046235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 -- 172.21.15.19:0/1980046235 >> 172.21.15.19:0/1980046235 conn(0x55559b754300 msgr2=0x55559bafcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 -- 172.21.15.19:0/1980046235 shutdown_connections 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 -- 172.21.15.19:0/1980046235 wait complete. 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 -- start start 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.385+0000 7fb4d5c24740 1 Processor -- start 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7372f0 0x55559bb0ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 0x55559bb120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55559b749790 con 0x55559b7601b0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55559ba1f890 con 0x55559b73f4d0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55559bb09210 con 0x55559b7372f0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4d5c24740 10 client.0 _ll_register_callbacks cb 0x55559ba22d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 0x55559bb120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 0x55559bb120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52158/0 (socket says 172.21.15.19:52158) 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 -- 172.21.15.19:0/3669785847 learned_addr learned my addr 172.21.15.19:0/3669785847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7372f0 0x55559bb0ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 -- 172.21.15.19:0/3669785847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7372f0 msgr2=0x55559bb0ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7372f0 0x55559bb0ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 -- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 msgr2=0x55559bb0b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.386+0000 7fb4cd30f640 1 -- 172.21.15.19:0/3669785847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559b844620 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 0x55559bb120f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0035000 tx=0x7fb4c00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a5ee556b24820 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4a7fff640 1 -- 172.21.15.19:0/3669785847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0045070 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4a7fff640 1 -- 172.21.15.19:0/3669785847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4c00307c0 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4bc0a8ef0 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4a7fff640 1 -- 172.21.15.19:0/3669785847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4c0004b40 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4d5c24740 10 client.17172 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55559bb15090 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.387+0000 7fb4a7fff640 1 -- 172.21.15.19:0/3669785847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4bc0a8ef0 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4a7fff640 1 -- 172.21.15.19:0/3669785847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb4c0005430 con 0x55559b7601b0 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4d5c24740 10 client.17172 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4d5c24740 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4d5c24740 20 client.17172 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b71ed80 0x55559bb18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4d5c24740 4 client.17172 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55559ba75b80 con 0x55559b71ed80 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b71ed80 0x55559bb18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b71ed80 0x55559bb18820 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d29a54f7da3401c server_cookie=d5221c6b106a630a in_seq=0 out_seq=0 2024-05-17T09:28:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.388+0000 7fb4a7fff640 10 client.17172 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4a7fff640 1 -- 172.21.15.19:0/3669785847 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55559ba75b80 con 0x55559b71ed80 2024-05-17T09:28:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4a7fff640 10 client.17172 handle_command_reply: tid=0 2024-05-17T09:28:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 client.17172 shutdown 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 msgr2=0x55559bb120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 0x55559bb120f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0035000 tx=0x7fb4c00095c0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 shutdown_connections 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b71ed80 0x55559bb18820 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7601b0 0x55559bb120f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4a7fff640 0 client.17172 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/3669785847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7372f0 0x55559bb0ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.389+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 >> 172.21.15.19:0/3669785847 conn(0x55559b754300 msgr2=0x55559baffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.390+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 shutdown_connections 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.390+0000 7fb4d5c24740 1 -- 172.21.15.19:0/3669785847 wait complete. 2024-05-17T09:28:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.390+0000 7fb4d5c24740 20 client.17172 trim_cache size 0 max 16384 2024-05-17T09:28:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 Processor -- start 2024-05-17T09:28:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 -- start start 2024-05-17T09:28:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb00240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb12010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb12550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81af380 con 0x55559b7372f0 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81acc50 con 0x55559b73f4d0 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.405+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81aaaa0 con 0x55559b7601b0 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb00240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb00240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52174/0 (socket says 172.21.15.19:52174) 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/519570894 learned_addr learned my addr 172.21.15.19:0/519570894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb12010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/519570894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 msgr2=0x55559bb12550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb12550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/519570894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 msgr2=0x55559bb12010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb12010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/519570894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559bb025a0 con 0x55559b7372f0 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb00240 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8104840 tx=0x7fb4b800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a4cc1af815880d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ae7fc640 1 -- 172.21.15.19:0/519570894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8005220 con 0x55559b7372f0 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ae7fc640 1 -- 172.21.15.19:0/519570894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b8004c30 con 0x55559b7372f0 2024-05-17T09:28:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.406+0000 7fb4ae7fc640 1 -- 172.21.15.19:0/519570894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b801d8d0 con 0x55559b7372f0 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 -- 172.21.15.19:0/519570894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 msgr2=0x55559bb00240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb00240 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8104840 tx=0x7fb4b800b390 comp rx=0 tx=0).stop 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 -- 172.21.15.19:0/519570894 shutdown_connections 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb12550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb12010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/519570894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb00240 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 -- 172.21.15.19:0/519570894 >> 172.21.15.19:0/519570894 conn(0x55559b754300 msgr2=0x55559baffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 -- 172.21.15.19:0/519570894 shutdown_connections 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 -- 172.21.15.19:0/519570894 wait complete. 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.407+0000 7fb4d5c24740 1 -- start start 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 Processor -- start 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb0fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81ab110 con 0x55559b7372f0 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81ac580 con 0x55559b73f4d0 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81ab7e0 con 0x55559b7601b0 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4d5c24740 10 client.0 _ll_register_callbacks cb 0x7fb4c0003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4ccb0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb0fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4ccb0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb0fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52188/0 (socket says 172.21.15.19:52188) 2024-05-17T09:28:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/923314094 learned_addr learned my addr 172.21.15.19:0/923314094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.408+0000 7fb4cd30f640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/923314094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 msgr2=0x55559bb10eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb10eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 msgr2=0x55559bb0ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4ccb0e640 1 -- 172.21.15.19:0/923314094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559bb11470 con 0x55559b7372f0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4ccb0e640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb0fa40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8005890 tx=0x7fb4b8004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4e10caed2b1517 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4a7fff640 1 -- 172.21.15.19:0/923314094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004fe0 con 0x55559b7372f0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4a7fff640 1 -- 172.21.15.19:0/923314094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb4b8005180 con 0x55559b7372f0 2024-05-17T09:28:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b8004c30 con 0x55559b7372f0 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4a7fff640 1 -- 172.21.15.19:0/923314094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4b803a050 con 0x55559b7372f0 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4d5c24740 10 client.17178 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55559bb12700 con 0x55559b7372f0 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.409+0000 7fb4a7fff640 1 -- 172.21.15.19:0/923314094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b8004c30 con 0x55559b7372f0 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4a7fff640 1 -- 172.21.15.19:0/923314094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb4b802f020 con 0x55559b7372f0 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4d5c24740 10 client.17178 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4d5c24740 10 client.17178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4d5c24740 20 client.17178 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b6f41c0 0x55559bb831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4d5c24740 4 client.17178 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55559ba6f360 con 0x55559b6f41c0 2024-05-17T09:28:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b6f41c0 0x55559bb831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b6f41c0 0x55559bb831f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4274c4f8f7e2b7 server_cookie=a20c6d8b33a901f in_seq=0 out_seq=0 2024-05-17T09:28:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.410+0000 7fb4a7fff640 10 client.17178 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4a7fff640 1 -- 172.21.15.19:0/923314094 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55559ba6f360 con 0x55559b6f41c0 2024-05-17T09:28:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4a7fff640 10 client.17178 handle_command_reply: tid=0 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 client.17178 shutdown 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 msgr2=0x55559bb0fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb0fa40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8005890 tx=0x7fb4b8004e70 comp rx=0 tx=0).stop 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 shutdown_connections 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55559b6f41c0 0x55559bb831f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55559b7601b0 0x55559bb10eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4a7fff640 0 client.17178 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55559b73f4d0 0x55559bb0ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 --2- 172.21.15.19:0/923314094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55559b7372f0 0x55559bb0fa40 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 >> 172.21.15.19:0/923314094 conn(0x55559b754300 msgr2=0x55559bb051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 shutdown_connections 2024-05-17T09:28:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 1 -- 172.21.15.19:0/923314094 wait complete. 2024-05-17T09:28:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.434+0000 7fb4d5c24740 20 client.17178 trim_cache size 0 max 16384 2024-05-17T09:28:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:28:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:28:42.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4b00b2920 msgr2=0x7fb4b00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4b00b2920 0x7fb4b00b4e30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55559bb02570 tx=0x7fb4c002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 msgr2=0x7fb4c819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 0x7fb4c819a690 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d950 tx=0x7fb4bc00de10 comp rx=0 tx=0).stop 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 shutdown_connections 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4c8109b80 0x7fb4c81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb4b00b2920 0x7fb4b00b4e30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c8104e80 0x7fb4c819a690 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 --2- 172.21.15.19:0/3052421002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4c8103c40 0x7fb4c819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 >> 172.21.15.19:0/3052421002 conn(0x7fb4c80fee30 msgr2=0x7fb4c8105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 shutdown_connections 2024-05-17T09:28:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:42.435+0000 7fb4ced72640 1 -- 172.21.15.19:0/3052421002 wait complete. 2024-05-17T09:28:42.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.78 MB/sec execute 280 sec latency 6060.219 ms 2024-05-17T09:28:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:42 smithi079 ceph-mon[35502]: pgmap v706: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 881 KiB/s wr, 103 op/s 2024-05-17T09:28:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:42 smithi069 ceph-mon[35081]: pgmap v706: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 881 KiB/s wr, 103 op/s 2024-05-17T09:28:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:42 smithi019 ceph-mon[31620]: pgmap v706: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 881 KiB/s wr, 103 op/s 2024-05-17T09:28:43.437 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.78 MB/sec execute 281 sec latency 7060.306 ms 2024-05-17T09:28:44.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.77 MB/sec execute 282 sec latency 8060.436 ms 2024-05-17T09:28:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:44 smithi069 ceph-mon[35081]: pgmap v707: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 867 KiB/s wr, 100 op/s 2024-05-17T09:28:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:44 smithi019 ceph-mon[31620]: pgmap v707: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 867 KiB/s wr, 100 op/s 2024-05-17T09:28:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:44 smithi079 ceph-mon[35502]: pgmap v707: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 867 KiB/s wr, 100 op/s 2024-05-17T09:28:45.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.77 MB/sec execute 283 sec latency 9060.496 ms 2024-05-17T09:28:46.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.76 MB/sec execute 284 sec latency 10060.545 ms 2024-05-17T09:28:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:46 smithi069 ceph-mon[35081]: pgmap v708: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 576 KiB/s wr, 90 op/s 2024-05-17T09:28:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:46 smithi019 ceph-mon[31620]: pgmap v708: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 576 KiB/s wr, 90 op/s 2024-05-17T09:28:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:46 smithi079 ceph-mon[35502]: pgmap v708: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 576 KiB/s wr, 90 op/s 2024-05-17T09:28:47.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.75 MB/sec execute 285 sec latency 11060.600 ms 2024-05-17T09:28:48.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.75 MB/sec execute 286 sec latency 12060.657 ms 2024-05-17T09:28:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:48 smithi069 ceph-mon[35081]: pgmap v709: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 334 KiB/s wr, 54 op/s 2024-05-17T09:28:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:48 smithi019 ceph-mon[31620]: pgmap v709: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 334 KiB/s wr, 54 op/s 2024-05-17T09:28:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:48 smithi079 ceph-mon[35502]: pgmap v709: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 334 KiB/s wr, 54 op/s 2024-05-17T09:28:49.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:28:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:28:49.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.74 MB/sec execute 287 sec latency 13060.771 ms 2024-05-17T09:28:50.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.73 MB/sec execute 288 sec latency 14060.870 ms 2024-05-17T09:28:50.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:50 smithi019 ceph-mon[31620]: pgmap v710: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-05-17T09:28:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:50 smithi069 ceph-mon[35081]: pgmap v710: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-05-17T09:28:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:50 smithi079 ceph-mon[35502]: pgmap v710: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-05-17T09:28:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 -- 172.21.15.19:0/239311329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c108750 msgr2=0x7fc46c1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 --2- 172.21.15.19:0/239311329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c108750 0x7fc46c1100b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc46400b0d0 tx=0x7fc46400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 -- 172.21.15.19:0/239311329 shutdown_connections 2024-05-17T09:28:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 --2- 172.21.15.19:0/239311329 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc46c1034d0 0x7fc46c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 --2- 172.21.15.19:0/239311329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c108750 0x7fc46c1100b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 --2- 172.21.15.19:0/239311329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc46c107e00 0x7fc46c108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 -- 172.21.15.19:0/239311329 >> 172.21.15.19:0/239311329 conn(0x7fc46c0fef20 msgr2=0x7fc46c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 -- 172.21.15.19:0/239311329 shutdown_connections 2024-05-17T09:28:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 -- 172.21.15.19:0/239311329 wait complete. 2024-05-17T09:28:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.196+0000 7fc473dda700 1 Processor -- start 2024-05-17T09:28:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 -- start start 2024-05-17T09:28:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 0x7fc46c0ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc46c107e00 0x7fc46c0ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc46c108750 0x7fc46c0ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c113230 con 0x7fc46c1034d0 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1130b0 con 0x7fc46c107e00 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc473dda700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1133b0 con 0x7fc46c108750 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 0x7fc46c0ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 0x7fc46c0ff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45464/0 (socket says 172.21.15.19:45464) 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 -- 172.21.15.19:0/639764107 learned_addr learned my addr 172.21.15.19:0/639764107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc472377700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc46c108750 0x7fc46c0ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471375700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc46c107e00 0x7fc46c0ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 -- 172.21.15.19:0/639764107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc46c108750 msgr2=0x7fc46c0ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc46c108750 0x7fc46c0ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 -- 172.21.15.19:0/639764107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc46c107e00 msgr2=0x7fc46c0ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc46c107e00 0x7fc46c0ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.197+0000 7fc471b76700 1 -- 172.21.15.19:0/639764107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c1a90d0 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.198+0000 7fc471b76700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 0x7fc46c0ff400 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9a0 tx=0x7fc46800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.198+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46800cbd0 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.198+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc46800cd70 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.198+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468018680 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.198+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c1a93c0 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.198+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1a9900 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.199+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46c0512b0 con 0x7fc46c1034d0 2024-05-17T09:28:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.201+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc468010950 con 0x7fc46c1034d0 2024-05-17T09:28:51.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.201+0000 7fc462ffd700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4580b2880 0x7fc4580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:51.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.202+0000 7fc471375700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4580b2880 0x7fc4580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:51.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.202+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc4680e2b00 con 0x7fc46c1034d0 2024-05-17T09:28:51.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.202+0000 7fc471375700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4580b2880 0x7fc4580b4d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc46c1066d0 tx=0x7fc45c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:51.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.205+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4680aa470 con 0x7fc46c1034d0 2024-05-17T09:28:51.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.73 MB/sec execute 289 sec latency 15060.919 ms 2024-05-17T09:28:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.477+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc46c1a9b60 con 0x7fc46c1034d0 2024-05-17T09:28:51.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.478+0000 7fc462ffd700 1 -- 172.21.15.19:0/639764107 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7fc4680a9d70 con 0x7fc46c1034d0 2024-05-17T09:28:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:28:51.478 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4580b2880 msgr2=0x7fc4580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4580b2880 0x7fc4580b4d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc46c1066d0 tx=0x7fc45c009260 comp rx=0 tx=0).stop 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 msgr2=0x7fc46c0ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 0x7fc46c0ff400 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9a0 tx=0x7fc46800ed60 comp rx=0 tx=0).stop 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 shutdown_connections 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4580b2880 0x7fc4580b4d90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc46c108750 0x7fc46c0ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c1034d0 0x7fc46c0ff400 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 --2- 172.21.15.19:0/639764107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc46c107e00 0x7fc46c0ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:51.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 >> 172.21.15.19:0/639764107 conn(0x7fc46c0fef20 msgr2=0x7fc46c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:51.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 shutdown_connections 2024-05-17T09:28:51.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:51.481+0000 7fc473dda700 1 -- 172.21.15.19:0/639764107 wait complete. 2024-05-17T09:28:51.482 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:28:52.170 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 28f4e22 --release --await' 2024-05-17T09:28:52.171 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 28f4e22 --release --await 2024-05-17T09:28:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:28:52.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.72 MB/sec execute 290 sec latency 16060.968 ms 2024-05-17T09:28:52.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:52 smithi019 ceph-mon[31620]: pgmap v711: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-05-17T09:28:52.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:52 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/639764107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:52 smithi069 ceph-mon[35081]: pgmap v711: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-05-17T09:28:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:52 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/639764107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:52 smithi079 ceph-mon[35502]: pgmap v711: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-05-17T09:28:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:52 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/639764107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:53.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135954 1.72 MB/sec execute 291 sec latency 17061.062 ms 2024-05-17T09:28:54.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 -- 172.21.15.19:0/2477191461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5534107e00 msgr2=0x7f5534108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 --2- 172.21.15.19:0/2477191461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5534107e00 0x7f5534108210 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f55240099c0 tx=0x7f5524009cd0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 -- 172.21.15.19:0/2477191461 shutdown_connections 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 --2- 172.21.15.19:0/2477191461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 0x7f55341100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 --2- 172.21.15.19:0/2477191461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55341034d0 0x7f55341038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 --2- 172.21.15.19:0/2477191461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5534107e00 0x7f5534108210 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 -- 172.21.15.19:0/2477191461 >> 172.21.15.19:0/2477191461 conn(0x7f55340fef40 msgr2=0x7f5534101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 -- 172.21.15.19:0/2477191461 shutdown_connections 2024-05-17T09:28:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.248+0000 7f553c308700 1 -- 172.21.15.19:0/2477191461 wait complete. 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.249+0000 7f553c308700 1 Processor -- start 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.249+0000 7f553c308700 1 -- start start 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553c308700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 0x7f553419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553c308700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5534107e00 0x7f553419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553c308700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 0x7f553419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553c308700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5534113280 con 0x7f55341034d0 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553c308700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5534113100 con 0x7f5534107e00 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553c308700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5534113400 con 0x7f5534108750 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 0x7f553419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 0x7f553419bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60572/0 (socket says 172.21.15.19:60572) 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 -- 172.21.15.19:0/3733299031 learned_addr learned my addr 172.21.15.19:0/3733299031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f55398a3700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5534107e00 0x7f553419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a8a5700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 0x7f553419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 -- 172.21.15.19:0/3733299031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 msgr2=0x7f553419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 0x7f553419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 -- 172.21.15.19:0/3733299031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5534107e00 msgr2=0x7f553419c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5534107e00 0x7f553419c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 -- 172.21.15.19:0/3733299031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341a6f70 con 0x7f55341034d0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a8a5700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 0x7f553419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.250+0000 7f553a0a4700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 0x7f553419bc00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f553000c920 tx=0x7f553000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.251+0000 7f552b7fe700 1 -- 172.21.15.19:0/3733299031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530007b00 con 0x7f55341034d0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.251+0000 7f552b7fe700 1 -- 172.21.15.19:0/3733299031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5530004cc0 con 0x7f55341034d0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.251+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341a7260 con 0x7f55341034d0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.251+0000 7f552b7fe700 1 -- 172.21.15.19:0/3733299031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530005440 con 0x7f55341034d0 2024-05-17T09:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.251+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341a77c0 con 0x7f55341034d0 2024-05-17T09:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.254+0000 7f552b7fe700 1 -- 172.21.15.19:0/3733299031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5530004750 con 0x7f55341034d0 2024-05-17T09:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.254+0000 7f552b7fe700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55200b2900 0x7f55200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.254+0000 7f554ad1db80 1 Processor -- start 2024-05-17T09:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.254+0000 7f55398a3700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55200b2900 0x7f55200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f552b7fe700 1 -- 172.21.15.19:0/3733299031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f55300e20f0 con 0x7f55341034d0 2024-05-17T09:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 -- start start 2024-05-17T09:28:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e2c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246e26620 0x55c246e26a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c246d54d90 con 0x55c246ca7f40 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c246d44930 con 0x55c246e26620 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f554ad1db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c246d48970 con 0x55c246d254d0 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f55398a3700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55200b2900 0x7f55200b4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5524009990 tx=0x7f5524005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f55398a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f55398a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e2caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60584/0 (socket says 172.21.15.19:60584) 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f55398a3700 1 -- 172.21.15.19:0/2697361461 learned_addr learned my addr 172.21.15.19:0/2697361461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f553a0a4700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e2c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.255+0000 7f553a8a5700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246e26620 0x55c246e26a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f55398a3700 1 -- 172.21.15.19:0/2697361461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 msgr2=0x55c246e2c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f55398a3700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e2c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f55398a3700 1 -- 172.21.15.19:0/2697361461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246e26620 msgr2=0x55c246e26a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f55398a3700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246e26620 0x55c246e26a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f55398a3700 1 -- 172.21.15.19:0/2697361461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c246c949d0 con 0x55c246ca7f40 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f553a0a4700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e2c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f553a8a5700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246e26620 0x55c246e26a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f55398a3700 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e2caa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5524015000 tx=0x7f552401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfdb96ff73cac3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f5528d47700 1 -- 172.21.15.19:0/2697361461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524003680 con 0x55c246ca7f40 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f5528d47700 1 -- 172.21.15.19:0/2697361461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5524003cb0 con 0x55c246ca7f40 2024-05-17T09:28:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.256+0000 7f5528d47700 1 -- 172.21.15.19:0/2697361461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552402fc20 con 0x55c246ca7f40 2024-05-17T09:28:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 -- 172.21.15.19:0/2697361461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 msgr2=0x55c246e2caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e2caa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5524015000 tx=0x7f552401e000 comp rx=0 tx=0).stop 2024-05-17T09:28:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 -- 172.21.15.19:0/2697361461 shutdown_connections 2024-05-17T09:28:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e2c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246e26620 0x55c246e26a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2697361461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e2caa0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 -- 172.21.15.19:0/2697361461 >> 172.21.15.19:0/2697361461 conn(0x55c246ce6d80 msgr2=0x55c246e21520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 -- 172.21.15.19:0/2697361461 shutdown_connections 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 -- 172.21.15.19:0/2697361461 wait complete. 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.257+0000 7f554ad1db80 1 -- start start 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 Processor -- start 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246d254d0 0x55c246e307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246e26620 0x55c246e30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c246c4b0a0 con 0x55c246ca7f40 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c246ce68c0 con 0x55c246d254d0 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c246e27440 con 0x55c246e26620 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f554ad1db80 10 client.0 _ll_register_callbacks cb 0x55c246cdfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e30280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60598/0 (socket says 172.21.15.19:60598) 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 -- 172.21.15.19:0/2752512729 learned_addr learned my addr 172.21.15.19:0/2752512729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a8a5700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246e26620 0x55c246e30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f55398a3700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246d254d0 0x55c246e307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 -- 172.21.15.19:0/2752512729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246e26620 msgr2=0x55c246e30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246e26620 0x55c246e30d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 -- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246d254d0 msgr2=0x55c246e307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246d254d0 0x55c246e307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.258+0000 7f553a0a4700 1 -- 172.21.15.19:0/2752512729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c246ce3790 con 0x55c246ca7f40 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f553a0a4700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e30280 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f553001e000 tx=0x7f55300a8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1447443a54ad89f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f551dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f551dffb700 1 -- 172.21.15.19:0/2752512729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553001d040 con 0x55c246ca7f40 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f551dffb700 1 -- 172.21.15.19:0/2752512729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f55300b1f40 con 0x55c246ca7f40 2024-05-17T09:28:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5524003cb0 con 0x55c246ca7f40 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f551dffb700 1 -- 172.21.15.19:0/2752512729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55300a9a60 con 0x55c246ca7f40 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 10 client.17196 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c246e3daf0 con 0x55c246ca7f40 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f551dffb700 1 -- 172.21.15.19:0/2752512729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5524003cb0 con 0x55c246ca7f40 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f551dffb700 1 -- 172.21.15.19:0/2752512729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f55300a4cd0 con 0x55c246ca7f40 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 10 client.17196 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 10 client.17196 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 20 client.17196 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246e40080 0x55c246e424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.259+0000 7f554ad1db80 4 client.17196 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.260+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c246d56ce0 con 0x55c246e40080 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.260+0000 7f55398a3700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246e40080 0x55c246e424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.260+0000 7f551dffb700 10 client.17196 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.260+0000 7f55398a3700 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246e40080 0x55c246e424e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9c947f9b43dec2 server_cookie=2828779c50e81395 in_seq=0 out_seq=0 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f551dffb700 1 -- 172.21.15.19:0/2752512729 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c246d56ce0 con 0x55c246e40080 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f551dffb700 10 client.17196 handle_command_reply: tid=0 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 client.17196 shutdown 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 msgr2=0x55c246e30280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e30280 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f553001e000 tx=0x7f55300a8b50 comp rx=0 tx=0).stop 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 shutdown_connections 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246e26620 0x55c246e30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246e40080 0x55c246e424e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246ca7f40 0x55c246e30280 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 --2- 172.21.15.19:0/2752512729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246d254d0 0x55c246e307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 >> 172.21.15.19:0/2752512729 conn(0x55c246ce6d80 msgr2=0x55c246e1b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f551dffb700 0 client.17196 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 shutdown_connections 2024-05-17T09:28:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.261+0000 7f554ad1db80 1 -- 172.21.15.19:0/2752512729 wait complete. 2024-05-17T09:28:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.262+0000 7f554ad1db80 20 client.17196 trim_cache size 0 max 16384 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 Processor -- start 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 -- start start 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e3c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55341b2be0 con 0x55c246e26620 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55341b0470 con 0x55c246ca7f40 2024-05-17T09:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f554ad1db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55341ae2c0 con 0x55c246d254d0 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f553a8a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f553a8a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e3c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60612/0 (socket says 172.21.15.19:60612) 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f553a8a5700 1 -- 172.21.15.19:0/3048468778 learned_addr learned my addr 172.21.15.19:0/3048468778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.282+0000 7f55398a3700 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a0a4700 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a8a5700 1 -- 172.21.15.19:0/3048468778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 msgr2=0x55c246e26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a8a5700 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a8a5700 1 -- 172.21.15.19:0/3048468778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 msgr2=0x55c246e2b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a8a5700 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e2b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a8a5700 1 -- 172.21.15.19:0/3048468778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c246e1aee0 con 0x55c246e26620 2024-05-17T09:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f553a8a5700 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e3c100 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f552c00a500 tx=0x7f552c00a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f28b53756b6c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f551ffff700 1 -- 172.21.15.19:0/3048468778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c01a020 con 0x55c246e26620 2024-05-17T09:28:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f551ffff700 1 -- 172.21.15.19:0/3048468778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f552c016880 con 0x55c246e26620 2024-05-17T09:28:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.283+0000 7f551ffff700 1 -- 172.21.15.19:0/3048468778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c015710 con 0x55c246e26620 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 -- 172.21.15.19:0/3048468778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 msgr2=0x55c246e3c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e3c100 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f552c00a500 tx=0x7f552c00a530 comp rx=0 tx=0).stop 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 -- 172.21.15.19:0/3048468778 shutdown_connections 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e3c100 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 --2- 172.21.15.19:0/3048468778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 -- 172.21.15.19:0/3048468778 >> 172.21.15.19:0/3048468778 conn(0x55c246ce6d80 msgr2=0x55c246e31930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 -- 172.21.15.19:0/3048468778 shutdown_connections 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 -- 172.21.15.19:0/3048468778 wait complete. 2024-05-17T09:28:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 -- start start 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.284+0000 7f554ad1db80 1 Processor -- start 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e38a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55341ae930 con 0x55c246e26620 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55341afda0 con 0x55c246ca7f40 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55341af000 con 0x55c246d254d0 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f554ad1db80 10 client.0 _ll_register_callbacks cb 0x55c246cdfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e352c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60616/0 (socket says 172.21.15.19:60616) 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 -- 172.21.15.19:0/1339417153 learned_addr learned my addr 172.21.15.19:0/1339417153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a0a4700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f55398a3700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e38a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 -- 172.21.15.19:0/1339417153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 msgr2=0x55c246e38a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e38a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 -- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 msgr2=0x55c246e384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 -- 172.21.15.19:0/1339417153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c246e35960 con 0x55c246e26620 2024-05-17T09:28:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f553a8a5700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e352c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f552c007030 tx=0x7f552c016cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff4a88aaefaab45 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.285+0000 7f551dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f551dffb700 1 -- 172.21.15.19:0/1339417153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c01f830 con 0x55c246e26620 2024-05-17T09:28:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f551dffb700 1 -- 172.21.15.19:0/1339417153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f552c01f9d0 con 0x55c246e26620 2024-05-17T09:28:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f552c016880 con 0x55c246e26620 2024-05-17T09:28:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f551dffb700 1 -- 172.21.15.19:0/1339417153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f552c01fbd0 con 0x55c246e26620 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 10 client.17202 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c246eab4e0 con 0x55c246e26620 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f551dffb700 1 -- 172.21.15.19:0/1339417153 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f552c016880 con 0x55c246e26620 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f551dffb700 1 -- 172.21.15.19:0/1339417153 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f552c026370 con 0x55c246e26620 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 10 client.17202 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 10 client.17202 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 20 client.17202 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246eabb00 0x55c246eadf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 4 client.17202 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "28f4e22", "release": true, "await": true}] 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.286+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "28f4e22", "release": true, "await": true}) -- 0x55c246d504c0 con 0x55c246eabb00 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.287+0000 7f553a0a4700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246eabb00 0x55c246eadf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.287+0000 7f553a0a4700 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246eabb00 0x55c246eadf20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77c571e2a899a50 server_cookie=3a3dd9defc5ed680 in_seq=0 out_seq=0 2024-05-17T09:28:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.287+0000 7f551dffb700 10 client.17202 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f551dffb700 1 -- 172.21.15.19:0/1339417153 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c246d504c0 con 0x55c246eabb00 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f551dffb700 10 client.17202 handle_command_reply: tid=0 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 client.17202 shutdown 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 msgr2=0x55c246e352c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e352c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f552c007030 tx=0x7f552c016cc0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 shutdown_connections 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c246eabb00 0x55c246eadf20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c246e26620 0x55c246e352c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c246ca7f40 0x55c246e384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f551dffb700 0 client.17202 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 --2- 172.21.15.19:0/1339417153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c246d254d0 0x55c246e38a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 >> 172.21.15.19:0/1339417153 conn(0x55c246ce6d80 msgr2=0x55c246e25da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 shutdown_connections 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 1 -- 172.21.15.19:0/1339417153 wait complete. 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.291+0000 7f554ad1db80 20 client.17202 trim_cache size 0 max 16384 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 32, 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "28f4e22": { 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 32, 2024-05-17T09:28:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 141.6, 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:28:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:28:54.299 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55200b2900 msgr2=0x7f55200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55200b2900 0x7f55200b4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5524009990 tx=0x7f5524005fb0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 msgr2=0x7f553419bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 0x7f553419bc00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f553000c920 tx=0x7f553000cc30 comp rx=0 tx=0).stop 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 shutdown_connections 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f55200b2900 0x7f55200b4e10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5534108750 0x7f553419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55341034d0 0x7f553419bc00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 --2- 172.21.15.19:0/3733299031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5534107e00 0x7f553419c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.292+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 >> 172.21.15.19:0/3733299031 conn(0x7f55340fef40 msgr2=0x7f55341096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.293+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 shutdown_connections 2024-05-17T09:28:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:54.293+0000 7f553c308700 1 -- 172.21.15.19:0/3733299031 wait complete. 2024-05-17T09:28:54.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 136004 1.71 MB/sec execute 292 sec latency 17913.487 ms 2024-05-17T09:28:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:54 smithi069 ceph-mon[35081]: pgmap v712: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-05-17T09:28:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:54 smithi019 ceph-mon[31620]: pgmap v712: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-05-17T09:28:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:54 smithi079 ceph-mon[35502]: pgmap v712: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-05-17T09:28:55.064 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '28f4e22', total seconds elapsed: 27.5 2024-05-17T09:28:55.064 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.6 2024-05-17T09:28:55.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:28:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:28:55.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 136553 1.72 MB/sec execute 293 sec latency 16.421 ms 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 -- 172.21.15.19:0/2324846287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10104ea0 msgr2=0x7f3f101052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/2324846287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10104ea0 0x7f3f101052b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009900 tx=0x7f3f0002a410 comp rx=0 tx=0).stop 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 -- 172.21.15.19:0/2324846287 shutdown_connections 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/2324846287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f10109ba0 0x7f3f1010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/2324846287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10104ea0 0x7f3f101052b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/2324846287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10103c60 0x7f3f10104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 -- 172.21.15.19:0/2324846287 >> 172.21.15.19:0/2324846287 conn(0x7f3f100fee50 msgr2=0x7f3f10101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 -- 172.21.15.19:0/2324846287 shutdown_connections 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.541+0000 7f3f17b5e640 1 -- 172.21.15.19:0/2324846287 wait complete. 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 Processor -- start 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 -- start start 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 0x7f3f1019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10104ea0 0x7f3f1019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f10109ba0 0x7f3f101a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010f640 con 0x7f3f10103c60 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010f4c0 con 0x7f3f10104ea0 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f17b5e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010f7c0 con 0x7f3f10109ba0 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 0x7f3f1019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 0x7f3f1019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60632/0 (socket says 172.21.15.19:60632) 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f158fa640 1 -- 172.21.15.19:0/973506820 learned_addr learned my addr 172.21.15.19:0/973506820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f160fb640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f10109ba0 0x7f3f101a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.542+0000 7f3f150f9640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10104ea0 0x7f3f1019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f158fa640 1 -- 172.21.15.19:0/973506820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f10109ba0 msgr2=0x7f3f101a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f158fa640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f10109ba0 0x7f3f101a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f158fa640 1 -- 172.21.15.19:0/973506820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10104ea0 msgr2=0x7f3f1019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f158fa640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10104ea0 0x7f3f1019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f158fa640 1 -- 172.21.15.19:0/973506820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f101a1cc0 con 0x7f3f10103c60 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f158fa640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 0x7f3f1019a180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00e9f0 tx=0x7f3f0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f06ffd640 1 -- 172.21.15.19:0/973506820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00ccc0 con 0x7f3f10103c60 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f101a1fb0 con 0x7f3f10103c60 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f06ffd640 1 -- 172.21.15.19:0/973506820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0c0044b0 con 0x7f3f10103c60 2024-05-17T09:28:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f06ffd640 1 -- 172.21.15.19:0/973506820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c010430 con 0x7f3f10103c60 2024-05-17T09:28:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.543+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a2410 con 0x7f3f10103c60 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.547+0000 7f3f1ea10740 1 Processor -- start 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.547+0000 7f3f1ea10740 1 -- start start 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.547+0000 7f3f06ffd640 1 -- 172.21.15.19:0/973506820 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f0c003f90 con 0x7f3f10103c60 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.547+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e6b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6bc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556e8e627c30 con 0x556e8e2f34d0 2024-05-17T09:28:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556e8e6177d0 con 0x556e8e3141b0 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556e8e61b810 con 0x556e8e2eb2f0 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41010/0 (socket says 172.21.15.19:41010) 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 -- 172.21.15.19:0/4190684350 learned_addr learned my addr 172.21.15.19:0/4190684350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f150f9640 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6bc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f160fb640 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e6b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 -- 172.21.15.19:0/4190684350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 msgr2=0x556e8e6b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e6b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 -- 172.21.15.19:0/4190684350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 msgr2=0x556e8e6bc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6bc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 -- 172.21.15.19:0/4190684350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8e32c210 con 0x556e8e2eb2f0 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f158fa640 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6b5920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0e8a80 tx=0x7f3f0c0e8f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e217e59b98b348 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.548+0000 7f3f06ffd640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3efc0b28a0 0x7f3efc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.549+0000 7f3f160fb640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3efc0b28a0 0x7f3efc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.549+0000 7f3efbfff640 1 -- 172.21.15.19:0/4190684350 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0f1070 con 0x556e8e2eb2f0 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.549+0000 7f3efbfff640 1 -- 172.21.15.19:0/4190684350 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0c07a4b0 con 0x556e8e2eb2f0 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.549+0000 7f3efbfff640 1 -- 172.21.15.19:0/4190684350 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c07a6a0 con 0x556e8e2eb2f0 2024-05-17T09:28:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.549+0000 7f3f06ffd640 1 -- 172.21.15.19:0/973506820 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f0c0e28e0 con 0x7f3f10103c60 2024-05-17T09:28:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 -- 172.21.15.19:0/4190684350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 msgr2=0x556e8e6b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f160fb640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3efc0b28a0 0x7f3efc0b4db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x556e8e6b6570 tx=0x7f3f08009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6b5920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0e8a80 tx=0x7f3f0c0e8f40 comp rx=0 tx=0).stop 2024-05-17T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 -- 172.21.15.19:0/4190684350 shutdown_connections 2024-05-17T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6bc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6b5920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/4190684350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e6b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 -- 172.21.15.19:0/4190684350 >> 172.21.15.19:0/4190684350 conn(0x556e8e308300 msgr2=0x556e8e6b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 -- 172.21.15.19:0/4190684350 shutdown_connections 2024-05-17T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 -- 172.21.15.19:0/4190684350 wait complete. 2024-05-17T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 -- start start 2024-05-17T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.550+0000 7f3f1ea10740 1 Processor -- start 2024-05-17T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e3141b0 0x556e8e6c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556e8e2fd790 con 0x556e8e3141b0 2024-05-17T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556e8e5d3890 con 0x556e8e2f34d0 2024-05-17T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556e8e6bd270 con 0x556e8e2eb2f0 2024-05-17T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f1ea10740 10 client.0 _ll_register_callbacks cb 0x556e8e5d6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f150f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6becf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41020/0 (socket says 172.21.15.19:41020) 2024-05-17T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f158fa640 1 -- 172.21.15.19:0/3044295034 learned_addr learned my addr 172.21.15.19:0/3044295034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f160fb640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e3141b0 0x556e8e6c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f158fa640 1 -- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 msgr2=0x556e8e6bf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f158fa640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6bf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.551+0000 7f3f158fa640 1 -- 172.21.15.19:0/3044295034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e3141b0 msgr2=0x556e8e6c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f158fa640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e3141b0 0x556e8e6c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f158fa640 1 -- 172.21.15.19:0/3044295034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8e3f8620 con 0x556e8e2eb2f0 2024-05-17T09:28:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f160fb640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e3141b0 0x556e8e6c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f158fa640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6becf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0e0080 tx=0x7f3f0c0e2be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99b6087a9524ae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3ef97fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:28:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3ef97fa640 1 -- 172.21.15.19:0/3044295034 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0f1070 con 0x556e8e2eb2f0 2024-05-17T09:28:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3ef97fa640 1 -- 172.21.15.19:0/3044295034 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f0c0a6140 con 0x556e8e2eb2f0 2024-05-17T09:28:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c07a4b0 con 0x556e8e2eb2f0 2024-05-17T09:28:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3ef97fa640 1 -- 172.21.15.19:0/3044295034 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c00ce60 con 0x556e8e2eb2f0 2024-05-17T09:28:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f1ea10740 10 client.26824 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e8e6c9050 con 0x556e8e2eb2f0 2024-05-17T09:28:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3ef97fa640 1 -- 172.21.15.19:0/3044295034 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0c07a4b0 con 0x556e8e2eb2f0 2024-05-17T09:28:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.552+0000 7f3ef97fa640 1 -- 172.21.15.19:0/3044295034 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3f0c0eac90 con 0x556e8e2eb2f0 2024-05-17T09:28:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f1ea10740 10 client.26824 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f1ea10740 10 client.26824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f1ea10740 20 client.26824 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2d2d80 0x556e8e6cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f1ea10740 4 client.26824 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:28:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e8e629b80 con 0x556e8e2d2d80 2024-05-17T09:28:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.553+0000 7f3f150f9640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2d2d80 0x556e8e6cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3ef97fa640 10 client.26824 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3f150f9640 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2d2d80 0x556e8e6cc3b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf3fb86ae409e9a server_cookie=17f2255ad51cb785 in_seq=0 out_seq=0 2024-05-17T09:28:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3ef97fa640 1 -- 172.21.15.19:0/3044295034 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556e8e629b80 con 0x556e8e2d2d80 2024-05-17T09:28:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3ef97fa640 10 client.26824 handle_command_reply: tid=0 2024-05-17T09:28:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3f1ea10740 1 client.26824 shutdown 2024-05-17T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 msgr2=0x556e8e6becf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.554+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6becf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0e0080 tx=0x7f3f0c0e2be0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 shutdown_connections 2024-05-17T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2d2d80 0x556e8e6cc3b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e3141b0 0x556e8e6c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3044295034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6becf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3ef97fa640 0 client.26824 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 >> 172.21.15.19:0/3044295034 conn(0x556e8e308300 msgr2=0x556e8e6afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 shutdown_connections 2024-05-17T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3044295034 wait complete. 2024-05-17T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.555+0000 7f3f1ea10740 20 client.26824 trim_cache size 0 max 16384 2024-05-17T09:28:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 Processor -- start 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 -- start start 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 0x556e8e6befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e3141b0 0x556e8e6c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101af390 con 0x556e8e2eb2f0 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101acc60 con 0x556e8e2f34d0 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.571+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101aaab0 con 0x556e8e3141b0 2024-05-17T09:28:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 0x556e8e6befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f158fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 0x556e8e6befb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60650/0 (socket says 172.21.15.19:60650) 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f158fa640 1 -- 172.21.15.19:0/3673595246 learned_addr learned my addr 172.21.15.19:0/3673595246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f160fb640 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e3141b0 0x556e8e6c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 -- 172.21.15.19:0/3673595246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e3141b0 msgr2=0x556e8e6c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e3141b0 0x556e8e6c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 -- 172.21.15.19:0/3673595246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 msgr2=0x556e8e6befb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 0x556e8e6befb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 -- 172.21.15.19:0/3673595246 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8e6bc650 con 0x556e8e2f34d0 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f158fa640 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 0x556e8e6befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3f150f9640 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6c60b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005760 tx=0x7f3f00005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554807dd82d9fc9a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3efb7fe640 1 -- 172.21.15.19:0/3673595246 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00035040 con 0x556e8e2f34d0 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3efb7fe640 1 -- 172.21.15.19:0/3673595246 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0004b070 con 0x556e8e2f34d0 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.572+0000 7f3efb7fe640 1 -- 172.21.15.19:0/3673595246 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f000467d0 con 0x556e8e2f34d0 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3673595246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 msgr2=0x556e8e6c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6c60b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005760 tx=0x7f3f00005fd0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3673595246 shutdown_connections 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e3141b0 0x556e8e6c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e2f34d0 0x556e8e6c60b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/3673595246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2eb2f0 0x556e8e6befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3673595246 >> 172.21.15.19:0/3673595246 conn(0x556e8e308300 msgr2=0x556e8e6b9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3673595246 shutdown_connections 2024-05-17T09:28:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.573+0000 7f3f1ea10740 1 -- 172.21.15.19:0/3673595246 wait complete. 2024-05-17T09:28:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 -- start start 2024-05-17T09:28:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 Processor -- start 2024-05-17T09:28:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e732910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ab120 con 0x556e8e2f34d0 2024-05-17T09:28:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ac590 con 0x556e8e3141b0 2024-05-17T09:28:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ab7f0 con 0x556e8e2eb2f0 2024-05-17T09:28:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f1ea10740 10 client.0 _ll_register_callbacks cb 0x7f3f0c0e8900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:28:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f150f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f150f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60660/0 (socket says 172.21.15.19:60660) 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f150f9640 1 -- 172.21.15.19:0/2531314346 learned_addr learned my addr 172.21.15.19:0/2531314346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f160fb640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e732910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.574+0000 7f3f158fa640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f150f9640 1 -- 172.21.15.19:0/2531314346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 msgr2=0x556e8e6c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f150f9640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f150f9640 1 -- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 msgr2=0x556e8e732910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f150f9640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e732910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f150f9640 1 -- 172.21.15.19:0/2531314346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8e732e50 con 0x556e8e2f34d0 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f158fa640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:28:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f150f9640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6c3e00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3f00048040 tx=0x7f3f000040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae3cc75f77b31ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3ef97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3ef97fa640 1 -- 172.21.15.19:0/2531314346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00035040 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3ef97fa640 1 -- 172.21.15.19:0/2531314346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f00031b50 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0004b070 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3ef97fa640 1 -- 172.21.15.19:0/2531314346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f00062050 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f1ea10740 10 client.17220 fetch_fsmap learned FSMap version 32 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.575+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e8e734050 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3ef97fa640 1 -- 172.21.15.19:0/2531314346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0004b070 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3ef97fa640 1 -- 172.21.15.19:0/2531314346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3f000037b0 con 0x556e8e2f34d0 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f1ea10740 10 client.17220 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f1ea10740 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f1ea10740 20 client.17220 populate_metadata read hostname 'smithi019' 2024-05-17T09:28:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2a81c0 0x556e8e736a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f1ea10740 4 client.17220 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:28:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e8e623360 con 0x556e8e2a81c0 2024-05-17T09:28:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.576+0000 7f3f158fa640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2a81c0 0x556e8e736a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.577+0000 7f3ef97fa640 10 client.17220 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.577+0000 7f3f158fa640 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2a81c0 0x556e8e736a20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12da464da5124fe4 server_cookie=93db9d171ee090db in_seq=0 out_seq=0 2024-05-17T09:28:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3ef97fa640 1 -- 172.21.15.19:0/2531314346 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556e8e623360 con 0x556e8e2a81c0 2024-05-17T09:28:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3ef97fa640 10 client.17220 handle_command_reply: tid=0 2024-05-17T09:28:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 client.17220 shutdown 2024-05-17T09:28:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 msgr2=0x556e8e6c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6c3e00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3f00048040 tx=0x7f3f000040f0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 shutdown_connections 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556e8e2a81c0 0x556e8e736a20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e8e3141b0 0x556e8e732910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556e8e2f34d0 0x556e8e6c3e00 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3ef97fa640 0 client.17220 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 --2- 172.21.15.19:0/2531314346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e8e2eb2f0 0x556e8e6c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 >> 172.21.15.19:0/2531314346 conn(0x556e8e308300 msgr2=0x556e8e6bdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 shutdown_connections 2024-05-17T09:28:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 1 -- 172.21.15.19:0/2531314346 wait complete. 2024-05-17T09:28:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.597+0000 7f3f1ea10740 20 client.17220 trim_cache size 0 max 16384 2024-05-17T09:28:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:28:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:28:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3efc0b28a0 msgr2=0x7f3efc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3efc0b28a0 0x7f3efc0b4db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x556e8e6b6570 tx=0x7f3f08009500 comp rx=0 tx=0).stop 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 msgr2=0x7f3f1019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 0x7f3f1019a180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00e9f0 tx=0x7f3f0c00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 shutdown_connections 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f10109ba0 0x7f3f101a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10104ea0 0x7f3f1019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3efc0b28a0 0x7f3efc0b4db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 --2- 172.21.15.19:0/973506820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f10103c60 0x7f3f1019a180 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 >> 172.21.15.19:0/973506820 conn(0x7f3f100fee50 msgr2=0x7f3f10105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 shutdown_connections 2024-05-17T09:28:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:28:55.598+0000 7f3f17b5e640 1 -- 172.21.15.19:0/973506820 wait complete. 2024-05-17T09:28:56.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 137184 1.72 MB/sec execute 294 sec latency 15.325 ms 2024-05-17T09:28:56.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:56 smithi019 ceph-mon[31620]: pgmap v713: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-17T09:28:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:56 smithi069 ceph-mon[35081]: pgmap v713: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-17T09:28:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:56 smithi079 ceph-mon[35502]: pgmap v713: 129 pgs: 129 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 -- 172.21.15.19:0/2750419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 msgr2=0x7f987c109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 --2- 172.21.15.19:0/2750419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c109300 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f98700099c0 tx=0x7f9870009cd0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 -- 172.21.15.19:0/2750419871 shutdown_connections 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 --2- 172.21.15.19:0/2750419871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c109840 0x7f987c1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 --2- 172.21.15.19:0/2750419871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987c1045c0 0x7f987c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 --2- 172.21.15.19:0/2750419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c109300 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.168+0000 7f9881de3700 1 -- 172.21.15.19:0/2750419871 >> 172.21.15.19:0/2750419871 conn(0x7f987c0ffe70 msgr2=0x7f987c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:57.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.169+0000 7f9881de3700 1 -- 172.21.15.19:0/2750419871 shutdown_connections 2024-05-17T09:28:57.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.169+0000 7f9881de3700 1 -- 172.21.15.19:0/2750419871 wait complete. 2024-05-17T09:28:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.169+0000 7f9881de3700 1 Processor -- start 2024-05-17T09:28:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 -- start start 2024-05-17T09:28:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c1045c0 0x7f987c1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987c109840 0x7f987c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f987c114300 con 0x7f987c108ef0 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f987c114180 con 0x7f987c109840 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f9881de3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f987c114480 con 0x7f987c1045c0 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f987affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f987b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c1045c0 0x7f987c1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f987affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c1a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60678/0 (socket says 172.21.15.19:60678) 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f987affd700 1 -- 172.21.15.19:0/710109218 learned_addr learned my addr 172.21.15.19:0/710109218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.170+0000 7f987bfff700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987c109840 0x7f987c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f987b7fe700 1 -- 172.21.15.19:0/710109218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987c109840 msgr2=0x7f987c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f987b7fe700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987c109840 0x7f987c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f987b7fe700 1 -- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 msgr2=0x7f987c1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f987b7fe700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f987b7fe700 1 -- 172.21.15.19:0/710109218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c19b270 con 0x7f987c1045c0 2024-05-17T09:28:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f987b7fe700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c1045c0 0x7f987c1a09b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f986c00e9f0 tx=0x7f986c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c00cbd0 con 0x7f987c1045c0 2024-05-17T09:28:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c19b560 con 0x7f987c1045c0 2024-05-17T09:28:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f986c00cd70 con 0x7f987c1045c0 2024-05-17T09:28:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1a7760 con 0x7f987c1045c0 2024-05-17T09:28:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.171+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c018680 con 0x7f987c1045c0 2024-05-17T09:28:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.173+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c105a70 con 0x7f987c1045c0 2024-05-17T09:28:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.174+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f986c010430 con 0x7f987c1045c0 2024-05-17T09:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.175+0000 7f9878ff9700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98640b2880 0x7f98640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.175+0000 7f987affd700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98640b2880 0x7f98640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.175+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f986c0e2790 con 0x7f987c1045c0 2024-05-17T09:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.176+0000 7f987affd700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98640b2880 0x7f98640b4d90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f987c19c270 tx=0x7f987001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.180+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f986c0aa100 con 0x7f987c1045c0 2024-05-17T09:28:57.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 137744 1.73 MB/sec execute 295 sec latency 15.777 ms 2024-05-17T09:28:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.484+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f987c19bfe0 con 0x7f987c1045c0 2024-05-17T09:28:57.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.485+0000 7f9878ff9700 1 -- 172.21.15.19:0/710109218 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f986c0b2180 con 0x7f987c1045c0 2024-05-17T09:28:57.486 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:28:57.486 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:28:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98640b2880 msgr2=0x7f98640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98640b2880 0x7f98640b4d90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f987c19c270 tx=0x7f987001a040 comp rx=0 tx=0).stop 2024-05-17T09:28:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c1045c0 msgr2=0x7f987c1a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:28:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c1045c0 0x7f987c1a09b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f986c00e9f0 tx=0x7f986c00ed00 comp rx=0 tx=0).stop 2024-05-17T09:28:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 shutdown_connections 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98640b2880 0x7f98640b4d90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f987c1045c0 0x7f987c1a09b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987c109840 0x7f987c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 --2- 172.21.15.19:0/710109218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987c108ef0 0x7f987c1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 >> 172.21.15.19:0/710109218 conn(0x7f987c0ffe70 msgr2=0x7f987c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 shutdown_connections 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:28:57.488+0000 7f9881de3700 1 -- 172.21.15.19:0/710109218 wait complete. 2024-05-17T09:28:57.489 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:28:58.177 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 141.6 --await-for 45.0' 2024-05-17T09:28:58.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 141.6 --await-for 45.0 2024-05-17T09:28:58.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138187 1.73 MB/sec execute 296 sec latency 14.332 ms 2024-05-17T09:28:58.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:58 smithi019 ceph-mon[31620]: pgmap v714: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-05-17T09:28:58.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:28:58 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/710109218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:28:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:58 smithi069 ceph-mon[35081]: pgmap v714: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-05-17T09:28:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:28:58 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/710109218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:58 smithi079 ceph-mon[35502]: pgmap v714: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-05-17T09:28:58.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:28:58 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/710109218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:28:59.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138645 1.74 MB/sec execute 297 sec latency 16.181 ms 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 -- 172.21.15.19:0/3115203063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 msgr2=0x7f2d601081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 --2- 172.21.15.19:0/3115203063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 0x7f2d601081f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098f0 tx=0x7f2d54009c00 comp rx=0 tx=0).stop 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 -- 172.21.15.19:0/3115203063 shutdown_connections 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 --2- 172.21.15.19:0/3115203063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d60108730 0x7f2d60110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 --2- 172.21.15.19:0/3115203063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d601034b0 0x7f2d601038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 --2- 172.21.15.19:0/3115203063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 0x7f2d601081f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 -- 172.21.15.19:0/3115203063 >> 172.21.15.19:0/3115203063 conn(0x7f2d600fedb0 msgr2=0x7f2d601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.256+0000 7f2d65b24700 1 -- 172.21.15.19:0/3115203063 shutdown_connections 2024-05-17T09:29:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.257+0000 7f2d65b24700 1 -- 172.21.15.19:0/3115203063 wait complete. 2024-05-17T09:29:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.257+0000 7f2d65b24700 1 Processor -- start 2024-05-17T09:29:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 -- start start 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 0x7f2d6019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 0x7f2d6019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d60108730 0x7f2d6019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601131f0 con 0x7f2d60108730 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60113070 con 0x7f2d60107de0 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d65b24700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60113370 con 0x7f2d601034b0 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 0x7f2d6019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 0x7f2d6019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41086/0 (socket says 172.21.15.19:41086) 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 0x7f2d6019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/1834495269 learned_addr learned my addr 172.21.15.19:0/1834495269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d60108730 0x7f2d6019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/1834495269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 msgr2=0x7f2d6019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 0x7f2d6019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d60108730 msgr2=0x7f2d6019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d60108730 0x7f2d6019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.258+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/1834495269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6019fbe0 con 0x7f2d601034b0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.259+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 0x7f2d6019ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009db0 tx=0x7f2d5000c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.259+0000 7f2d5cff9700 1 -- 172.21.15.19:0/1834495269 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50004830 con 0x7f2d601034b0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.259+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6019fed0 con 0x7f2d601034b0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.259+0000 7f2d5cff9700 1 -- 172.21.15.19:0/1834495269 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d50012070 con 0x7f2d601034b0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.259+0000 7f2d5cff9700 1 -- 172.21.15.19:0/1834495269 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50005280 con 0x7f2d601034b0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.259+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d600ff1c0 con 0x7f2d601034b0 2024-05-17T09:29:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.262+0000 7f2d5cff9700 1 -- 172.21.15.19:0/1834495269 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d50005420 con 0x7f2d601034b0 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5cff9700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2d480b2880 0x7f2d480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 Processor -- start 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5effd700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2d480b2880 0x7f2d480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 -- start start 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5cff9700 1 -- 172.21.15.19:0/1834495269 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2d500e6320 con 0x7f2d601034b0 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d1185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d118b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f66d040e20 con 0x55f66d011560 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f66d0309c0 con 0x55f66d1126b0 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d74539b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f66d034a00 con 0x55f66cf93fd0 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d1185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d1185f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60702/0 (socket says 172.21.15.19:60702) 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/673023806 learned_addr learned my addr 172.21.15.19:0/673023806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.263+0000 7f2d5effd700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2d480b2880 0x7f2d480b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098c0 tx=0x7f2d54019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5effd700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d118b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/673023806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 msgr2=0x55f66d118b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d118b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/673023806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 msgr2=0x55f66d112ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d112ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/673023806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66cf80a60 con 0x55f66d011560 2024-05-17T09:29:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5effd700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d118b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:29:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d112ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d1185f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2d500a9610 tx=0x7f2d500124b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62584ed79de0bdea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d45ffb700 1 -- 172.21.15.19:0/673023806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500adc90 con 0x55f66d011560 2024-05-17T09:29:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d45ffb700 1 -- 172.21.15.19:0/673023806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d500ad590 con 0x55f66d011560 2024-05-17T09:29:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.264+0000 7f2d45ffb700 1 -- 172.21.15.19:0/673023806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500a9820 con 0x55f66d011560 2024-05-17T09:29:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 -- 172.21.15.19:0/673023806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 msgr2=0x55f66d1185f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d1185f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2d500a9610 tx=0x7f2d500124b0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 -- 172.21.15.19:0/673023806 shutdown_connections 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d112ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d1185f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 --2- 172.21.15.19:0/673023806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d118b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 -- 172.21.15.19:0/673023806 >> 172.21.15.19:0/673023806 conn(0x55f66cfd2e10 msgr2=0x55f66d10d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 -- 172.21.15.19:0/673023806 shutdown_connections 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.265+0000 7f2d74539b80 1 -- 172.21.15.19:0/673023806 wait complete. 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 -- start start 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 Processor -- start 2024-05-17T09:29:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d11c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 0x55f66d11c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d1126b0 0x55f66d11cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f66cf37130 con 0x55f66d1126b0 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f66cfd2950 con 0x55f66d011560 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f66d128460 con 0x55f66cf93fd0 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d74539b80 10 client.0 _ll_register_callbacks cb 0x55f66cfcbab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d5effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 0x55f66d11c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d5ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d1126b0 0x55f66d11cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d5effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 0x55f66d11c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54920/0 (socket says 172.21.15.19:54920) 2024-05-17T09:29:00.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.266+0000 7f2d5effd700 1 -- 172.21.15.19:0/2719459254 learned_addr learned my addr 172.21.15.19:0/2719459254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d11c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5effd700 1 -- 172.21.15.19:0/2719459254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 msgr2=0x55f66d11c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5effd700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d11c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5effd700 1 -- 172.21.15.19:0/2719459254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d1126b0 msgr2=0x55f66d11cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5effd700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d1126b0 0x55f66d11cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5effd700 1 -- 172.21.15.19:0/2719459254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66cfcf820 con 0x55f66d011560 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d11c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d5effd700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 0x55f66d11c830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009680 tx=0x7f2d54009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94165bf2bf8650b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d3effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d3effd700 1 -- 172.21.15.19:0/2719459254 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54033060 con 0x55f66d011560 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d3effd700 1 -- 172.21.15.19:0/2719459254 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2d54005610 con 0x55f66d011560 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.267+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d500ad590 con 0x55f66d011560 2024-05-17T09:29:00.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d3effd700 1 -- 172.21.15.19:0/2719459254 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d54003680 con 0x55f66d011560 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 10 client.26861 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f66d129d40 con 0x55f66d011560 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d3effd700 1 -- 172.21.15.19:0/2719459254 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d500ad590 con 0x55f66d011560 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d3effd700 1 -- 172.21.15.19:0/2719459254 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2d5400b8d0 con 0x55f66d011560 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 10 client.26861 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 10 client.26861 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 20 client.26861 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d12c380 0x55f66d12e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 4 client.26861 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:29:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f66d042d70 con 0x55f66d12c380 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.268+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d12c380 0x55f66d12e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.269+0000 7f2d3effd700 10 client.26861 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.269+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d12c380 0x55f66d12e7a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9faa8f546ede78c server_cookie=10605aef0a62e889 in_seq=0 out_seq=0 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d3effd700 1 -- 172.21.15.19:0/2719459254 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f66d042d70 con 0x55f66d12c380 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d3effd700 10 client.26861 handle_command_reply: tid=0 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 client.26861 shutdown 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 msgr2=0x55f66d11c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 0x55f66d11c830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009680 tx=0x7f2d54009260 comp rx=0 tx=0).stop 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 shutdown_connections 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d11c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d12c380 0x55f66d12e7a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d1126b0 0x55f66d11cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 --2- 172.21.15.19:0/2719459254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d011560 0x55f66d11c830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d3effd700 0 client.26861 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 >> 172.21.15.19:0/2719459254 conn(0x55f66cfd2e10 msgr2=0x55f66d107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 shutdown_connections 2024-05-17T09:29:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 1 -- 172.21.15.19:0/2719459254 wait complete. 2024-05-17T09:29:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.270+0000 7f2d74539b80 20 client.26861 trim_cache size 0 max 16384 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 Processor -- start 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 -- start start 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 0x55f66d116930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66d011560 0x55f66d111db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d133f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b09d0 con 0x55f66cf93fd0 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601ae260 con 0x55f66d1126b0 2024-05-17T09:29:00.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d74539b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601ac0b0 con 0x55f66d011560 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 0x55f66d116930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 0x55f66d116930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60716/0 (socket says 172.21.15.19:60716) 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/3228260974 learned_addr learned my addr 172.21.15.19:0/3228260974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5effd700 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66d011560 0x55f66d111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/3228260974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66d011560 msgr2=0x55f66d111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66d011560 0x55f66d111db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/3228260974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 msgr2=0x55f66d133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.294+0000 7f2d5f7fe700 1 -- 172.21.15.19:0/3228260974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66d11f0f0 con 0x55f66cf93fd0 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.295+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.295+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 0x55f66d116930 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2d500e3de0 tx=0x7f2d500a9580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b36c037e26d5c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.295+0000 7f2d457fa700 1 -- 172.21.15.19:0/3228260974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500cc070 con 0x55f66cf93fd0 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.295+0000 7f2d457fa700 1 -- 172.21.15.19:0/3228260974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d500ad590 con 0x55f66cf93fd0 2024-05-17T09:29:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.295+0000 7f2d457fa700 1 -- 172.21.15.19:0/3228260974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50012220 con 0x55f66cf93fd0 2024-05-17T09:29:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 -- 172.21.15.19:0/3228260974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 msgr2=0x55f66d116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 0x55f66d116930 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2d500e3de0 tx=0x7f2d500a9580 comp rx=0 tx=0).stop 2024-05-17T09:29:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 -- 172.21.15.19:0/3228260974 shutdown_connections 2024-05-17T09:29:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66cf93fd0 0x55f66d116930 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 --2- 172.21.15.19:0/3228260974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66d011560 0x55f66d111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 -- 172.21.15.19:0/3228260974 >> 172.21.15.19:0/3228260974 conn(0x55f66cfd2e10 msgr2=0x55f66d1116a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 -- 172.21.15.19:0/3228260974 shutdown_connections 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 -- 172.21.15.19:0/3228260974 wait complete. 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.296+0000 7f2d74539b80 1 -- start start 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 Processor -- start 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d121990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d121ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d196e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601ac720 con 0x55f66d011560 2024-05-17T09:29:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601adb90 con 0x55f66d1126b0 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601acdf0 con 0x55f66cf93fd0 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d74539b80 10 client.0 _ll_register_callbacks cb 0x55f66cfcbab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d5effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d121ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d5ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d5effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d121ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60724/0 (socket says 172.21.15.19:60724) 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d5effd700 1 -- 172.21.15.19:0/214070904 learned_addr learned my addr 172.21.15.19:0/214070904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.297+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5ffff700 1 -- 172.21.15.19:0/214070904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 msgr2=0x55f66d121990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d121990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5ffff700 1 -- 172.21.15.19:0/214070904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 msgr2=0x55f66d121ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d121ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5ffff700 1 -- 172.21.15.19:0/214070904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66d197390 con 0x55f66d1126b0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5effd700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d121ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d5ffff700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d196e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d58007cb0 tx=0x7f2d580114d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50247695d3615cc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d3effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d3effd700 1 -- 172.21.15.19:0/214070904 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011c60 con 0x55f66d1126b0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d500ad590 con 0x55f66d1126b0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d3effd700 1 -- 172.21.15.19:0/214070904 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2d58011e00 con 0x55f66d1126b0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d3effd700 1 -- 172.21.15.19:0/214070904 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d5802b050 con 0x55f66d1126b0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d74539b80 10 client.26867 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.298+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f66d198620 con 0x55f66d1126b0 2024-05-17T09:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d3effd700 1 -- 172.21.15.19:0/214070904 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d500ad590 con 0x55f66d1126b0 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d3effd700 1 -- 172.21.15.19:0/214070904 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2d58016540 con 0x55f66d1126b0 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d74539b80 10 client.26867 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d74539b80 10 client.26867 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d74539b80 20 client.26867 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d74539b80 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d198da0 0x55f66d19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d74539b80 4 client.26867 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.6, "await_for": 45.0}] 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.6, "await_for": 45.0}) -- 0x55f66d03c550 con 0x55f66d198da0 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d198da0 0x55f66d19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.299+0000 7f2d3effd700 10 client.26867 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.300+0000 7f2d5f7fe700 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d198da0 0x55f66d19b1c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d0676bfd2ae4be server_cookie=806ecb5dd229d8 in_seq=0 out_seq=0 2024-05-17T09:29:00.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d3effd700 1 -- 172.21.15.19:0/214070904 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f66d03c550 con 0x55f66d198da0 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d3effd700 10 client.26867 handle_command_reply: tid=0 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 client.26867 shutdown 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 msgr2=0x55f66d196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d196e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d58007cb0 tx=0x7f2d580114d0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 shutdown_connections 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f66cf93fd0 0x55f66d121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f66d198da0 0x55f66d19b1c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66d1126b0 0x55f66d196e00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 --2- 172.21.15.19:0/214070904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f66d011560 0x55f66d121ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d3effd700 0 client.26867 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 >> 172.21.15.19:0/214070904 conn(0x55f66cfd2e10 msgr2=0x55f66d116690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 shutdown_connections 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 1 -- 172.21.15.19:0/214070904 wait complete. 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.379+0000 7f2d74539b80 20 client.26867 trim_cache size 0 max 16384 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 34, 2024-05-17T09:29:00.381 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "4fb28607": { 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 34, 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:29:00.382 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 141.6, 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:00.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2d480b2880 msgr2=0x7f2d480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2d480b2880 0x7f2d480b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098c0 tx=0x7f2d54019040 comp rx=0 tx=0).stop 2024-05-17T09:29:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 msgr2=0x7f2d6019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 0x7f2d6019ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009db0 tx=0x7f2d5000c5b0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 shutdown_connections 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2d480b2880 0x7f2d480b4d90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d601034b0 0x7f2d6019ad10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d60108730 0x7f2d6019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 --2- 172.21.15.19:0/1834495269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d60107de0 0x7f2d6019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 >> 172.21.15.19:0/1834495269 conn(0x7f2d600fedb0 msgr2=0x7f2d6006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:00.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 shutdown_connections 2024-05-17T09:29:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:00.380+0000 7f2d65b24700 1 -- 172.21.15.19:0/1834495269 wait complete. 2024-05-17T09:29:00.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.73 MB/sec execute 298 sec latency 84.217 ms 2024-05-17T09:29:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:00 smithi069 ceph-mon[35081]: pgmap v715: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 281 KiB/s wr, 36 op/s 2024-05-17T09:29:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:00 smithi019 ceph-mon[31620]: pgmap v715: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 281 KiB/s wr, 36 op/s 2024-05-17T09:29:00.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:00 smithi079 ceph-mon[35502]: pgmap v715: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 281 KiB/s wr, 36 op/s 2024-05-17T09:29:01.393 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4fb28607', quiesce took 6.3 seconds. Will release after: 15.3 2024-05-17T09:29:01.438 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.73 MB/sec execute 299 sec latency 1045.361 ms 2024-05-17T09:29:02.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.72 MB/sec execute 300 sec latency 2045.451 ms 2024-05-17T09:29:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:02 smithi069 ceph-mon[35081]: pgmap v716: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 358 KiB/s wr, 61 op/s 2024-05-17T09:29:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:02 smithi019 ceph-mon[31620]: pgmap v716: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 358 KiB/s wr, 61 op/s 2024-05-17T09:29:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:02 smithi079 ceph-mon[35502]: pgmap v716: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 358 KiB/s wr, 61 op/s 2024-05-17T09:29:03.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.72 MB/sec execute 301 sec latency 3045.491 ms 2024-05-17T09:29:04.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.71 MB/sec execute 302 sec latency 4045.601 ms 2024-05-17T09:29:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:04 smithi069 ceph-mon[35081]: pgmap v717: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 549 KiB/s wr, 66 op/s 2024-05-17T09:29:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:04 smithi019 ceph-mon[31620]: pgmap v717: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 549 KiB/s wr, 66 op/s 2024-05-17T09:29:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:04 smithi079 ceph-mon[35502]: pgmap v717: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 549 KiB/s wr, 66 op/s 2024-05-17T09:29:05.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.71 MB/sec execute 303 sec latency 5045.665 ms 2024-05-17T09:29:06.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.70 MB/sec execute 304 sec latency 6045.823 ms 2024-05-17T09:29:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:06 smithi069 ceph-mon[35081]: pgmap v718: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 636 KiB/s wr, 77 op/s 2024-05-17T09:29:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:06 smithi019 ceph-mon[31620]: pgmap v718: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 636 KiB/s wr, 77 op/s 2024-05-17T09:29:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:06 smithi079 ceph-mon[35502]: pgmap v718: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 636 KiB/s wr, 77 op/s 2024-05-17T09:29:07.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.69 MB/sec execute 305 sec latency 7045.935 ms 2024-05-17T09:29:08.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.69 MB/sec execute 306 sec latency 8046.064 ms 2024-05-17T09:29:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:08 smithi069 ceph-mon[35081]: pgmap v719: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 637 KiB/s wr, 78 op/s 2024-05-17T09:29:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:08 smithi019 ceph-mon[31620]: pgmap v719: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 637 KiB/s wr, 78 op/s 2024-05-17T09:29:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2059548853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8109ba0 msgr2=0x7fc2c810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2059548853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8109ba0 0x7fc2c810bfc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009900 tx=0x7fc2c002a410 comp rx=0 tx=0).stop 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2059548853 shutdown_connections 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2059548853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8109ba0 0x7fc2c810bfc0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2059548853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8104ea0 0x7fc2c81052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2059548853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c8103c60 0x7fc2c8104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2059548853 >> 172.21.15.19:0/2059548853 conn(0x7fc2c80fee50 msgr2=0x7fc2c8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2059548853 shutdown_connections 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.714+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2059548853 wait complete. 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 Processor -- start 2024-05-17T09:29:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 -- start start 2024-05-17T09:29:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 0x7fc2c819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c8104ea0 0x7fc2c819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8109ba0 0x7fc2c81a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c810f620 con 0x7fc2c8103c60 2024-05-17T09:29:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c810f4a0 con 0x7fc2c8109ba0 2024-05-17T09:29:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2ceba8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c810f7a0 con 0x7fc2c8104ea0 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2cc944640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 0x7fc2c819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2cc944640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 0x7fc2c819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54486/0 (socket says 172.21.15.19:54486) 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2cc944640 1 -- 172.21.15.19:0/2820497067 learned_addr learned my addr 172.21.15.19:0/2820497067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c8104ea0 0x7fc2c819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.715+0000 7fc2cd145640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8109ba0 0x7fc2c81a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cc944640 1 -- 172.21.15.19:0/2820497067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c8104ea0 msgr2=0x7fc2c819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cc944640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c8104ea0 0x7fc2c819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cc944640 1 -- 172.21.15.19:0/2820497067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8109ba0 msgr2=0x7fc2c81a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cc944640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8109ba0 0x7fc2c81a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cc944640 1 -- 172.21.15.19:0/2820497067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c81a1ca0 con 0x7fc2c8103c60 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cd145640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8109ba0 0x7fc2c81a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:29:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2cc944640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 0x7fc2c819a160 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005fd0 tx=0x7fc2b800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2c5ffb640 1 -- 172.21.15.19:0/2820497067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8017070 con 0x7fc2c8103c60 2024-05-17T09:29:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c81a1f90 con 0x7fc2c8103c60 2024-05-17T09:29:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2c5ffb640 1 -- 172.21.15.19:0/2820497067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b8009c30 con 0x7fc2c8103c60 2024-05-17T09:29:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2c5ffb640 1 -- 172.21.15.19:0/2820497067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8003c90 con 0x7fc2c8103c60 2024-05-17T09:29:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.716+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c81a23f0 con 0x7fc2c8103c60 2024-05-17T09:29:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 Processor -- start 2024-05-17T09:29:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 -- start start 2024-05-17T09:29:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb186573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb1828d2f0 0x55bb18657920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb1865e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb185c9c30 con 0x55bb182b61b0 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb185b97d0 con 0x55bb182954d0 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bb185bd810 con 0x55bb1828d2f0 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2c5ffb640 1 -- 172.21.15.19:0/2820497067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2b8003e30 con 0x7fc2c8103c60 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb186573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb186573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54500/0 (socket says 172.21.15.19:54500) 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 -- 172.21.15.19:0/1395997556 learned_addr learned my addr 172.21.15.19:0/1395997556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb1865e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cc944640 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb1828d2f0 0x55bb18657920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 -- 172.21.15.19:0/1395997556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb1828d2f0 msgr2=0x55bb18657920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb1828d2f0 0x55bb18657920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 -- 172.21.15.19:0/1395997556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 msgr2=0x55bb1865e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.720+0000 7fc2cd145640 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb1865e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2cd145640 1 -- 172.21.15.19:0/1395997556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb182ce210 con 0x55bb182b61b0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2cd145640 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb186573a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0006010 tx=0x7fc2c0031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f50c16260aa02 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2beffd640 1 -- 172.21.15.19:0/1395997556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0035030 con 0x55bb182b61b0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2beffd640 1 -- 172.21.15.19:0/1395997556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c0004030 con 0x55bb182b61b0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2beffd640 1 -- 172.21.15.19:0/1395997556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003a10 con 0x55bb182b61b0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2c5ffb640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2b40b28a0 0x7fc2b40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2cc944640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2b40b28a0 0x7fc2b40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.721+0000 7fc2c5ffb640 1 -- 172.21.15.19:0/2820497067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2b80e22b0 con 0x7fc2c8103c60 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 -- 172.21.15.19:0/1395997556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 msgr2=0x55bb186573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2cc944640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2b40b28a0 0x7fc2b40b4db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55bb18658780 tx=0x7fc2b800bf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb186573a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0006010 tx=0x7fc2c0031580 comp rx=0 tx=0).stop 2024-05-17T09:29:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 -- 172.21.15.19:0/1395997556 shutdown_connections 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb1865e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb1828d2f0 0x55bb18657920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/1395997556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb186573a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 -- 172.21.15.19:0/1395997556 >> 172.21.15.19:0/1395997556 conn(0x55bb182aa300 msgr2=0x55bb18652d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 -- 172.21.15.19:0/1395997556 shutdown_connections 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 -- 172.21.15.19:0/1395997556 wait complete. 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 -- start start 2024-05-17T09:29:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.722+0000 7fc2d5a59740 1 Processor -- start 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb1828d2f0 0x55bb18660c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb186611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb18668100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb1829f790 con 0x55bb182b61b0 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb18575890 con 0x55bb1828d2f0 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bb1865f210 con 0x55bb182954d0 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2d5a59740 10 client.0 _ll_register_callbacks cb 0x55bb18578d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb18668100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb18668100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54512/0 (socket says 172.21.15.19:54512) 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2cd145640 1 -- 172.21.15.19:0/3379383996 learned_addr learned my addr 172.21.15.19:0/3379383996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb186611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb1828d2f0 0x55bb18660c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2cd145640 1 -- 172.21.15.19:0/3379383996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 msgr2=0x55bb186611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.723+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb186611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2cd145640 1 -- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb1828d2f0 msgr2=0x55bb18660c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb1828d2f0 0x55bb18660c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2cd145640 1 -- 172.21.15.19:0/3379383996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1839a620 con 0x55bb182b61b0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb18668100 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0005910 tx=0x7fc2c002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e53c48ca1cc5f1d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2a7fff640 1 -- 172.21.15.19:0/3379383996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0035030 con 0x55bb182b61b0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2a7fff640 1 -- 172.21.15.19:0/3379383996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2c0031a70 con 0x55bb182b61b0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c0004030 con 0x55bb182b61b0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2a7fff640 1 -- 172.21.15.19:0/3379383996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2c005a050 con 0x55bb182b61b0 2024-05-17T09:29:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2d5a59740 10 client.17256 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bb1866b030 con 0x55bb182b61b0 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.724+0000 7fc2a7fff640 1 -- 172.21.15.19:0/3379383996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c0004030 con 0x55bb182b61b0 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2a7fff640 1 -- 172.21.15.19:0/3379383996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc2c003a040 con 0x55bb182b61b0 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2d5a59740 10 client.17256 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2d5a59740 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2d5a59740 20 client.17256 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb18274d80 0x55bb1866e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2d5a59740 4 client.17256 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb185cbb80 con 0x55bb18274d80 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb18274d80 0x55bb1866e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2a7fff640 10 client.17256 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.725+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb18274d80 0x55bb1866e390 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478836ff618949d server_cookie=fb282fdc2a1f7f00 in_seq=0 out_seq=0 2024-05-17T09:29:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2a7fff640 1 -- 172.21.15.19:0/3379383996 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bb185cbb80 con 0x55bb18274d80 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2a7fff640 10 client.17256 handle_command_reply: tid=0 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 client.17256 shutdown 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 msgr2=0x55bb18668100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb18668100 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0005910 tx=0x7fc2c002ab10 comp rx=0 tx=0).stop 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 shutdown_connections 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb18274d80 0x55bb1866e390 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb182b61b0 0x55bb18668100 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb186611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3379383996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb1828d2f0 0x55bb18660c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2a7fff640 0 client.17256 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.726+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 >> 172.21.15.19:0/3379383996 conn(0x55bb182aa300 msgr2=0x55bb18655c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.727+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 shutdown_connections 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.727+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3379383996 wait complete. 2024-05-17T09:29:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.727+0000 7fc2d5a59740 20 client.17256 trim_cache size 0 max 16384 2024-05-17T09:29:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 Processor -- start 2024-05-17T09:29:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 -- start start 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb18673ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb18668020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182b61b0 0x55bb18673ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c81af1d0 con 0x55bb1828d2f0 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c81acaa0 con 0x55bb182954d0 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c81aa8f0 con 0x55bb182b61b0 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.742+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb18673ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb18673ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54524/0 (socket says 172.21.15.19:54524) 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 -- 172.21.15.19:0/3306790614 learned_addr learned my addr 172.21.15.19:0/3306790614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182b61b0 0x55bb18673ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb18668020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 -- 172.21.15.19:0/3306790614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182b61b0 msgr2=0x55bb18673ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182b61b0 0x55bb18673ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 -- 172.21.15.19:0/3306790614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 msgr2=0x55bb18668020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb18668020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 -- 172.21.15.19:0/3306790614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1865ebd0 con 0x55bb1828d2f0 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb18668020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2cd145640 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb18673ab0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc2c002a930 tx=0x7fc2c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad4b5db9a8d895d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2be7fc640 1 -- 172.21.15.19:0/3306790614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0035040 con 0x55bb1828d2f0 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2be7fc640 1 -- 172.21.15.19:0/3306790614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c004e040 con 0x55bb1828d2f0 2024-05-17T09:29:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.743+0000 7fc2be7fc640 1 -- 172.21.15.19:0/3306790614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c003c470 con 0x55bb1828d2f0 2024-05-17T09:29:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3306790614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 msgr2=0x55bb18673ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb18673ab0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc2c002a930 tx=0x7fc2c0009260 comp rx=0 tx=0).stop 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3306790614 shutdown_connections 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182b61b0 0x55bb18673ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182954d0 0x55bb18668020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/3306790614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb18673ab0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3306790614 >> 172.21.15.19:0/3306790614 conn(0x55bb182aa300 msgr2=0x55bb1865a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3306790614 shutdown_connections 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 -- 172.21.15.19:0/3306790614 wait complete. 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.744+0000 7fc2d5a59740 1 -- start start 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 Processor -- start 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb186677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb18667ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182b61b0 0x55bb186d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c81aaf60 con 0x55bb1828d2f0 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c81ac3d0 con 0x55bb182b61b0 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c81ab630 con 0x55bb182954d0 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2d5a59740 10 client.0 _ll_register_callbacks cb 0x7fc2c0003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb186677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb186677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54536/0 (socket says 172.21.15.19:54536) 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 -- 172.21.15.19:0/2426146008 learned_addr learned my addr 172.21.15.19:0/2426146008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb18667ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 -- 172.21.15.19:0/2426146008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 msgr2=0x55bb18667ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb18667ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 -- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182b61b0 msgr2=0x55bb186d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182b61b0 0x55bb186d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.745+0000 7fc2cd145640 1 -- 172.21.15.19:0/2426146008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb186d55e0 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2cd145640 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb186677a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0001eb0 tx=0x7fc2c0003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824866beb5ae94b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2a7fff640 1 -- 172.21.15.19:0/2426146008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0035040 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2a7fff640 1 -- 172.21.15.19:0/2426146008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2c0044500 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c004e040 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2a7fff640 1 -- 172.21.15.19:0/2426146008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2c003c880 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2d5a59740 10 client.17262 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bb186d67e0 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2a7fff640 1 -- 172.21.15.19:0/2426146008 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2c004e040 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2a7fff640 1 -- 172.21.15.19:0/2426146008 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc2c0069030 con 0x55bb1828d2f0 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.746+0000 7fc2d5a59740 10 client.17262 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2d5a59740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2d5a59740 20 client.17262 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb1824a1c0 0x55bb186d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2d5a59740 4 client.17262 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:29:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb185c5360 con 0x55bb1824a1c0 2024-05-17T09:29:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb1824a1c0 0x55bb186d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2a7fff640 10 client.17262 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.747+0000 7fc2c7fff640 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb1824a1c0 0x55bb186d9200 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb434d3ef1ff54ab server_cookie=5d90f9163b5fea94 in_seq=0 out_seq=0 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2a7fff640 1 -- 172.21.15.19:0/2426146008 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb185c5360 con 0x55bb1824a1c0 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2a7fff640 10 client.17262 handle_command_reply: tid=0 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 client.17262 shutdown 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 msgr2=0x55bb186677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb186677a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0001eb0 tx=0x7fc2c0003e60 comp rx=0 tx=0).stop 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 shutdown_connections 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bb1824a1c0 0x55bb186d9200 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb182b61b0 0x55bb186d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2a7fff640 0 client.17262 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bb182954d0 0x55bb18667ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 --2- 172.21.15.19:0/2426146008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb1828d2f0 0x55bb186677a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.771+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 >> 172.21.15.19:0/2426146008 conn(0x55bb182aa300 msgr2=0x55bb1865b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 shutdown_connections 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2d5a59740 1 -- 172.21.15.19:0/2426146008 wait complete. 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2d5a59740 20 client.17262 trim_cache size 0 max 16384 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:29:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:29:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2b40b28a0 msgr2=0x7fc2b40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2b40b28a0 0x7fc2b40b4db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55bb18658780 tx=0x7fc2b800bf60 comp rx=0 tx=0).stop 2024-05-17T09:29:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 msgr2=0x7fc2c819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 0x7fc2c819a160 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005fd0 tx=0x7fc2b800dde0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 shutdown_connections 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c8109ba0 0x7fc2c81a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2c8104ea0 0x7fc2c819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2b40b28a0 0x7fc2b40b4db0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 --2- 172.21.15.19:0/2820497067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2c8103c60 0x7fc2c819a160 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 >> 172.21.15.19:0/2820497067 conn(0x7fc2c80fee50 msgr2=0x7fc2c8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.772+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 shutdown_connections 2024-05-17T09:29:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:08.773+0000 7fc2ceba8640 1 -- 172.21.15.19:0/2820497067 wait complete. 2024-05-17T09:29:08.775 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:08 smithi079 ceph-mon[35502]: pgmap v719: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 637 KiB/s wr, 78 op/s 2024-05-17T09:29:09.439 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.68 MB/sec execute 307 sec latency 9046.187 ms 2024-05-17T09:29:10.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.68 MB/sec execute 308 sec latency 10046.305 ms 2024-05-17T09:29:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:10 smithi069 ceph-mon[35081]: pgmap v720: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 575 KiB/s wr, 71 op/s 2024-05-17T09:29:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:10 smithi019 ceph-mon[31620]: pgmap v720: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 575 KiB/s wr, 71 op/s 2024-05-17T09:29:10.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:10 smithi079 ceph-mon[35502]: pgmap v720: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 575 KiB/s wr, 71 op/s 2024-05-17T09:29:11.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.67 MB/sec execute 309 sec latency 11046.443 ms 2024-05-17T09:29:12.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.67 MB/sec execute 310 sec latency 12046.554 ms 2024-05-17T09:29:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:12 smithi069 ceph-mon[35081]: pgmap v721: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 366 KiB/s wr, 53 op/s 2024-05-17T09:29:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:12 smithi019 ceph-mon[31620]: pgmap v721: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 366 KiB/s wr, 53 op/s 2024-05-17T09:29:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:12 smithi079 ceph-mon[35502]: pgmap v721: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 366 KiB/s wr, 53 op/s 2024-05-17T09:29:13.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.66 MB/sec execute 311 sec latency 13046.687 ms 2024-05-17T09:29:14.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.66 MB/sec execute 312 sec latency 14046.764 ms 2024-05-17T09:29:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:14 smithi069 ceph-mon[35081]: pgmap v722: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 287 KiB/s wr, 27 op/s 2024-05-17T09:29:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:14 smithi019 ceph-mon[31620]: pgmap v722: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 287 KiB/s wr, 27 op/s 2024-05-17T09:29:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:14 smithi079 ceph-mon[35502]: pgmap v722: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 287 KiB/s wr, 27 op/s 2024-05-17T09:29:15.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.65 MB/sec execute 313 sec latency 15046.874 ms 2024-05-17T09:29:16.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.65 MB/sec execute 314 sec latency 16046.982 ms 2024-05-17T09:29:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:16 smithi069 ceph-mon[35081]: pgmap v723: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 97 KiB/s wr, 23 op/s 2024-05-17T09:29:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:16 smithi019 ceph-mon[31620]: pgmap v723: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 97 KiB/s wr, 23 op/s 2024-05-17T09:29:16.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:29:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:16 smithi079 ceph-mon[35502]: pgmap v723: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 97 KiB/s wr, 23 op/s 2024-05-17T09:29:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:29:17.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.64 MB/sec execute 315 sec latency 17047.080 ms 2024-05-17T09:29:18.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.63 MB/sec execute 316 sec latency 18047.171 ms 2024-05-17T09:29:18.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:18 smithi019 ceph-mon[31620]: pgmap v724: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:29:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:18 smithi069 ceph-mon[35081]: pgmap v724: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:29:18.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1370295855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab0105dc0 msgr2=0x7f8ab010e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1370295855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab0105dc0 0x7f8ab010e310 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009990 tx=0x7f8aa4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1370295855 shutdown_connections 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1370295855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ab0105470 0x7f8ab0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1370295855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1370295855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab0105dc0 0x7f8ab010e310 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1370295855 >> 172.21.15.19:0/1370295855 conn(0x7f8ab00fef20 msgr2=0x7f8ab0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1370295855 shutdown_connections 2024-05-17T09:29:18.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.762+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1370295855 wait complete. 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 Processor -- start 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 -- start start 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ab0105470 0x7f8ab0103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ab0105dc0 0x7f8ab01036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab01142f0 con 0x7f8ab010e8e0 2024-05-17T09:29:18.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0114170 con 0x7f8ab0105470 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.763+0000 7f8ab87f8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0114470 con 0x7f8ab0105dc0 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0103be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48896/0 (socket says 172.21.15.19:48896) 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab5d93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ab0105dc0 0x7f8ab01036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6594700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ab0105470 0x7f8ab0103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 -- 172.21.15.19:0/1472558966 learned_addr learned my addr 172.21.15.19:0/1472558966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 -- 172.21.15.19:0/1472558966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ab0105dc0 msgr2=0x7f8ab01036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ab0105dc0 0x7f8ab01036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 -- 172.21.15.19:0/1472558966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ab0105470 msgr2=0x7f8ab0103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ab0105470 0x7f8ab0103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 -- 172.21.15.19:0/1472558966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01a90a0 con 0x7f8ab010e8e0 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab6d95700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0103be0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d980 tx=0x7f8aa800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:18.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8017070 con 0x7f8ab010e8e0 2024-05-17T09:29:18.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8aa8005020 con 0x7f8ab010e8e0 2024-05-17T09:29:18.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.764+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01a9390 con 0x7f8ab010e8e0 2024-05-17T09:29:18.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.765+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800f980 con 0x7f8ab010e8e0 2024-05-17T09:29:18.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.765+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab01a98f0 con 0x7f8ab010e8e0 2024-05-17T09:29:18.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.766+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab00512b0 con 0x7f8ab010e8e0 2024-05-17T09:29:18.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.767+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8aa8004a30 con 0x7f8ab010e8e0 2024-05-17T09:29:18.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.768+0000 7f8aaf7fe700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a980b2900 0x7f8a980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:18.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.768+0000 7f8ab6594700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a980b2900 0x7f8a980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:18.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.768+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8aa80e22a0 con 0x7f8ab010e8e0 2024-05-17T09:29:18.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.768+0000 7f8ab6594700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a980b2900 0x7f8a980b4e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00060b0 tx=0x7f8aa000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:18 smithi079 ceph-mon[35502]: pgmap v724: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:29:18.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:18.773+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8aa8020020 con 0x7f8ab010e8e0 2024-05-17T09:29:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.040+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ab002e5e0 con 0x7f8ab010e8e0 2024-05-17T09:29:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.041+0000 7f8aaf7fe700 1 -- 172.21.15.19:0/1472558966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f8aa80a9bb0 con 0x7f8ab010e8e0 2024-05-17T09:29:19.041 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:29:19.042 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a980b2900 msgr2=0x7f8a980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a980b2900 0x7f8a980b4e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00060b0 tx=0x7f8aa000c040 comp rx=0 tx=0).stop 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 msgr2=0x7f8ab0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0103be0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d980 tx=0x7f8aa800dc90 comp rx=0 tx=0).stop 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 shutdown_connections 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab010e8e0 0x7f8ab0103be0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ab0105470 0x7f8ab0103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a980b2900 0x7f8a980b4e10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 --2- 172.21.15.19:0/1472558966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ab0105dc0 0x7f8ab01036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 >> 172.21.15.19:0/1472558966 conn(0x7f8ab00fef20 msgr2=0x7f8ab0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 shutdown_connections 2024-05-17T09:29:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:19.044+0000 7f8ab87f8700 1 -- 172.21.15.19:0/1472558966 wait complete. 2024-05-17T09:29:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:29:19.440 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.63 MB/sec execute 317 sec latency 19047.231 ms 2024-05-17T09:29:19.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:19 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1472558966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:29:19.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:19 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1472558966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:29:19.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:19 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1472558966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:29:19.794 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 4fb28607 --release --await' 2024-05-17T09:29:19.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 4fb28607 --release --await 2024-05-17T09:29:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:29:20.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.62 MB/sec execute 318 sec latency 20047.292 ms 2024-05-17T09:29:20.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:20 smithi019 ceph-mon[31620]: pgmap v725: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 12 op/s 2024-05-17T09:29:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:20 smithi069 ceph-mon[35081]: pgmap v725: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 12 op/s 2024-05-17T09:29:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:20 smithi079 ceph-mon[35502]: pgmap v725: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 12 op/s 2024-05-17T09:29:21.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138840 1.62 MB/sec execute 319 sec latency 21047.400 ms 2024-05-17T09:29:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.773+0000 7fae9a97c700 1 -- 172.21.15.19:0/10693646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 msgr2=0x7fae941100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.773+0000 7fae9a97c700 1 --2- 172.21.15.19:0/10693646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae941100a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b0d0 tx=0x7fae8c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.773+0000 7fae9a97c700 1 -- 172.21.15.19:0/10693646 shutdown_connections 2024-05-17T09:29:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.773+0000 7fae9a97c700 1 --2- 172.21.15.19:0/10693646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae941100a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.773+0000 7fae9a97c700 1 --2- 172.21.15.19:0/10693646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae94103510 0x7fae94103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.773+0000 7fae9a97c700 1 --2- 172.21.15.19:0/10693646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae94107df0 0x7fae94108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.774+0000 7fae9a97c700 1 -- 172.21.15.19:0/10693646 >> 172.21.15.19:0/10693646 conn(0x7fae940fef80 msgr2=0x7fae941013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.774+0000 7fae9a97c700 1 -- 172.21.15.19:0/10693646 shutdown_connections 2024-05-17T09:29:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.774+0000 7fae9a97c700 1 -- 172.21.15.19:0/10693646 wait complete. 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 Processor -- start 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 -- start start 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae94103510 0x7fae9419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae94107df0 0x7fae9419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae9419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae94113240 con 0x7fae94108740 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae941130c0 con 0x7fae94103510 2024-05-17T09:29:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae9a97c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fae941133c0 con 0x7fae94107df0 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae98f19700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae9419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae98f19700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae9419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48916/0 (socket says 172.21.15.19:48916) 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae98f19700 1 -- 172.21.15.19:0/4100859093 learned_addr learned my addr 172.21.15.19:0/4100859093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae93fff700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae94103510 0x7fae9419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae8bfff700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae94107df0 0x7fae9419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.775+0000 7fae98f19700 1 -- 172.21.15.19:0/4100859093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae94107df0 msgr2=0x7fae9419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae98f19700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae94107df0 0x7fae9419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae98f19700 1 -- 172.21.15.19:0/4100859093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae94103510 msgr2=0x7fae9419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae98f19700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae94103510 0x7fae9419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae98f19700 1 -- 172.21.15.19:0/4100859093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae941a6eb0 con 0x7fae94108740 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae98f19700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae9419c600 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009850 tx=0x7fae8c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae91ffb700 1 -- 172.21.15.19:0/4100859093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01f070 con 0x7fae94108740 2024-05-17T09:29:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae91ffb700 1 -- 172.21.15.19:0/4100859093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae8c004c50 con 0x7fae94108740 2024-05-17T09:29:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae91ffb700 1 -- 172.21.15.19:0/4100859093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01b690 con 0x7fae94108740 2024-05-17T09:29:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae941a7140 con 0x7fae94108740 2024-05-17T09:29:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.776+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941a75f0 con 0x7fae94108740 2024-05-17T09:29:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.779+0000 7fae91ffb700 1 -- 172.21.15.19:0/4100859093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae8c022070 con 0x7fae94108740 2024-05-17T09:29:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7fae91ffb700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fae800b2900 0x7fae800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7fae93fff700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fae800b2900 0x7fae800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7fae91ffb700 1 -- 172.21.15.19:0/4100859093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fae8c00f020 con 0x7fae94108740 2024-05-17T09:29:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 Processor -- start 2024-05-17T09:29:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 -- start start 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 0x55f5ad01bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5ace98f40 0x55f5ad01c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7fae93fff700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fae800b2900 0x7fae800b4e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fae84009c60 tx=0x7fae84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad020890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5acf45d90 con 0x55f5acf164d0 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f5acf35930 con 0x55f5ace98f40 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.780+0000 7faea9391b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f5acf39970 con 0x55f5ad020480 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 0x55f5ad01bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 0x55f5ad01bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48922/0 (socket says 172.21.15.19:48922) 2024-05-17T09:29:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 -- 172.21.15.19:0/1074183037 learned_addr learned my addr 172.21.15.19:0/1074183037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae8bfff700 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5ace98f40 0x55f5ad01c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 -- 172.21.15.19:0/1074183037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 msgr2=0x55f5ad020890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae98f19700 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad020890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad020890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 -- 172.21.15.19:0/1074183037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5ace98f40 msgr2=0x55f5ad01c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5ace98f40 0x55f5ad01c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 -- 172.21.15.19:0/1074183037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ace859d0 con 0x55f5acf164d0 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae93fff700 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 0x55f5ad01bcf0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fae8400e4e0 tx=0x7fae84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c933115f4b751b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae8a7fc700 1 -- 172.21.15.19:0/1074183037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84026430 con 0x55f5acf164d0 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae8a7fc700 1 -- 172.21.15.19:0/1074183037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae8401c8a0 con 0x55f5acf164d0 2024-05-17T09:29:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.781+0000 7fae8a7fc700 1 -- 172.21.15.19:0/1074183037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8401d8b0 con 0x55f5acf164d0 2024-05-17T09:29:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 -- 172.21.15.19:0/1074183037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 msgr2=0x55f5ad01bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 0x55f5ad01bcf0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fae8400e4e0 tx=0x7fae84014000 comp rx=0 tx=0).stop 2024-05-17T09:29:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 -- 172.21.15.19:0/1074183037 shutdown_connections 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad020890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5acf164d0 0x55f5ad01bcf0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 --2- 172.21.15.19:0/1074183037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5ace98f40 0x55f5ad01c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 -- 172.21.15.19:0/1074183037 >> 172.21.15.19:0/1074183037 conn(0x55f5aced7d80 msgr2=0x55f5ad012520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 -- 172.21.15.19:0/1074183037 shutdown_connections 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.782+0000 7faea9391b80 1 -- 172.21.15.19:0/1074183037 wait complete. 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 -- start start 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 Processor -- start 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ace98f40 0x55f5ad025730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad025c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ad020480 0x55f5ad0261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f5ace3c0a0 con 0x55f5ad020480 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f5aced78c0 con 0x55f5acf164d0 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad021230 con 0x55f5ace98f40 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.783+0000 7faea9391b80 10 client.0 _ll_register_callbacks cb 0x55f5aced0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad025c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad025c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36248/0 (socket says 172.21.15.19:36248) 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae8bfff700 1 -- 172.21.15.19:0/790560230 learned_addr learned my addr 172.21.15.19:0/790560230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ad020480 0x55f5ad0261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae93fff700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ace98f40 0x55f5ad025730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 -- 172.21.15.19:0/790560230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ace98f40 msgr2=0x55f5ad025730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ace98f40 0x55f5ad025730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 -- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 msgr2=0x55f5ad025c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad025c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 -- 172.21.15.19:0/790560230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5aced4790 con 0x55f5ad020480 2024-05-17T09:29:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae98f19700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ad020480 0x55f5ad0261b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0e0db0 tx=0x7fae8c0a60f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a594ae74f268749 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae737fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae737fe700 1 -- 172.21.15.19:0/790560230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0aa470 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7fae737fe700 1 -- 172.21.15.19:0/790560230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fae8c0e3360 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.784+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8401c8a0 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7fae737fe700 1 -- 172.21.15.19:0/790560230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae8c0ab060 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 10 client.17280 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f5ad033010 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7fae737fe700 1 -- 172.21.15.19:0/790560230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae8401c8a0 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7fae737fe700 1 -- 172.21.15.19:0/790560230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fae8c011070 con 0x55f5ad020480 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 10 client.17280 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 10 client.17280 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 20 client.17280 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0355d0 0x55f5ad037a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 4 client.17280 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:29:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.785+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5acf47ce0 con 0x55f5ad0355d0 2024-05-17T09:29:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.786+0000 7fae93fff700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0355d0 0x55f5ad037a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.786+0000 7fae737fe700 10 client.17280 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.786+0000 7fae93fff700 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0355d0 0x55f5ad037a60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbe77b86454d6b4 server_cookie=d15f1f6732943006 in_seq=0 out_seq=0 2024-05-17T09:29:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7fae737fe700 1 -- 172.21.15.19:0/790560230 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f5acf47ce0 con 0x55f5ad0355d0 2024-05-17T09:29:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7fae737fe700 10 client.17280 handle_command_reply: tid=0 2024-05-17T09:29:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 client.17280 shutdown 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ad020480 msgr2=0x55f5ad0261b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ad020480 0x55f5ad0261b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0e0db0 tx=0x7fae8c0a60f0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 shutdown_connections 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ace98f40 0x55f5ad025730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0355d0 0x55f5ad037a60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ad020480 0x55f5ad0261b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 --2- 172.21.15.19:0/790560230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad025c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7fae737fe700 0 client.17280 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.787+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 >> 172.21.15.19:0/790560230 conn(0x55f5aced7d80 msgr2=0x55f5ad016610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.788+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 shutdown_connections 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.788+0000 7faea9391b80 1 -- 172.21.15.19:0/790560230 wait complete. 2024-05-17T09:29:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.788+0000 7faea9391b80 20 client.17280 trim_cache size 0 max 16384 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 Processor -- start 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 -- start start 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad025ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad024730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad026de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae941b0970 con 0x55f5ace98f40 2024-05-17T09:29:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae941ae200 con 0x55f5acf164d0 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.808+0000 7faea9391b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fae941ac050 con 0x55f5ad020480 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad025ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad025ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48940/0 (socket says 172.21.15.19:48940) 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 -- 172.21.15.19:0/1128938523 learned_addr learned my addr 172.21.15.19:0/1128938523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae8bfff700 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad024730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae98f19700 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad026de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 -- 172.21.15.19:0/1128938523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 msgr2=0x55f5ad026de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad026de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 -- 172.21.15.19:0/1128938523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 msgr2=0x55f5ad024730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad024730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 -- 172.21.15.19:0/1128938523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ad0223c0 con 0x55f5ace98f40 2024-05-17T09:29:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae8bfff700 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad024730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae93fff700 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad025ff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fae8400eb80 tx=0x7fae8401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52aff189efab1ba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae89ffb700 1 -- 172.21.15.19:0/1128938523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8400ed70 con 0x55f5ace98f40 2024-05-17T09:29:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae89ffb700 1 -- 172.21.15.19:0/1128938523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae840054d0 con 0x55f5ace98f40 2024-05-17T09:29:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.809+0000 7fae89ffb700 1 -- 172.21.15.19:0/1128938523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84003dd0 con 0x55f5ace98f40 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 -- 172.21.15.19:0/1128938523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 msgr2=0x55f5ad025ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad025ff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fae8400eb80 tx=0x7fae8401c410 comp rx=0 tx=0).stop 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 -- 172.21.15.19:0/1128938523 shutdown_connections 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad026de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad025ff0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 --2- 172.21.15.19:0/1128938523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad024730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.810+0000 7faea9391b80 1 -- 172.21.15.19:0/1128938523 >> 172.21.15.19:0/1128938523 conn(0x55f5aced7d80 msgr2=0x55f5ad024020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.811+0000 7faea9391b80 1 -- 172.21.15.19:0/1128938523 shutdown_connections 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.811+0000 7faea9391b80 1 -- 172.21.15.19:0/1128938523 wait complete. 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.811+0000 7faea9391b80 1 -- start start 2024-05-17T09:29:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.811+0000 7faea9391b80 1 Processor -- start 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.811+0000 7faea9391b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad029dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.811+0000 7faea9391b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad02a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7faea9391b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad0990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7faea9391b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae941ac6c0 con 0x55f5ace98f40 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7faea9391b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae941adb30 con 0x55f5acf164d0 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7faea9391b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fae941acd90 con 0x55f5ad020480 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7faea9391b80 10 client.0 _ll_register_callbacks cb 0x55f5aced0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad029dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad029dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48944/0 (socket says 172.21.15.19:48944) 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 -- 172.21.15.19:0/1308573530 learned_addr learned my addr 172.21.15.19:0/1308573530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae8bfff700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad02a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae98f19700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad0990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 -- 172.21.15.19:0/1308573530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 msgr2=0x55f5ad0990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad0990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 -- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 msgr2=0x55f5ad02a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad02a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 -- 172.21.15.19:0/1308573530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ad099650 con 0x55f5ace98f40 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae93fff700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad029dc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fae84011ab0 tx=0x7fae8400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a344036aff7c5f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae737fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.812+0000 7fae737fe700 1 -- 172.21.15.19:0/1308573530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84005180 con 0x55f5ace98f40 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae840054d0 con 0x55f5ace98f40 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7fae737fe700 1 -- 172.21.15.19:0/1308573530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fae84003680 con 0x55f5ace98f40 2024-05-17T09:29:21.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7fae737fe700 1 -- 172.21.15.19:0/1308573530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae84005320 con 0x55f5ace98f40 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 10 client.17286 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f5ad0a0990 con 0x55f5ace98f40 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7fae737fe700 1 -- 172.21.15.19:0/1308573530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae840054d0 con 0x55f5ace98f40 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7fae737fe700 1 -- 172.21.15.19:0/1308573530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fae8401d480 con 0x55f5ace98f40 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 10 client.17286 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 10 client.17286 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 20 client.17286 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0a0d60 0x55f5ad0a3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 4 client.17286 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4fb28607", "release": true, "await": true}] 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.813+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4fb28607", "release": true, "await": true}) -- 0x55f5acf414c0 con 0x55f5ad0a0d60 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.814+0000 7fae8bfff700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0a0d60 0x55f5ad0a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.814+0000 7fae8bfff700 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0a0d60 0x55f5ad0a3180 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05ff918f83942de server_cookie=3725fad66057015f in_seq=0 out_seq=0 2024-05-17T09:29:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.814+0000 7fae737fe700 10 client.17286 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7fae737fe700 1 -- 172.21.15.19:0/1308573530 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f5acf414c0 con 0x55f5ad0a0d60 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7fae737fe700 10 client.17286 handle_command_reply: tid=0 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 client.17286 shutdown 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 msgr2=0x55f5ad029dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad029dc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fae84011ab0 tx=0x7fae8400eed0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 shutdown_connections 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5ad020480 0x55f5ad0990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f5ad0a0d60 0x55f5ad0a3180 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f5ace98f40 0x55f5ad029dc0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 --2- 172.21.15.19:0/1308573530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5acf164d0 0x55f5ad02a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 >> 172.21.15.19:0/1308573530 conn(0x55f5aced7d80 msgr2=0x55f5ad00ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7fae737fe700 0 client.17286 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 shutdown_connections 2024-05-17T09:29:21.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 1 -- 172.21.15.19:0/1308573530 wait complete. 2024-05-17T09:29:21.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.818+0000 7faea9391b80 20 client.17286 trim_cache size 0 max 16384 2024-05-17T09:29:21.822 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:29:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:29:21.822 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 36, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "4fb28607": { 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 36, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 141.6, 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:29:21.823 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:29:21.824 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:29:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fae800b2900 msgr2=0x7fae800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fae800b2900 0x7fae800b4e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fae84009c60 tx=0x7fae84009260 comp rx=0 tx=0).stop 2024-05-17T09:29:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 msgr2=0x7fae9419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae9419c600 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009850 tx=0x7fae8c007a20 comp rx=0 tx=0).stop 2024-05-17T09:29:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 shutdown_connections 2024-05-17T09:29:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae9419c600 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae94103510 0x7fae9419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fae800b2900 0x7fae800b4e10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 --2- 172.21.15.19:0/4100859093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae94107df0 0x7fae9419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 >> 172.21.15.19:0/4100859093 conn(0x7fae940fef80 msgr2=0x7fae94107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 shutdown_connections 2024-05-17T09:29:21.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:29:21.819+0000 7fae9a97c700 1 -- 172.21.15.19:0/4100859093 wait complete. 2024-05-17T09:29:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 -- 172.21.15.19:0/171909028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4103c60 msgr2=0x7f61f4104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 --2- 172.21.15.19:0/171909028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4103c60 0x7f61f4104960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009860 tx=0x7f61e402a340 comp rx=0 tx=0).stop 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 -- 172.21.15.19:0/171909028 shutdown_connections 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 --2- 172.21.15.19:0/171909028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4109ba0 0x7f61f410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 --2- 172.21.15.19:0/171909028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4104ea0 0x7f61f41052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 --2- 172.21.15.19:0/171909028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4103c60 0x7f61f4104960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 -- 172.21.15.19:0/171909028 >> 172.21.15.19:0/171909028 conn(0x7f61f40fee50 msgr2=0x7f61f4101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 -- 172.21.15.19:0/171909028 shutdown_connections 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.890+0000 7f61f8f66640 1 -- 172.21.15.19:0/171909028 wait complete. 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 Processor -- start 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 -- start start 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4103c60 0x7f61f419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 0x7f61f419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4109ba0 0x7f61f41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410f5f0 con 0x7f61f4104ea0 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410f470 con 0x7f61f4103c60 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f8f66640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410f770 con 0x7f61f4109ba0 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 0x7f61f419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 0x7f61f419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48970/0 (socket says 172.21.15.19:48970) 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f1d9b640 1 -- 172.21.15.19:0/1577476355 learned_addr learned my addr 172.21.15.19:0/1577476355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f259c640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4103c60 0x7f61f419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4109ba0 0x7f61f41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f1d9b640 1 -- 172.21.15.19:0/1577476355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4109ba0 msgr2=0x7f61f41a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.891+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4109ba0 0x7f61f41a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f1d9b640 1 -- 172.21.15.19:0/1577476355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4103c60 msgr2=0x7f61f419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4103c60 0x7f61f419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f1d9b640 1 -- 172.21.15.19:0/1577476355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f41a1d30 con 0x7f61f4104ea0 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f259c640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4103c60 0x7f61f419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4109ba0 0x7f61f41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 0x7f61f419a730 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f61e800e9c0 tx=0x7f61e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61e37fe640 1 -- 172.21.15.19:0/1577476355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e800ccc0 con 0x7f61f4104ea0 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f41a1fc0 con 0x7f61f4104ea0 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61e37fe640 1 -- 172.21.15.19:0/1577476355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e80054b0 con 0x7f61f4104ea0 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61e37fe640 1 -- 172.21.15.19:0/1577476355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8010430 con 0x7f61f4104ea0 2024-05-17T09:29:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.892+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f41a24d0 con 0x7f61f4104ea0 2024-05-17T09:29:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 Processor -- start 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 -- start start 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6467b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f6467bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64682c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f645e9c30 con 0x559f642b54d0 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f645d97d0 con 0x559f642d61b0 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61ffe16740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f645dd810 con 0x559f642ad2f0 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.895+0000 7f61f2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6467b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6467b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36294/0 (socket says 172.21.15.19:36294) 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 -- 172.21.15.19:0/275533144 learned_addr learned my addr 172.21.15.19:0/275533144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61e37fe640 1 -- 172.21.15.19:0/1577476355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f61e8005020 con 0x7f61f4104ea0 2024-05-17T09:29:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f259c640 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f6467bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 -- 172.21.15.19:0/275533144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 msgr2=0x559f6467bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f6467bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 -- 172.21.15.19:0/275533144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 msgr2=0x559f64682c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64682c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 -- 172.21.15.19:0/275533144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f642ee210 con 0x559f642d61b0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6467b7b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61ec010450 tx=0x7f61ec010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e3d4aa55b2229f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61e0ff9640 1 -- 172.21.15.19:0/275533144 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec017070 con 0x559f642d61b0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61e0ff9640 1 -- 172.21.15.19:0/275533144 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61ec011bf0 con 0x559f642d61b0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.896+0000 7f61e0ff9640 1 -- 172.21.15.19:0/275533144 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec01ac00 con 0x559f642d61b0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61e37fe640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61dc0b28a0 0x7f61dc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61f259c640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61dc0b28a0 0x7f61dc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61e37fe640 1 -- 172.21.15.19:0/1577476355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f61e80e2a60 con 0x7f61f4104ea0 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 -- 172.21.15.19:0/275533144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 msgr2=0x559f6467b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6467b7b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61ec010450 tx=0x7f61ec010910 comp rx=0 tx=0).stop 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61f259c640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61dc0b28a0 0x7f61dc0b4db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f61e402a860 tx=0x7f61e402bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 -- 172.21.15.19:0/275533144 shutdown_connections 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64682c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f6467bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 --2- 172.21.15.19:0/275533144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6467b7b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 -- 172.21.15.19:0/275533144 >> 172.21.15.19:0/275533144 conn(0x559f642ca300 msgr2=0x559f64672d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.897+0000 7f61ffe16740 1 -- 172.21.15.19:0/275533144 shutdown_connections 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 -- 172.21.15.19:0/275533144 wait complete. 2024-05-17T09:29:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 -- start start 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 Processor -- start 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f64685150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64685690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6468c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f642bf790 con 0x559f642b54d0 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f64595890 con 0x559f642d61b0 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f64683680 con 0x559f642ad2f0 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61ffe16740 10 client.0 _ll_register_callbacks cb 0x559f64598d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64685690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64685690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48982/0 (socket says 172.21.15.19:48982) 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61f1d9b640 1 -- 172.21.15.19:0/3251072392 learned_addr learned my addr 172.21.15.19:0/3251072392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.898+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6468c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f259c640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f64685150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f1d9b640 1 -- 172.21.15.19:0/3251072392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 msgr2=0x559f64685150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f64685150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f1d9b640 1 -- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 msgr2=0x559f6468c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6468c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f1d9b640 1 -- 172.21.15.19:0/3251072392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f643ba620 con 0x559f642b54d0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6468c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64685690 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f61e80a99d0 tx=0x7f61e80a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3bd4ed3d42dbce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61d9ffb640 1 -- 172.21.15.19:0/3251072392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e80c1060 con 0x559f642b54d0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61d9ffb640 1 -- 172.21.15.19:0/3251072392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f61e80a8f30 con 0x559f642b54d0 2024-05-17T09:29:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61ec011bf0 con 0x559f642b54d0 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61d9ffb640 1 -- 172.21.15.19:0/3251072392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f61e806e050 con 0x559f642b54d0 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61ffe16740 10 client.17295 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.899+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f6468f590 con 0x559f642b54d0 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61d9ffb640 1 -- 172.21.15.19:0/3251072392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61ec011bf0 con 0x559f642b54d0 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61d9ffb640 1 -- 172.21.15.19:0/3251072392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f61e80a89a0 con 0x559f642b54d0 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61ffe16740 10 client.17295 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61ffe16740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61ffe16740 20 client.17295 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f64294d80 0x559f64692c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61ffe16740 4 client.17295 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f645ebb80 con 0x559f64294d80 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.900+0000 7f61f259c640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f64294d80 0x559f64692c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61d9ffb640 10 client.17295 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61f259c640 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f64294d80 0x559f64692c70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc117023496f62dd server_cookie=863a86c7bfe56165 in_seq=0 out_seq=0 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61d9ffb640 1 -- 172.21.15.19:0/3251072392 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f645ebb80 con 0x559f64294d80 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61d9ffb640 10 client.17295 handle_command_reply: tid=0 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61ffe16740 1 client.17295 shutdown 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 msgr2=0x559f64685690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.901+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64685690 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f61e80a99d0 tx=0x7f61e80a9500 comp rx=0 tx=0).stop 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 shutdown_connections 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f64294d80 0x559f64692c70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642d61b0 0x559f6468c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64685690 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3251072392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642ad2f0 0x559f64685150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 >> 172.21.15.19:0/3251072392 conn(0x559f642ca300 msgr2=0x559f64671d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61d9ffb640 0 client.17295 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 shutdown_connections 2024-05-17T09:29:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 1 -- 172.21.15.19:0/3251072392 wait complete. 2024-05-17T09:29:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.902+0000 7f61ffe16740 20 client.17295 trim_cache size 0 max 16384 2024-05-17T09:29:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 Processor -- start 2024-05-17T09:29:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 -- start start 2024-05-17T09:29:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f6467fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f6468c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f64676600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41ad140 con 0x559f642b54d0 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41aaa10 con 0x559f642ad2f0 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61ffe16740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41a8860 con 0x559f642d61b0 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f6468c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f6468c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48996/0 (socket says 172.21.15.19:48996) 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f6467fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61f1d9b640 1 -- 172.21.15.19:0/3656961577 learned_addr learned my addr 172.21.15.19:0/3656961577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.919+0000 7f61f2d9d640 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f64676600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61f259c640 1 -- 172.21.15.19:0/3656961577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 msgr2=0x559f64676600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61f259c640 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f64676600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61f259c640 1 -- 172.21.15.19:0/3656961577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 msgr2=0x559f6468c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61f259c640 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f6468c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61f259c640 1 -- 172.21.15.19:0/3656961577 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6466e540 con 0x559f642ad2f0 2024-05-17T09:29:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61f259c640 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f6467fd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61f4104860 tx=0x7f61e403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633c24a6762579aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61dbfff640 1 -- 172.21.15.19:0/3656961577 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4037040 con 0x559f642ad2f0 2024-05-17T09:29:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61dbfff640 1 -- 172.21.15.19:0/3656961577 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e4032c80 con 0x559f642ad2f0 2024-05-17T09:29:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.920+0000 7f61dbfff640 1 -- 172.21.15.19:0/3656961577 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4003ae0 con 0x559f642ad2f0 2024-05-17T09:29:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 -- 172.21.15.19:0/3656961577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 msgr2=0x559f6467fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f6467fd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61f4104860 tx=0x7f61e403e000 comp rx=0 tx=0).stop 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 -- 172.21.15.19:0/3656961577 shutdown_connections 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f64676600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f6468c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 --2- 172.21.15.19:0/3656961577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f6467fd00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 -- 172.21.15.19:0/3656961577 >> 172.21.15.19:0/3656961577 conn(0x559f642ca300 msgr2=0x559f646844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 -- 172.21.15.19:0/3656961577 shutdown_connections 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.921+0000 7f61ffe16740 1 -- 172.21.15.19:0/3656961577 wait complete. 2024-05-17T09:29:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 -- start start 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 Processor -- start 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f64689650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64689b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f6468aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64689b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41a8ed0 con 0x559f642b54d0 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41aa340 con 0x559f642ad2f0 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64689b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49008/0 (socket says 172.21.15.19:49008) 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41a95a0 con 0x559f642d61b0 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f1d9b640 1 -- 172.21.15.19:0/713572890 learned_addr learned my addr 172.21.15.19:0/713572890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f259c640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f64689650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61ffe16740 10 client.0 _ll_register_callbacks cb 0x7f61e80a9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f1d9b640 1 -- 172.21.15.19:0/713572890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 msgr2=0x559f6468aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f6468aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.922+0000 7f61f1d9b640 1 -- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 msgr2=0x559f64689650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f64689650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61f1d9b640 1 -- 172.21.15.19:0/713572890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6468b080 con 0x559f642b54d0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61f259c640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f64689650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61f1d9b640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64689b90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f61e80a9c60 tx=0x7f61e80094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50471ef619682e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61d9ffb640 1 -- 172.21.15.19:0/713572890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e80c1070 con 0x559f642b54d0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61d9ffb640 1 -- 172.21.15.19:0/713572890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f61e80a62c0 con 0x559f642b54d0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e4032c80 con 0x559f642b54d0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61d9ffb640 1 -- 172.21.15.19:0/713572890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f61e800ce60 con 0x559f642b54d0 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61ffe16740 10 client.17301 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f6468c380 con 0x559f642b54d0 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.923+0000 7f61d9ffb640 1 -- 172.21.15.19:0/713572890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e4032c80 con 0x559f642b54d0 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61d9ffb640 1 -- 172.21.15.19:0/713572890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f61e80af210 con 0x559f642b54d0 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61ffe16740 10 client.17301 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61ffe16740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61ffe16740 20 client.17301 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61ffe16740 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f6426a1c0 0x559f646fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61ffe16740 4 client.17301 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f645e5360 con 0x559f6426a1c0 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61f259c640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f6426a1c0 0x559f646fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61f259c640 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f6426a1c0 0x559f646fd440 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27814aa79493f6bd server_cookie=7ab628172f2d6369 in_seq=0 out_seq=0 2024-05-17T09:29:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.924+0000 7f61d9ffb640 10 client.17301 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61d9ffb640 1 -- 172.21.15.19:0/713572890 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559f645e5360 con 0x559f6426a1c0 2024-05-17T09:29:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61d9ffb640 10 client.17301 handle_command_reply: tid=0 2024-05-17T09:29:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 client.17301 shutdown 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 msgr2=0x559f64689b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64689b90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f61e80a9c60 tx=0x7f61e80094c0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 shutdown_connections 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f6426a1c0 0x559f646fd440 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f642d61b0 0x559f6468aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f642b54d0 0x559f64689b90 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 --2- 172.21.15.19:0/713572890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f642ad2f0 0x559f64689650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61d9ffb640 0 client.17301 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 >> 172.21.15.19:0/713572890 conn(0x559f642ca300 msgr2=0x559f64684210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 shutdown_connections 2024-05-17T09:29:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 1 -- 172.21.15.19:0/713572890 wait complete. 2024-05-17T09:29:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.945+0000 7f61ffe16740 20 client.17301 trim_cache size 0 max 16384 2024-05-17T09:29:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:29:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:29:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61dc0b28a0 msgr2=0x7f61dc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61dc0b28a0 0x7f61dc0b4db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f61e402a860 tx=0x7f61e402bf90 comp rx=0 tx=0).stop 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 msgr2=0x7f61f419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 0x7f61f419a730 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f61e800e9c0 tx=0x7f61e800ee80 comp rx=0 tx=0).stop 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 shutdown_connections 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f4109ba0 0x7f61f41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f61dc0b28a0 0x7f61dc0b4db0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61f4104ea0 0x7f61f419a730 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 --2- 172.21.15.19:0/1577476355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61f4103c60 0x7f61f419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 >> 172.21.15.19:0/1577476355 conn(0x7f61f40fee50 msgr2=0x7f61f4105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 shutdown_connections 2024-05-17T09:29:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:21.946+0000 7f61f8f66640 1 -- 172.21.15.19:0/1577476355 wait complete. 2024-05-17T09:29:22.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 139051 1.62 MB/sec execute 320 sec latency 21428.409 ms 2024-05-17T09:29:22.553 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:22 smithi019 ceph-mon[31620]: pgmap v726: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-17T09:29:22.588 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4fb28607', total seconds elapsed: 27.5 2024-05-17T09:29:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:22 smithi069 ceph-mon[35081]: pgmap v726: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-17T09:29:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:22 smithi079 ceph-mon[35502]: pgmap v726: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-17T09:29:23.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 139529 1.62 MB/sec execute 321 sec latency 12.457 ms 2024-05-17T09:29:24.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 140016 1.62 MB/sec execute 322 sec latency 15.938 ms 2024-05-17T09:29:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:24 smithi069 ceph-mon[35081]: pgmap v727: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-17T09:29:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:24 smithi019 ceph-mon[31620]: pgmap v727: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-17T09:29:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:24 smithi079 ceph-mon[35502]: pgmap v727: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-17T09:29:25.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 140675 1.62 MB/sec execute 323 sec latency 14.672 ms 2024-05-17T09:29:26.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 141241 1.63 MB/sec execute 324 sec latency 15.762 ms 2024-05-17T09:29:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:26 smithi069 ceph-mon[35081]: pgmap v728: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 73 KiB/s wr, 17 op/s 2024-05-17T09:29:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:26 smithi019 ceph-mon[31620]: pgmap v728: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 73 KiB/s wr, 17 op/s 2024-05-17T09:29:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:26 smithi079 ceph-mon[35502]: pgmap v728: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 73 KiB/s wr, 17 op/s 2024-05-17T09:29:27.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 141699 1.63 MB/sec execute 325 sec latency 16.579 ms 2024-05-17T09:29:28.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 142155 1.64 MB/sec execute 326 sec latency 16.034 ms 2024-05-17T09:29:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:28 smithi069 ceph-mon[35081]: pgmap v729: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 368 KiB/s wr, 43 op/s 2024-05-17T09:29:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:28 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:29:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:28 smithi019 ceph-mon[31620]: pgmap v729: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 368 KiB/s wr, 43 op/s 2024-05-17T09:29:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:28 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:29:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:28 smithi079 ceph-mon[35502]: pgmap v729: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 368 KiB/s wr, 43 op/s 2024-05-17T09:29:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:28 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:29:29.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 142386 1.64 MB/sec execute 327 sec latency 15.570 ms 2024-05-17T09:29:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:29 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:29 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:29 smithi079 ceph-mon[35502]: pgmap v730: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 611 KiB/s wr, 66 op/s 2024-05-17T09:29:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:29 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:29:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:29 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:29:29.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:29 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:29 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:29 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:29 smithi069 ceph-mon[35081]: pgmap v730: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 611 KiB/s wr, 66 op/s 2024-05-17T09:29:29.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:29 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:29:29.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:29 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:29:29.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:29 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:29 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:29 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:29.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:29 smithi019 ceph-mon[31620]: pgmap v730: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 611 KiB/s wr, 66 op/s 2024-05-17T09:29:29.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:29 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:29:29.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:29 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:29:29.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:29 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:29:30.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 142784 1.64 MB/sec execute 328 sec latency 16.965 ms 2024-05-17T09:29:31.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 143303 1.64 MB/sec execute 329 sec latency 18.900 ms 2024-05-17T09:29:32.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 143878 1.64 MB/sec execute 330 sec latency 16.507 ms 2024-05-17T09:29:32.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:32 smithi019 ceph-mon[31620]: pgmap v731: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 708 KiB/s wr, 91 op/s 2024-05-17T09:29:32.473 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:32 smithi079 ceph-mon[35502]: pgmap v731: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 708 KiB/s wr, 91 op/s 2024-05-17T09:29:32.632 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:32 smithi069 ceph-mon[35081]: pgmap v731: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 708 KiB/s wr, 91 op/s 2024-05-17T09:29:33.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 144382 1.64 MB/sec execute 331 sec latency 15.495 ms 2024-05-17T09:29:34.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 145112 1.65 MB/sec execute 332 sec latency 15.310 ms 2024-05-17T09:29:34.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:34 smithi019 ceph-mon[31620]: pgmap v732: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 881 KiB/s wr, 100 op/s 2024-05-17T09:29:34.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:34 smithi079 ceph-mon[35502]: pgmap v732: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 881 KiB/s wr, 100 op/s 2024-05-17T09:29:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:34 smithi069 ceph-mon[35081]: pgmap v732: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 881 KiB/s wr, 100 op/s 2024-05-17T09:29:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/736562907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc01099f0 msgr2=0x7fbcc010be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/736562907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc01099f0 0x7fbcc010be10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009ca0 tx=0x7fbcb802a8c0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/736562907 shutdown_connections 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/736562907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc01099f0 0x7fbcc010be10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/736562907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcc0107090 0x7fbcc01094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/736562907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc0106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/736562907 >> 172.21.15.19:0/736562907 conn(0x7fbcc00fdd30 msgr2=0x7fbcc01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/736562907 shutdown_connections 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.052+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/736562907 wait complete. 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 Processor -- start 2024-05-17T09:29:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 -- start start 2024-05-17T09:29:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 0x7fbcc019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcc01099f0 0x7fbcc01a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010f650 con 0x7fbcc0107090 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010f4d0 con 0x7fbcc01099f0 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc7e0c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010f7d0 con 0x7fbcc0104730 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 0x7fbcc019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55518/0 (socket says 172.21.15.19:55518) 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/2468392126 learned_addr learned my addr 172.21.15.19:0/2468392126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 0x7fbcc019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36502/0 (socket says 172.21.15.19:36502) 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcc01099f0 0x7fbcc01a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/2468392126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcc01099f0 msgr2=0x7fbcc01a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcc01099f0 0x7fbcc01a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 msgr2=0x7fbcc019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 0x7fbcc019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/2468392126 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc01a1cb0 con 0x7fbcc0104730 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.053+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 0x7fbcc019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.054+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc019a170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9c0 tx=0x7fbcbc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.054+0000 7fbcb6ffd640 1 -- 172.21.15.19:0/2468392126 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00cd80 con 0x7fbcc0104730 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.054+0000 7fbcb6ffd640 1 -- 172.21.15.19:0/2468392126 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcbc0054b0 con 0x7fbcc0104730 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.054+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc01a1fa0 con 0x7fbcc0104730 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.054+0000 7fbcb6ffd640 1 -- 172.21.15.19:0/2468392126 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc010430 con 0x7fbcc0104730 2024-05-17T09:29:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.054+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a2400 con 0x7fbcc0104730 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.057+0000 7fbcced99740 1 Processor -- start 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 -- start start 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f381b0 0x561a292d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f0f2f0 0x561a292d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a2924bc30 con 0x561a28f381b0 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561a2923b7d0 con 0x561a28f0f2f0 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcced99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561a2923f810 con 0x561a28f174d0 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcb6ffd640 1 -- 172.21.15.19:0/2468392126 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbcbc004f20 con 0x7fbcc0104730 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55526/0 (socket says 172.21.15.19:55526) 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcc53a7640 1 -- 172.21.15.19:0/700719446 learned_addr learned my addr 172.21.15.19:0/700719446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f0f2f0 0x561a292d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.058+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f381b0 0x561a292d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc53a7640 1 -- 172.21.15.19:0/700719446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f0f2f0 msgr2=0x561a292d9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f0f2f0 0x561a292d9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc53a7640 1 -- 172.21.15.19:0/700719446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f381b0 msgr2=0x561a292d93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f381b0 0x561a292d93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc53a7640 1 -- 172.21.15.19:0/700719446 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a28f50210 con 0x561a28f174d0 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e0820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00114f0 tx=0x7fbcb00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff942961b204f4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcabfff640 1 -- 172.21.15.19:0/700719446 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0005020 con 0x561a28f174d0 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcabfff640 1 -- 172.21.15.19:0/700719446 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcb00047a0 con 0x561a28f174d0 2024-05-17T09:29:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcabfff640 1 -- 172.21.15.19:0/700719446 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000f820 con 0x561a28f174d0 2024-05-17T09:29:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcb6ffd640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcac0b6cb0 0x7fbcac0b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcac0b6cb0 0x7fbcac0b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.059+0000 7fbcb6ffd640 1 -- 172.21.15.19:0/2468392126 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbcbc0e2bf0 con 0x7fbcc0104730 2024-05-17T09:29:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 -- 172.21.15.19:0/700719446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 msgr2=0x561a292e0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcac0b6cb0 0x7fbcac0b91c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb8031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e0820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00114f0 tx=0x7fbcb00119b0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 -- 172.21.15.19:0/700719446 shutdown_connections 2024-05-17T09:29:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e0820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f0f2f0 0x561a292d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 --2- 172.21.15.19:0/700719446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f381b0 0x561a292d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 -- 172.21.15.19:0/700719446 >> 172.21.15.19:0/700719446 conn(0x561a28f2c300 msgr2=0x561a292d4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 -- 172.21.15.19:0/700719446 shutdown_connections 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.060+0000 7fbcced99740 1 -- 172.21.15.19:0/700719446 wait complete. 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 -- start start 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 Processor -- start 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 0x561a292e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f381b0 0x561a292ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a28f21790 con 0x561a28f0f2f0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561a291f7890 con 0x561a28f381b0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.061+0000 7fbcced99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561a292e1270 con 0x561a28f174d0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcced99740 10 client.0 _ll_register_callbacks cb 0x561a291fad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 0x561a292e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 0x561a292e2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36514/0 (socket says 172.21.15.19:36514) 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/396598738 learned_addr learned my addr 172.21.15.19:0/396598738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f381b0 0x561a292ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/396598738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 msgr2=0x561a292e31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f381b0 msgr2=0x561a292ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f381b0 0x561a292ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 -- 172.21.15.19:0/396598738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2901c620 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 0x561a292e2cb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0a5e80 tx=0x7fbcbc0a9c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49dfd45ee48be48 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbca97fa640 1 -- 172.21.15.19:0/396598738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0aa500 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbca97fa640 1 -- 172.21.15.19:0/396598738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbcbc0a63f0 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.062+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb00047a0 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbca97fa640 1 -- 172.21.15.19:0/396598738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc0c9050 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 10 client.17313 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a292ed0f0 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbca97fa640 1 -- 172.21.15.19:0/396598738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcb00047a0 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbca97fa640 1 -- 172.21.15.19:0/396598738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbcbc0a5770 con 0x561a28f0f2f0 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 10 client.17313 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 20 client.17313 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28ef6d80 0x561a292f0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 4 client.17313 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2924db80 con 0x561a28ef6d80 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.063+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28ef6d80 0x561a292f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.064+0000 7fbca97fa640 10 client.17313 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.064+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28ef6d80 0x561a292f0860 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996594afd27573c4 server_cookie=2ea82a3f0a3b265e in_seq=0 out_seq=0 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.064+0000 7fbca97fa640 1 -- 172.21.15.19:0/396598738 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561a2924db80 con 0x561a28ef6d80 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.064+0000 7fbca97fa640 10 client.17313 handle_command_reply: tid=0 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.064+0000 7fbcced99740 1 client.17313 shutdown 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 msgr2=0x561a292e2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 0x561a292e2cb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0a5e80 tx=0x7fbcbc0a9c90 comp rx=0 tx=0).stop 2024-05-17T09:29:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 shutdown_connections 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28ef6d80 0x561a292f0860 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28f381b0 0x561a292ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f174d0 0x561a292e31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbca97fa640 0 client.17313 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 --2- 172.21.15.19:0/396598738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f0f2f0 0x561a292e2cb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 >> 172.21.15.19:0/396598738 conn(0x561a28f2c300 msgr2=0x561a292d3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 shutdown_connections 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 1 -- 172.21.15.19:0/396598738 wait complete. 2024-05-17T09:29:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.065+0000 7fbcced99740 20 client.17313 trim_cache size 0 max 16384 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.082+0000 7fbcced99740 1 Processor -- start 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.082+0000 7fbcced99740 1 -- start start 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcced99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcced99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcced99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcced99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01ad060 con 0x561a28f174d0 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcced99740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01aa930 con 0x561a28ef6d80 2024-05-17T09:29:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcced99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01a8780 con 0x561a28f0f2f0 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292ea0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55546/0 (socket says 172.21.15.19:55546) 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292d3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:38016/0 (socket says 172.21.15.19:38016) 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc53a7640 1 -- 172.21.15.19:0/296549976 learned_addr learned my addr 172.21.15.19:0/296549976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 -- 172.21.15.19:0/296549976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 msgr2=0x561a292ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 -- 172.21.15.19:0/296549976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 msgr2=0x561a292d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292d3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 -- 172.21.15.19:0/296549976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a292ee5b0 con 0x561a28f174d0 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.083+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292dc410 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbcb802aeb0 tx=0x7fbcb8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f077373c03bd7423 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.084+0000 7fbcab7fe640 1 -- 172.21.15.19:0/296549976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8003b70 con 0x561a28f174d0 2024-05-17T09:29:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.084+0000 7fbcab7fe640 1 -- 172.21.15.19:0/296549976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcb8036050 con 0x561a28f174d0 2024-05-17T09:29:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.084+0000 7fbcab7fe640 1 -- 172.21.15.19:0/296549976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8030820 con 0x561a28f174d0 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 -- 172.21.15.19:0/296549976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 msgr2=0x561a292dc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292dc410 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbcb802aeb0 tx=0x7fbcb8044000 comp rx=0 tx=0).stop 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 -- 172.21.15.19:0/296549976 shutdown_connections 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292dc410 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 --2- 172.21.15.19:0/296549976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 -- 172.21.15.19:0/296549976 >> 172.21.15.19:0/296549976 conn(0x561a28f2c300 msgr2=0x561a292d3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 -- 172.21.15.19:0/296549976 shutdown_connections 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 -- 172.21.15.19:0/296549976 wait complete. 2024-05-17T09:29:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 -- start start 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.085+0000 7fbcced99740 1 Processor -- start 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01a8df0 con 0x561a28f174d0 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01aa260 con 0x561a28ef6d80 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01a94c0 con 0x561a28f0f2f0 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcced99740 10 client.0 _ll_register_callbacks cb 0x7fbcbc0a5cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc63a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc63a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292e8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36524/0 (socket says 172.21.15.19:36524) 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc5ba8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:38022/0 (socket says 172.21.15.19:38022) 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc63a9640 1 -- 172.21.15.19:0/2876640646 learned_addr learned my addr 172.21.15.19:0/2876640646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 -- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 msgr2=0x561a292e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 -- 172.21.15.19:0/2876640646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 msgr2=0x561a292e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 -- 172.21.15.19:0/2876640646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a292e9140 con 0x561a28f0f2f0 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc63a9640 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbcc53a7640 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292e7c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0014040 tx=0x7fbcb0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faf1a52c3188455 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.086+0000 7fbca97fa640 1 -- 172.21.15.19:0/2876640646 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0035060 con 0x561a28f0f2f0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb8036050 con 0x561a28f0f2f0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbca97fa640 1 -- 172.21.15.19:0/2876640646 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbcb0016040 con 0x561a28f0f2f0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbca97fa640 1 -- 172.21.15.19:0/2876640646 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcb003c050 con 0x561a28f0f2f0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 10 client.26920 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a292ea440 con 0x561a28f0f2f0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbca97fa640 1 -- 172.21.15.19:0/2876640646 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcb8036050 con 0x561a28f0f2f0 2024-05-17T09:29:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbca97fa640 1 -- 172.21.15.19:0/2876640646 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbcb0004040 con 0x561a28f0f2f0 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 10 client.26920 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 20 client.26920 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28f381b0 0x561a2935b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 4 client.26920 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a29247360 con 0x561a28f381b0 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.087+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28f381b0 0x561a2935b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.088+0000 7fbca97fa640 10 client.26920 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.088+0000 7fbcc5ba8640 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28f381b0 0x561a2935b5c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e240ddbc98af32 server_cookie=9fefe358fcb0b253 in_seq=0 out_seq=0 2024-05-17T09:29:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.107+0000 7fbca97fa640 1 -- 172.21.15.19:0/2876640646 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a29247360 con 0x561a28f381b0 2024-05-17T09:29:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.107+0000 7fbca97fa640 10 client.26920 handle_command_reply: tid=0 2024-05-17T09:29:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.107+0000 7fbcced99740 1 client.26920 shutdown 2024-05-17T09:29:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.107+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 msgr2=0x561a292e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.107+0000 7fbcced99740 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292e7c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0014040 tx=0x7fbcb0003820 comp rx=0 tx=0).stop 2024-05-17T09:29:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 shutdown_connections 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561a28f381b0 0x561a2935b5c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a28f174d0 0x561a292e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a28f0f2f0 0x561a292e7c50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbca97fa640 0 client.26920 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 --2- 172.21.15.19:0/2876640646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a28ef6d80 0x561a292e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 >> 172.21.15.19:0/2876640646 conn(0x561a28f2c300 msgr2=0x561a292e3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 shutdown_connections 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 1 -- 172.21.15.19:0/2876640646 wait complete. 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcced99740 20 client.26920 trim_cache size 0 max 16384 2024-05-17T09:29:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:29:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:29:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:29:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcac0b6cb0 msgr2=0x7fbcac0b91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcac0b6cb0 0x7fbcac0b91c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb8031000 comp rx=0 tx=0).stop 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 msgr2=0x7fbcc019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc019a170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9c0 tx=0x7fbcbc00ee80 comp rx=0 tx=0).stop 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 shutdown_connections 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcc01099f0 0x7fbcc01a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbcac0b6cb0 0x7fbcac0b91c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc0107090 0x7fbcc019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 --2- 172.21.15.19:0/2468392126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcc0104730 0x7fbcc019a170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.108+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 >> 172.21.15.19:0/2468392126 conn(0x7fbcc00fdd30 msgr2=0x7fbcc0105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.109+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 shutdown_connections 2024-05-17T09:29:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:35.109+0000 7fbcc7e0c640 1 -- 172.21.15.19:0/2468392126 wait complete. 2024-05-17T09:29:35.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 145488 1.66 MB/sec execute 333 sec latency 15.641 ms 2024-05-17T09:29:36.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 145821 1.66 MB/sec execute 334 sec latency 12.824 ms 2024-05-17T09:29:36.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:36 smithi019 ceph-mon[31620]: pgmap v733: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-17T09:29:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:36 smithi079 ceph-mon[35502]: pgmap v733: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-17T09:29:36.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:36 smithi069 ceph-mon[35081]: pgmap v733: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-17T09:29:37.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 146059 1.66 MB/sec execute 335 sec latency 15.926 ms 2024-05-17T09:29:38.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 146502 1.66 MB/sec execute 336 sec latency 65.030 ms 2024-05-17T09:29:38.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:38 smithi079 ceph-mon[35502]: pgmap v734: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:29:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:38 smithi069 ceph-mon[35081]: pgmap v734: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:29:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:38 smithi019 ceph-mon[31620]: pgmap v734: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-17T09:29:39.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 146989 1.66 MB/sec execute 337 sec latency 52.458 ms 2024-05-17T09:29:40.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 147602 1.66 MB/sec execute 338 sec latency 18.726 ms 2024-05-17T09:29:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:40 smithi079 ceph-mon[35502]: pgmap v735: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:29:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:40 smithi069 ceph-mon[35081]: pgmap v735: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:29:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:40 smithi019 ceph-mon[31620]: pgmap v735: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:29:41.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 148057 1.66 MB/sec execute 339 sec latency 15.839 ms 2024-05-17T09:29:42.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 148755 1.67 MB/sec execute 340 sec latency 15.977 ms 2024-05-17T09:29:42.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:42 smithi079 ceph-mon[35502]: pgmap v736: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-05-17T09:29:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:42 smithi069 ceph-mon[35081]: pgmap v736: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-05-17T09:29:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:42 smithi019 ceph-mon[31620]: pgmap v736: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-05-17T09:29:43.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 149148 1.68 MB/sec execute 341 sec latency 18.974 ms 2024-05-17T09:29:44.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 149412 1.68 MB/sec execute 342 sec latency 18.876 ms 2024-05-17T09:29:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:44 smithi079 ceph-mon[35502]: pgmap v737: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:29:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:44 smithi069 ceph-mon[35081]: pgmap v737: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:29:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:44 smithi019 ceph-mon[31620]: pgmap v737: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-17T09:29:45.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 149689 1.67 MB/sec execute 343 sec latency 15.096 ms 2024-05-17T09:29:46.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 150097 1.67 MB/sec execute 344 sec latency 125.818 ms 2024-05-17T09:29:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:46 smithi079 ceph-mon[35502]: pgmap v738: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:29:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:46 smithi069 ceph-mon[35081]: pgmap v738: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:29:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:46 smithi019 ceph-mon[31620]: pgmap v738: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:29:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 150625 1.68 MB/sec execute 345 sec latency 20.313 ms 2024-05-17T09:29:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 -- 172.21.15.19:0/1715976768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58109ba0 msgr2=0x7fea5810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 --2- 172.21.15.19:0/1715976768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58109ba0 0x7fea5810bfc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fea50009ca0 tx=0x7fea5002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 -- 172.21.15.19:0/1715976768 shutdown_connections 2024-05-17T09:29:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 --2- 172.21.15.19:0/1715976768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58109ba0 0x7fea5810bfc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 --2- 172.21.15.19:0/1715976768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58104ea0 0x7fea581052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 --2- 172.21.15.19:0/1715976768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea58104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 -- 172.21.15.19:0/1715976768 >> 172.21.15.19:0/1715976768 conn(0x7fea580fee50 msgr2=0x7fea58101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 -- 172.21.15.19:0/1715976768 shutdown_connections 2024-05-17T09:29:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.221+0000 7fea5e613640 1 -- 172.21.15.19:0/1715976768 wait complete. 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 Processor -- start 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 -- start start 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea581a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58104ea0 0x7fea581a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58109ba0 0x7fea581a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea580769e0 con 0x7fea58104ea0 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea58076860 con 0x7fea58103c60 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea5e613640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea58076b60 con 0x7fea58109ba0 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.222+0000 7fea57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea581a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea581a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:41060/0 (socket says 172.21.15.19:41060) 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 -- 172.21.15.19:0/3000068627 learned_addr learned my addr 172.21.15.19:0/3000068627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea577fe640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58104ea0 0x7fea581a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58109ba0 0x7fea581a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 -- 172.21.15.19:0/3000068627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58109ba0 msgr2=0x7fea581a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58109ba0 0x7fea581a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 -- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58104ea0 msgr2=0x7fea581a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58104ea0 0x7fea581a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 -- 172.21.15.19:0/3000068627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea581a82a0 con 0x7fea58103c60 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.223+0000 7fea57fff640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea581a0760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e9f0 tx=0x7fea4800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.224+0000 7fea557fa640 1 -- 172.21.15.19:0/3000068627 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4800ccc0 con 0x7fea58103c60 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.224+0000 7fea557fa640 1 -- 172.21.15.19:0/3000068627 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea480044b0 con 0x7fea58103c60 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.224+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581a8590 con 0x7fea58103c60 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.224+0000 7fea557fa640 1 -- 172.21.15.19:0/3000068627 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48010430 con 0x7fea58103c60 2024-05-17T09:29:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.224+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a8b30 con 0x7fea58103c60 2024-05-17T09:29:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.228+0000 7fea6559d740 1 Processor -- start 2024-05-17T09:29:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.228+0000 7fea557fa640 1 -- 172.21.15.19:0/3000068627 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea48004020 con 0x7fea58103c60 2024-05-17T09:29:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.228+0000 7fea6559d740 1 -- start start 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea6559d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41531f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea6559d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41531f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea6559d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415326820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea6559d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c415291c30 con 0x55c414f5d4d0 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea6559d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c4152817d0 con 0x55c414f7e1b0 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea6559d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c415285810 con 0x55c414f552f0 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea57fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41531f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea5cbb0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41531f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea57fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41531f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37022/0 (socket says 172.21.15.19:37022) 2024-05-17T09:29:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea57fff640 1 -- 172.21.15.19:0/2175618421 learned_addr learned my addr 172.21.15.19:0/2175618421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea577fe640 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415326820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea5cbb0640 1 -- 172.21.15.19:0/2175618421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 msgr2=0x55c41531f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.229+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41531f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea5cbb0640 1 -- 172.21.15.19:0/2175618421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 msgr2=0x55c415326820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415326820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea5cbb0640 1 -- 172.21.15.19:0/2175618421 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c414f96210 con 0x55c414f7e1b0 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea577fe640 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415326820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea557fa640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea400b2920 0x7fea400b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41531f3a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea50031920 tx=0x7fea50009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbe13e99126060c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.230+0000 7fea557fa640 1 -- 172.21.15.19:0/3000068627 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fea480e28d0 con 0x7fea58103c60 2024-05-17T09:29:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.231+0000 7fea57fff640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea400b2920 0x7fea400b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.231+0000 7fea3e7fc640 1 -- 172.21.15.19:0/2175618421 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50031a80 con 0x55c414f7e1b0 2024-05-17T09:29:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.231+0000 7fea3e7fc640 1 -- 172.21.15.19:0/2175618421 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea50036030 con 0x55c414f7e1b0 2024-05-17T09:29:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.231+0000 7fea3e7fc640 1 -- 172.21.15.19:0/2175618421 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50004330 con 0x55c414f7e1b0 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 -- 172.21.15.19:0/2175618421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 msgr2=0x55c41531f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41531f3a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea50031920 tx=0x7fea50009800 comp rx=0 tx=0).stop 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 -- 172.21.15.19:0/2175618421 shutdown_connections 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415326820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41531f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 --2- 172.21.15.19:0/2175618421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41531f3a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.232+0000 7fea6559d740 1 -- 172.21.15.19:0/2175618421 >> 172.21.15.19:0/2175618421 conn(0x55c414f72300 msgr2=0x55c41531ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.233+0000 7fea57fff640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea400b2920 0x7fea400b4e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea480a9a70 tx=0x7fea480e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.233+0000 7fea6559d740 1 -- 172.21.15.19:0/2175618421 shutdown_connections 2024-05-17T09:29:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.233+0000 7fea6559d740 1 -- 172.21.15.19:0/2175618421 wait complete. 2024-05-17T09:29:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.233+0000 7fea6559d740 1 -- start start 2024-05-17T09:29:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 Processor -- start 2024-05-17T09:29:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c415328b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415329070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea5cbb0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c415328b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea5cbb0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c415328b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37034/0 (socket says 172.21.15.19:37034) 2024-05-17T09:29:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea5cbb0640 1 -- 172.21.15.19:0/474366284 learned_addr learned my addr 172.21.15.19:0/474366284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c414f67790 con 0x55c414f5d4d0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea577fe640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415329070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c41523d890 con 0x55c414f7e1b0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c415327090 con 0x55c414f552f0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.234+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea6559d740 10 client.0 _ll_register_callbacks cb 0x55c415240d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea577fe640 1 -- 172.21.15.19:0/474366284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 msgr2=0x55c415328b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea577fe640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c415328b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea577fe640 1 -- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 msgr2=0x55c41532ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea577fe640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea577fe640 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c415062620 con 0x55c414f5d4d0 2024-05-17T09:29:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c415328b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:29:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea577fe640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415329070 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea581a1800 tx=0x7fea4c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25af2d418907803 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea377fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea377fe640 1 -- 172.21.15.19:0/474366284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004020 con 0x55c414f5d4d0 2024-05-17T09:29:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea377fe640 1 -- 172.21.15.19:0/474366284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fea4c0041c0 con 0x55c414f5d4d0 2024-05-17T09:29:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50036030 con 0x55c414f5d4d0 2024-05-17T09:29:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.235+0000 7fea377fe640 1 -- 172.21.15.19:0/474366284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea4c025050 con 0x55c414f5d4d0 2024-05-17T09:29:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 10 client.17331 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c415332f70 con 0x55c414f5d4d0 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea377fe640 1 -- 172.21.15.19:0/474366284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea50036030 con 0x55c414f5d4d0 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea377fe640 1 -- 172.21.15.19:0/474366284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fea4c003820 con 0x55c414f5d4d0 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 10 client.17331 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 20 client.17331 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f3cd80 0x55c415336bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 4 client.17331 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c415293b80 con 0x55c414f3cd80 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.236+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f3cd80 0x55c415336bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.237+0000 7fea377fe640 10 client.17331 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.237+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f3cd80 0x55c415336bb0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462cd401b9735b83 server_cookie=6f12cadc393ef1c5 in_seq=0 out_seq=0 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea377fe640 1 -- 172.21.15.19:0/474366284 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c415293b80 con 0x55c414f3cd80 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea377fe640 10 client.17331 handle_command_reply: tid=0 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 client.17331 shutdown 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 msgr2=0x55c415329070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415329070 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea581a1800 tx=0x7fea4c012b60 comp rx=0 tx=0).stop 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 shutdown_connections 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f3cd80 0x55c415336bb0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415329070 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 --2- 172.21.15.19:0/474366284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c415328b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 >> 172.21.15.19:0/474366284 conn(0x55c414f72300 msgr2=0x55c415319d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea377fe640 0 client.17331 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 shutdown_connections 2024-05-17T09:29:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 1 -- 172.21.15.19:0/474366284 wait complete. 2024-05-17T09:29:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.238+0000 7fea6559d740 20 client.17331 trim_cache size 0 max 16384 2024-05-17T09:29:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 Processor -- start 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 -- start start 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 0x55c415319a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f5d4d0 0x55c41532fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f7e1b0 0x55c415330430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b5870 con 0x55c414f552f0 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b3140 con 0x55c414f5d4d0 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea6559d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b0f90 con 0x55c414f7e1b0 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 0x55c415319a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 0x55c415319a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33666/0 (socket says 172.21.15.19:33666) 2024-05-17T09:29:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 -- 172.21.15.19:0/1251171262 learned_addr learned my addr 172.21.15.19:0/1251171262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea577fe640 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f5d4d0 0x55c41532fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 -- 172.21.15.19:0/1251171262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f7e1b0 msgr2=0x55c415330430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f7e1b0 0x55c415330430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 -- 172.21.15.19:0/1251171262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f5d4d0 msgr2=0x55c41532fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f5d4d0 0x55c41532fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 -- 172.21.15.19:0/1251171262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c415329c70 con 0x55c414f552f0 2024-05-17T09:29:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.257+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 0x55c415319a40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fea5002aeb0 tx=0x7fea50009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b3ab160a7d7963 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.258+0000 7fea3dffb640 1 -- 172.21.15.19:0/1251171262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50058070 con 0x55c414f552f0 2024-05-17T09:29:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.258+0000 7fea3dffb640 1 -- 172.21.15.19:0/1251171262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea50036040 con 0x55c414f552f0 2024-05-17T09:29:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.258+0000 7fea3dffb640 1 -- 172.21.15.19:0/1251171262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea500043c0 con 0x55c414f552f0 2024-05-17T09:29:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 -- 172.21.15.19:0/1251171262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 msgr2=0x55c415319a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 0x55c415319a40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fea5002aeb0 tx=0x7fea50009640 comp rx=0 tx=0).stop 2024-05-17T09:29:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 -- 172.21.15.19:0/1251171262 shutdown_connections 2024-05-17T09:29:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f7e1b0 0x55c415330430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f5d4d0 0x55c41532fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 --2- 172.21.15.19:0/1251171262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f552f0 0x55c415319a40 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 -- 172.21.15.19:0/1251171262 >> 172.21.15.19:0/1251171262 conn(0x55c414f72300 msgr2=0x55c415319330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 -- 172.21.15.19:0/1251171262 shutdown_connections 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 -- 172.21.15.19:0/1251171262 wait complete. 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 -- start start 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.259+0000 7fea6559d740 1 Processor -- start 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41532fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415330330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b1600 con 0x55c414f5d4d0 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b2a70 con 0x55c414f7e1b0 2024-05-17T09:29:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea581b1cd0 con 0x55c414f552f0 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea6559d740 10 client.0 _ll_register_callbacks cb 0x7fea4c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415330330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415330330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33670/0 (socket says 172.21.15.19:33670) 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 -- 172.21.15.19:0/1550069049 learned_addr learned my addr 172.21.15.19:0/1550069049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41532fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 -- 172.21.15.19:0/1550069049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 msgr2=0x55c41532fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41532fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 -- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 msgr2=0x55c41532c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 -- 172.21.15.19:0/1550069049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41532d130 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea577fe640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415330330 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fea4c005eb0 tx=0x7fea4c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8353eea56963bcf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea377fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.260+0000 7fea377fe640 1 -- 172.21.15.19:0/1550069049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c003e60 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea377fe640 1 -- 172.21.15.19:0/1550069049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fea4c005280 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50036040 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea377fe640 1 -- 172.21.15.19:0/1550069049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea4c036050 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 10 client.17337 fetch_fsmap learned FSMap version 32 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c41539dfb0 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea377fe640 1 -- 172.21.15.19:0/1550069049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea50036040 con 0x55c414f5d4d0 2024-05-17T09:29:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea377fe640 1 -- 172.21.15.19:0/1550069049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fea4c01c470 con 0x55c414f5d4d0 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 10 client.17337 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 20 client.17337 populate_metadata read hostname 'smithi019' 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f121c0 0x55c4153a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 4 client.17337 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.261+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c41528d360 con 0x55c414f121c0 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.262+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f121c0 0x55c4153a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.262+0000 7fea377fe640 10 client.17337 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.262+0000 7fea5cbb0640 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f121c0 0x55c4153a09c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a79df9464a5f81 server_cookie=90366f990ad9ecb2 in_seq=0 out_seq=0 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea377fe640 1 -- 172.21.15.19:0/1550069049 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c41528d360 con 0x55c414f121c0 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea377fe640 10 client.17337 handle_command_reply: tid=0 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 client.17337 shutdown 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 msgr2=0x55c415330330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415330330 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fea4c005eb0 tx=0x7fea4c003820 comp rx=0 tx=0).stop 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 shutdown_connections 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c414f121c0 0x55c4153a09c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c414f7e1b0 0x55c41532c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c414f5d4d0 0x55c415330330 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 --2- 172.21.15.19:0/1550069049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c414f552f0 0x55c41532fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 >> 172.21.15.19:0/1550069049 conn(0x55c414f72300 msgr2=0x55c415319330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea377fe640 0 client.17337 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 shutdown_connections 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 1 -- 172.21.15.19:0/1550069049 wait complete. 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.287+0000 7fea6559d740 20 client.17337 trim_cache size 0 max 16384 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:29:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:29:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea400b2920 msgr2=0x7fea400b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea400b2920 0x7fea400b4e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea480a9a70 tx=0x7fea480e8530 comp rx=0 tx=0).stop 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 msgr2=0x7fea581a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea581a0760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e9f0 tx=0x7fea4800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 shutdown_connections 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58109ba0 0x7fea581a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fea400b2920 0x7fea400b4e30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58104ea0 0x7fea581a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 --2- 172.21.15.19:0/3000068627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea58103c60 0x7fea581a0760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 >> 172.21.15.19:0/3000068627 conn(0x7fea580fee50 msgr2=0x7fea58105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 shutdown_connections 2024-05-17T09:29:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:29:48.288+0000 7fea5e613640 1 -- 172.21.15.19:0/3000068627 wait complete. 2024-05-17T09:29:48.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 151294 1.68 MB/sec execute 346 sec latency 22.894 ms 2024-05-17T09:29:48.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:48 smithi079 ceph-mon[35502]: pgmap v739: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:29:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:48 smithi069 ceph-mon[35081]: pgmap v739: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:29:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:48 smithi019 ceph-mon[31620]: pgmap v739: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:29:49.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 151622 1.68 MB/sec execute 347 sec latency 56.265 ms 2024-05-17T09:29:50.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 152312 1.69 MB/sec execute 348 sec latency 16.416 ms 2024-05-17T09:29:50.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:50 smithi079 ceph-mon[35502]: pgmap v740: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-05-17T09:29:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:50 smithi069 ceph-mon[35081]: pgmap v740: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-05-17T09:29:50.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:50 smithi019 ceph-mon[31620]: pgmap v740: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-05-17T09:29:51.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 152759 1.70 MB/sec execute 349 sec latency 14.896 ms 2024-05-17T09:29:52.441 INFO:tasks.workunit.client.0.smithi019.stdout: 1 152974 1.69 MB/sec execute 350 sec latency 15.642 ms 2024-05-17T09:29:52.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:52 smithi019 ceph-mon[31620]: pgmap v741: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-05-17T09:29:52.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:52 smithi079 ceph-mon[35502]: pgmap v741: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-05-17T09:29:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:52 smithi069 ceph-mon[35081]: pgmap v741: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-05-17T09:29:53.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 153295 1.69 MB/sec execute 351 sec latency 13.863 ms 2024-05-17T09:29:54.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 153771 1.69 MB/sec execute 352 sec latency 13.690 ms 2024-05-17T09:29:54.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:54 smithi079 ceph-mon[35502]: pgmap v742: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-17T09:29:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:54 smithi069 ceph-mon[35081]: pgmap v742: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-17T09:29:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:54 smithi019 ceph-mon[31620]: pgmap v742: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-17T09:29:55.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 154357 1.70 MB/sec execute 353 sec latency 19.795 ms 2024-05-17T09:29:56.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 154915 1.70 MB/sec execute 354 sec latency 15.660 ms 2024-05-17T09:29:56.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:56 smithi079 ceph-mon[35502]: pgmap v743: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:29:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:56 smithi069 ceph-mon[35081]: pgmap v743: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:29:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:56 smithi019 ceph-mon[31620]: pgmap v743: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-17T09:29:57.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 155464 1.71 MB/sec execute 355 sec latency 16.469 ms 2024-05-17T09:29:58.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 155884 1.71 MB/sec execute 356 sec latency 16.739 ms 2024-05-17T09:29:58.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:29:58 smithi079 ceph-mon[35502]: pgmap v744: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:29:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:29:58 smithi069 ceph-mon[35081]: pgmap v744: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:29:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:29:58 smithi019 ceph-mon[31620]: pgmap v744: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:29:59.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 156341 1.71 MB/sec execute 357 sec latency 14.532 ms 2024-05-17T09:30:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:29:59.999+0000 7f49787cc700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:30:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:29:59.999+0000 7f49787cc700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-17T09:30:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:29:59.999+0000 7f49787cc700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2024-05-17T09:30:00.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-mon[31620]: pgmap v745: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:30:00.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-mon[31620]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:30:00.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-mon[31620]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-17T09:30:00.443 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:00 smithi019 ceph-mon[31620]: mds.h(mds.2): Metadata damage detected 2024-05-17T09:30:00.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 156559 1.71 MB/sec execute 358 sec latency 17.648 ms 2024-05-17T09:30:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:00 smithi079 ceph-mon[35502]: pgmap v745: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:30:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:00 smithi079 ceph-mon[35502]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:30:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:00 smithi079 ceph-mon[35502]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-17T09:30:00.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:00 smithi079 ceph-mon[35502]: mds.h(mds.2): Metadata damage detected 2024-05-17T09:30:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:00 smithi069 ceph-mon[35081]: pgmap v745: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:30:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:00 smithi069 ceph-mon[35081]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:30:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:00 smithi069 ceph-mon[35081]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-17T09:30:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:00 smithi069 ceph-mon[35081]: mds.h(mds.2): Metadata damage detected 2024-05-17T09:30:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 -- 172.21.15.19:0/1401661182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c102b60 msgr2=0x7f404c103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 --2- 172.21.15.19:0/1401661182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c102b60 0x7f404c103860 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f403c009900 tx=0x7f403c02a410 comp rx=0 tx=0).stop 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 -- 172.21.15.19:0/1401661182 shutdown_connections 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 --2- 172.21.15.19:0/1401661182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c108aa0 0x7f404c10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 --2- 172.21.15.19:0/1401661182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 0x7f404c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 --2- 172.21.15.19:0/1401661182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c102b60 0x7f404c103860 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 -- 172.21.15.19:0/1401661182 >> 172.21.15.19:0/1401661182 conn(0x7f404c0fdd50 msgr2=0x7f404c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 -- 172.21.15.19:0/1401661182 shutdown_connections 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 -- 172.21.15.19:0/1401661182 wait complete. 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.400+0000 7f40524fa640 1 Processor -- start 2024-05-17T09:30:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 -- start start 2024-05-17T09:30:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 0x7f404c06bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 0x7f404c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 0x7f404c1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10e510 con 0x7f404c108aa0 2024-05-17T09:30:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10e390 con 0x7f404c102b60 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f40524fa640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10e690 con 0x7f404c103da0 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f404bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 0x7f404c06bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f4050a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 0x7f404c1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f4050a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 0x7f404c1a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42202/0 (socket says 172.21.15.19:42202) 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f404bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 0x7f404c06bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52904/0 (socket says 172.21.15.19:52904) 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f404b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 0x7f404c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.401+0000 7f4050a97640 1 -- 172.21.15.19:0/1646186107 learned_addr learned my addr 172.21.15.19:0/1646186107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f4050a97640 1 -- 172.21.15.19:0/1646186107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 msgr2=0x7f404c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f4050a97640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 0x7f404c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f4050a97640 1 -- 172.21.15.19:0/1646186107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 msgr2=0x7f404c06bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f4050a97640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 0x7f404c06bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f4050a97640 1 -- 172.21.15.19:0/1646186107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c1a1d50 con 0x7f404c108aa0 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f404bfff640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 0x7f404c06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 0x7f404c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f4050a97640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 0x7f404c1a1530 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f404400eb10 tx=0x7f404400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f40497fa640 1 -- 172.21.15.19:0/1646186107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044004b20 con 0x7f404c108aa0 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f40497fa640 1 -- 172.21.15.19:0/1646186107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4044004cc0 con 0x7f404c108aa0 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1a1fe0 con 0x7f404c108aa0 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f40497fa640 1 -- 172.21.15.19:0/1646186107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044010690 con 0x7f404c108aa0 2024-05-17T09:30:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.402+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a24a0 con 0x7f404c108aa0 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 Processor -- start 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40497fa640 1 -- 172.21.15.19:0/1646186107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4044005020 con 0x7f404c108aa0 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 -- start start 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48fe57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48fe5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e48f53c30 con 0x559e48c1f4d0 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559e48f437d0 con 0x559e48c401b0 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f40593a8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e48f47810 con 0x559e48c172f0 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f404bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48fe5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f404bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48fe5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37960/0 (socket says 172.21.15.19:37960) 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f404bfff640 1 -- 172.21.15.19:0/1948752946 learned_addr learned my addr 172.21.15.19:0/1948752946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.405+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48fe57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404bfff640 1 -- 172.21.15.19:0/1948752946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 msgr2=0x559e48fe57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404bfff640 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48fe57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404bfff640 1 -- 172.21.15.19:0/1948752946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 msgr2=0x559e48fecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404bfff640 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404bfff640 1 -- 172.21.15.19:0/1948752946 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e48c58210 con 0x559e48c172f0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404bfff640 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48fe5d30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f403c005ab0 tx=0x7f403c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f990c75b39e14514 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f40497fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f40340b28a0 0x7f40340b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f40327fc640 1 -- 172.21.15.19:0/1948752946 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c035030 con 0x559e48c172f0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f40327fc640 1 -- 172.21.15.19:0/1948752946 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c040070 con 0x559e48c172f0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f40340b28a0 0x7f40340b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.406+0000 7f40327fc640 1 -- 172.21.15.19:0/1948752946 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c003af0 con 0x559e48c172f0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40497fa640 1 -- 172.21.15.19:0/1646186107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f40440e2ae0 con 0x7f404c108aa0 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 -- 172.21.15.19:0/1948752946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 msgr2=0x559e48fe5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48fe5d30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f403c005ab0 tx=0x7f403c005d50 comp rx=0 tx=0).stop 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f40340b28a0 0x7f40340b4db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x559e48fe6980 tx=0x7f40400096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 -- 172.21.15.19:0/1948752946 shutdown_connections 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48fe5d30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 --2- 172.21.15.19:0/1948752946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48fe57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 -- 172.21.15.19:0/1948752946 >> 172.21.15.19:0/1948752946 conn(0x559e48c34300 msgr2=0x559e48fdcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.407+0000 7f40593a8740 1 -- 172.21.15.19:0/1948752946 shutdown_connections 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 -- 172.21.15.19:0/1948752946 wait complete. 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 -- start start 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 Processor -- start 2024-05-17T09:30:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 0x559e48fef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c1f4d0 0x559e48fef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48ff6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.408+0000 7f40593a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e48c29790 con 0x559e48c172f0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f40593a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559e48eff890 con 0x559e48c401b0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f40593a8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e48fed620 con 0x559e48c1f4d0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 0x559e48fef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f40593a8740 10 client.0 _ll_register_callbacks cb 0x559e48f02d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 0x559e48fef0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42218/0 (socket says 172.21.15.19:42218) 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 -- 172.21.15.19:0/3953398157 learned_addr learned my addr 172.21.15.19:0/3953398157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c1f4d0 0x559e48fef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f4050a97640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48ff6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 -- 172.21.15.19:0/3953398157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c1f4d0 msgr2=0x559e48fef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c1f4d0 0x559e48fef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 -- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 msgr2=0x559e48ff6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48ff6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 -- 172.21.15.19:0/3953398157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e48d24620 con 0x559e48c172f0 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c1f4d0 0x559e48fef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f404bfff640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 0x559e48fef0a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f403c0057c0 tx=0x7f403c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cf66b92787b3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f402b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f402b7fe640 1 -- 172.21.15.19:0/3953398157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c052060 con 0x559e48c172f0 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c040070 con 0x559e48c172f0 2024-05-17T09:30:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.409+0000 7f402b7fe640 1 -- 172.21.15.19:0/3953398157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f403c004240 con 0x559e48c172f0 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f402b7fe640 1 -- 172.21.15.19:0/3953398157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f403c059050 con 0x559e48c172f0 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 10 client.17349 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e48ff9450 con 0x559e48c172f0 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f402b7fe640 1 -- 172.21.15.19:0/3953398157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c040070 con 0x559e48c172f0 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f402b7fe640 1 -- 172.21.15.19:0/3953398157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f403c03c440 con 0x559e48c172f0 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 10 client.17349 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 20 client.17349 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48bfed80 0x559e48ffc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 4 client.17349 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:30:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e48f55b80 con 0x559e48bfed80 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.410+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48bfed80 0x559e48ffc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.411+0000 7f402b7fe640 10 client.17349 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.411+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48bfed80 0x559e48ffc7b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708dbc96907b42b3 server_cookie=36f905ee52057e21 in_seq=0 out_seq=0 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f402b7fe640 1 -- 172.21.15.19:0/3953398157 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e48f55b80 con 0x559e48bfed80 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f402b7fe640 10 client.17349 handle_command_reply: tid=0 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 client.17349 shutdown 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 msgr2=0x559e48fef0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 0x559e48fef0a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f403c0057c0 tx=0x7f403c02a410 comp rx=0 tx=0).stop 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 shutdown_connections 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48bfed80 0x559e48ffc7b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f402b7fe640 0 client.17349 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c401b0 0x559e48ff6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c1f4d0 0x559e48fef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 --2- 172.21.15.19:0/3953398157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c172f0 0x559e48fef0a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 >> 172.21.15.19:0/3953398157 conn(0x559e48c34300 msgr2=0x559e48fdfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 shutdown_connections 2024-05-17T09:30:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 1 -- 172.21.15.19:0/3953398157 wait complete. 2024-05-17T09:30:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.412+0000 7f40593a8740 20 client.17349 trim_cache size 0 max 16384 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 Processor -- start 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 -- start start 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48bfed80 0x559e48fe9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c172f0 0x559e48ff6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1ad130 con 0x559e48c1f4d0 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1aaa00 con 0x559e48c172f0 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f40593a8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1a8850 con 0x559e48bfed80 2024-05-17T09:30:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.429+0000 7f4050a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48bfed80 0x559e48fe9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f4050a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42232/0 (socket says 172.21.15.19:42232) 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f4050a97640 1 -- 172.21.15.19:0/1644716031 learned_addr learned my addr 172.21.15.19:0/1644716031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404b7fe640 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c172f0 0x559e48ff6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 -- 172.21.15.19:0/1644716031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c172f0 msgr2=0x559e48ff6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c172f0 0x559e48ff6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 -- 172.21.15.19:0/1644716031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 msgr2=0x559e48fef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 -- 172.21.15.19:0/1644716031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e48fed010 con 0x559e48bfed80 2024-05-17T09:30:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f404bfff640 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48bfed80 0x559e48fe9cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f403c02a930 tx=0x7f403c02ab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ecc3b8fe999d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f4031ffb640 1 -- 172.21.15.19:0/1644716031 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c052050 con 0x559e48bfed80 2024-05-17T09:30:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f4031ffb640 1 -- 172.21.15.19:0/1644716031 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c040040 con 0x559e48bfed80 2024-05-17T09:30:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.430+0000 7f4031ffb640 1 -- 172.21.15.19:0/1644716031 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c035040 con 0x559e48bfed80 2024-05-17T09:30:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 -- 172.21.15.19:0/1644716031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48bfed80 msgr2=0x559e48fe9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48bfed80 0x559e48fe9cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f403c02a930 tx=0x7f403c02ab90 comp rx=0 tx=0).stop 2024-05-17T09:30:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 -- 172.21.15.19:0/1644716031 shutdown_connections 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48c1f4d0 0x559e48fef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c172f0 0x559e48ff6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 --2- 172.21.15.19:0/1644716031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48bfed80 0x559e48fe9cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 -- 172.21.15.19:0/1644716031 >> 172.21.15.19:0/1644716031 conn(0x559e48c34300 msgr2=0x559e48feec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 -- 172.21.15.19:0/1644716031 shutdown_connections 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.431+0000 7f40593a8740 1 -- 172.21.15.19:0/1644716031 wait complete. 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 -- start start 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 Processor -- start 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 0x559e48ff6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48ff4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c1f4d0 0x559e48ff5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1a8ec0 con 0x559e48bfed80 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1aa330 con 0x559e48c1f4d0 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1a9590 con 0x559e48c172f0 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f40593a8740 10 client.0 _ll_register_callbacks cb 0x7f403c009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f404bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 0x559e48ff6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f404b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48ff4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f404bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 0x559e48ff6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42236/0 (socket says 172.21.15.19:42236) 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f404bfff640 1 -- 172.21.15.19:0/3795647146 learned_addr learned my addr 172.21.15.19:0/3795647146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.432+0000 7f4050a97640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c1f4d0 0x559e48ff5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404b7fe640 1 -- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c1f4d0 msgr2=0x559e48ff5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c1f4d0 0x559e48ff5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404b7fe640 1 -- 172.21.15.19:0/3795647146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 msgr2=0x559e48ff6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 0x559e48ff6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404b7fe640 1 -- 172.21.15.19:0/3795647146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e48ff58e0 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404bfff640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 0x559e48ff6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f404b7fe640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48ff4b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x559e48ffce90 tx=0x7f4040004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=726884620df627ed server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f402b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f402b7fe640 1 -- 172.21.15.19:0/3795647146 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040019040 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f402b7fe640 1 -- 172.21.15.19:0/3795647146 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f404002e940 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c040040 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f402b7fe640 1 -- 172.21.15.19:0/3795647146 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f404002eb40 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f40593a8740 10 client.26953 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.433+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e49063f10 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f402b7fe640 1 -- 172.21.15.19:0/3795647146 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c040040 con 0x559e48c172f0 2024-05-17T09:30:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f402b7fe640 1 -- 172.21.15.19:0/3795647146 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4040004a40 con 0x559e48c172f0 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f40593a8740 10 client.26953 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f40593a8740 10 client.26953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f40593a8740 20 client.26953 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f40593a8740 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48c401b0 0x559e49066870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f40593a8740 4 client.26953 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e48f4f360 con 0x559e48c401b0 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f404bfff640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48c401b0 0x559e49066870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.434+0000 7f402b7fe640 10 client.26953 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.435+0000 7f404bfff640 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48c401b0 0x559e49066870 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd88cd8a6af34c0 server_cookie=a0d118b4fc1833d2 in_seq=0 out_seq=0 2024-05-17T09:30:01.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 156902 1.71 MB/sec execute 359 sec latency 18.017 ms 2024-05-17T09:30:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.461+0000 7f402b7fe640 1 -- 172.21.15.19:0/3795647146 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e48f4f360 con 0x559e48c401b0 2024-05-17T09:30:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.461+0000 7f402b7fe640 10 client.26953 handle_command_reply: tid=0 2024-05-17T09:30:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 client.26953 shutdown 2024-05-17T09:30:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 msgr2=0x559e48ff4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48ff4b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x559e48ffce90 tx=0x7f4040004730 comp rx=0 tx=0).stop 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 shutdown_connections 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559e48c401b0 0x559e49066870 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e48c1f4d0 0x559e48ff5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e48c172f0 0x559e48ff4b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f402b7fe640 0 client.26953 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 --2- 172.21.15.19:0/3795647146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e48bfed80 0x559e48ff6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 >> 172.21.15.19:0/3795647146 conn(0x559e48c34300 msgr2=0x559e48fef650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 shutdown_connections 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 1 -- 172.21.15.19:0/3795647146 wait complete. 2024-05-17T09:30:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.462+0000 7f40593a8740 20 client.26953 trim_cache size 0 max 16384 2024-05-17T09:30:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:30:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:30:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:30:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f40340b28a0 msgr2=0x7f40340b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f40340b28a0 0x7f40340b4db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x559e48fe6980 tx=0x7f40400096a0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 msgr2=0x7f404c1a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 0x7f404c1a1530 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f404400eb10 tx=0x7f404400c2d0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 shutdown_connections 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f40340b28a0 0x7f40340b4db0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f404c108aa0 0x7f404c1a1530 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c103da0 0x7f404c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 --2- 172.21.15.19:0/1646186107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102b60 0x7f404c06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 >> 172.21.15.19:0/1646186107 conn(0x7f404c0fdd50 msgr2=0x7f404c104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 shutdown_connections 2024-05-17T09:30:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:01.463+0000 7f40524fa640 1 -- 172.21.15.19:0/1646186107 wait complete. 2024-05-17T09:30:02.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 157332 1.71 MB/sec execute 360 sec latency 16.280 ms 2024-05-17T09:30:02.482 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:02 smithi069 ceph-mon[35081]: pgmap v746: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-05-17T09:30:02.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:02 smithi079 ceph-mon[35502]: pgmap v746: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-05-17T09:30:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:02 smithi019 ceph-mon[31620]: pgmap v746: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-05-17T09:30:03.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 157919 1.71 MB/sec execute 361 sec latency 16.167 ms 2024-05-17T09:30:04.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 158483 1.71 MB/sec execute 362 sec latency 16.930 ms 2024-05-17T09:30:04.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:04 smithi079 ceph-mon[35502]: pgmap v747: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-17T09:30:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:04 smithi069 ceph-mon[35081]: pgmap v747: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-17T09:30:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:04 smithi019 ceph-mon[31620]: pgmap v747: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-17T09:30:05.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 159028 1.72 MB/sec execute 363 sec latency 15.356 ms 2024-05-17T09:30:06.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 159442 1.72 MB/sec execute 364 sec latency 16.321 ms 2024-05-17T09:30:06.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:06 smithi079 ceph-mon[35502]: pgmap v748: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:30:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:06 smithi069 ceph-mon[35081]: pgmap v748: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:30:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:06 smithi019 ceph-mon[31620]: pgmap v748: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:30:07.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 159886 1.73 MB/sec execute 365 sec latency 26.254 ms 2024-05-17T09:30:08.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 160099 1.73 MB/sec execute 366 sec latency 20.069 ms 2024-05-17T09:30:08.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:08 smithi079 ceph-mon[35502]: pgmap v749: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:30:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:08 smithi069 ceph-mon[35081]: pgmap v749: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:30:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:08 smithi019 ceph-mon[31620]: pgmap v749: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:30:09.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 160422 1.72 MB/sec execute 367 sec latency 17.199 ms 2024-05-17T09:30:10.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 160866 1.72 MB/sec execute 368 sec latency 14.249 ms 2024-05-17T09:30:10.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:10 smithi079 ceph-mon[35502]: pgmap v750: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:30:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:10 smithi069 ceph-mon[35081]: pgmap v750: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:30:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:10 smithi019 ceph-mon[31620]: pgmap v750: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:30:11.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 161434 1.73 MB/sec execute 369 sec latency 16.557 ms 2024-05-17T09:30:12.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 162012 1.73 MB/sec execute 370 sec latency 15.950 ms 2024-05-17T09:30:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:12 smithi079 ceph-mon[35502]: pgmap v751: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-05-17T09:30:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:12 smithi069 ceph-mon[35081]: pgmap v751: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-05-17T09:30:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:12 smithi019 ceph-mon[31620]: pgmap v751: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-05-17T09:30:13.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 162564 1.74 MB/sec execute 371 sec latency 15.715 ms 2024-05-17T09:30:14.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 162990 1.74 MB/sec execute 372 sec latency 15.665 ms 2024-05-17T09:30:14.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:14 smithi079 ceph-mon[35502]: pgmap v752: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 -- 172.21.15.19:0/129361103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d581099f0 msgr2=0x7f1d5810be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 --2- 172.21.15.19:0/129361103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d581099f0 0x7f1d5810be10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009ca0 tx=0x7f1d5002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 -- 172.21.15.19:0/129361103 shutdown_connections 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 --2- 172.21.15.19:0/129361103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d581099f0 0x7f1d5810be10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 --2- 172.21.15.19:0/129361103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d58107090 0x7f1d581094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 --2- 172.21.15.19:0/129361103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d58104730 0x7f1d58106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 -- 172.21.15.19:0/129361103 >> 172.21.15.19:0/129361103 conn(0x7f1d580fdd30 msgr2=0x7f1d581001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 -- 172.21.15.19:0/129361103 shutdown_connections 2024-05-17T09:30:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.573+0000 7f1d60669640 1 -- 172.21.15.19:0/129361103 wait complete. 2024-05-17T09:30:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 Processor -- start 2024-05-17T09:30:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 -- start start 2024-05-17T09:30:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d58104730 0x7f1d5819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 0x7f1d5819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d581099f0 0x7f1d581a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5810f670 con 0x7f1d58107090 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5810f4f0 con 0x7f1d581099f0 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.574+0000 7f1d60669640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5810f7f0 con 0x7f1d58104730 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 0x7f1d5819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 0x7f1d5819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60970/0 (socket says 172.21.15.19:60970) 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 -- 172.21.15.19:0/860301790 learned_addr learned my addr 172.21.15.19:0/860301790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5e405640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d58104730 0x7f1d5819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d581099f0 0x7f1d581a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 -- 172.21.15.19:0/860301790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d58104730 msgr2=0x7f1d5819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d58104730 0x7f1d5819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 -- 172.21.15.19:0/860301790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d581099f0 msgr2=0x7f1d581a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d581099f0 0x7f1d581a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 -- 172.21.15.19:0/860301790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d581a1cc0 con 0x7f1d58107090 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d581099f0 0x7f1d581a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 0x7f1d5819a6c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e9f0 tx=0x7f1d4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d4f7fe640 1 -- 172.21.15.19:0/860301790 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800cd80 con 0x7f1d58107090 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d581a1fb0 con 0x7f1d58107090 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d4f7fe640 1 -- 172.21.15.19:0/860301790 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d480054b0 con 0x7f1d58107090 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d4f7fe640 1 -- 172.21.15.19:0/860301790 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48010430 con 0x7f1d58107090 2024-05-17T09:30:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.575+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a2410 con 0x7f1d58107090 2024-05-17T09:30:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.578+0000 7f1d675f5740 1 Processor -- start 2024-05-17T09:30:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.578+0000 7f1d675f5740 1 -- start start 2024-05-17T09:30:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.578+0000 7f1d4f7fe640 1 -- 172.21.15.19:0/860301790 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d480046c0 con 0x7f1d58107090 2024-05-17T09:30:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.578+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 0x5573869f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866324d0 0x5573869fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d675f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557386966c30 con 0x5573866324d0 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d675f5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5573869567d0 con 0x5573866531b0 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d675f5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55738695a810 con 0x55738662a2f0 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866324d0 0x5573869fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 0x5573869f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 0x5573869f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59060/0 (socket says 172.21.15.19:59060) 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 -- 172.21.15.19:0/2921227614 learned_addr learned my addr 172.21.15.19:0/2921227614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5e405640 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 -- 172.21.15.19:0/2921227614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 msgr2=0x5573869f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 -- 172.21.15.19:0/2921227614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866324d0 msgr2=0x5573869fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866324d0 0x5573869fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 -- 172.21.15.19:0/2921227614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738666b210 con 0x5573866531b0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.579+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 0x5573869f43a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d50035000 tx=0x7f1d50009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e319b39fcc6e392 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.580+0000 7f1d4f7fe640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d440b28a0 0x7f1d440b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.580+0000 7f1d4cbfe640 1 -- 172.21.15.19:0/2921227614 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50031a80 con 0x5573866531b0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.580+0000 7f1d5e405640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d440b28a0 0x7f1d440b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.580+0000 7f1d4cbfe640 1 -- 172.21.15.19:0/2921227614 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d50036040 con 0x5573866531b0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.580+0000 7f1d4cbfe640 1 -- 172.21.15.19:0/2921227614 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500043c0 con 0x5573866531b0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.580+0000 7f1d4f7fe640 1 -- 172.21.15.19:0/860301790 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1d480e29b0 con 0x7f1d58107090 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 -- 172.21.15.19:0/2921227614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 msgr2=0x5573869f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d5e405640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d440b28a0 0x7f1d440b4db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5573869f5780 tx=0x7f1d54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 0x5573869f43a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d50035000 tx=0x7f1d50009910 comp rx=0 tx=0).stop 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 -- 172.21.15.19:0/2921227614 shutdown_connections 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866324d0 0x5573869fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2921227614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866531b0 0x5573869f43a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 -- 172.21.15.19:0/2921227614 >> 172.21.15.19:0/2921227614 conn(0x557386647300 msgr2=0x5573869efd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 -- 172.21.15.19:0/2921227614 shutdown_connections 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 -- 172.21.15.19:0/2921227614 wait complete. 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 -- start start 2024-05-17T09:30:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.581+0000 7f1d675f5740 1 Processor -- start 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x5573869fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x5573869fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55738663c790 con 0x55738662a2f0 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557386912890 con 0x5573866324d0 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5573869fc210 con 0x5573866531b0 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d675f5740 10 client.0 _ll_register_callbacks cb 0x557386915d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5e405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x5573869fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x5573869fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5e405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x5573869fdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60996/0 (socket says 172.21.15.19:60996) 2024-05-17T09:30:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5e405640 1 -- 172.21.15.19:0/1656599101 learned_addr learned my addr 172.21.15.19:0/1656599101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5dc04640 1 -- 172.21.15.19:0/1656599101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 msgr2=0x557386a050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5dc04640 1 -- 172.21.15.19:0/1656599101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 msgr2=0x5573869fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x5573869fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5dc04640 1 -- 172.21.15.19:0/1656599101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557386737620 con 0x5573866324d0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5e405640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x5573869fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.582+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x5573869fe190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d480e0150 tx=0x7f1d480a5ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd8bbc8e8c186bd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d41ffb640 1 -- 172.21.15.19:0/1656599101 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480e2e60 con 0x5573866324d0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d41ffb640 1 -- 172.21.15.19:0/1656599101 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1d480b2800 con 0x5573866324d0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d50036040 con 0x5573866324d0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d41ffb640 1 -- 172.21.15.19:0/1656599101 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1d480a9390 con 0x5573866324d0 2024-05-17T09:30:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 10 client.26984 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557386a08090 con 0x5573866324d0 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d41ffb640 1 -- 172.21.15.19:0/1656599101 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d50036040 con 0x5573866324d0 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d41ffb640 1 -- 172.21.15.19:0/1656599101 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1d480c4120 con 0x5573866324d0 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 10 client.26984 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 10 client.26984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 20 client.26984 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557386611d80 0x557386a0b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 4 client.26984 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:30:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.583+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557386968b80 con 0x557386611d80 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.584+0000 7f1d5e405640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557386611d80 0x557386a0b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.584+0000 7f1d5e405640 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557386611d80 0x557386a0b770 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d992ae5cf50f93 server_cookie=234101097452d0b5 in_seq=0 out_seq=0 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.584+0000 7f1d41ffb640 10 client.26984 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d41ffb640 1 -- 172.21.15.19:0/1656599101 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557386968b80 con 0x557386611d80 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d41ffb640 10 client.26984 handle_command_reply: tid=0 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 client.26984 shutdown 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 msgr2=0x5573869fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x5573869fe190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d480e0150 tx=0x7f1d480a5ae0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 shutdown_connections 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557386611d80 0x557386a0b770 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x5573869fe190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d41ffb640 0 client.26984 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 --2- 172.21.15.19:0/1656599101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x5573869fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 >> 172.21.15.19:0/1656599101 conn(0x557386647300 msgr2=0x5573869f2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 shutdown_connections 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 1 -- 172.21.15.19:0/1656599101 wait complete. 2024-05-17T09:30:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.585+0000 7f1d675f5740 20 client.26984 trim_cache size 0 max 16384 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.602+0000 7f1d675f5740 1 Processor -- start 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.602+0000 7f1d675f5740 1 -- start start 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a05010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866531b0 0x5573869f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d675f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581af290 con 0x5573866531b0 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d675f5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581acb60 con 0x5573866324d0 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d675f5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581aa9b0 con 0x55738662a2f0 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866531b0 0x5573869f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a05010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a05010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59080/0 (socket says 172.21.15.19:59080) 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5dc04640 1 -- 172.21.15.19:0/4086777723 learned_addr learned my addr 172.21.15.19:0/4086777723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5e405640 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 -- 172.21.15.19:0/4086777723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 msgr2=0x5573869f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 -- 172.21.15.19:0/4086777723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 msgr2=0x557386a05010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a05010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 -- 172.21.15.19:0/4086777723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573869fe4f0 con 0x5573866531b0 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a05010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.603+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866531b0 0x5573869f3070 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1d5002a900 tx=0x7f1d5003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c496c505fdcca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.604+0000 7f1d43fff640 1 -- 172.21.15.19:0/4086777723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5004a040 con 0x5573866531b0 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.604+0000 7f1d43fff640 1 -- 172.21.15.19:0/4086777723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d50053070 con 0x5573866531b0 2024-05-17T09:30:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.604+0000 7f1d43fff640 1 -- 172.21.15.19:0/4086777723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500042a0 con 0x5573866531b0 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.604+0000 7f1d675f5740 1 -- 172.21.15.19:0/4086777723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866531b0 msgr2=0x5573869f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.604+0000 7f1d675f5740 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866531b0 0x5573869f3070 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1d5002a900 tx=0x7f1d5003e040 comp rx=0 tx=0).stop 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.604+0000 7f1d675f5740 1 -- 172.21.15.19:0/4086777723 shutdown_connections 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573866531b0 0x5573869f3070 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a05010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 --2- 172.21.15.19:0/4086777723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55738662a2f0 0x5573869f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- 172.21.15.19:0/4086777723 >> 172.21.15.19:0/4086777723 conn(0x557386647300 msgr2=0x5573869ef680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- 172.21.15.19:0/4086777723 shutdown_connections 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- 172.21.15.19:0/4086777723 wait complete. 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- start start 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 Processor -- start 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x557386a01e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a71e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581ab020 con 0x55738662a2f0 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581ac490 con 0x5573866324d0 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581ab6f0 con 0x5573866531b0 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d675f5740 10 client.0 _ll_register_callbacks cb 0x7f1d4800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.605+0000 7f1d5e405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x557386a01e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5e405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x557386a01e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32792/0 (socket says 172.21.15.19:32792) 2024-05-17T09:30:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5e405640 1 -- 172.21.15.19:0/2227400326 learned_addr learned my addr 172.21.15.19:0/2227400326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5ec06640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a71e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 -- 172.21.15.19:0/2227400326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 msgr2=0x557386a71e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a71e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 -- 172.21.15.19:0/2227400326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 msgr2=0x557386a01e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x557386a01e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 -- 172.21.15.19:0/2227400326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557386a02a00 con 0x5573866324d0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5e405640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x557386a01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d5dc04640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a023c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d480e0470 tx=0x7f1d48005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae92ac8a04b33e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d41ffb640 1 -- 172.21.15.19:0/2227400326 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480a53b0 con 0x5573866324d0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d41ffb640 1 -- 172.21.15.19:0/2227400326 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1d48005210 con 0x5573866324d0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d50053070 con 0x5573866324d0 2024-05-17T09:30:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d41ffb640 1 -- 172.21.15.19:0/2227400326 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1d480a9730 con 0x5573866324d0 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d675f5740 10 client.26990 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.606+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557386a73450 con 0x5573866324d0 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d41ffb640 1 -- 172.21.15.19:0/2227400326 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d50053070 con 0x5573866324d0 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d41ffb640 1 -- 172.21.15.19:0/2227400326 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1d480a5620 con 0x5573866324d0 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d675f5740 10 client.26990 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d675f5740 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d675f5740 20 client.26990 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5573865e71c0 0x557386a75df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d675f5740 4 client.26990 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557386962360 con 0x5573865e71c0 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d5e405640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5573865e71c0 0x557386a75df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d41ffb640 10 client.26990 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.607+0000 7f1d5e405640 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5573865e71c0 0x557386a75df0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690cae9e5b0707ab server_cookie=2e8bc2ce7916ae3c in_seq=0 out_seq=0 2024-05-17T09:30:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.630+0000 7f1d41ffb640 1 -- 172.21.15.19:0/2227400326 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557386962360 con 0x5573865e71c0 2024-05-17T09:30:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.630+0000 7f1d41ffb640 10 client.26990 handle_command_reply: tid=0 2024-05-17T09:30:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.630+0000 7f1d675f5740 1 client.26990 shutdown 2024-05-17T09:30:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 msgr2=0x557386a023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a023c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d480e0470 tx=0x7f1d48005ab0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 shutdown_connections 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5573865e71c0 0x557386a75df0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573866531b0 0x557386a71e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573866324d0 0x557386a023c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 --2- 172.21.15.19:0/2227400326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55738662a2f0 0x557386a01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d41ffb640 0 client.26990 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 >> 172.21.15.19:0/2227400326 conn(0x557386647300 msgr2=0x5573869f77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 shutdown_connections 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 1 -- 172.21.15.19:0/2227400326 wait complete. 2024-05-17T09:30:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d675f5740 20 client.26990 trim_cache size 0 max 16384 2024-05-17T09:30:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:30:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:30:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d440b28a0 msgr2=0x7f1d440b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d440b28a0 0x7f1d440b4db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5573869f5780 tx=0x7f1d54005f90 comp rx=0 tx=0).stop 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 msgr2=0x7f1d5819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 0x7f1d5819a6c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e9f0 tx=0x7f1d4800edf0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 shutdown_connections 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d581099f0 0x7f1d581a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d440b28a0 0x7f1d440b4db0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d58107090 0x7f1d5819a6c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.631+0000 7f1d60669640 1 --2- 172.21.15.19:0/860301790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d58104730 0x7f1d5819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.632+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 >> 172.21.15.19:0/860301790 conn(0x7f1d580fdd30 msgr2=0x7f1d581051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.632+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 shutdown_connections 2024-05-17T09:30:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:14.632+0000 7f1d60669640 1 -- 172.21.15.19:0/860301790 wait complete. 2024-05-17T09:30:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:14 smithi069 ceph-mon[35081]: pgmap v752: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-17T09:30:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:14 smithi019 ceph-mon[31620]: pgmap v752: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-17T09:30:15.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 163432 1.74 MB/sec execute 373 sec latency 14.821 ms 2024-05-17T09:30:16.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 163655 1.74 MB/sec execute 374 sec latency 15.614 ms 2024-05-17T09:30:16.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:16 smithi079 ceph-mon[35502]: pgmap v753: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:30:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:16 smithi069 ceph-mon[35081]: pgmap v753: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:30:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:16 smithi019 ceph-mon[31620]: pgmap v753: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:30:17.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 164045 1.74 MB/sec execute 375 sec latency 62.192 ms 2024-05-17T09:30:18.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 164566 1.74 MB/sec execute 376 sec latency 17.485 ms 2024-05-17T09:30:18.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:18 smithi079 ceph-mon[35502]: pgmap v754: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:30:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:18 smithi069 ceph-mon[35081]: pgmap v754: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:30:18.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:18 smithi019 ceph-mon[31620]: pgmap v754: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:30:19.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 165109 1.74 MB/sec execute 377 sec latency 15.920 ms 2024-05-17T09:30:20.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 165637 1.74 MB/sec execute 378 sec latency 16.858 ms 2024-05-17T09:30:20.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:20 smithi079 ceph-mon[35502]: pgmap v755: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:30:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:20 smithi069 ceph-mon[35081]: pgmap v755: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:30:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:20 smithi019 ceph-mon[31620]: pgmap v755: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-17T09:30:21.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 166384 1.75 MB/sec execute 379 sec latency 16.135 ms 2024-05-17T09:30:22.442 INFO:tasks.workunit.client.0.smithi019.stdout: 1 166801 1.76 MB/sec execute 380 sec latency 15.767 ms 2024-05-17T09:30:22.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:22 smithi079 ceph-mon[35502]: pgmap v756: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-05-17T09:30:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:22 smithi069 ceph-mon[35081]: pgmap v756: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-05-17T09:30:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:22 smithi019 ceph-mon[31620]: pgmap v756: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-05-17T09:30:23.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 167117 1.76 MB/sec execute 381 sec latency 13.640 ms 2024-05-17T09:30:24.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 167397 1.75 MB/sec execute 382 sec latency 43.600 ms 2024-05-17T09:30:24.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:24 smithi079 ceph-mon[35502]: pgmap v757: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:30:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:24 smithi069 ceph-mon[35081]: pgmap v757: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:30:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:24 smithi019 ceph-mon[31620]: pgmap v757: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-17T09:30:25.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 167896 1.75 MB/sec execute 383 sec latency 17.091 ms 2024-05-17T09:30:26.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 168424 1.76 MB/sec execute 384 sec latency 18.699 ms 2024-05-17T09:30:26.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:26 smithi079 ceph-mon[35502]: pgmap v758: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:30:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:26 smithi069 ceph-mon[35081]: pgmap v758: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:30:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:26 smithi019 ceph-mon[31620]: pgmap v758: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:30:27.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 169067 1.76 MB/sec execute 385 sec latency 22.259 ms 2024-05-17T09:30:27.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 -- 172.21.15.19:0/464991085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 msgr2=0x7fdedc103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 --2- 172.21.15.19:0/464991085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc103820 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fded0009900 tx=0x7fded002a410 comp rx=0 tx=0).stop 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 -- 172.21.15.19:0/464991085 shutdown_connections 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 --2- 172.21.15.19:0/464991085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdedc108a60 0x7fdedc10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 --2- 172.21.15.19:0/464991085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdedc103d60 0x7fdedc104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 --2- 172.21.15.19:0/464991085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc103820 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 -- 172.21.15.19:0/464991085 >> 172.21.15.19:0/464991085 conn(0x7fdedc0fdd10 msgr2=0x7fdedc1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 -- 172.21.15.19:0/464991085 shutdown_connections 2024-05-17T09:30:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.736+0000 7fdee2c69640 1 -- 172.21.15.19:0/464991085 wait complete. 2024-05-17T09:30:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.737+0000 7fdee2c69640 1 Processor -- start 2024-05-17T09:30:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.737+0000 7fdee2c69640 1 -- start start 2024-05-17T09:30:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.737+0000 7fdee2c69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.737+0000 7fdee2c69640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdedc103d60 0x7fdedc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.737+0000 7fdee2c69640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdedc108a60 0x7fdedc1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.737+0000 7fdee0a05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee2c69640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e500 con 0x7fdedc102b20 2024-05-17T09:30:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee2c69640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e380 con 0x7fdedc108a60 2024-05-17T09:30:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc075960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37562/0 (socket says 172.21.15.19:37562) 2024-05-17T09:30:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 -- 172.21.15.19:0/2192083816 learned_addr learned my addr 172.21.15.19:0/2192083816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e680 con 0x7fdedc103d60 2024-05-17T09:30:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee1206640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdedc108a60 0x7fdedc1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdedc103d60 0x7fdedc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 -- 172.21.15.19:0/2192083816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdedc103d60 msgr2=0x7fdedc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdedc103d60 0x7fdedc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 -- 172.21.15.19:0/2192083816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdedc108a60 msgr2=0x7fdedc1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdedc108a60 0x7fdedc1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 -- 172.21.15.19:0/2192083816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc1a1d00 con 0x7fdedc102b20 2024-05-17T09:30:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.738+0000 7fdee0a05640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc075960 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fded0005760 tx=0x7fded0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.739+0000 7fded9ffb640 1 -- 172.21.15.19:0/2192083816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded003f070 con 0x7fdedc102b20 2024-05-17T09:30:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.739+0000 7fded9ffb640 1 -- 172.21.15.19:0/2192083816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded0031ba0 con 0x7fdedc102b20 2024-05-17T09:30:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.739+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc1a1f90 con 0x7fdedc102b20 2024-05-17T09:30:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.739+0000 7fded9ffb640 1 -- 172.21.15.19:0/2192083816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0035030 con 0x7fdedc102b20 2024-05-17T09:30:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.739+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc1a2390 con 0x7fdedc102b20 2024-05-17T09:30:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 Processor -- start 2024-05-17T09:30:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 -- start start 2024-05-17T09:30:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77242f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c7724317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772433f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c772396c30 con 0x55c7720831b0 2024-05-17T09:30:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7723867d0 con 0x55c7720624d0 2024-05-17T09:30:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c77238a810 con 0x55c77205a2f0 2024-05-17T09:30:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.743+0000 7fded9ffb640 1 -- 172.21.15.19:0/2192083816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fded0042070 con 0x7fdedc102b20 2024-05-17T09:30:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77242f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77242f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37576/0 (socket says 172.21.15.19:37576) 2024-05-17T09:30:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 -- 172.21.15.19:0/2792568737 learned_addr learned my addr 172.21.15.19:0/2792568737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772433f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee0a05640 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c7724317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 -- 172.21.15.19:0/2792568737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 msgr2=0x55c7724317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c7724317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 -- 172.21.15.19:0/2792568737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 msgr2=0x55c772433f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772433f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 -- 172.21.15.19:0/2792568737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77209b210 con 0x55c7720831b0 2024-05-17T09:30:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772433f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdee1206640 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77242f200 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fded4011440 tx=0x7fded4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386d1871230be27d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.744+0000 7fdec6ffd640 1 -- 172.21.15.19:0/2792568737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4003c30 con 0x55c7720831b0 2024-05-17T09:30:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.745+0000 7fdec6ffd640 1 -- 172.21.15.19:0/2792568737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded4003dd0 con 0x55c7720831b0 2024-05-17T09:30:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.745+0000 7fdec6ffd640 1 -- 172.21.15.19:0/2792568737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded400f8e0 con 0x55c7720831b0 2024-05-17T09:30:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.745+0000 7fded9ffb640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdecc0b28a0 0x7fdecc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.745+0000 7fdee0a05640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdecc0b28a0 0x7fdecc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.745+0000 7fded9ffb640 1 -- 172.21.15.19:0/2192083816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fded0103080 con 0x7fdedc102b20 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2792568737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 msgr2=0x55c77242f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee0a05640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdecc0b28a0 0x7fdecc0b4db0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fded00c7560 tx=0x7fded00b2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77242f200 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fded4011440 tx=0x7fded4011900 comp rx=0 tx=0).stop 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2792568737 shutdown_connections 2024-05-17T09:30:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772433f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c7724317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2792568737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77242f200 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2792568737 >> 172.21.15.19:0/2792568737 conn(0x55c772077300 msgr2=0x55c77241fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2792568737 shutdown_connections 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2792568737 wait complete. 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 -- start start 2024-05-17T09:30:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.746+0000 7fdee9bf0740 1 Processor -- start 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c772436530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772436a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77243d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c77206c790 con 0x55c7720831b0 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c772342890 con 0x55c7720624d0 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c772434a60 con 0x55c77205a2f0 2024-05-17T09:30:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee9bf0740 10 client.0 _ll_register_callbacks cb 0x55c772345d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee1206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77243d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee1206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77243d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37582/0 (socket says 172.21.15.19:37582) 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee1206640 1 -- 172.21.15.19:0/2111888578 learned_addr learned my addr 172.21.15.19:0/2111888578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772436a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee1206640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c772436530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 -- 172.21.15.19:0/2111888578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 msgr2=0x55c772436530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c772436530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 -- 172.21.15.19:0/2111888578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 msgr2=0x55c77243d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77243d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 -- 172.21.15.19:0/2111888578 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c772167620 con 0x55c7720624d0 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdee1206640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77243d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772436a70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdedc075300 tx=0x7fdec8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476c2d8f0096a832 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.747+0000 7fdebbfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdebbfff640 1 -- 172.21.15.19:0/2111888578 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8004030 con 0x55c7720624d0 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded4003dd0 con 0x55c7720624d0 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdebbfff640 1 -- 172.21.15.19:0/2111888578 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdec80041d0 con 0x55c7720624d0 2024-05-17T09:30:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdebbfff640 1 -- 172.21.15.19:0/2111888578 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdec8025050 con 0x55c7720624d0 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 10 client.27002 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c772440a50 con 0x55c7720624d0 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdebbfff640 1 -- 172.21.15.19:0/2111888578 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fded4003dd0 con 0x55c7720624d0 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdebbfff640 1 -- 172.21.15.19:0/2111888578 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdec8003880 con 0x55c7720624d0 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 10 client.27002 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 20 client.27002 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c772041d80 0x55c772444230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 4 client.27002 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.748+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c772398b80 con 0x55c772041d80 2024-05-17T09:30:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.749+0000 7fdee1206640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c772041d80 0x55c772444230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.749+0000 7fdee1206640 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c772041d80 0x55c772444230 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd50c1d04c8a739 server_cookie=50dba8a7c6ce121a in_seq=0 out_seq=0 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.749+0000 7fdebbfff640 10 client.27002 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdebbfff640 1 -- 172.21.15.19:0/2111888578 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c772398b80 con 0x55c772041d80 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdebbfff640 10 client.27002 handle_command_reply: tid=0 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 client.27002 shutdown 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 msgr2=0x55c772436a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772436a70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdedc075300 tx=0x7fdec8012b60 comp rx=0 tx=0).stop 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 shutdown_connections 2024-05-17T09:30:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c772041d80 0x55c772444230 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720831b0 0x55c77243d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720624d0 0x55c772436a70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2111888578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c772436530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 >> 172.21.15.19:0/2111888578 conn(0x55c772077300 msgr2=0x55c77241ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdebbfff640 0 client.27002 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 shutdown_connections 2024-05-17T09:30:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2111888578 wait complete. 2024-05-17T09:30:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.750+0000 7fdee9bf0740 20 client.27002 trim_cache size 0 max 16384 2024-05-17T09:30:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 Processor -- start 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 -- start start 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c77242e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 0x55c77243d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1ad060 con 0x55c7720624d0 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1aa930 con 0x55c7720831b0 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1a8780 con 0x55c77205a2f0 2024-05-17T09:30:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 0x55c77243d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 0x55c77243d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37586/0 (socket says 172.21.15.19:37586) 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdedbfff640 1 -- 172.21.15.19:0/2915207242 learned_addr learned my addr 172.21.15.19:0/2915207242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee1206640 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.767+0000 7fdee1206640 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c77242e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdedbfff640 1 -- 172.21.15.19:0/2915207242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 msgr2=0x55c77242e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c77242e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdedbfff640 1 -- 172.21.15.19:0/2915207242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 msgr2=0x55c77243de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdedbfff640 1 -- 172.21.15.19:0/2915207242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c772436290 con 0x55c7720624d0 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdee1206640 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 0x55c77243d8f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdec8000c00 tx=0x7fdec800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5420ca961926ae40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdec67fc640 1 -- 172.21.15.19:0/2915207242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec802e070 con 0x55c7720624d0 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdec67fc640 1 -- 172.21.15.19:0/2915207242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdec800f920 con 0x55c7720624d0 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.768+0000 7fdec67fc640 1 -- 172.21.15.19:0/2915207242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8022040 con 0x55c7720624d0 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2915207242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 msgr2=0x55c77243d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 0x55c77243d8f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdec8000c00 tx=0x7fdec800f810 comp rx=0 tx=0).stop 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2915207242 shutdown_connections 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7720624d0 0x55c77243d8f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2915207242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c77205a2f0 0x55c77242e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2915207242 >> 172.21.15.19:0/2915207242 conn(0x55c772077300 msgr2=0x55c77242cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2915207242 shutdown_connections 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2915207242 wait complete. 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 -- start start 2024-05-17T09:30:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.769+0000 7fdee9bf0740 1 Processor -- start 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 0x55c77243aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c7720624d0 0x55c77243b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1a8df0 con 0x55c77205a2f0 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1aa260 con 0x55c7720831b0 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1a94c0 con 0x55c7720624d0 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee9bf0740 10 client.0 _ll_register_callbacks cb 0x7fdec8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 0x55c77243aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 0x55c77243aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37598/0 (socket says 172.21.15.19:37598) 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 -- 172.21.15.19:0/2720902695 learned_addr learned my addr 172.21.15.19:0/2720902695 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c7720624d0 0x55c77243b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 -- 172.21.15.19:0/2720902695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c7720624d0 msgr2=0x55c77243b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c7720624d0 0x55c77243b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 -- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 msgr2=0x55c77243c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdee1206640 1 -- 172.21.15.19:0/2720902695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77243c9d0 con 0x55c77205a2f0 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.770+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c7720624d0 0x55c77243b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee1206640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 0x55c77243aff0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fded4004960 tx=0x7fded4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa61432425968de server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdebbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdebbfff640 1 -- 172.21.15.19:0/2720902695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4003d40 con 0x55c77205a2f0 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdebbfff640 1 -- 172.21.15.19:0/2720902695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fded4004250 con 0x55c77205a2f0 2024-05-17T09:30:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec800f920 con 0x55c77205a2f0 2024-05-17T09:30:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdebbfff640 1 -- 172.21.15.19:0/2720902695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fded4044050 con 0x55c77205a2f0 2024-05-17T09:30:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 10 client.17391 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c77243dcd0 con 0x55c77205a2f0 2024-05-17T09:30:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdebbfff640 1 -- 172.21.15.19:0/2720902695 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdec800f920 con 0x55c77205a2f0 2024-05-17T09:30:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdebbfff640 1 -- 172.21.15.19:0/2720902695 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fded4025040 con 0x55c77205a2f0 2024-05-17T09:30:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 10 client.17391 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 20 client.17391 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.771+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c7720171c0 0x55c7724aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.772+0000 7fdee9bf0740 4 client.17391 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:30:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.772+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c772392360 con 0x55c7720171c0 2024-05-17T09:30:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.772+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c7720171c0 0x55c7724aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.772+0000 7fdebbfff640 10 client.17391 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.772+0000 7fdedbfff640 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c7720171c0 0x55c7724aead0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910998e456057fde server_cookie=2e22f5d439e73de7 in_seq=0 out_seq=0 2024-05-17T09:30:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdebbfff640 1 -- 172.21.15.19:0/2720902695 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c772392360 con 0x55c7720171c0 2024-05-17T09:30:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdebbfff640 10 client.17391 handle_command_reply: tid=0 2024-05-17T09:30:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 client.17391 shutdown 2024-05-17T09:30:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 msgr2=0x55c77243aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 0x55c77243aff0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fded4004960 tx=0x7fded4011900 comp rx=0 tx=0).stop 2024-05-17T09:30:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 shutdown_connections 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c7720171c0 0x55c7724aead0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7720831b0 0x55c77243c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c7720624d0 0x55c77243b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 --2- 172.21.15.19:0/2720902695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77205a2f0 0x55c77243aff0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdebbfff640 0 client.17391 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 >> 172.21.15.19:0/2720902695 conn(0x55c772077300 msgr2=0x55c77241f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 shutdown_connections 2024-05-17T09:30:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 1 -- 172.21.15.19:0/2720902695 wait complete. 2024-05-17T09:30:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.797+0000 7fdee9bf0740 20 client.17391 trim_cache size 0 max 16384 2024-05-17T09:30:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:30:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:30:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdecc0b28a0 msgr2=0x7fdecc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdecc0b28a0 0x7fdecc0b4db0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fded00c7560 tx=0x7fded00b2040 comp rx=0 tx=0).stop 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 msgr2=0x7fdedc075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc075960 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fded0005760 tx=0x7fded0003c10 comp rx=0 tx=0).stop 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 shutdown_connections 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdedc108a60 0x7fdedc1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdedc103d60 0x7fdedc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdecc0b28a0 0x7fdecc0b4db0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 --2- 172.21.15.19:0/2192083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdedc102b20 0x7fdedc075960 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 >> 172.21.15.19:0/2192083816 conn(0x7fdedc0fdd10 msgr2=0x7fdedc104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 shutdown_connections 2024-05-17T09:30:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:27.798+0000 7fdee2c69640 1 -- 172.21.15.19:0/2192083816 wait complete. 2024-05-17T09:30:28.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 169631 1.77 MB/sec execute 386 sec latency 17.519 ms 2024-05-17T09:30:28.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:28 smithi079 ceph-mon[35502]: pgmap v759: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:30:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:28 smithi069 ceph-mon[35081]: pgmap v759: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:30:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:28 smithi019 ceph-mon[31620]: pgmap v759: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-17T09:30:29.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 170059 1.77 MB/sec execute 387 sec latency 59.130 ms 2024-05-17T09:30:30.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 170507 1.77 MB/sec execute 388 sec latency 18.880 ms 2024-05-17T09:30:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:30 smithi079 ceph-mon[35502]: pgmap v760: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-17T09:30:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:30:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:30:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:30:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:30:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:30 smithi069 ceph-mon[35081]: pgmap v760: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-17T09:30:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:30:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:30:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:30:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:30:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:30 smithi019 ceph-mon[31620]: pgmap v760: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-17T09:30:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:30:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:30:30.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:30:30.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:30:31.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 170744 1.77 MB/sec execute 389 sec latency 15.164 ms 2024-05-17T09:30:32.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 171106 1.77 MB/sec execute 390 sec latency 16.146 ms 2024-05-17T09:30:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:32 smithi079 ceph-mon[35502]: pgmap v761: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-05-17T09:30:32.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:32 smithi069 ceph-mon[35081]: pgmap v761: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-05-17T09:30:32.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:32 smithi019 ceph-mon[31620]: pgmap v761: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-05-17T09:30:33.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 171660 1.77 MB/sec execute 391 sec latency 14.963 ms 2024-05-17T09:30:34.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 172182 1.77 MB/sec execute 392 sec latency 17.112 ms 2024-05-17T09:30:34.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:34 smithi079 ceph-mon[35502]: pgmap v762: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:30:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:34 smithi069 ceph-mon[35081]: pgmap v762: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:30:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:34 smithi019 ceph-mon[31620]: pgmap v762: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:30:35.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 172715 1.77 MB/sec execute 393 sec latency 16.431 ms 2024-05-17T09:30:36.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 173460 1.78 MB/sec execute 394 sec latency 13.611 ms 2024-05-17T09:30:36.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:36 smithi079 ceph-mon[35502]: pgmap v763: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-05-17T09:30:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:36 smithi069 ceph-mon[35081]: pgmap v763: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-05-17T09:30:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:36 smithi019 ceph-mon[31620]: pgmap v763: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-05-17T09:30:37.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 173841 1.79 MB/sec execute 395 sec latency 20.016 ms 2024-05-17T09:30:38.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 174190 1.79 MB/sec execute 396 sec latency 15.759 ms 2024-05-17T09:30:38.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:38 smithi079 ceph-mon[35502]: pgmap v764: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:30:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:38 smithi069 ceph-mon[35081]: pgmap v764: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:30:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:38 smithi019 ceph-mon[31620]: pgmap v764: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-17T09:30:39.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 174439 1.78 MB/sec execute 397 sec latency 15.213 ms 2024-05-17T09:30:40.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 174919 1.78 MB/sec execute 398 sec latency 16.339 ms 2024-05-17T09:30:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:40 smithi079 ceph-mon[35502]: pgmap v765: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:30:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:40 smithi069 ceph-mon[35081]: pgmap v765: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:30:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:40 smithi019 ceph-mon[31620]: pgmap v765: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3023038397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 msgr2=0x7fdb6410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3023038397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 0x7fdb6410bde0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009ca0 tx=0x7fdb5c02a8c0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3023038397 shutdown_connections 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3023038397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 0x7fdb6410bde0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3023038397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb64109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3023038397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb64104350 0x7fdb64106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3023038397 >> 172.21.15.19:0/3023038397 conn(0x7fdb640fdd30 msgr2=0x7fdb641001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3023038397 shutdown_connections 2024-05-17T09:30:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.905+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3023038397 wait complete. 2024-05-17T09:30:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 Processor -- start 2024-05-17T09:30:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 -- start start 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb64104350 0x7fdb6419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb6419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 0x7fdb641a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6410f660 con 0x7fdb641098d0 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6410f4e0 con 0x7fdb64106d60 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.906+0000 7fdb6b4eb640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6410f7e0 con 0x7fdb64104350 2024-05-17T09:30:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb6419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb69a88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 0x7fdb641a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb6419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36340/0 (socket says 172.21.15.19:36340) 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 -- 172.21.15.19:0/3043174495 learned_addr learned my addr 172.21.15.19:0/3043174495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb69287640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb64104350 0x7fdb6419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 -- 172.21.15.19:0/3043174495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb64104350 msgr2=0x7fdb6419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb64104350 0x7fdb6419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 -- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 msgr2=0x7fdb641a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 0x7fdb641a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 -- 172.21.15.19:0/3043174495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb641a1cb0 con 0x7fdb64106d60 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb68a86640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb6419a6b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e9f0 tx=0x7fdb5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb5a7fc640 1 -- 172.21.15.19:0/3043174495 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400cd80 con 0x7fdb64106d60 2024-05-17T09:30:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb641a1fa0 con 0x7fdb64106d60 2024-05-17T09:30:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb5a7fc640 1 -- 172.21.15.19:0/3043174495 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb540108c0 con 0x7fdb64106d60 2024-05-17T09:30:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.907+0000 7fdb5a7fc640 1 -- 172.21.15.19:0/3043174495 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540187c0 con 0x7fdb64106d60 2024-05-17T09:30:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.908+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb641a2460 con 0x7fdb64106d60 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.910+0000 7fdb7239d740 1 Processor -- start 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.910+0000 7fdb7239d740 1 -- start start 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 0x55fdb9e353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e35920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb7239d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9da7c30 con 0x55fdb9a941b0 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb7239d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9d977d0 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb7239d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9d9b810 con 0x55fdb9a734d0 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb5a7fc640 1 -- 172.21.15.19:0/3043174495 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdb54010430 con 0x7fdb64106d60 2024-05-17T09:30:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69a88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 0x55fdb9e353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e35920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69a88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 0x55fdb9e353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41908/0 (socket says 172.21.15.19:41908) 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69a88640 1 -- 172.21.15.19:0/1131210380 learned_addr learned my addr 172.21.15.19:0/1131210380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e35920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36350/0 (socket says 172.21.15.19:36350) 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb68a86640 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 -- 172.21.15.19:0/1131210380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 msgr2=0x55fdb9e3c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 -- 172.21.15.19:0/1131210380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 msgr2=0x55fdb9e353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 0x55fdb9e353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69287640 1 -- 172.21.15.19:0/1131210380 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb9aac210 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.911+0000 7fdb69a88640 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 0x55fdb9e353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb69287640 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e35920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb6406b030 tx=0x7fdb6000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad70ee5cecc6adb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb5a7fc640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb500b2920 0x7fdb500b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb4f7fe640 1 -- 172.21.15.19:0/1131210380 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60003bd0 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb4f7fe640 1 -- 172.21.15.19:0/1131210380 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb60015070 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb4f7fe640 1 -- 172.21.15.19:0/1131210380 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004b70 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb5a7fc640 1 -- 172.21.15.19:0/3043174495 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdb540e28f0 con 0x7fdb64106d60 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.912+0000 7fdb69a88640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb500b2920 0x7fdb500b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 -- 172.21.15.19:0/1131210380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 msgr2=0x55fdb9e35920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e35920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb6406b030 tx=0x7fdb6000bb20 comp rx=0 tx=0).stop 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb69a88640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb500b2920 0x7fdb500b4e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55fdb9e36570 tx=0x7fdb5c031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 -- 172.21.15.19:0/1131210380 shutdown_connections 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e35920 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 --2- 172.21.15.19:0/1131210380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a941b0 0x55fdb9e353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 -- 172.21.15.19:0/1131210380 >> 172.21.15.19:0/1131210380 conn(0x55fdb9a88300 msgr2=0x55fdb9e30d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 -- 172.21.15.19:0/1131210380 shutdown_connections 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 -- 172.21.15.19:0/1131210380 wait complete. 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 -- start start 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.913+0000 7fdb7239d740 1 Processor -- start 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e3edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a941b0 0x55fdb9e46190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9a7d790 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9d53890 con 0x55fdb9a941b0 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9e3d270 con 0x55fdb9a734d0 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb7239d740 10 client.0 _ll_register_callbacks cb 0x55fdb9d56d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e3edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e3edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41912/0 (socket says 172.21.15.19:41912) 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 -- 172.21.15.19:0/3277743775 learned_addr learned my addr 172.21.15.19:0/3277743775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69a88640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a941b0 0x55fdb9e46190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb68a86640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 -- 172.21.15.19:0/3277743775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 msgr2=0x55fdb9e3f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 -- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a941b0 msgr2=0x55fdb9e46190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a941b0 0x55fdb9e46190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69287640 1 -- 172.21.15.19:0/3277743775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb9b78620 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.914+0000 7fdb69a88640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a941b0 0x55fdb9e46190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb69287640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e3edd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb60004650 tx=0x7fdb6000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6016d36974c98228 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb4cff9640 1 -- 172.21.15.19:0/3277743775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60028060 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb4cff9640 1 -- 172.21.15.19:0/3277743775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdb60011640 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb60015070 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb4cff9640 1 -- 172.21.15.19:0/3277743775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb6002f050 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 10 client.17403 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fdb9e490c0 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb4cff9640 1 -- 172.21.15.19:0/3277743775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb60015070 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb4cff9640 1 -- 172.21.15.19:0/3277743775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdb60022070 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 10 client.17403 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 20 client.17403 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.915+0000 7fdb7239d740 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a52d80 0x55fdb9e4c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.916+0000 7fdb7239d740 4 client.17403 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.916+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb9da9b80 con 0x55fdb9a52d80 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.916+0000 7fdb68a86640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a52d80 0x55fdb9e4c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.916+0000 7fdb4cff9640 10 client.17403 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.916+0000 7fdb68a86640 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a52d80 0x55fdb9e4c530 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=852370d09efba1e1 server_cookie=d6ecea2edc552b89 in_seq=0 out_seq=0 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb4cff9640 1 -- 172.21.15.19:0/3277743775 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fdb9da9b80 con 0x55fdb9a52d80 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb4cff9640 10 client.17403 handle_command_reply: tid=0 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 client.17403 shutdown 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 msgr2=0x55fdb9e3edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e3edd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb60004650 tx=0x7fdb6000bf80 comp rx=0 tx=0).stop 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 shutdown_connections 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a52d80 0x55fdb9e4c530 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a941b0 0x55fdb9e46190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e3f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb4cff9640 0 client.17403 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 --2- 172.21.15.19:0/3277743775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e3edd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 >> 172.21.15.19:0/3277743775 conn(0x55fdb9a88300 msgr2=0x55fdb9e33c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 shutdown_connections 2024-05-17T09:30:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 1 -- 172.21.15.19:0/3277743775 wait complete. 2024-05-17T09:30:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.917+0000 7fdb7239d740 20 client.17403 trim_cache size 0 max 16384 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 Processor -- start 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 -- start start 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 0x55fdb9e398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e45ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a734d0 0x55fdb9e50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641af2f0 con 0x55fdb9a734d0 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641acbc0 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb7239d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641aaa10 con 0x55fdb9a88300 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 0x55fdb9e398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69a88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a734d0 0x55fdb9e50c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 0x55fdb9e398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:48102/0 (socket says 172.21.15.19:48102) 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 -- 172.21.15.19:0/4112052229 learned_addr learned my addr 172.21.15.19:0/4112052229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb68a86640 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e45ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 -- 172.21.15.19:0/4112052229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 msgr2=0x55fdb9e45ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e45ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 -- 172.21.15.19:0/4112052229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a734d0 msgr2=0x55fdb9e50c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a734d0 0x55fdb9e50c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 -- 172.21.15.19:0/4112052229 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb9e312c0 con 0x55fdb9a88300 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.935+0000 7fdb69287640 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 0x55fdb9e398f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000bb60 tx=0x7fdb60005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6dda8818961955 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb4effd640 1 -- 172.21.15.19:0/4112052229 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60028050 con 0x55fdb9a88300 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb4effd640 1 -- 172.21.15.19:0/4112052229 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb60015070 con 0x55fdb9a88300 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb4effd640 1 -- 172.21.15.19:0/4112052229 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004980 con 0x55fdb9a88300 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 -- 172.21.15.19:0/4112052229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 msgr2=0x55fdb9e398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 0x55fdb9e398f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000bb60 tx=0x7fdb60005c50 comp rx=0 tx=0).stop 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 -- 172.21.15.19:0/4112052229 shutdown_connections 2024-05-17T09:30:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a734d0 0x55fdb9e50c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 --2- 172.21.15.19:0/4112052229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a88300 0x55fdb9e398f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 -- 172.21.15.19:0/4112052229 >> 172.21.15.19:0/4112052229 conn(0x55fdb9e3f4e0 msgr2=0x55fdb9e34560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 -- 172.21.15.19:0/4112052229 shutdown_connections 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.936+0000 7fdb7239d740 1 -- 172.21.15.19:0/4112052229 wait complete. 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 -- start start 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 Processor -- start 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 0x55fdb9e449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641ab080 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641ac4f0 con 0x55fdb9a88300 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641ab750 con 0x55fdb9a734d0 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb7239d740 10 client.0 _ll_register_callbacks cb 0x7fdb600044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.937+0000 7fdb69287640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e43530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69a88640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 0x55fdb9e449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e43530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41922/0 (socket says 172.21.15.19:41922) 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69a88640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 0x55fdb9e449a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36362/0 (socket says 172.21.15.19:36362) 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 -- 172.21.15.19:0/663756463 learned_addr learned my addr 172.21.15.19:0/663756463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb68a86640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 -- 172.21.15.19:0/663756463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 msgr2=0x55fdb9e43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e43a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 -- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 msgr2=0x55fdb9e449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 0x55fdb9e449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 -- 172.21.15.19:0/663756463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb9e44f60 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69a88640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 0x55fdb9e449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb69287640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e43530 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdb6001b040 tx=0x7fdb6000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db00a53ea2cbadb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb4cff9640 1 -- 172.21.15.19:0/663756463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60028030 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb4cff9640 1 -- 172.21.15.19:0/663756463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdb60019cd0 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.938+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb60015070 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb4cff9640 1 -- 172.21.15.19:0/663756463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb60049050 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 10 client.17409 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fdb9e461f0 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb4cff9640 1 -- 172.21.15.19:0/663756463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb60015070 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb4cff9640 1 -- 172.21.15.19:0/663756463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdb60044030 con 0x55fdb9a6b2f0 2024-05-17T09:30:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 10 client.17409 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 10 client.17409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 20 client.17409 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a941b0 0x55fdb9eb7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 4 client.17409 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdb9da3360 con 0x55fdb9a941b0 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.939+0000 7fdb68a86640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a941b0 0x55fdb9eb7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.940+0000 7fdb4cff9640 10 client.17409 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.940+0000 7fdb68a86640 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a941b0 0x55fdb9eb7030 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45aa35b9bcb2c22d server_cookie=a9bf8ea897623184 in_seq=0 out_seq=0 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.964+0000 7fdb4cff9640 1 -- 172.21.15.19:0/663756463 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fdb9da3360 con 0x55fdb9a941b0 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.964+0000 7fdb4cff9640 10 client.17409 handle_command_reply: tid=0 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.964+0000 7fdb7239d740 1 client.17409 shutdown 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 msgr2=0x55fdb9e43530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e43530 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdb6001b040 tx=0x7fdb6000bf80 comp rx=0 tx=0).stop 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 shutdown_connections 2024-05-17T09:30:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fdb9a941b0 0x55fdb9eb7030 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb9a88300 0x55fdb9e449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb4cff9640 0 client.17409 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdb9a734d0 0x55fdb9e43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 --2- 172.21.15.19:0/663756463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdb9a6b2f0 0x55fdb9e43530 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 >> 172.21.15.19:0/663756463 conn(0x55fdb9e3f4e0 msgr2=0x55fdb9e342c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 shutdown_connections 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 1 -- 172.21.15.19:0/663756463 wait complete. 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb7239d740 20 client.17409 trim_cache size 0 max 16384 2024-05-17T09:30:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:30:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:30:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:30:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb500b2920 msgr2=0x7fdb500b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.965+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb500b2920 0x7fdb500b4e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55fdb9e36570 tx=0x7fdb5c031000 comp rx=0 tx=0).stop 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 msgr2=0x7fdb6419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb6419a6b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e9f0 tx=0x7fdb5400eeb0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 shutdown_connections 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdb500b2920 0x7fdb500b4e30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb641098d0 0x7fdb641a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb64106d60 0x7fdb6419a6b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 --2- 172.21.15.19:0/3043174495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb64104350 0x7fdb6419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 >> 172.21.15.19:0/3043174495 conn(0x7fdb640fdd30 msgr2=0x7fdb64105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 shutdown_connections 2024-05-17T09:30:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:40.966+0000 7fdb6b4eb640 1 -- 172.21.15.19:0/3043174495 wait complete. 2024-05-17T09:30:41.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 175440 1.79 MB/sec execute 399 sec latency 22.390 ms 2024-05-17T09:30:42.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 176047 1.78 MB/sec execute 400 sec latency 22.748 ms 2024-05-17T09:30:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:42 smithi079 ceph-mon[35502]: pgmap v766: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-17T09:30:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:42 smithi069 ceph-mon[35081]: pgmap v766: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-17T09:30:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:42 smithi019 ceph-mon[31620]: pgmap v766: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-17T09:30:42.817 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:30:42.818 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.8 2024-05-17T09:30:42.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:30:43.092 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:30:43.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 176579 1.79 MB/sec execute 401 sec latency 15.493 ms 2024-05-17T09:30:44.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 177141 1.80 MB/sec execute 402 sec latency 15.242 ms 2024-05-17T09:30:44.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:44 smithi019 ceph-mon[31620]: pgmap v767: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-17T09:30:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:44 smithi069 ceph-mon[35081]: pgmap v767: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-17T09:30:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:44 smithi079 ceph-mon[35502]: pgmap v767: 129 pgs: 129 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-17T09:30:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 -- 172.21.15.19:0/3471228049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 msgr2=0x7f3c50103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 --2- 172.21.15.19:0/3471228049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c50103fb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009990 tx=0x7f3c4c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 -- 172.21.15.19:0/3471228049 shutdown_connections 2024-05-17T09:30:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 --2- 172.21.15.19:0/3471228049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c501087a0 0x7f3c501100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 --2- 172.21.15.19:0/3471228049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c50103fb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 --2- 172.21.15.19:0/3471228049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 0x7f3c50108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.018+0000 7f3c576bd700 1 -- 172.21.15.19:0/3471228049 >> 172.21.15.19:0/3471228049 conn(0x7f3c500fe4f0 msgr2=0x7f3c50100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.019+0000 7f3c576bd700 1 -- 172.21.15.19:0/3471228049 shutdown_connections 2024-05-17T09:30:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.019+0000 7f3c576bd700 1 -- 172.21.15.19:0/3471228049 wait complete. 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.019+0000 7f3c576bd700 1 Processor -- start 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.019+0000 7f3c576bd700 1 -- start start 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c576bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c5019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c576bd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 0x7f3c5019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c576bd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c501087a0 0x7f3c5019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c576bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50113240 con 0x7f3c50103ba0 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c576bd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501130c0 con 0x7f3c501087a0 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c576bd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501133c0 con 0x7f3c50107e50 2024-05-17T09:30:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c5019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c5019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53602/0 (socket says 172.21.15.19:53602) 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 -- 172.21.15.19:0/2066512836 learned_addr learned my addr 172.21.15.19:0/2066512836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c54c58700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 0x7f3c5019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55c5a700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c501087a0 0x7f3c5019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 -- 172.21.15.19:0/2066512836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 msgr2=0x7f3c5019c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 0x7f3c5019c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 -- 172.21.15.19:0/2066512836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c501087a0 msgr2=0x7f3c5019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c501087a0 0x7f3c5019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 -- 172.21.15.19:0/2066512836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c501a6ef0 con 0x7f3c50103ba0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c54c58700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 0x7f3c5019c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.020+0000 7f3c55459700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c5019bba0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006010 tx=0x7f3c4c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.021+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c027070 con 0x7f3c50103ba0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.021+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c4c022470 con 0x7f3c50103ba0 2024-05-17T09:30:45.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.021+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c501a7180 con 0x7f3c50103ba0 2024-05-17T09:30:45.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.021+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c017650 con 0x7f3c50103ba0 2024-05-17T09:30:45.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.021+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c501a75d0 con 0x7f3c50103ba0 2024-05-17T09:30:45.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.022+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c500512b0 con 0x7f3c50103ba0 2024-05-17T09:30:45.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.024+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3c4c02a070 con 0x7f3c50103ba0 2024-05-17T09:30:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.024+0000 7f3c467fc700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c3c0b2880 0x7f3c3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.025+0000 7f3c54c58700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c3c0b2880 0x7f3c3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.025+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c4c0e37f0 con 0x7f3c50103ba0 2024-05-17T09:30:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.025+0000 7f3c54c58700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c3c0b2880 0x7f3c3c0b4d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c501065b0 tx=0x7f3c400073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.028+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c4c05fdf0 con 0x7f3c50103ba0 2024-05-17T09:30:45.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.330+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c5002e5e0 con 0x7f3c50103ba0 2024-05-17T09:30:45.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.331+0000 7f3c467fc700 1 -- 172.21.15.19:0/2066512836 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f3c4c013070 con 0x7f3c50103ba0 2024-05-17T09:30:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:30:45.331 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:30:45.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c3c0b2880 msgr2=0x7f3c3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:45.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c3c0b2880 0x7f3c3c0b4d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c501065b0 tx=0x7f3c400073d0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 msgr2=0x7f3c5019bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:45.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c5019bba0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006010 tx=0x7f3c4c0057c0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 shutdown_connections 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c501087a0 0x7f3c5019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c50103ba0 0x7f3c5019bba0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3c3c0b2880 0x7f3c3c0b4d90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 --2- 172.21.15.19:0/2066512836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c50107e50 0x7f3c5019c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 >> 172.21.15.19:0/2066512836 conn(0x7f3c500fe4f0 msgr2=0x7f3c501014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 shutdown_connections 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:45.333+0000 7f3c576bd700 1 -- 172.21.15.19:0/2066512836 wait complete. 2024-05-17T09:30:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:30:45.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 177598 1.80 MB/sec execute 403 sec latency 13.619 ms 2024-05-17T09:30:45.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:45 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2066512836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:30:45.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:45 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2066512836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:30:45.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:45 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2066512836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:30:46.170 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.8 --await-for 45.0' 2024-05-17T09:30:46.170 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.8 --await-for 45.0 2024-05-17T09:30:46.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:46 smithi019 ceph-mon[31620]: pgmap v768: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-05-17T09:30:46.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 177821 1.80 MB/sec execute 404 sec latency 13.458 ms 2024-05-17T09:30:46.453 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:30:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:46 smithi069 ceph-mon[35081]: pgmap v768: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-05-17T09:30:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:46 smithi079 ceph-mon[35502]: pgmap v768: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-05-17T09:30:47.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178192 1.80 MB/sec execute 405 sec latency 15.101 ms 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 -- 172.21.15.19:0/204674722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1045e0 msgr2=0x7f693c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 --2- 172.21.15.19:0/204674722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1045e0 0x7f693c1049f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f692c0099c0 tx=0x7f692c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 -- 172.21.15.19:0/204674722 shutdown_connections 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 --2- 172.21.15.19:0/204674722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c109860 0x7f693c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 --2- 172.21.15.19:0/204674722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1045e0 0x7f693c1049f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 --2- 172.21.15.19:0/204674722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f693c108f10 0x7f693c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 -- 172.21.15.19:0/204674722 >> 172.21.15.19:0/204674722 conn(0x7f693c0fffd0 msgr2=0x7f693c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 -- 172.21.15.19:0/204674722 shutdown_connections 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.203+0000 7f6942591700 1 -- 172.21.15.19:0/204674722 wait complete. 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 Processor -- start 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 -- start start 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f693c1045e0 0x7f693c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 0x7f693c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c109860 0x7f693c0764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f693c114390 con 0x7f693c108f10 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f693c114210 con 0x7f693c109860 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6942591700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f693c114510 con 0x7f693c1045e0 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 0x7f693c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 0x7f693c075fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53612/0 (socket says 172.21.15.19:53612) 2024-05-17T09:30:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 -- 172.21.15.19:0/1467835893 learned_addr learned my addr 172.21.15.19:0/1467835893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f693bfff700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f693c1045e0 0x7f693c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6940b2e700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c109860 0x7f693c0764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 -- 172.21.15.19:0/1467835893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f693c1045e0 msgr2=0x7f693c075a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f693c1045e0 0x7f693c075a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 -- 172.21.15.19:0/1467835893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c109860 msgr2=0x7f693c0764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c109860 0x7f693c0764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.204+0000 7f6933fff700 1 -- 172.21.15.19:0/1467835893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1a9170 con 0x7f693c108f10 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.205+0000 7f6933fff700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 0x7f693c075fb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f692400e970 tx=0x7f692400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.205+0000 7f6939ffb700 1 -- 172.21.15.19:0/1467835893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692400cbd0 con 0x7f693c108f10 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.205+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1a9460 con 0x7f693c108f10 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.205+0000 7f6939ffb700 1 -- 172.21.15.19:0/1467835893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f692400cd70 con 0x7f693c108f10 2024-05-17T09:30:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.205+0000 7f6939ffb700 1 -- 172.21.15.19:0/1467835893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240055e0 con 0x7f693c108f10 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.205+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a99a0 con 0x7f693c108f10 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.207+0000 7f6939ffb700 1 -- 172.21.15.19:0/1467835893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6924004750 con 0x7f693c108f10 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.208+0000 7f6939ffb700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69280b2880 0x7f69280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.208+0000 7f6939ffb700 1 -- 172.21.15.19:0/1467835893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f69240e27b0 con 0x7f693c108f10 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.208+0000 7f693bfff700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69280b2880 0x7f69280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f693bfff700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69280b2880 0x7f69280b4d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f692c000c00 tx=0x7f692c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 Processor -- start 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 -- start start 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580eeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559158016e20 con 0x5591580e86b0 2024-05-17T09:30:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591580069c0 con 0x559157fe7560 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55915800aa00 con 0x559157f69fd0 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6940b2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6940b2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580e8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53626/0 (socket says 172.21.15.19:53626) 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6940b2e700 1 -- 172.21.15.19:0/4124340198 learned_addr learned my addr 172.21.15.19:0/4124340198 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f693bfff700 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.209+0000 7f6933fff700 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580eeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f6940b2e700 1 -- 172.21.15.19:0/4124340198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 msgr2=0x5591580eeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f6940b2e700 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580eeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f6940b2e700 1 -- 172.21.15.19:0/4124340198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 msgr2=0x5591580ee5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f6940b2e700 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580ee5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f6940b2e700 1 -- 172.21.15.19:0/4124340198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559157f56a60 con 0x5591580e86b0 2024-05-17T09:30:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f6940b2e700 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580e8ac0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6934009d50 tx=0x7f69340077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64a6373c268ea4d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f69327fc700 1 -- 172.21.15.19:0/4124340198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934003a00 con 0x5591580e86b0 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f69327fc700 1 -- 172.21.15.19:0/4124340198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6934003ba0 con 0x5591580e86b0 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.210+0000 7f69327fc700 1 -- 172.21.15.19:0/4124340198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934014650 con 0x5591580e86b0 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 -- 172.21.15.19:0/4124340198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 msgr2=0x5591580e8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580e8ac0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6934009d50 tx=0x7f69340077e0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 -- 172.21.15.19:0/4124340198 shutdown_connections 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580e8ac0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580ee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/4124340198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580eeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 -- 172.21.15.19:0/4124340198 >> 172.21.15.19:0/4124340198 conn(0x559157fa8e10 msgr2=0x5591580e35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 -- 172.21.15.19:0/4124340198 shutdown_connections 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 -- 172.21.15.19:0/4124340198 wait complete. 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.211+0000 7f6950fa6b80 1 -- start start 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 Processor -- start 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157f69fd0 0x5591580f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157fe7560 0x5591580f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559157f0d130 con 0x5591580e86b0 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559157fa8950 con 0x559157f69fd0 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5591580fe460 con 0x559157fe7560 2024-05-17T09:30:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6950fa6b80 10 client.0 _ll_register_callbacks cb 0x559157fa1ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6940b2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6940b2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53628/0 (socket says 172.21.15.19:53628) 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6933fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157fe7560 0x5591580f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6940b2e700 1 -- 172.21.15.19:0/3903346393 learned_addr learned my addr 172.21.15.19:0/3903346393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f693bfff700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157f69fd0 0x5591580f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6933fff700 1 -- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157f69fd0 msgr2=0x5591580f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6933fff700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157f69fd0 0x5591580f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6933fff700 1 -- 172.21.15.19:0/3903346393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 msgr2=0x5591580f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6933fff700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6933fff700 1 -- 172.21.15.19:0/3903346393 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559157fa5820 con 0x559157fe7560 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.212+0000 7f6940b2e700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6933fff700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157fe7560 0x5591580f2730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69240a6c90 tx=0x7f6924004cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bae905e147c481 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f691b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f691b7fe700 1 -- 172.21.15.19:0/3903346393 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240b21c0 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6934003ba0 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f691b7fe700 1 -- 172.21.15.19:0/3903346393 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f69240b2600 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f691b7fe700 1 -- 172.21.15.19:0/3903346393 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69240a9530 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 10 client.27025 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5591580ffd30 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f691b7fe700 1 -- 172.21.15.19:0/3903346393 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6934003ba0 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f691b7fe700 1 -- 172.21.15.19:0/3903346393 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f692406f1f0 con 0x559157fe7560 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 10 client.27025 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 10 client.27025 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 20 client.27025 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5591581022c0 0x5591581046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 4 client.27025 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.213+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559158018d70 con 0x5591581022c0 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.214+0000 7f693bfff700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5591581022c0 0x5591581046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.214+0000 7f693bfff700 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5591581022c0 0x5591581046e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e568fef50ad4cf46 server_cookie=44ce964f9aa36ea8 in_seq=0 out_seq=0 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.214+0000 7f691b7fe700 10 client.27025 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f691b7fe700 1 -- 172.21.15.19:0/3903346393 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559158018d70 con 0x5591581022c0 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f691b7fe700 10 client.27025 handle_command_reply: tid=0 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 client.27025 shutdown 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157fe7560 msgr2=0x5591580f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157fe7560 0x5591580f2730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69240a6c90 tx=0x7f6924004cc0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 shutdown_connections 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5591581022c0 0x5591581046e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591580e86b0 0x5591580f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157f69fd0 0x5591580f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f691b7fe700 0 client.27025 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3903346393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157fe7560 0x5591580f2730 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 >> 172.21.15.19:0/3903346393 conn(0x559157fa8e10 msgr2=0x5591580e76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 shutdown_connections 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3903346393 wait complete. 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.215+0000 7f6950fa6b80 20 client.27025 trim_cache size 0 max 16384 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.235+0000 7f6950fa6b80 1 Processor -- start 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.235+0000 7f6950fa6b80 1 -- start start 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 0x5591580f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591580e86b0 0x5591580fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b4e80 con 0x559157f69fd0 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b2710 con 0x559157fe7560 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b0560 con 0x5591580e86b0 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 0x5591580f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 0x5591580f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53644/0 (socket says 172.21.15.19:53644) 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 -- 172.21.15.19:0/1309808796 learned_addr learned my addr 172.21.15.19:0/1309808796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6940b2e700 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591580e86b0 0x5591580fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6933fff700 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 -- 172.21.15.19:0/1309808796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591580e86b0 msgr2=0x5591580fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591580e86b0 0x5591580fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 -- 172.21.15.19:0/1309808796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 msgr2=0x5591580f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 -- 172.21.15.19:0/1309808796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591580db670 con 0x559157f69fd0 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f6940b2e700 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591580e86b0 0x5591580fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.236+0000 7f693bfff700 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 0x5591580f42d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f692c0098e0 tx=0x7f692c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b542c57f105fdad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6931ffb700 1 -- 172.21.15.19:0/1309808796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c003e50 con 0x559157f69fd0 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6931ffb700 1 -- 172.21.15.19:0/1309808796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f692c00bc20 con 0x559157f69fd0 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6931ffb700 1 -- 172.21.15.19:0/1309808796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c00fb20 con 0x559157f69fd0 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 -- 172.21.15.19:0/1309808796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 msgr2=0x5591580f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 0x5591580f42d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f692c0098e0 tx=0x7f692c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 -- 172.21.15.19:0/1309808796 shutdown_connections 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5591580e86b0 0x5591580fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157f69fd0 0x5591580f42d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/1309808796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559157fe7560 0x5591580f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 -- 172.21.15.19:0/1309808796 >> 172.21.15.19:0/1309808796 conn(0x559157fa8e10 msgr2=0x5591580fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 -- 172.21.15.19:0/1309808796 shutdown_connections 2024-05-17T09:30:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.237+0000 7f6950fa6b80 1 -- 172.21.15.19:0/1309808796 wait complete. 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 -- start start 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 Processor -- start 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580f7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 0x5591580f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591580e86b0 0x5591580f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b0bd0 con 0x559157fe7560 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b2040 con 0x5591580e86b0 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b12a0 con 0x559157f69fd0 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6950fa6b80 10 client.0 _ll_register_callbacks cb 0x559157fa1ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 0x5591580f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 0x5591580f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53652/0 (socket says 172.21.15.19:53652) 2024-05-17T09:30:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6933fff700 1 -- 172.21.15.19:0/3778315649 learned_addr learned my addr 172.21.15.19:0/3778315649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.238+0000 7f6940b2e700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591580e86b0 0x5591580f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f693bfff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6933fff700 1 -- 172.21.15.19:0/3778315649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 msgr2=0x5591580f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6933fff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6933fff700 1 -- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591580e86b0 msgr2=0x5591580f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6933fff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591580e86b0 0x5591580f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6933fff700 1 -- 172.21.15.19:0/3778315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915816ce50 con 0x559157fe7560 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6940b2e700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591580e86b0 0x5591580f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f693bfff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6933fff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 0x5591580f78d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f69240a5aa0 tx=0x7f692400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2694e21356ee0db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f691b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f691b7fe700 1 -- 172.21.15.19:0/3778315649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692406d070 con 0x559157fe7560 2024-05-17T09:30:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f691b7fe700 1 -- 172.21.15.19:0/3778315649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f69240a6010 con 0x559157fe7560 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692c00bc20 con 0x559157fe7560 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f691b7fe700 1 -- 172.21.15.19:0/3778315649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69240b2880 con 0x559157fe7560 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6950fa6b80 10 client.17433 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55915816e070 con 0x559157fe7560 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.239+0000 7f691b7fe700 1 -- 172.21.15.19:0/3778315649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6924086030 con 0x559157fe7560 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f691b7fe700 1 -- 172.21.15.19:0/3778315649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f69240a5390 con 0x559157fe7560 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f6950fa6b80 10 client.17433 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f6950fa6b80 10 client.17433 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f6950fa6b80 20 client.17433 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55915816e790 0x559158170bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f6950fa6b80 4 client.17433 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.8, "await_for": 45.0}] 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.8, "await_for": 45.0}) -- 0x559158012550 con 0x55915816e790 2024-05-17T09:30:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f693bfff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55915816e790 0x559158170bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 38, 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "27b5ba51": { 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 38, 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:30:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.8, 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:30:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f693bfff700 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55915816e790 0x559158170bb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151e2c7e8b7def61 server_cookie=985c5a134ccc6bf7 in_seq=0 out_seq=0 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.240+0000 7f691b7fe700 10 client.17433 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.314+0000 7f691b7fe700 1 -- 172.21.15.19:0/3778315649 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559158012550 con 0x55915816e790 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.314+0000 7f691b7fe700 10 client.17433 handle_command_reply: tid=0 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.314+0000 7f6950fa6b80 1 client.17433 shutdown 2024-05-17T09:30:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 msgr2=0x5591580f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 0x5591580f78d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f69240a5aa0 tx=0x7f692400cf20 comp rx=0 tx=0).stop 2024-05-17T09:30:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 shutdown_connections 2024-05-17T09:30:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559157f69fd0 0x5591580f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55915816e790 0x559158170bb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591580e86b0 0x5591580f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 --2- 172.21.15.19:0/3778315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559157fe7560 0x5591580f78d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f691b7fe700 0 client.17433 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 >> 172.21.15.19:0/3778315649 conn(0x559157fa8e10 msgr2=0x5591580f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 shutdown_connections 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 1 -- 172.21.15.19:0/3778315649 wait complete. 2024-05-17T09:30:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.315+0000 7f6950fa6b80 20 client.17433 trim_cache size 0 max 16384 2024-05-17T09:30:48.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69280b2880 msgr2=0x7f69280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69280b2880 0x7f69280b4d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f692c000c00 tx=0x7f692c01a040 comp rx=0 tx=0).stop 2024-05-17T09:30:48.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 msgr2=0x7f693c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 0x7f693c075fb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f692400e970 tx=0x7f692400ec80 comp rx=0 tx=0).stop 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 shutdown_connections 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f69280b2880 0x7f69280b4d90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f693c1045e0 0x7f693c075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c109860 0x7f693c0764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 --2- 172.21.15.19:0/1467835893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c108f10 0x7f693c075fb0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 >> 172.21.15.19:0/1467835893 conn(0x7f693c0fffd0 msgr2=0x7f693c1072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 shutdown_connections 2024-05-17T09:30:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:30:48.316+0000 7f6942591700 1 -- 172.21.15.19:0/1467835893 wait complete. 2024-05-17T09:30:48.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.80 MB/sec execute 406 sec latency 104.400 ms 2024-05-17T09:30:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:48 smithi069 ceph-mon[35081]: pgmap v769: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:30:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:48 smithi019 ceph-mon[31620]: pgmap v769: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:30:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:48 smithi079 ceph-mon[35502]: pgmap v769: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:30:49.121 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27b5ba51', quiesce took 6.3 seconds. Will release after: 16.5 2024-05-17T09:30:49.443 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.79 MB/sec execute 407 sec latency 1104.482 ms 2024-05-17T09:30:50.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.79 MB/sec execute 408 sec latency 2104.586 ms 2024-05-17T09:30:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:50 smithi069 ceph-mon[35081]: pgmap v770: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:30:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:50 smithi019 ceph-mon[31620]: pgmap v770: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:30:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:50 smithi079 ceph-mon[35502]: pgmap v770: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:30:51.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.78 MB/sec execute 409 sec latency 3104.643 ms 2024-05-17T09:30:52.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.78 MB/sec execute 410 sec latency 4104.747 ms 2024-05-17T09:30:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:52 smithi069 ceph-mon[35081]: pgmap v771: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:30:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:52 smithi019 ceph-mon[31620]: pgmap v771: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:30:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:52 smithi079 ceph-mon[35502]: pgmap v771: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:30:53.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.77 MB/sec execute 411 sec latency 5104.858 ms 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 -- 172.21.15.19:0/410140845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 msgr2=0x7fa9d4103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/410140845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d4103820 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009900 tx=0x7fa9c402a410 comp rx=0 tx=0).stop 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 -- 172.21.15.19:0/410140845 shutdown_connections 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/410140845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d4108a60 0x7fa9d410ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/410140845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d4103d60 0x7fa9d4104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/410140845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d4103820 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 -- 172.21.15.19:0/410140845 >> 172.21.15.19:0/410140845 conn(0x7fa9d40fdd10 msgr2=0x7fa9d41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 -- 172.21.15.19:0/410140845 shutdown_connections 2024-05-17T09:30:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.080+0000 7fa9d9f72640 1 -- 172.21.15.19:0/410140845 wait complete. 2024-05-17T09:30:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.081+0000 7fa9d9f72640 1 Processor -- start 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.081+0000 7fa9d9f72640 1 -- start start 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d9f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d9f72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d4103d60 0x7fa9d419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d9f72640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d4108a60 0x7fa9d41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d9f72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d410e4d0 con 0x7fa9d4102b20 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d9f72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d410e350 con 0x7fa9d4103d60 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d9f72640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d410e650 con 0x7fa9d4108a60 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52772/0 (socket says 172.21.15.19:52772) 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 -- 172.21.15.19:0/1316129006 learned_addr learned my addr 172.21.15.19:0/1316129006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d4103d60 0x7fa9d419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d4108a60 0x7fa9d41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 -- 172.21.15.19:0/1316129006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d4108a60 msgr2=0x7fa9d41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d4108a60 0x7fa9d41a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 -- 172.21.15.19:0/1316129006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d4103d60 msgr2=0x7fa9d419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d4103d60 0x7fa9d419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 -- 172.21.15.19:0/1316129006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d41a1d10 con 0x7fa9d4102b20 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.082+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d419a1d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4005760 tx=0x7fa9c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.083+0000 7fa9d0ff9640 1 -- 172.21.15.19:0/1316129006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c403f030 con 0x7fa9d4102b20 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.083+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d41a1fa0 con 0x7fa9d4102b20 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.083+0000 7fa9d0ff9640 1 -- 172.21.15.19:0/1316129006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9c4004440 con 0x7fa9d4102b20 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.083+0000 7fa9d0ff9640 1 -- 172.21.15.19:0/1316129006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4035070 con 0x7fa9d4102b20 2024-05-17T09:30:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.083+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a2400 con 0x7fa9d4102b20 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.085+0000 7fa9e0e27740 1 Processor -- start 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.085+0000 7fa9e0e27740 1 -- start start 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.085+0000 7fa9d0ff9640 1 -- 172.21.15.19:0/1316129006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa9c4042030 con 0x7fa9d4102b20 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d31920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d38820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c589ca3c30 con 0x55c58996f4d0 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c589c937d0 con 0x55c5899672f0 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c589c97810 con 0x55c5899901b0 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d38820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d38820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52780/0 (socket says 172.21.15.19:52780) 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/3100864757 learned_addr learned my addr 172.21.15.19:0/3100864757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d31920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/3100864757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 msgr2=0x55c589d313a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d313a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/3100864757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 msgr2=0x55c589d31920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d31920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/3100864757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5899a8210 con 0x55c58996f4d0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d31920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.086+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d38820 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800bcb0 tx=0x7fa9c80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690929a90cf939d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9d0ff9640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9bc0b28a0 0x7fa9bc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9b9ffb640 1 -- 172.21.15.19:0/3100864757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c801c480 con 0x55c58996f4d0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9bc0b28a0 0x7fa9bc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9b9ffb640 1 -- 172.21.15.19:0/3100864757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9c801cab0 con 0x55c58996f4d0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9b9ffb640 1 -- 172.21.15.19:0/3100864757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8011b90 con 0x55c58996f4d0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9d0ff9640 1 -- 172.21.15.19:0/1316129006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa9c4102930 con 0x7fa9d4102b20 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9e0e27740 1 -- 172.21.15.19:0/3100864757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 msgr2=0x55c589d38820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9bc0b28a0 0x7fa9bc0b4db0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55c589d32570 tx=0x7fa9cc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.087+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d38820 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800bcb0 tx=0x7fa9c80107a0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 -- 172.21.15.19:0/3100864757 shutdown_connections 2024-05-17T09:30:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d38820 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d31920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/3100864757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 -- 172.21.15.19:0/3100864757 >> 172.21.15.19:0/3100864757 conn(0x55c589984300 msgr2=0x55c589d2cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 -- 172.21.15.19:0/3100864757 shutdown_connections 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 -- 172.21.15.19:0/3100864757 wait complete. 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 -- start start 2024-05-17T09:30:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.088+0000 7fa9e0e27740 1 Processor -- start 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c589979790 con 0x55c58996f4d0 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c589c4f890 con 0x55c5899672f0 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c589d39270 con 0x55c5899901b0 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9e0e27740 10 client.0 _ll_register_callbacks cb 0x55c589c52d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52786/0 (socket says 172.21.15.19:52786) 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2615063656 learned_addr learned my addr 172.21.15.19:0/2615063656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2615063656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 msgr2=0x55c589d42190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d42190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 msgr2=0x55c589d3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2615063656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c589a74620 con 0x55c58996f4d0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3b230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa93a09dfa0a644 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.089+0000 7fa9b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2615063656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c802a060 con 0x55c58996f4d0 2024-05-17T09:30:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2615063656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa9c8004030 con 0x55c58996f4d0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c801cab0 con 0x55c58996f4d0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2615063656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c8031050 con 0x55c58996f4d0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 10 client.17445 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c589d45140 con 0x55c58996f4d0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2615063656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9c801cab0 con 0x55c58996f4d0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2615063656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa9c8011b00 con 0x55c58996f4d0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 10 client.17445 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 20 client.17445 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c58994ed80 0x55c589d485b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 4 client.17445 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.090+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c589ca5b80 con 0x55c58994ed80 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.091+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c58994ed80 0x55c589d485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.091+0000 7fa9b2ffd640 10 client.17445 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.091+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c58994ed80 0x55c589d485b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990c10d56a1c86bf server_cookie=ec60b6d0d1859004 in_seq=0 out_seq=0 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2615063656 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c589ca5b80 con 0x55c58994ed80 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9b2ffd640 10 client.17445 handle_command_reply: tid=0 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 client.17445 shutdown 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 msgr2=0x55c589d3b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3b230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c800bfd0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 shutdown_connections 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c58994ed80 0x55c589d485b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899901b0 0x55c589d42190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3b230 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2615063656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899672f0 0x55c589d3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9b2ffd640 0 client.17445 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 >> 172.21.15.19:0/2615063656 conn(0x55c589984300 msgr2=0x55c589d2bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 shutdown_connections 2024-05-17T09:30:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2615063656 wait complete. 2024-05-17T09:30:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.092+0000 7fa9e0e27740 20 client.17445 trim_cache size 0 max 16384 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.107+0000 7fa9e0e27740 1 Processor -- start 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.107+0000 7fa9e0e27740 1 -- start start 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 0x55c589d2ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c58996f4d0 0x55c589d420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589d425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41af200 con 0x55c5899672f0 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41acad0 con 0x55c5899901b0 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41aa920 con 0x55c58996f4d0 2024-05-17T09:30:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 0x55c589d2ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 0x55c589d2ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52790/0 (socket says 172.21.15.19:52790) 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 -- 172.21.15.19:0/2234978570 learned_addr learned my addr 172.21.15.19:0/2234978570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c58996f4d0 0x55c589d420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 -- 172.21.15.19:0/2234978570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c58996f4d0 msgr2=0x55c589d420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c58996f4d0 0x55c589d420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 -- 172.21.15.19:0/2234978570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 msgr2=0x55c589d425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589d425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589d425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 -- 172.21.15.19:0/2234978570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c589d3b8f0 con 0x55c5899672f0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589d425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.108+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 0x55c589d2ba40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40c9950 tx=0x7fa9c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5ec95f207270cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9b97fa640 1 -- 172.21.15.19:0/2234978570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4049040 con 0x55c5899672f0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9b97fa640 1 -- 172.21.15.19:0/2234978570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9c40eae50 con 0x55c5899672f0 2024-05-17T09:30:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9b97fa640 1 -- 172.21.15.19:0/2234978570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40d2a00 con 0x55c5899672f0 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2234978570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 msgr2=0x55c589d2ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 0x55c589d2ba40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40c9950 tx=0x7fa9c4000bc0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2234978570 shutdown_connections 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589d425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c58996f4d0 0x55c589d420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2234978570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5899672f0 0x55c589d2ba40 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2234978570 >> 172.21.15.19:0/2234978570 conn(0x55c589984300 msgr2=0x55c589d2b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2234978570 shutdown_connections 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.109+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2234978570 wait complete. 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 -- start start 2024-05-17T09:30:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 Processor -- start 2024-05-17T09:30:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899672f0 0x55c589d3f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589daebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41aaf90 con 0x55c58996f4d0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41ac400 con 0x55c5899901b0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41ab660 con 0x55c5899672f0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9e0e27740 10 client.0 _ll_register_callbacks cb 0x7fa9c8005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.110+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52796/0 (socket says 172.21.15.19:52796) 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2052047636 learned_addr learned my addr 172.21.15.19:0/2052047636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899672f0 0x55c589d3f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589daebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2052047636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899672f0 msgr2=0x55c589d3f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899672f0 0x55c589d3f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 msgr2=0x55c589daebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:30:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589daebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 -- 172.21.15.19:0/2052047636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c589daf100 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d3fff640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589daebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9d2ffd640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3fd30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8011010 tx=0x7fa9c800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b81d5bc83ff025 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2052047636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c802a070 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2052047636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa9c8022040 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c40eae50 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2052047636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c8041050 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9e0e27740 10 client.17451 fetch_fsmap learned FSMap version 32 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.111+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c589db0190 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2052047636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9c40eae50 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2052047636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa9c80036c0 con 0x55c58996f4d0 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9e0e27740 10 client.17451 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:30:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9e0e27740 10 client.17451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9e0e27740 20 client.17451 populate_metadata read hostname 'smithi019' 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c5899241c0 0x55c589db2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9e0e27740 4 client.17451 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c589c9f360 con 0x55c5899241c0 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.112+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c5899241c0 0x55c589db2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.113+0000 7fa9b2ffd640 10 client.17451 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.113+0000 7fa9d37fe640 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c5899241c0 0x55c589db2b80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e70c2e3c06097b server_cookie=6ec454697bf41cdc in_seq=0 out_seq=0 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9b2ffd640 1 -- 172.21.15.19:0/2052047636 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c589c9f360 con 0x55c5899241c0 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9b2ffd640 10 client.17451 handle_command_reply: tid=0 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 client.17451 shutdown 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 msgr2=0x55c589d3fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3fd30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8011010 tx=0x7fa9c800b7a0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 shutdown_connections 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55c5899241c0 0x55c589db2b80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5899901b0 0x55c589daebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c58996f4d0 0x55c589d3fd30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 --2- 172.21.15.19:0/2052047636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5899672f0 0x55c589d3f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9b2ffd640 0 client.17451 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 >> 172.21.15.19:0/2052047636 conn(0x55c589984300 msgr2=0x55c589d2b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 shutdown_connections 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.140+0000 7fa9e0e27740 1 -- 172.21.15.19:0/2052047636 wait complete. 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9e0e27740 20 client.17451 trim_cache size 0 max 16384 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:30:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:30:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9bc0b28a0 msgr2=0x7fa9bc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9bc0b28a0 0x7fa9bc0b4db0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55c589d32570 tx=0x7fa9cc009500 comp rx=0 tx=0).stop 2024-05-17T09:30:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 msgr2=0x7fa9d419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:30:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d419a1d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4005760 tx=0x7fa9c4004040 comp rx=0 tx=0).stop 2024-05-17T09:30:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 shutdown_connections 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d4108a60 0x7fa9d41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d4103d60 0x7fa9d419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa9bc0b28a0 0x7fa9bc0b4db0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 --2- 172.21.15.19:0/1316129006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d4102b20 0x7fa9d419a1d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 >> 172.21.15.19:0/1316129006 conn(0x7fa9d40fdd10 msgr2=0x7fa9d4109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 shutdown_connections 2024-05-17T09:30:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:30:54.141+0000 7fa9d9f72640 1 -- 172.21.15.19:0/1316129006 wait complete. 2024-05-17T09:30:54.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.77 MB/sec execute 412 sec latency 6104.923 ms 2024-05-17T09:30:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:54 smithi069 ceph-mon[35081]: pgmap v772: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-05-17T09:30:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:54 smithi019 ceph-mon[31620]: pgmap v772: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-05-17T09:30:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:54 smithi079 ceph-mon[35502]: pgmap v772: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-05-17T09:30:55.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.77 MB/sec execute 413 sec latency 7105.031 ms 2024-05-17T09:30:56.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.76 MB/sec execute 414 sec latency 8105.138 ms 2024-05-17T09:30:56.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:56 smithi069 ceph-mon[35081]: pgmap v773: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 876 KiB/s wr, 106 op/s 2024-05-17T09:30:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:56 smithi019 ceph-mon[31620]: pgmap v773: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 876 KiB/s wr, 106 op/s 2024-05-17T09:30:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:56 smithi079 ceph-mon[35502]: pgmap v773: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 876 KiB/s wr, 106 op/s 2024-05-17T09:30:57.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.76 MB/sec execute 415 sec latency 9105.202 ms 2024-05-17T09:30:58.444 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.75 MB/sec execute 416 sec latency 10105.343 ms 2024-05-17T09:30:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:30:58 smithi069 ceph-mon[35081]: pgmap v774: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 715 KiB/s wr, 82 op/s 2024-05-17T09:30:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:30:58 smithi019 ceph-mon[31620]: pgmap v774: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 715 KiB/s wr, 82 op/s 2024-05-17T09:30:58.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:30:58 smithi079 ceph-mon[35502]: pgmap v774: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 715 KiB/s wr, 82 op/s 2024-05-17T09:30:59.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.75 MB/sec execute 417 sec latency 11105.456 ms 2024-05-17T09:31:00.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.74 MB/sec execute 418 sec latency 12105.566 ms 2024-05-17T09:31:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:00 smithi069 ceph-mon[35081]: pgmap v775: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 341 KiB/s wr, 56 op/s 2024-05-17T09:31:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:00 smithi019 ceph-mon[31620]: pgmap v775: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 341 KiB/s wr, 56 op/s 2024-05-17T09:31:00.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:00 smithi079 ceph-mon[35502]: pgmap v775: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 341 KiB/s wr, 56 op/s 2024-05-17T09:31:01.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.74 MB/sec execute 419 sec latency 13105.661 ms 2024-05-17T09:31:02.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.74 MB/sec execute 420 sec latency 14105.760 ms 2024-05-17T09:31:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:02 smithi069 ceph-mon[35081]: pgmap v776: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 122 KiB/s wr, 24 op/s 2024-05-17T09:31:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:02 smithi019 ceph-mon[31620]: pgmap v776: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 122 KiB/s wr, 24 op/s 2024-05-17T09:31:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:02 smithi079 ceph-mon[35502]: pgmap v776: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 122 KiB/s wr, 24 op/s 2024-05-17T09:31:03.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.73 MB/sec execute 421 sec latency 15105.875 ms 2024-05-17T09:31:04.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.73 MB/sec execute 422 sec latency 16105.989 ms 2024-05-17T09:31:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:04 smithi069 ceph-mon[35081]: pgmap v777: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-05-17T09:31:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:04 smithi019 ceph-mon[31620]: pgmap v777: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-05-17T09:31:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:04 smithi079 ceph-mon[35502]: pgmap v777: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-05-17T09:31:05.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.72 MB/sec execute 423 sec latency 17106.084 ms 2024-05-17T09:31:05.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:31:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:31:06.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.72 MB/sec execute 424 sec latency 18106.132 ms 2024-05-17T09:31:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:06 smithi069 ceph-mon[35081]: pgmap v778: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-17T09:31:06.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:06 smithi019 ceph-mon[31620]: pgmap v778: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-17T09:31:06.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:06 smithi079 ceph-mon[35502]: pgmap v778: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 -- 172.21.15.19:0/2381147330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8103d80 msgr2=0x7f8fe8104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/2381147330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8103d80 0x7f8fe8104190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009900 tx=0x7f8fd802a410 comp rx=0 tx=0).stop 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 -- 172.21.15.19:0/2381147330 shutdown_connections 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/2381147330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8108a80 0x7f8fe810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/2381147330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8103d80 0x7f8fe8104190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/2381147330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8102b40 0x7f8fe8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 -- 172.21.15.19:0/2381147330 >> 172.21.15.19:0/2381147330 conn(0x7f8fe80fdd30 msgr2=0x7f8fe81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 -- 172.21.15.19:0/2381147330 shutdown_connections 2024-05-17T09:31:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.247+0000 7f8fef3c0640 1 -- 172.21.15.19:0/2381147330 wait complete. 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 Processor -- start 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 -- start start 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 0x7f8fe819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8103d80 0x7f8fe819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8108a80 0x7f8fe81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e4f0 con 0x7f8fe8102b40 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e370 con 0x7f8fe8103d80 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fef3c0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e670 con 0x7f8fe8108a80 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 0x7f8fe819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 0x7f8fe819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47540/0 (socket says 172.21.15.19:47540) 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 -- 172.21.15.19:0/4128408230 learned_addr learned my addr 172.21.15.19:0/4128408230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fec95b640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8103d80 0x7f8fe819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed95d640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8108a80 0x7f8fe81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 -- 172.21.15.19:0/4128408230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8108a80 msgr2=0x7f8fe81a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8108a80 0x7f8fe81a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 -- 172.21.15.19:0/4128408230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8103d80 msgr2=0x7f8fe819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8103d80 0x7f8fe819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed15c640 1 -- 172.21.15.19:0/4128408230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe81a1d00 con 0x7f8fe8102b40 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fec95b640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8103d80 0x7f8fe819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.248+0000 7f8fed95d640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8108a80 0x7f8fe81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.249+0000 7f8fed15c640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 0x7f8fe819a1c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005fd0 tx=0x7f8fe400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.249+0000 7f8fde7fc640 1 -- 172.21.15.19:0/4128408230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4017070 con 0x7f8fe8102b40 2024-05-17T09:31:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.249+0000 7f8fde7fc640 1 -- 172.21.15.19:0/4128408230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fe4009c30 con 0x7f8fe8102b40 2024-05-17T09:31:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.249+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81a1ff0 con 0x7f8fe8102b40 2024-05-17T09:31:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.249+0000 7f8fde7fc640 1 -- 172.21.15.19:0/4128408230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4003c90 con 0x7f8fe8102b40 2024-05-17T09:31:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.249+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81a24b0 con 0x7f8fe8102b40 2024-05-17T09:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 Processor -- start 2024-05-17T09:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 -- start start 2024-05-17T09:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8fde7fc640 1 -- 172.21.15.19:0/4128408230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8fe4003e30 con 0x7f8fe8102b40 2024-05-17T09:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc18c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdbe2f0 0x555dfc18cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 0x555dfc193c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555dfc0fac30 con 0x555dfbdc64d0 2024-05-17T09:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555dfc0ea7d0 con 0x555dfbde71b0 2024-05-17T09:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8ff6271740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555dfc0ee810 con 0x555dfbdbe2f0 2024-05-17T09:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8fec95b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 0x555dfc193c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8fec95b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 0x555dfc193c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47544/0 (socket says 172.21.15.19:47544) 2024-05-17T09:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8fec95b640 1 -- 172.21.15.19:0/3435308111 learned_addr learned my addr 172.21.15.19:0/3435308111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8fed15c640 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdbe2f0 0x555dfc18cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.252+0000 7f8fed95d640 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc18c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fec95b640 1 -- 172.21.15.19:0/3435308111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdbe2f0 msgr2=0x555dfc18cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fec95b640 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdbe2f0 0x555dfc18cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fec95b640 1 -- 172.21.15.19:0/3435308111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 msgr2=0x555dfc18c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fec95b640 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc18c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fec95b640 1 -- 172.21.15.19:0/3435308111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfbdff210 con 0x555dfbdc64d0 2024-05-17T09:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fed95d640 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc18c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fec95b640 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 0x555dfc193c30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8006010 tx=0x7f8fd8031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07dc9c2829e2fa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fde7fc640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8fd40b2920 0x7f8fd40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fd37fe640 1 -- 172.21.15.19:0/3435308111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8035030 con 0x555dfbdc64d0 2024-05-17T09:31:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fd37fe640 1 -- 172.21.15.19:0/3435308111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fd8004030 con 0x555dfbdc64d0 2024-05-17T09:31:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fd37fe640 1 -- 172.21.15.19:0/3435308111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8003a80 con 0x555dfbdc64d0 2024-05-17T09:31:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fed95d640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8fd40b2920 0x7f8fd40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.253+0000 7f8fde7fc640 1 -- 172.21.15.19:0/4128408230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8fe40e22c0 con 0x7f8fe8102b40 2024-05-17T09:31:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 -- 172.21.15.19:0/3435308111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 msgr2=0x555dfc193c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8fed95d640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8fd40b2920 0x7f8fd40b4e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x555dfc18d980 tx=0x7f8fe0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 0x555dfc193c30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8006010 tx=0x7f8fd8031580 comp rx=0 tx=0).stop 2024-05-17T09:31:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 -- 172.21.15.19:0/3435308111 shutdown_connections 2024-05-17T09:31:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdc64d0 0x555dfc193c30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdbe2f0 0x555dfc18cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 --2- 172.21.15.19:0/3435308111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc18c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 -- 172.21.15.19:0/3435308111 >> 172.21.15.19:0/3435308111 conn(0x555dfbddb300 msgr2=0x555dfc183d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 -- 172.21.15.19:0/3435308111 shutdown_connections 2024-05-17T09:31:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.254+0000 7f8ff6271740 1 -- 172.21.15.19:0/3435308111 wait complete. 2024-05-17T09:31:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 -- start start 2024-05-17T09:31:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 Processor -- start 2024-05-17T09:31:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdbe2f0 0x555dfc196070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc1965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 0x555dfc19d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555dfbdd0790 con 0x555dfbde71b0 2024-05-17T09:31:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555dfc0a6890 con 0x555dfbdbe2f0 2024-05-17T09:31:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555dfc194620 con 0x555dfbdc64d0 2024-05-17T09:31:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8ff6271740 10 client.0 _ll_register_callbacks cb 0x555dfc0a9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8fed95d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 0x555dfc19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8fed95d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 0x555dfc19d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47556/0 (socket says 172.21.15.19:47556) 2024-05-17T09:31:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8fed95d640 1 -- 172.21.15.19:0/2917970424 learned_addr learned my addr 172.21.15.19:0/2917970424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8fec95b640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc1965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.255+0000 7f8fed15c640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdbe2f0 0x555dfc196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed95d640 1 -- 172.21.15.19:0/2917970424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 msgr2=0x555dfc1965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed95d640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc1965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed95d640 1 -- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdbe2f0 msgr2=0x555dfc196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed95d640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdbe2f0 0x555dfc196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed95d640 1 -- 172.21.15.19:0/2917970424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfbecb620 con 0x555dfbde71b0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fec95b640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc1965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed15c640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdbe2f0 0x555dfc196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fed95d640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 0x555dfc19d510 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0012010 tx=0x7f8fe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45cd52323ea51e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/2917970424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000fd60 con 0x555dfbde71b0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/2917970424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8fe001f420 con 0x555dfbde71b0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd8004030 con 0x555dfbde71b0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/2917970424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fe0030050 con 0x555dfbde71b0 2024-05-17T09:31:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8ff6271740 10 client.17463 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555dfc1a0510 con 0x555dfbde71b0 2024-05-17T09:31:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.256+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/2917970424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fd8004030 con 0x555dfbde71b0 2024-05-17T09:31:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/2917970424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8fe001d4d0 con 0x555dfbde71b0 2024-05-17T09:31:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8ff6271740 10 client.17463 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8ff6271740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8ff6271740 20 client.17463 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8ff6271740 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbda5d80 0x555dfc1a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8ff6271740 4 client.17463 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dfc0fcb80 con 0x555dfbda5d80 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.257+0000 7f8fed15c640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbda5d80 0x555dfc1a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8fd0ff9640 10 client.17463 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8fed15c640 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbda5d80 0x555dfc1a3d60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0e7cf144210a00 server_cookie=ba0a511c8e7fd3c2 in_seq=0 out_seq=0 2024-05-17T09:31:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/2917970424 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555dfc0fcb80 con 0x555dfbda5d80 2024-05-17T09:31:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8fd0ff9640 10 client.17463 handle_command_reply: tid=0 2024-05-17T09:31:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8ff6271740 1 client.17463 shutdown 2024-05-17T09:31:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 msgr2=0x555dfc19d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8ff6271740 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 0x555dfc19d510 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0012010 tx=0x7f8fe0004a40 comp rx=0 tx=0).stop 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 shutdown_connections 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8ff6271740 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbda5d80 0x555dfc1a3d60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.258+0000 7f8ff6271740 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbde71b0 0x555dfc19d510 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8ff6271740 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc1965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8ff6271740 1 --2- 172.21.15.19:0/2917970424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdbe2f0 0x555dfc196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8fd0ff9640 0 client.17463 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 >> 172.21.15.19:0/2917970424 conn(0x555dfbddb300 msgr2=0x555dfc186c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 shutdown_connections 2024-05-17T09:31:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8ff6271740 1 -- 172.21.15.19:0/2917970424 wait complete. 2024-05-17T09:31:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.259+0000 7f8ff6271740 20 client.17463 trim_cache size 0 max 16384 2024-05-17T09:31:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.274+0000 7f8ff6271740 1 Processor -- start 2024-05-17T09:31:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.274+0000 7f8ff6271740 1 -- start start 2024-05-17T09:31:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc187600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdc64d0 0x555dfc19d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbde71b0 0x555dfc19d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8ff6271740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81ad140 con 0x555dfbdbe2f0 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8ff6271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81aaa10 con 0x555dfbdc64d0 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8ff6271740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81a8860 con 0x555dfbde71b0 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc187600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc187600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47566/0 (socket says 172.21.15.19:47566) 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 -- 172.21.15.19:0/843937544 learned_addr learned my addr 172.21.15.19:0/843937544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 -- 172.21.15.19:0/843937544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbde71b0 msgr2=0x555dfc19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbde71b0 0x555dfc19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fec95b640 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdc64d0 0x555dfc19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 -- 172.21.15.19:0/843937544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdc64d0 msgr2=0x555dfc19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdc64d0 0x555dfc19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 -- 172.21.15.19:0/843937544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfc196c70 con 0x555dfbdbe2f0 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fed15c640 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc187600 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40dfa10 tx=0x7f8fe40e5e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abd605de30eb4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fd2ffd640 1 -- 172.21.15.19:0/843937544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40035f0 con 0x555dfbdbe2f0 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fd2ffd640 1 -- 172.21.15.19:0/843937544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fe40a9040 con 0x555dfbdbe2f0 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.275+0000 7f8fd2ffd640 1 -- 172.21.15.19:0/843937544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40b2390 con 0x555dfbdbe2f0 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 -- 172.21.15.19:0/843937544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 msgr2=0x555dfc187600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc187600 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40dfa10 tx=0x7f8fe40e5e80 comp rx=0 tx=0).stop 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 -- 172.21.15.19:0/843937544 shutdown_connections 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbde71b0 0x555dfc19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbdc64d0 0x555dfc19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 --2- 172.21.15.19:0/843937544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc187600 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 -- 172.21.15.19:0/843937544 >> 172.21.15.19:0/843937544 conn(0x555dfbddb300 msgr2=0x555dfc190680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 -- 172.21.15.19:0/843937544 shutdown_connections 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 -- 172.21.15.19:0/843937544 wait complete. 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.276+0000 7f8ff6271740 1 -- start start 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 Processor -- start 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc19d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc19d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc20a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81a8ed0 con 0x555dfbdbe2f0 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81aa340 con 0x555dfbde71b0 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81a95a0 con 0x555dfbdc64d0 2024-05-17T09:31:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8ff6271740 10 client.0 _ll_register_callbacks cb 0x7f8fe00125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc19d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47580/0 (socket says 172.21.15.19:47580) 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 -- 172.21.15.19:0/1717464087 learned_addr learned my addr 172.21.15.19:0/1717464087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fec95b640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 -- 172.21.15.19:0/1717464087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 msgr2=0x555dfc19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed95d640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc20a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 -- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 msgr2=0x555dfc20a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc20a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed15c640 1 -- 172.21.15.19:0/1717464087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfc20ae20 con 0x555dfbdbe2f0 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fec95b640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.277+0000 7f8fed95d640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc20a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fed15c640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc19d230 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40a55b0 tx=0x7f8fe40036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3accc0c9c9c8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/1717464087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4009de0 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/1717464087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8fe4091050 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe40a9040 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/1717464087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fe40b2390 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 10 client.17469 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555dfc20c120 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/1717464087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fe40a9040 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/1717464087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8fe40a4ea0 con 0x555dfbdbe2f0 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 10 client.17469 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 20 client.17469 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbd7b1c0 0x555dfc20ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 4 client.17469 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:31:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.278+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555dfc0f6360 con 0x555dfbd7b1c0 2024-05-17T09:31:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.279+0000 7f8fec95b640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbd7b1c0 0x555dfc20ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.279+0000 7f8fd0ff9640 10 client.17469 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.279+0000 7f8fec95b640 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbd7b1c0 0x555dfc20ea10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedb2b19326c075e server_cookie=dc0ecd582f5f174c in_seq=0 out_seq=0 2024-05-17T09:31:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8fd0ff9640 1 -- 172.21.15.19:0/1717464087 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555dfc0f6360 con 0x555dfbd7b1c0 2024-05-17T09:31:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8fd0ff9640 10 client.17469 handle_command_reply: tid=0 2024-05-17T09:31:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 client.17469 shutdown 2024-05-17T09:31:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 msgr2=0x555dfc19d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc19d230 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40a55b0 tx=0x7f8fe40036a0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 shutdown_connections 2024-05-17T09:31:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555dfbd7b1c0 0x555dfc20ea10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555dfbde71b0 0x555dfc20a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555dfbdc64d0 0x555dfc19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8fd0ff9640 0 client.17469 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 --2- 172.21.15.19:0/1717464087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555dfbdbe2f0 0x555dfc19d230 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 >> 172.21.15.19:0/1717464087 conn(0x555dfbddb300 msgr2=0x555dfc1a8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 shutdown_connections 2024-05-17T09:31:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 1 -- 172.21.15.19:0/1717464087 wait complete. 2024-05-17T09:31:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.306+0000 7f8ff6271740 20 client.17469 trim_cache size 0 max 16384 2024-05-17T09:31:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:31:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:31:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:31:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8fd40b2920 msgr2=0x7f8fd40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8fd40b2920 0x7f8fd40b4e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x555dfc18d980 tx=0x7f8fe0009500 comp rx=0 tx=0).stop 2024-05-17T09:31:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 msgr2=0x7f8fe819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 0x7f8fe819a1c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005fd0 tx=0x7f8fe400dde0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 shutdown_connections 2024-05-17T09:31:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe8108a80 0x7f8fe81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fe8103d80 0x7f8fe819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8fd40b2920 0x7f8fd40b4e30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 --2- 172.21.15.19:0/4128408230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe8102b40 0x7f8fe819a1c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 >> 172.21.15.19:0/4128408230 conn(0x7f8fe80fdd30 msgr2=0x7f8fe8109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 shutdown_connections 2024-05-17T09:31:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:07.307+0000 7f8fef3c0640 1 -- 172.21.15.19:0/4128408230 wait complete. 2024-05-17T09:31:07.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.72 MB/sec execute 425 sec latency 19106.196 ms 2024-05-17T09:31:07.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 -- 172.21.15.19:0/1718146359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f4108750 msgr2=0x7f36f41100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 --2- 172.21.15.19:0/1718146359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f4108750 0x7f36f41100b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00b0d0 tx=0x7f36ec00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 -- 172.21.15.19:0/1718146359 shutdown_connections 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 --2- 172.21.15.19:0/1718146359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36f41034d0 0x7f36f41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 --2- 172.21.15.19:0/1718146359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f4108750 0x7f36f41100b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 --2- 172.21.15.19:0/1718146359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36f4107e00 0x7f36f4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 -- 172.21.15.19:0/1718146359 >> 172.21.15.19:0/1718146359 conn(0x7f36f40fef20 msgr2=0x7f36f4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 -- 172.21.15.19:0/1718146359 shutdown_connections 2024-05-17T09:31:07.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.747+0000 7f36fb2de700 1 -- 172.21.15.19:0/1718146359 wait complete. 2024-05-17T09:31:07.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.748+0000 7f36fb2de700 1 Processor -- start 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.748+0000 7f36fb2de700 1 -- start start 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36fb2de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 0x7f36f4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36fb2de700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36f4107e00 0x7f36f419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36fb2de700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36f4108750 0x7f36f419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36fb2de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4113230 con 0x7f36f41034d0 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36fb2de700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36f41130b0 con 0x7f36f4108750 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36fb2de700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36f41133b0 con 0x7f36f4107e00 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 0x7f36f4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 0x7f36f4199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47598/0 (socket says 172.21.15.19:47598) 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 -- 172.21.15.19:0/3466822455 learned_addr learned my addr 172.21.15.19:0/3466822455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f8879700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36f4107e00 0x7f36f419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f987b700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36f4108750 0x7f36f419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 -- 172.21.15.19:0/3466822455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36f4107e00 msgr2=0x7f36f419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36f4107e00 0x7f36f419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 -- 172.21.15.19:0/3466822455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36f4108750 msgr2=0x7f36f419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36f4108750 0x7f36f419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 -- 172.21.15.19:0/3466822455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f419da10 con 0x7f36f41034d0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36f907a700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 0x7f36f4199ca0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.749+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0016070 con 0x7f36f41034d0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.750+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f419dd00 con 0x7f36f41034d0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.750+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f000fd20 con 0x7f36f41034d0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.750+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0019360 con 0x7f36f41034d0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.750+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f419e240 con 0x7f36f41034d0 2024-05-17T09:31:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.751+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f40512b0 con 0x7f36f41034d0 2024-05-17T09:31:07.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.752+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f36f0019070 con 0x7f36f41034d0 2024-05-17T09:31:07.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.753+0000 7f36ea7fc700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36e00b2880 0x7f36e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:07.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.753+0000 7f36f8879700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36e00b2880 0x7f36e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:07.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.753+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f36f00e2050 con 0x7f36f41034d0 2024-05-17T09:31:07.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.754+0000 7f36f8879700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36e00b2880 0x7f36e00b4d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f36f4106740 tx=0x7f36e4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:07.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:07.757+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36f00a99c0 con 0x7f36f41034d0 2024-05-17T09:31:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.031+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36f410b1e0 con 0x7f36f41034d0 2024-05-17T09:31:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.032+0000 7f36ea7fc700 1 -- 172.21.15.19:0/3466822455 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f36f00a92c0 con 0x7f36f41034d0 2024-05-17T09:31:08.032 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:31:08.033 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.034+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36e00b2880 msgr2=0x7f36e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.034+0000 7f36fb2de700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36e00b2880 0x7f36e00b4d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f36f4106740 tx=0x7f36e4010040 comp rx=0 tx=0).stop 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 msgr2=0x7f36f4199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 0x7f36f4199ca0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000dc60 comp rx=0 tx=0).stop 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 shutdown_connections 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36f4108750 0x7f36f419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f41034d0 0x7f36f4199ca0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36e00b2880 0x7f36e00b4d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 --2- 172.21.15.19:0/3466822455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36f4107e00 0x7f36f419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 >> 172.21.15.19:0/3466822455 conn(0x7f36f40fef20 msgr2=0x7f36f41075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 shutdown_connections 2024-05-17T09:31:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:08.035+0000 7f36fb2de700 1 -- 172.21.15.19:0/3466822455 wait complete. 2024-05-17T09:31:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:31:08.445 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.71 MB/sec execute 426 sec latency 20106.332 ms 2024-05-17T09:31:08.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:08 smithi019 ceph-mon[31620]: pgmap v779: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 KiB/s wr, 10 op/s 2024-05-17T09:31:08.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:08 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3466822455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:08 smithi069 ceph-mon[35081]: pgmap v779: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 KiB/s wr, 10 op/s 2024-05-17T09:31:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:08 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3466822455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:08.705 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 27b5ba51 --release --await' 2024-05-17T09:31:08.705 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 27b5ba51 --release --await 2024-05-17T09:31:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:08 smithi079 ceph-mon[35502]: pgmap v779: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 KiB/s wr, 10 op/s 2024-05-17T09:31:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:08 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3466822455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:31:09.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.71 MB/sec execute 427 sec latency 21106.446 ms 2024-05-17T09:31:10.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178548 1.70 MB/sec execute 428 sec latency 22106.798 ms 2024-05-17T09:31:10.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:10 smithi019 ceph-mon[31620]: pgmap v780: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:31:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:10 smithi069 ceph-mon[35081]: pgmap v780: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:31:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:10 smithi079 ceph-mon[35502]: pgmap v780: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-17T09:31:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 -- 172.21.15.19:0/4224116197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374108220 msgr2=0x7f6374108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 --2- 172.21.15.19:0/4224116197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374108220 0x7f6374108630 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6370009990 tx=0x7f6370009ca0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 -- 172.21.15.19:0/4224116197 shutdown_connections 2024-05-17T09:31:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 --2- 172.21.15.19:0/4224116197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6374104310 0x7f6374110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 --2- 172.21.15.19:0/4224116197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374108220 0x7f6374108630 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 --2- 172.21.15.19:0/4224116197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63741039c0 0x7f6374103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 -- 172.21.15.19:0/4224116197 >> 172.21.15.19:0/4224116197 conn(0x7f63740fe500 msgr2=0x7f6374100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 -- 172.21.15.19:0/4224116197 shutdown_connections 2024-05-17T09:31:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.863+0000 7f637b8c4700 1 -- 172.21.15.19:0/4224116197 wait complete. 2024-05-17T09:31:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 Processor -- start 2024-05-17T09:31:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 -- start start 2024-05-17T09:31:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 0x7f637419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6374104310 0x7f637419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6374108220 0x7f637419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6374113240 con 0x7f63741039c0 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63741130c0 con 0x7f6374104310 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f637b8c4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63741133c0 con 0x7f6374108220 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f6379660700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 0x7f637419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f6379660700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 0x7f637419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47610/0 (socket says 172.21.15.19:47610) 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.864+0000 7f6379660700 1 -- 172.21.15.19:0/1982661619 learned_addr learned my addr 172.21.15.19:0/1982661619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379e61700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6374108220 0x7f637419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6374104310 0x7f637419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379660700 1 -- 172.21.15.19:0/1982661619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6374108220 msgr2=0x7f637419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379660700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6374108220 0x7f637419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379660700 1 -- 172.21.15.19:0/1982661619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6374104310 msgr2=0x7f637419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379660700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6374104310 0x7f637419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379660700 1 -- 172.21.15.19:0/1982661619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419fc90 con 0x7f63741039c0 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f6379660700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 0x7f637419adc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6370006010 tx=0x7f63700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f636a7fc700 1 -- 172.21.15.19:0/1982661619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370027070 con 0x7f63741039c0 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f636a7fc700 1 -- 172.21.15.19:0/1982661619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f637000fb00 con 0x7f63741039c0 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637419ff20 con 0x7f63741039c0 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f636a7fc700 1 -- 172.21.15.19:0/1982661619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63700229d0 con 0x7f63741039c0 2024-05-17T09:31:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.865+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a0400 con 0x7f63741039c0 2024-05-17T09:31:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.868+0000 7f636a7fc700 1 -- 172.21.15.19:0/1982661619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f637002a070 con 0x7f63741039c0 2024-05-17T09:31:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.869+0000 7f636a7fc700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f63600b2880 0x7f63600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.869+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f63600b2880 0x7f63600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.869+0000 7f636a7fc700 1 -- 172.21.15.19:0/1982661619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f63700e33e0 con 0x7f63741039c0 2024-05-17T09:31:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.869+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f63600b2880 0x7f63600b4d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6374109950 tx=0x7f636400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.869+0000 7f638a2d9b80 1 Processor -- start 2024-05-17T09:31:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 -- start start 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc051774d0 0x55bc0527ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc0527d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc05281890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc051a6d90 con 0x55bc050f9f40 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bc05196930 con 0x55bc051774d0 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc0519a970 con 0x55bc05281480 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc0527d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc0527d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47620/0 (socket says 172.21.15.19:47620) 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 -- 172.21.15.19:0/904554516 learned_addr learned my addr 172.21.15.19:0/904554516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6379e61700 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc05281890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6379660700 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc051774d0 0x55bc0527ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 -- 172.21.15.19:0/904554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 msgr2=0x55bc05281890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc05281890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 -- 172.21.15.19:0/904554516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc051774d0 msgr2=0x55bc0527ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc051774d0 0x55bc0527ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.870+0000 7f6378e5f700 1 -- 172.21.15.19:0/904554516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc050e69d0 con 0x55bc050f9f40 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.871+0000 7f6378e5f700 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc0527d250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6364010f70 tx=0x7f6364013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478e67ae0e39524b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.871+0000 7f635f7fe700 1 -- 172.21.15.19:0/904554516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400be60 con 0x55bc050f9f40 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.871+0000 7f635f7fe700 1 -- 172.21.15.19:0/904554516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6364011db0 con 0x55bc050f9f40 2024-05-17T09:31:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.871+0000 7f635f7fe700 1 -- 172.21.15.19:0/904554516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401c630 con 0x55bc050f9f40 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 -- 172.21.15.19:0/904554516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 msgr2=0x55bc0527d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc0527d250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6364010f70 tx=0x7f6364013000 comp rx=0 tx=0).stop 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 -- 172.21.15.19:0/904554516 shutdown_connections 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc05281890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc051774d0 0x55bc0527ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/904554516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc0527d250 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 -- 172.21.15.19:0/904554516 >> 172.21.15.19:0/904554516 conn(0x55bc05138d80 msgr2=0x55bc05273520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 -- 172.21.15.19:0/904554516 shutdown_connections 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 -- 172.21.15.19:0/904554516 wait complete. 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 -- start start 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.872+0000 7f638a2d9b80 1 Processor -- start 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc05286650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc051774d0 0x55bc05286b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc052870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc0509d0a0 con 0x55bc050f9f40 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bc051388c0 con 0x55bc05281480 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc05282230 con 0x55bc051774d0 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f638a2d9b80 10 client.0 _ll_register_callbacks cb 0x55bc05131a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc05286650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc05286650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47626/0 (socket says 172.21.15.19:47626) 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 -- 172.21.15.19:0/1990025492 learned_addr learned my addr 172.21.15.19:0/1990025492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379e61700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc052870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc051774d0 0x55bc05286b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 -- 172.21.15.19:0/1990025492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc051774d0 msgr2=0x55bc05286b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc051774d0 0x55bc05286b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 -- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 msgr2=0x55bc052870d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc052870d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.873+0000 7f6379660700 1 -- 172.21.15.19:0/1990025492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc05135790 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f6379660700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc05286650 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f63700a78f0 tx=0x7f6370005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a459069f1021fcbb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f635cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f635cff9700 1 -- 172.21.15.19:0/1990025492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63700aad80 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f635cff9700 1 -- 172.21.15.19:0/1990025492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f63700b31d0 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6364011db0 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f635cff9700 1 -- 172.21.15.19:0/1990025492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f63700aa680 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 10 client.17487 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc05293f70 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f635cff9700 1 -- 172.21.15.19:0/1990025492 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6364011db0 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f635cff9700 1 -- 172.21.15.19:0/1990025492 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f63700afbc0 con 0x55bc050f9f40 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 10 client.17487 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 10 client.17487 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 20 client.17487 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:10.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc05296440 0x55bc052988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 4 client.17487 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.874+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc051a8ce0 con 0x55bc05296440 2024-05-17T09:31:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.875+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc05296440 0x55bc052988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.875+0000 7f635cff9700 10 client.17487 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.875+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc05296440 0x55bc052988e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a957dd24699254f2 server_cookie=bea44695ab9dfa1a in_seq=0 out_seq=0 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f635cff9700 1 -- 172.21.15.19:0/1990025492 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bc051a8ce0 con 0x55bc05296440 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f635cff9700 10 client.17487 handle_command_reply: tid=0 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 client.17487 shutdown 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 msgr2=0x55bc05286650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc05286650 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f63700a78f0 tx=0x7f6370005a00 comp rx=0 tx=0).stop 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 shutdown_connections 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc05296440 0x55bc052988e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc052870d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc050f9f40 0x55bc05286650 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1990025492 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc051774d0 0x55bc05286b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 >> 172.21.15.19:0/1990025492 conn(0x55bc05138d80 msgr2=0x55bc0526d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f635cff9700 0 client.17487 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 shutdown_connections 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1990025492 wait complete. 2024-05-17T09:31:10.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.876+0000 7f638a2d9b80 20 client.17487 trim_cache size 0 max 16384 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 Processor -- start 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 -- start start 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc050f9f40 0x55bc05285860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc05292610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc0529e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b09b0 con 0x55bc051774d0 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ae240 con 0x55bc05281480 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ac090 con 0x55bc050f9f40 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f6378e5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc05292610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f6378e5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc05292610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47642/0 (socket says 172.21.15.19:47642) 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f6378e5f700 1 -- 172.21.15.19:0/1604881059 learned_addr learned my addr 172.21.15.19:0/1604881059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f6379660700 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc050f9f40 0x55bc05285860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.897+0000 7f6379e61700 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc0529e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f6378e5f700 1 -- 172.21.15.19:0/1604881059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc050f9f40 msgr2=0x55bc05285860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc050f9f40 0x55bc05285860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f6378e5f700 1 -- 172.21.15.19:0/1604881059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 msgr2=0x55bc0529e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc0529e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f6378e5f700 1 -- 172.21.15.19:0/1604881059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc05284be0 con 0x55bc051774d0 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f6378e5f700 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc05292610 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x55bc05274ca0 tx=0x7f6364012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2467a961958b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f635effd700 1 -- 172.21.15.19:0/1604881059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364005300 con 0x55bc051774d0 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f635effd700 1 -- 172.21.15.19:0/1604881059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63640054a0 con 0x55bc051774d0 2024-05-17T09:31:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.898+0000 7f635effd700 1 -- 172.21.15.19:0/1604881059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401e680 con 0x55bc051774d0 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1604881059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 msgr2=0x55bc05292610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc05292610 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x55bc05274ca0 tx=0x7f6364012f90 comp rx=0 tx=0).stop 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1604881059 shutdown_connections 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc050f9f40 0x55bc05285860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc05281480 0x55bc0529e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/1604881059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc05292610 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1604881059 >> 172.21.15.19:0/1604881059 conn(0x55bc05138d80 msgr2=0x55bc05277c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1604881059 shutdown_connections 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 -- 172.21.15.19:0/1604881059 wait complete. 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 -- start start 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.899+0000 7f638a2d9b80 1 Processor -- start 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc050f9f40 0x55bc0528b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc0528bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc0528c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ac700 con 0x55bc051774d0 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63741adb70 con 0x55bc050f9f40 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63741acdd0 con 0x55bc05281480 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f638a2d9b80 10 client.0 _ll_register_callbacks cb 0x55bc05131a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc0528bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc0528bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47648/0 (socket says 172.21.15.19:47648) 2024-05-17T09:31:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 -- 172.21.15.19:0/197410918 learned_addr learned my addr 172.21.15.19:0/197410918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6379e61700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc0528c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6379660700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc050f9f40 0x55bc0528b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 -- 172.21.15.19:0/197410918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 msgr2=0x55bc0528c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc0528c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 -- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc050f9f40 msgr2=0x55bc0528b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc050f9f40 0x55bc0528b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 -- 172.21.15.19:0/197410918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc05300f80 con 0x55bc051774d0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f6378e5f700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc0528bb20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f63640059d0 tx=0x7f63640056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb3a72d4e7e1094 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.900+0000 7f635cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f635cff9700 1 -- 172.21.15.19:0/197410918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364004690 con 0x55bc051774d0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f635cff9700 1 -- 172.21.15.19:0/197410918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6364004830 con 0x55bc051774d0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63640054a0 con 0x55bc051774d0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f635cff9700 1 -- 172.21.15.19:0/197410918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6364004a70 con 0x55bc051774d0 2024-05-17T09:31:10.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 10 client.17493 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc053021f0 con 0x55bc051774d0 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f635cff9700 1 -- 172.21.15.19:0/197410918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63640054a0 con 0x55bc051774d0 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f635cff9700 1 -- 172.21.15.19:0/197410918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6364033450 con 0x55bc051774d0 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 10 client.17493 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 10 client.17493 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 20 client.17493 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc053026c0 0x55bc05304ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 4 client.17493 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27b5ba51", "release": true, "await": true}] 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.901+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27b5ba51", "release": true, "await": true}) -- 0x55bc051a24c0 con 0x55bc053026c0 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.902+0000 7f6379660700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc053026c0 0x55bc05304ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.902+0000 7f6379660700 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc053026c0 0x55bc05304ae0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f13549e7fe9a79 server_cookie=c0d215c53de17038 in_seq=0 out_seq=0 2024-05-17T09:31:10.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.902+0000 7f635cff9700 10 client.17493 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:10.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f635cff9700 1 -- 172.21.15.19:0/197410918 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bc051a24c0 con 0x55bc053026c0 2024-05-17T09:31:10.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f635cff9700 10 client.17493 handle_command_reply: tid=0 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 client.17493 shutdown 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 msgr2=0x55bc0528bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc0528bb20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f63640059d0 tx=0x7f63640056e0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 shutdown_connections 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc05281480 0x55bc0528c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc053026c0 0x55bc05304ae0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc050f9f40 0x55bc0528b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 --2- 172.21.15.19:0/197410918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc051774d0 0x55bc0528bb20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 >> 172.21.15.19:0/197410918 conn(0x55bc05138d80 msgr2=0x55bc05285f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f635cff9700 0 client.17493 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 shutdown_connections 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 1 -- 172.21.15.19:0/197410918 wait complete. 2024-05-17T09:31:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.906+0000 7f638a2d9b80 20 client.17493 trim_cache size 0 max 16384 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 40, 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "27b5ba51": { 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 40, 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:31:10.908 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.8, 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:10.909 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:31:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f63600b2880 msgr2=0x7f63600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f63600b2880 0x7f63600b4d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6374109950 tx=0x7f636400b410 comp rx=0 tx=0).stop 2024-05-17T09:31:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 msgr2=0x7f637419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 0x7f637419adc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6370006010 tx=0x7f63700047b0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 shutdown_connections 2024-05-17T09:31:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f63600b2880 0x7f63600b4d90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6374108220 0x7f637419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63741039c0 0x7f637419adc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 --2- 172.21.15.19:0/1982661619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6374104310 0x7f637419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 >> 172.21.15.19:0/1982661619 conn(0x7f63740fe500 msgr2=0x7f6374102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 shutdown_connections 2024-05-17T09:31:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:10.907+0000 7f637b8c4700 1 -- 172.21.15.19:0/1982661619 wait complete. 2024-05-17T09:31:11.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178864 1.71 MB/sec execute 429 sec latency 22566.674 ms 2024-05-17T09:31:11.664 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27b5ba51', total seconds elapsed: 28.8 2024-05-17T09:31:11.664 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.8 2024-05-17T09:31:11.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:31:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:31:12.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 179430 1.70 MB/sec execute 430 sec latency 74.049 ms 2024-05-17T09:31:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:12 smithi069 ceph-mon[35081]: pgmap v781: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-05-17T09:31:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:12 smithi019 ceph-mon[31620]: pgmap v781: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-05-17T09:31:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:12 smithi079 ceph-mon[35502]: pgmap v781: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-05-17T09:31:13.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 179942 1.71 MB/sec execute 431 sec latency 75.746 ms 2024-05-17T09:31:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 -- 172.21.15.19:0/626483925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 msgr2=0x7efd6c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 --2- 172.21.15.19:0/626483925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c102dc0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efd60009990 tx=0x7efd60009ca0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 -- 172.21.15.19:0/626483925 shutdown_connections 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 --2- 172.21.15.19:0/626483925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c1089e0 0x7efd6c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 --2- 172.21.15.19:0/626483925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c103300 0x7efd6c10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 --2- 172.21.15.19:0/626483925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c102dc0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.392+0000 7efd728d6700 1 -- 172.21.15.19:0/626483925 >> 172.21.15.19:0/626483925 conn(0x7efd6c0fe510 msgr2=0x7efd6c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.393+0000 7efd728d6700 1 -- 172.21.15.19:0/626483925 shutdown_connections 2024-05-17T09:31:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.393+0000 7efd728d6700 1 -- 172.21.15.19:0/626483925 wait complete. 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.393+0000 7efd728d6700 1 Processor -- start 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.393+0000 7efd728d6700 1 -- start start 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd728d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd728d6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c103300 0x7efd6c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd728d6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c1089e0 0x7efd6c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd728d6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c112190 con 0x7efd6c1029b0 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd728d6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c112010 con 0x7efd6c1089e0 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd728d6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c112310 con 0x7efd6c103300 2024-05-17T09:31:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44862/0 (socket says 172.21.15.19:44862) 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 -- 172.21.15.19:0/524047673 learned_addr learned my addr 172.21.15.19:0/524047673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6b7fe700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c103300 0x7efd6c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd70e73700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c1089e0 0x7efd6c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 -- 172.21.15.19:0/524047673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c103300 msgr2=0x7efd6c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c103300 0x7efd6c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 -- 172.21.15.19:0/524047673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c1089e0 msgr2=0x7efd6c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c1089e0 0x7efd6c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.394+0000 7efd6bfff700 1 -- 172.21.15.19:0/524047673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1a1e20 con 0x7efd6c1029b0 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd6b7fe700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c103300 0x7efd6c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd6bfff700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c19cf50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00d950 tx=0x7efd5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c016070 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c1a2110 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd5c004830 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c005420 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.395+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1a9a30 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.396+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c1040f0 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.398+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efd5c0055c0 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.399+0000 7efd697fa700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd540b2880 0x7efd540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.399+0000 7efd6b7fe700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd540b2880 0x7efd540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.399+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efd5c0e2750 con 0x7efd6c1029b0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.400+0000 7efd6b7fe700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd540b2880 0x7efd540b4d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efd6000b2e0 tx=0x7efd60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.403+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd5c01f020 con 0x7efd6c1029b0 2024-05-17T09:31:14.459 INFO:tasks.workunit.client.0.smithi019.stdout: 1 180647 1.72 MB/sec execute 432 sec latency 19.142 ms 2024-05-17T09:31:14.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:14 smithi019 ceph-mon[31620]: pgmap v782: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-05-17T09:31:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:14 smithi069 ceph-mon[35081]: pgmap v782: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-05-17T09:31:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.728+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6c19e2b0 con 0x7efd6c1029b0 2024-05-17T09:31:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.729+0000 7efd697fa700 1 -- 172.21.15.19:0/524047673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7efd5c0aa0c0 con 0x7efd6c1029b0 2024-05-17T09:31:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:31:14.730 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:31:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd540b2880 msgr2=0x7efd540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd540b2880 0x7efd540b4d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efd6000b2e0 tx=0x7efd60005fb0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 msgr2=0x7efd6c19cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c19cf50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00d950 tx=0x7efd5c00dc60 comp rx=0 tx=0).stop 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 shutdown_connections 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c1089e0 0x7efd6c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1029b0 0x7efd6c19cf50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd540b2880 0x7efd540b4d90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 --2- 172.21.15.19:0/524047673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd6c103300 0x7efd6c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 >> 172.21.15.19:0/524047673 conn(0x7efd6c0fe510 msgr2=0x7efd6c107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 shutdown_connections 2024-05-17T09:31:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:14.733+0000 7efd728d6700 1 -- 172.21.15.19:0/524047673 wait complete. 2024-05-17T09:31:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:31:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:14 smithi079 ceph-mon[35502]: pgmap v782: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-05-17T09:31:15.445 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.8 --await-for 45.0' 2024-05-17T09:31:15.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.8 --await-for 45.0 2024-05-17T09:31:15.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181027 1.72 MB/sec execute 433 sec latency 18.681 ms 2024-05-17T09:31:15.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:15 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/524047673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:15.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:15 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/524047673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:15.729 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:31:15.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:15 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/524047673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:16.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181304 1.72 MB/sec execute 434 sec latency 18.095 ms 2024-05-17T09:31:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:16 smithi069 ceph-mon[35081]: pgmap v783: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 166 KiB/s wr, 29 op/s 2024-05-17T09:31:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:16 smithi019 ceph-mon[31620]: pgmap v783: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 166 KiB/s wr, 29 op/s 2024-05-17T09:31:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:16 smithi079 ceph-mon[35502]: pgmap v783: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 166 KiB/s wr, 29 op/s 2024-05-17T09:31:17.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181591 1.72 MB/sec execute 435 sec latency 15.567 ms 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 -- 172.21.15.19:0/3600682682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 msgr2=0x7f7418110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 --2- 172.21.15.19:0/3600682682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f7418110090 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f741000b0d0 tx=0x7f741000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 -- 172.21.15.19:0/3600682682 shutdown_connections 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 --2- 172.21.15.19:0/3600682682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74181034b0 0x7f74181038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 --2- 172.21.15.19:0/3600682682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f7418110090 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 --2- 172.21.15.19:0/3600682682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7418107de0 0x7f74181081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 -- 172.21.15.19:0/3600682682 >> 172.21.15.19:0/3600682682 conn(0x7f74180fedb0 msgr2=0x7f74181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 -- 172.21.15.19:0/3600682682 shutdown_connections 2024-05-17T09:31:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.636+0000 7f741d070700 1 -- 172.21.15.19:0/3600682682 wait complete. 2024-05-17T09:31:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.637+0000 7f741d070700 1 Processor -- start 2024-05-17T09:31:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.637+0000 7f741d070700 1 -- start start 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741d070700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 0x7f741819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741d070700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7418107de0 0x7f741819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741d070700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f741819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741d070700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7418113210 con 0x7f7418108730 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741d070700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7418113090 con 0x7f74181034b0 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741d070700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7418113390 con 0x7f7418107de0 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f7416d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 0x7f741819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f741819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f7416d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 0x7f741819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34900/0 (socket says 172.21.15.19:34900) 2024-05-17T09:31:17.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f7416d9d700 1 -- 172.21.15.19:0/3218290177 learned_addr learned my addr 172.21.15.19:0/3218290177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f740e59c700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7418107de0 0x7f741819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 -- 172.21.15.19:0/3218290177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7418107de0 msgr2=0x7f741819c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7418107de0 0x7f741819c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 -- 172.21.15.19:0/3218290177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 msgr2=0x7f741819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 0x7f741819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 -- 172.21.15.19:0/3218290177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74181a6ee0 con 0x7f7418108730 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f7416d9d700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 0x7f741819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f740e59c700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7418107de0 0x7f741819c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.638+0000 7f741759e700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f741819c630 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f741000b0d0 tx=0x7f7410007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.639+0000 7f7414d99700 1 -- 172.21.15.19:0/3218290177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741001f070 con 0x7f7418108730 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.639+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74181a7170 con 0x7f7418108730 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.639+0000 7f7414d99700 1 -- 172.21.15.19:0/3218290177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7410004c50 con 0x7f7418108730 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.639+0000 7f7414d99700 1 -- 172.21.15.19:0/3218290177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741001b430 con 0x7f7418108730 2024-05-17T09:31:17.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.639+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181a75c0 con 0x7f7418108730 2024-05-17T09:31:17.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.642+0000 7f7414d99700 1 -- 172.21.15.19:0/3218290177 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7410022070 con 0x7f7418108730 2024-05-17T09:31:17.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.643+0000 7f7414d99700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f74040b2880 0x7f74040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.643+0000 7f7416d9d700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f74040b2880 0x7f74040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.643+0000 7f7414d99700 1 -- 172.21.15.19:0/3218290177 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f741000f020 con 0x7f7418108730 2024-05-17T09:31:17.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.643+0000 7f742ba85b80 1 Processor -- start 2024-05-17T09:31:17.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.643+0000 7f742ba85b80 1 -- start start 2024-05-17T09:31:17.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.643+0000 7f7416d9d700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f74040b2880 0x7f74040b4d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7408011550 tx=0x7f7408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcfdb560 0x5619dd0e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0e5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f742ba85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5619dd00ae20 con 0x5619dd0e5510 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f742ba85b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619dcffa9c0 con 0x5619dcfdb560 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f742ba85b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5619dcffea00 con 0x5619dcf5dfd0 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0e5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44894/0 (socket says 172.21.15.19:44894) 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 -- 172.21.15.19:0/2711315620 learned_addr learned my addr 172.21.15.19:0/2711315620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f740e59c700 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f7416d9d700 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcfdb560 0x5619dd0e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 -- 172.21.15.19:0/2711315620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcf5dfd0 msgr2=0x5619dd0e12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 -- 172.21.15.19:0/2711315620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcfdb560 msgr2=0x5619dd0e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcfdb560 0x5619dd0e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f741759e700 1 -- 172.21.15.19:0/2711315620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619dcf4aa60 con 0x5619dd0e5510 2024-05-17T09:31:17.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.644+0000 7f7416d9d700 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcfdb560 0x5619dd0e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.645+0000 7f741759e700 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0e5920 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f74100aa540 tx=0x7f74100099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f68b78e0a3517b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.645+0000 7f740d59a700 1 -- 172.21.15.19:0/2711315620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100a9fb0 con 0x5619dd0e5510 2024-05-17T09:31:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.645+0000 7f740d59a700 1 -- 172.21.15.19:0/2711315620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7410004e00 con 0x5619dd0e5510 2024-05-17T09:31:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.645+0000 7f740d59a700 1 -- 172.21.15.19:0/2711315620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100b1020 con 0x5619dd0e5510 2024-05-17T09:31:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 -- 172.21.15.19:0/2711315620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 msgr2=0x5619dd0e5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0e5920 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f74100aa540 tx=0x7f74100099e0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 -- 172.21.15.19:0/2711315620 shutdown_connections 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0e5920 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcfdb560 0x5619dd0e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2711315620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 -- 172.21.15.19:0/2711315620 >> 172.21.15.19:0/2711315620 conn(0x5619dcf9ce10 msgr2=0x5619dd0d75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 -- 172.21.15.19:0/2711315620 shutdown_connections 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 -- 172.21.15.19:0/2711315620 wait complete. 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.646+0000 7f742ba85b80 1 -- start start 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 Processor -- start 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dd0e5510 0x5619dd0ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5619dcf01130 con 0x5619dcf5dfd0 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619dcf9c950 con 0x5619dd0e5510 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5619dd0e6080 con 0x5619dcfdb560 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f742ba85b80 10 client.0 _ll_register_callbacks cb 0x5619dcf95ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f740e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f740e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39598/0 (socket says 172.21.15.19:39598) 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f740e59c700 1 -- 172.21.15.19:0/2766917676 learned_addr learned my addr 172.21.15.19:0/2766917676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f7416d9d700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.647+0000 7f741759e700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dd0e5510 0x5619dd0ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f740e59c700 1 -- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dd0e5510 msgr2=0x5619dd0ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f740e59c700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dd0e5510 0x5619dd0ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f740e59c700 1 -- 172.21.15.19:0/2766917676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcf5dfd0 msgr2=0x5619dd0f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f740e59c700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f740e59c700 1 -- 172.21.15.19:0/2766917676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619dcf99820 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f741759e700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dd0e5510 0x5619dd0ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f7416d9d700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f740e59c700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f0690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7400012d20 tx=0x7f740000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3587171e19e1c2e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f73fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f73fe7fc700 1 -- 172.21.15.19:0/2766917676 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740000fbd0 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f73fe7fc700 1 -- 172.21.15.19:0/2766917676 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f740000fd70 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619dd00cd70 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f73fe7fc700 1 -- 172.21.15.19:0/2766917676 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7400024050 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f742ba85b80 10 client.27100 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5619dd0f7dd0 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.648+0000 7f73fe7fc700 1 -- 172.21.15.19:0/2766917676 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5619dd00cd70 con 0x5619dcfdb560 2024-05-17T09:31:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f73fe7fc700 1 -- 172.21.15.19:0/2766917676 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7400003ba0 con 0x5619dcfdb560 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f742ba85b80 10 client.27100 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f742ba85b80 10 client.27100 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f742ba85b80 20 client.27100 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd0fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f742ba85b80 4 client.27100 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619dd006550 con 0x5619dd0fa410 2024-05-17T09:31:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.649+0000 7f7416d9d700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd0fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f7416d9d700 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd0fc8b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96513656024a1c97 server_cookie=7dd0714b883c2b9f in_seq=0 out_seq=0 2024-05-17T09:31:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f73fe7fc700 10 client.27100 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:17.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f73fe7fc700 1 -- 172.21.15.19:0/2766917676 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5619dd006550 con 0x5619dd0fa410 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f73fe7fc700 10 client.27100 handle_command_reply: tid=0 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f742ba85b80 1 client.27100 shutdown 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 msgr2=0x5619dd0f0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f0690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7400012d20 tx=0x7f740000f6a0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.650+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 shutdown_connections 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd0fc8b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dd0e5510 0x5619dd0ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f73fe7fc700 0 client.27100 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 --2- 172.21.15.19:0/2766917676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f0690 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 >> 172.21.15.19:0/2766917676 conn(0x5619dcf9ce10 msgr2=0x5619dd0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 shutdown_connections 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 1 -- 172.21.15.19:0/2766917676 wait complete. 2024-05-17T09:31:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.651+0000 7f742ba85b80 20 client.27100 trim_cache size 0 max 16384 2024-05-17T09:31:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 Processor -- start 2024-05-17T09:31:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 -- start start 2024-05-17T09:31:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 0x5619dd0eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dd0e5510 0x5619dd0e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f74181b08d0 con 0x5619dcfdb560 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ae160 con 0x5619dcf5dfd0 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f742ba85b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74181abfb0 con 0x5619dd0e5510 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f740e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 0x5619dd0eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f740e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 0x5619dd0eaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44924/0 (socket says 172.21.15.19:44924) 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f740e59c700 1 -- 172.21.15.19:0/3756923892 learned_addr learned my addr 172.21.15.19:0/3756923892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f7416d9d700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.673+0000 7f741759e700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dd0e5510 0x5619dd0e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f741759e700 1 -- 172.21.15.19:0/3756923892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 msgr2=0x5619dd0e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f741759e700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f741759e700 1 -- 172.21.15.19:0/3756923892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 msgr2=0x5619dd0eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f741759e700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 0x5619dd0eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f741759e700 1 -- 172.21.15.19:0/3756923892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619dd0e5310 con 0x5619dd0e5510 2024-05-17T09:31:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f740e59c700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 0x5619dd0eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f7416d9d700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f741759e700 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dd0e5510 0x5619dd0e7df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f74100aa540 tx=0x7f74100b1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf0e6da83c4d996 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f740cd99700 1 -- 172.21.15.19:0/3756923892 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100cd110 con 0x5619dd0e5510 2024-05-17T09:31:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f740cd99700 1 -- 172.21.15.19:0/3756923892 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74100aa840 con 0x5619dd0e5510 2024-05-17T09:31:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.674+0000 7f740cd99700 1 -- 172.21.15.19:0/3756923892 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410093110 con 0x5619dd0e5510 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 -- 172.21.15.19:0/3756923892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dd0e5510 msgr2=0x5619dd0e7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dd0e5510 0x5619dd0e7df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f74100aa540 tx=0x7f74100b1040 comp rx=0 tx=0).stop 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 -- 172.21.15.19:0/3756923892 shutdown_connections 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dd0e5510 0x5619dd0e7df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 --2- 172.21.15.19:0/3756923892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dcfdb560 0x5619dd0eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 -- 172.21.15.19:0/3756923892 >> 172.21.15.19:0/3756923892 conn(0x5619dcf9ce10 msgr2=0x5619dd0e76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 -- 172.21.15.19:0/3756923892 shutdown_connections 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.675+0000 7f742ba85b80 1 -- 172.21.15.19:0/3756923892 wait complete. 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 -- start start 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 Processor -- start 2024-05-17T09:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ac620 con 0x5619dd0e5510 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ada90 con 0x5619dcf5dfd0 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74181accf0 con 0x5619dcfdb560 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f742ba85b80 10 client.0 _ll_register_callbacks cb 0x5619dcf95ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f740e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f7416d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f740e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39618/0 (socket says 172.21.15.19:39618) 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f740e59c700 1 -- 172.21.15.19:0/1213800015 learned_addr learned my addr 172.21.15.19:0/1213800015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.676+0000 7f741759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f740e59c700 1 -- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 msgr2=0x5619dd0f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f740e59c700 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f740e59c700 1 -- 172.21.15.19:0/1213800015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 msgr2=0x5619dd0f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f740e59c700 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f740e59c700 1 -- 172.21.15.19:0/1213800015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619dd0f3940 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f741759e700 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f740e59c700 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f2e90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7400010480 tx=0x7f7400003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506333dd00386df0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f73fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f73fe7fc700 1 -- 172.21.15.19:0/1213800015 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740002c070 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619dce7d3e0 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f73fe7fc700 1 -- 172.21.15.19:0/1213800015 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7400003ab0 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f73fe7fc700 1 -- 172.21.15.19:0/1213800015 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7400038050 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f742ba85b80 10 client.27106 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.677+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5619dd165c40 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f73fe7fc700 1 -- 172.21.15.19:0/1213800015 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5619dce7d3e0 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f73fe7fc700 1 -- 172.21.15.19:0/1213800015 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7400010700 con 0x5619dcfdb560 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f742ba85b80 10 client.27106 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f742ba85b80 10 client.27106 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f742ba85b80 20 client.27106 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f742ba85b80 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd1682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f742ba85b80 4 client.27106 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.8, "await_for": 45.0}] 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.8, "await_for": 45.0}) -- 0x5619dcfceb70 con 0x5619dd0fa410 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.678+0000 7f7416d9d700 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd1682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.679+0000 7f7416d9d700 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd1682e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4c4e1cae0880a6 server_cookie=4fa351fe182c05b5 in_seq=0 out_seq=0 2024-05-17T09:31:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.679+0000 7f73fe7fc700 10 client.27106 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:17.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f73fe7fc700 1 -- 172.21.15.19:0/1213800015 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5619dcfceb70 con 0x5619dd0fa410 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f73fe7fc700 10 client.27106 handle_command_reply: tid=0 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 client.27106 shutdown 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 msgr2=0x5619dd0f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f2e90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7400010480 tx=0x7f7400003680 comp rx=0 tx=0).stop 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 shutdown_connections 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619dd0fa410 0x5619dd1682e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619dd0e5510 0x5619dd0f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619dcf5dfd0 0x5619dd0f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f73fe7fc700 0 client.27106 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 --2- 172.21.15.19:0/1213800015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619dcfdb560 0x5619dd0f2e90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 >> 172.21.15.19:0/1213800015 conn(0x5619dcf9ce10 msgr2=0x5619dd0d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 shutdown_connections 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 1 -- 172.21.15.19:0/1213800015 wait complete. 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.746+0000 7f742ba85b80 20 client.27106 trim_cache size 0 max 16384 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 42, 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "23e7ba2f": { 2024-05-17T09:31:17.748 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 42, 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.8, 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:31:17.749 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:31:17.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:17.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:17.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:17.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:17.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:17.750 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f74040b2880 msgr2=0x7f74040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f74040b2880 0x7f74040b4d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7408011550 tx=0x7f7408009260 comp rx=0 tx=0).stop 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 msgr2=0x7f741819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f741819c630 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f741000b0d0 tx=0x7f7410007a00 comp rx=0 tx=0).stop 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 shutdown_connections 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7418108730 0x7f741819c630 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74181034b0 0x7f741819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f74040b2880 0x7f74040b4d90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 --2- 172.21.15.19:0/3218290177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7418107de0 0x7f741819c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 >> 172.21.15.19:0/3218290177 conn(0x7f74180fedb0 msgr2=0x7f74181075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 shutdown_connections 2024-05-17T09:31:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:17.747+0000 7f741d070700 1 -- 172.21.15.19:0/3218290177 wait complete. 2024-05-17T09:31:18.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.71 MB/sec execute 436 sec latency 701.348 ms 2024-05-17T09:31:18.483 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '23e7ba2f', quiesce took 6.8 seconds. Will release after: 16.0 2024-05-17T09:31:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:18 smithi069 ceph-mon[35081]: pgmap v784: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 421 KiB/s wr, 36 op/s 2024-05-17T09:31:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:18 smithi019 ceph-mon[31620]: pgmap v784: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 421 KiB/s wr, 36 op/s 2024-05-17T09:31:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:18 smithi079 ceph-mon[35502]: pgmap v784: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 421 KiB/s wr, 36 op/s 2024-05-17T09:31:19.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.71 MB/sec execute 437 sec latency 1701.471 ms 2024-05-17T09:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 -- 172.21.15.19:0/2263854280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac81098c0 msgr2=0x7f1ac810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/2263854280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac81098c0 0x7f1ac810bdd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009ca0 tx=0x7f1ac002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 -- 172.21.15.19:0/2263854280 shutdown_connections 2024-05-17T09:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/2263854280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac81098c0 0x7f1ac810bdd0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/2263854280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81024a0 0x7f1ac8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/2263854280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac8101fa0 0x7f1ac8106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 -- 172.21.15.19:0/2263854280 >> 172.21.15.19:0/2263854280 conn(0x7f1ac80fd4d0 msgr2=0x7f1ac80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 -- 172.21.15.19:0/2263854280 shutdown_connections 2024-05-17T09:31:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.422+0000 7f1acf2f9640 1 -- 172.21.15.19:0/2263854280 wait complete. 2024-05-17T09:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 Processor -- start 2024-05-17T09:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 -- start start 2024-05-17T09:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 0x7f1ac819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac81024a0 0x7f1ac819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81098c0 0x7f1ac81a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810f630 con 0x7f1ac8101fa0 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810f4b0 con 0x7f1ac81098c0 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acf2f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810f7b0 con 0x7f1ac81024a0 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 0x7f1ac819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 0x7f1ac819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44948/0 (socket says 172.21.15.19:44948) 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 -- 172.21.15.19:0/712198019 learned_addr learned my addr 172.21.15.19:0/712198019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acc894640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac81024a0 0x7f1ac819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd896640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81098c0 0x7f1ac81a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 -- 172.21.15.19:0/712198019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac81024a0 msgr2=0x7f1ac819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac81024a0 0x7f1ac819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 -- 172.21.15.19:0/712198019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81098c0 msgr2=0x7f1ac81a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81098c0 0x7f1ac81a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd095640 1 -- 172.21.15.19:0/712198019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81a3ec0 con 0x7f1ac8101fa0 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.423+0000 7f1acd896640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81098c0 0x7f1ac81a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.424+0000 7f1acd095640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 0x7f1ac819c380 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400e9f0 tx=0x7f1ac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.424+0000 7f1abe7fc640 1 -- 172.21.15.19:0/712198019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400cd80 con 0x7f1ac8101fa0 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.424+0000 7f1abe7fc640 1 -- 172.21.15.19:0/712198019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac40054b0 con 0x7f1ac8101fa0 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.424+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81a41b0 con 0x7f1ac8101fa0 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.424+0000 7f1abe7fc640 1 -- 172.21.15.19:0/712198019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4010430 con 0x7f1ac8101fa0 2024-05-17T09:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.424+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a46b0 con 0x7f1ac8101fa0 2024-05-17T09:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 Processor -- start 2024-05-17T09:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 -- start start 2024-05-17T09:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799836a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x55799836a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998371820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579982dcc30 con 0x557997fa84d0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5579982cc7d0 con 0x557997fc91b0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5579982d0810 con 0x557997fa02f0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1abe7fc640 1 -- 172.21.15.19:0/712198019 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ac40046c0 con 0x7f1ac8101fa0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1acc894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998371820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1acc894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998371820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44950/0 (socket says 172.21.15.19:44950) 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1acc894640 1 -- 172.21.15.19:0/3515914932 learned_addr learned my addr 172.21.15.19:0/3515914932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.427+0000 7f1acd896640 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799836a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acd095640 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x55799836a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acc894640 1 -- 172.21.15.19:0/3515914932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 msgr2=0x55799836a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acc894640 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x55799836a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acc894640 1 -- 172.21.15.19:0/3515914932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 msgr2=0x55799836a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acc894640 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799836a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acc894640 1 -- 172.21.15.19:0/3515914932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557997fe1210 con 0x557997fa84d0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acd095640 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x55799836a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acc894640 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998371820 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800bc60 tx=0x7f1ab800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f0e255c7773b38 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1ab37fe640 1 -- 172.21.15.19:0/3515914932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8015070 con 0x557997fa84d0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1ab37fe640 1 -- 172.21.15.19:0/3515914932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab801a8f0 con 0x557997fa84d0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1ab37fe640 1 -- 172.21.15.19:0/3515914932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8011800 con 0x557997fa84d0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1abe7fc640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ab40b28a0 0x7f1ab40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.428+0000 7f1acd896640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ab40b28a0 0x7f1ab40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1abe7fc640 1 -- 172.21.15.19:0/712198019 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1ac40e2a90 con 0x7f1ac8101fa0 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3515914932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 msgr2=0x557998371820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1acd896640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ab40b28a0 0x7f1ab40b4db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0006010 tx=0x7f1ac0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998371820 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800bc60 tx=0x7f1ab800f730 comp rx=0 tx=0).stop 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3515914932 shutdown_connections 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998371820 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x55799836a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3515914932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799836a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3515914932 >> 172.21.15.19:0/3515914932 conn(0x557997fbd300 msgr2=0x557998365d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3515914932 shutdown_connections 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.429+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3515914932 wait complete. 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 -- start start 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 Processor -- start 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x557998373cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998374230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799837b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557997fb2790 con 0x557997fa84d0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557998288890 con 0x557997fc91b0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557998372270 con 0x557997fa02f0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.430+0000 7f1ad61ad740 10 client.0 _ll_register_callbacks cb 0x55799828bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998374230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998374230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44958/0 (socket says 172.21.15.19:44958) 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 -- 172.21.15.19:0/1963880637 learned_addr learned my addr 172.21.15.19:0/1963880637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acd896640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799837b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acd095640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x557998373cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 -- 172.21.15.19:0/1963880637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 msgr2=0x557998373cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x557998373cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 -- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 msgr2=0x55799837b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799837b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 -- 172.21.15.19:0/1963880637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579980ad620 con 0x557997fa84d0 2024-05-17T09:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1acc894640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998374230 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab801a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f69a65ca4ad165a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.431+0000 7f1ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.434+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/1963880637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8028060 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.434+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/1963880637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1ab801acb0 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab801a8f0 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/1963880637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ab802f050 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 10 client.17529 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55799837e030 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/1963880637 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab801a8f0 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/1963880637 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1ab8004b10 con 0x557997fa84d0 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 10 client.17529 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 20 client.17529 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f87d80 0x557998381450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 4 client.17529 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.435+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579982deb80 con 0x557997f87d80 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.436+0000 7f1acd095640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f87d80 0x557998381450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.436+0000 7f1acd095640 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f87d80 0x557998381450 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e85930c2cde5543 server_cookie=56b54687537f1935 in_seq=0 out_seq=0 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.436+0000 7f1ab0ff9640 10 client.17529 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/1963880637 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5579982deb80 con 0x557997f87d80 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ab0ff9640 10 client.17529 handle_command_reply: tid=0 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 client.17529 shutdown 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 msgr2=0x557998374230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998374230 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab801a690 comp rx=0 tx=0).stop 2024-05-17T09:31:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 shutdown_connections 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f87d80 0x557998381450 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fc91b0 0x55799837b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998374230 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ab0ff9640 0 client.17529 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/1963880637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fa02f0 0x557998373cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 >> 172.21.15.19:0/1963880637 conn(0x557997fbd300 msgr2=0x557998364d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 shutdown_connections 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 1 -- 172.21.15.19:0/1963880637 wait complete. 2024-05-17T09:31:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.437+0000 7f1ad61ad740 20 client.17529 trim_cache size 0 max 16384 2024-05-17T09:31:20.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.70 MB/sec execute 438 sec latency 2701.520 ms 2024-05-17T09:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 Processor -- start 2024-05-17T09:31:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 -- start start 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 0x557998368b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa84d0 0x55799837b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x557998369070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b14a0 con 0x557997fa02f0 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81aed70 con 0x557997fa84d0 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81acbc0 con 0x557997fc91b0 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1acd095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 0x557998368b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1acd095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 0x557998368b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44974/0 (socket says 172.21.15.19:44974) 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.453+0000 7f1acd095640 1 -- 172.21.15.19:0/2590200567 learned_addr learned my addr 172.21.15.19:0/2590200567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acc894640 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa84d0 0x55799837b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd896640 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x557998369070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd095640 1 -- 172.21.15.19:0/2590200567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 msgr2=0x557998369070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd095640 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x557998369070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd095640 1 -- 172.21.15.19:0/2590200567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa84d0 msgr2=0x55799837b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd095640 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa84d0 0x55799837b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd095640 1 -- 172.21.15.19:0/2590200567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579983662d0 con 0x557997fa02f0 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acc894640 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa84d0 0x55799837b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1acd095640 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 0x557998368b30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40a6f70 tx=0x7f1ac40e00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a0e003ff253021 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1ab2ffd640 1 -- 172.21.15.19:0/2590200567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40aa400 con 0x557997fa02f0 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1ab2ffd640 1 -- 172.21.15.19:0/2590200567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac40a9d00 con 0x557997fa02f0 2024-05-17T09:31:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.454+0000 7f1ab2ffd640 1 -- 172.21.15.19:0/2590200567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40a9f60 con 0x557997fa02f0 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 -- 172.21.15.19:0/2590200567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 msgr2=0x557998368b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 0x557998368b30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40a6f70 tx=0x7f1ac40e00b0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 -- 172.21.15.19:0/2590200567 shutdown_connections 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x557998369070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa84d0 0x55799837b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/2590200567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa02f0 0x557998368b30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 -- 172.21.15.19:0/2590200567 >> 172.21.15.19:0/2590200567 conn(0x557997fbd300 msgr2=0x5579983656b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 -- 172.21.15.19:0/2590200567 shutdown_connections 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 -- 172.21.15.19:0/2590200567 wait complete. 2024-05-17T09:31:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 -- start start 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.455+0000 7f1ad61ad740 1 Processor -- start 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa02f0 0x5579983780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998378620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x5579983e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81ad230 con 0x557997fa84d0 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81ae6a0 con 0x557997fa02f0 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81ad900 con 0x557997fc91b0 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ad61ad740 10 client.0 _ll_register_callbacks cb 0x7f1ab800bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998378620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998378620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44982/0 (socket says 172.21.15.19:44982) 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 -- 172.21.15.19:0/3577995975 learned_addr learned my addr 172.21.15.19:0/3577995975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acd896640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x5579983e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acd095640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa02f0 0x5579983780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 -- 172.21.15.19:0/3577995975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 msgr2=0x5579983e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x5579983e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 -- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa02f0 msgr2=0x5579983780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa02f0 0x5579983780e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 -- 172.21.15.19:0/3577995975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579983e7fa0 con 0x557997fa84d0 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acd896640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x5579983e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acd095640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa02f0 0x5579983780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1acc894640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998378620 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8001eb0 tx=0x7f1ab800ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215518a4ae35f28a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.456+0000 7f1ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/3577995975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8028070 con 0x557997fa84d0 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/3577995975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1ab8011a30 con 0x557997fa84d0 2024-05-17T09:31:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac40a9d00 con 0x557997fa84d0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/3577995975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ab8015070 con 0x557997fa84d0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 10 client.17535 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5579983e9200 con 0x557997fa84d0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/3577995975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ac40a9d00 con 0x557997fa84d0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/3577995975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1ab8011c50 con 0x557997fa84d0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 10 client.17535 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 20 client.17535 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f5d1c0 0x5579983ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 4 client.17535 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.457+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55799836ad20 con 0x557997f5d1c0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.458+0000 7f1acd095640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f5d1c0 0x5579983ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.458+0000 7f1ab0ff9640 10 client.17535 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.458+0000 7f1acd095640 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f5d1c0 0x5579983ebc40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ed6e89f6354948 server_cookie=48dc0db42aa0635d in_seq=0 out_seq=0 2024-05-17T09:31:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.485+0000 7f1ab0ff9640 1 -- 172.21.15.19:0/3577995975 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55799836ad20 con 0x557997f5d1c0 2024-05-17T09:31:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ab0ff9640 10 client.17535 handle_command_reply: tid=0 2024-05-17T09:31:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 client.17535 shutdown 2024-05-17T09:31:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 msgr2=0x557998378620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998378620 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8001eb0 tx=0x7f1ab800ff60 comp rx=0 tx=0).stop 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 shutdown_connections 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557997f5d1c0 0x5579983ebc40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557997fc91b0 0x5579983e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557997fa84d0 0x557998378620 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ab0ff9640 0 client.17535 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 --2- 172.21.15.19:0/3577995975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557997fa02f0 0x5579983780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 >> 172.21.15.19:0/3577995975 conn(0x557997fbd300 msgr2=0x557998364330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 shutdown_connections 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 1 -- 172.21.15.19:0/3577995975 wait complete. 2024-05-17T09:31:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.486+0000 7f1ad61ad740 20 client.17535 trim_cache size 0 max 16384 2024-05-17T09:31:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:31:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:31:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:31:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ab40b28a0 msgr2=0x7f1ab40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ab40b28a0 0x7f1ab40b4db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0006010 tx=0x7f1ac0005c50 comp rx=0 tx=0).stop 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 msgr2=0x7f1ac819c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 0x7f1ac819c380 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400e9f0 tx=0x7f1ac400edf0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 shutdown_connections 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac81098c0 0x7f1ac81a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac81024a0 0x7f1ac819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ab40b28a0 0x7f1ab40b4db0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 --2- 172.21.15.19:0/712198019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac8101fa0 0x7f1ac819c380 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 >> 172.21.15.19:0/712198019 conn(0x7f1ac80fd4d0 msgr2=0x7f1ac80ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 shutdown_connections 2024-05-17T09:31:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:20.487+0000 7f1acf2f9640 1 -- 172.21.15.19:0/712198019 wait complete. 2024-05-17T09:31:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:20 smithi019 ceph-mon[31620]: pgmap v785: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 632 KiB/s wr, 79 op/s 2024-05-17T09:31:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:20 smithi079 ceph-mon[35502]: pgmap v785: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 632 KiB/s wr, 79 op/s 2024-05-17T09:31:20.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:20 smithi069 ceph-mon[35081]: pgmap v785: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 632 KiB/s wr, 79 op/s 2024-05-17T09:31:21.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.70 MB/sec execute 439 sec latency 3701.610 ms 2024-05-17T09:31:22.446 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.70 MB/sec execute 440 sec latency 4701.711 ms 2024-05-17T09:31:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:22 smithi019 ceph-mon[31620]: pgmap v786: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 656 KiB/s wr, 79 op/s 2024-05-17T09:31:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:22 smithi079 ceph-mon[35502]: pgmap v786: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 656 KiB/s wr, 79 op/s 2024-05-17T09:31:22.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:22 smithi069 ceph-mon[35081]: pgmap v786: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 656 KiB/s wr, 79 op/s 2024-05-17T09:31:23.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.69 MB/sec execute 441 sec latency 5701.797 ms 2024-05-17T09:31:24.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.69 MB/sec execute 442 sec latency 6701.845 ms 2024-05-17T09:31:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:24 smithi019 ceph-mon[31620]: pgmap v787: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 676 KiB/s wr, 84 op/s 2024-05-17T09:31:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:24 smithi079 ceph-mon[35502]: pgmap v787: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 676 KiB/s wr, 84 op/s 2024-05-17T09:31:24.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:24 smithi069 ceph-mon[35081]: pgmap v787: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 676 KiB/s wr, 84 op/s 2024-05-17T09:31:25.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.69 MB/sec execute 443 sec latency 7701.894 ms 2024-05-17T09:31:26.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.68 MB/sec execute 444 sec latency 8701.943 ms 2024-05-17T09:31:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:26 smithi079 ceph-mon[35502]: pgmap v788: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-05-17T09:31:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:26 smithi069 ceph-mon[35081]: pgmap v788: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-05-17T09:31:26.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:26 smithi019 ceph-mon[31620]: pgmap v788: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-05-17T09:31:27.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.68 MB/sec execute 445 sec latency 9701.992 ms 2024-05-17T09:31:28.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.67 MB/sec execute 446 sec latency 10702.053 ms 2024-05-17T09:31:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:28 smithi079 ceph-mon[35502]: pgmap v789: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 527 KiB/s wr, 69 op/s 2024-05-17T09:31:28.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:28 smithi069 ceph-mon[35081]: pgmap v789: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 527 KiB/s wr, 69 op/s 2024-05-17T09:31:28.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:28 smithi019 ceph-mon[31620]: pgmap v789: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 527 KiB/s wr, 69 op/s 2024-05-17T09:31:29.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.67 MB/sec execute 447 sec latency 11702.090 ms 2024-05-17T09:31:30.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.67 MB/sec execute 448 sec latency 12702.123 ms 2024-05-17T09:31:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:30 smithi079 ceph-mon[35502]: pgmap v790: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 273 KiB/s wr, 63 op/s 2024-05-17T09:31:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:30 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:31:30.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:30 smithi069 ceph-mon[35081]: pgmap v790: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 273 KiB/s wr, 63 op/s 2024-05-17T09:31:30.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:30 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:31:30.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:30 smithi019 ceph-mon[31620]: pgmap v790: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 273 KiB/s wr, 63 op/s 2024-05-17T09:31:30.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:30 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:31:31.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.66 MB/sec execute 449 sec latency 13702.210 ms 2024-05-17T09:31:31.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:31:31.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:31:31.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:31:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:31:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:31:31.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:31:31.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:31:31.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:31:31.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:31:32.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.66 MB/sec execute 450 sec latency 14702.293 ms 2024-05-17T09:31:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:32 smithi079 ceph-mon[35502]: pgmap v791: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 20 op/s 2024-05-17T09:31:32.831 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:32 smithi069 ceph-mon[35081]: pgmap v791: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 20 op/s 2024-05-17T09:31:32.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:32 smithi019 ceph-mon[31620]: pgmap v791: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 20 op/s 2024-05-17T09:31:33.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.66 MB/sec execute 451 sec latency 15702.420 ms 2024-05-17T09:31:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 -- 172.21.15.19:0/4001705838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c1018c0 msgr2=0x7f181c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 --2- 172.21.15.19:0/4001705838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c1018c0 0x7f181c101d30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f180c009900 tx=0x7f180c02a410 comp rx=0 tx=0).stop 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 -- 172.21.15.19:0/4001705838 shutdown_connections 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 --2- 172.21.15.19:0/4001705838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f181c1023c0 0x7f181c10be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 --2- 172.21.15.19:0/4001705838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c1018c0 0x7f181c101d30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 --2- 172.21.15.19:0/4001705838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c105210 0x7f181c105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 -- 172.21.15.19:0/4001705838 >> 172.21.15.19:0/4001705838 conn(0x7f181c0fd500 msgr2=0x7f181c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 -- 172.21.15.19:0/4001705838 shutdown_connections 2024-05-17T09:31:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.601+0000 7f18242d1640 1 -- 172.21.15.19:0/4001705838 wait complete. 2024-05-17T09:31:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 Processor -- start 2024-05-17T09:31:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 -- start start 2024-05-17T09:31:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f181c1018c0 0x7f181c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c1023c0 0x7f181c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 0x7f181c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10f630 con 0x7f181c105210 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10f4b0 con 0x7f181c1023c0 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f18242d1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10f7b0 con 0x7f181c1018c0 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f182286e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 0x7f181c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f182286e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 0x7f181c1a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48468/0 (socket says 172.21.15.19:48468) 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.602+0000 7f182286e640 1 -- 172.21.15.19:0/1685601747 learned_addr learned my addr 172.21.15.19:0/1685601747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182186c640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c1023c0 0x7f181c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182206d640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f181c1018c0 0x7f181c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182286e640 1 -- 172.21.15.19:0/1685601747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f181c1018c0 msgr2=0x7f181c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182286e640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f181c1018c0 0x7f181c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182286e640 1 -- 172.21.15.19:0/1685601747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c1023c0 msgr2=0x7f181c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182286e640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c1023c0 0x7f181c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182286e640 1 -- 172.21.15.19:0/1685601747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c1a1ce0 con 0x7f181c105210 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182186c640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c1023c0 0x7f181c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f182286e640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 0x7f181c1a1670 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f181400eb10 tx=0x7f181400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f18137fe640 1 -- 172.21.15.19:0/1685601747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814004b20 con 0x7f181c105210 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f18137fe640 1 -- 172.21.15.19:0/1685601747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1814004cc0 con 0x7f181c105210 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c1a1fd0 con 0x7f181c105210 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f18137fe640 1 -- 172.21.15.19:0/1685601747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814010690 con 0x7f181c105210 2024-05-17T09:31:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.603+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1a23e0 con 0x7f181c105210 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.607+0000 7f182b185740 1 Processor -- start 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.607+0000 7f18137fe640 1 -- 172.21.15.19:0/1685601747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1814005020 con 0x7f181c105210 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.607+0000 7f182b185740 1 -- start start 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.607+0000 7f182b185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182b185740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc82920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182b185740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 0x560a6bc89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182b185740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a6bbf4c30 con 0x560a6b8c04d0 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182b185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a6bbe47d0 con 0x560a6b8e11b0 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182b185740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560a6bbe8810 con 0x560a6b8b82f0 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 0x560a6bc89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 0x560a6bc89820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48484/0 (socket says 172.21.15.19:48484) 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 -- 172.21.15.19:0/1661603825 learned_addr learned my addr 172.21.15.19:0/1661603825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182206d640 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc82920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 -- 172.21.15.19:0/1661603825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 msgr2=0x560a6bc82920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc82920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 -- 172.21.15.19:0/1661603825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 msgr2=0x560a6bc823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182186c640 1 -- 172.21.15.19:0/1661603825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6b8f9210 con 0x560a6b8c04d0 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182206d640 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc82920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f18137fe640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18080b28a0 0x7f18080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.608+0000 7f182206d640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18080b28a0 0x7f18080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.609+0000 7f182186c640 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 0x560a6bc89820 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f180c0096c0 tx=0x7f180c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52314a76fb9099f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.609+0000 7f18137fe640 1 -- 172.21.15.19:0/1685601747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f18140e2cb0 con 0x7f181c105210 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.609+0000 7f1810bfe640 1 -- 172.21.15.19:0/1661603825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c035050 con 0x560a6b8c04d0 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.609+0000 7f1810bfe640 1 -- 172.21.15.19:0/1661603825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f180c0044c0 con 0x560a6b8c04d0 2024-05-17T09:31:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.609+0000 7f1810bfe640 1 -- 172.21.15.19:0/1661603825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c003c70 con 0x560a6b8c04d0 2024-05-17T09:31:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 -- 172.21.15.19:0/1661603825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 msgr2=0x560a6bc89820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 0x560a6bc89820 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f180c0096c0 tx=0x7f180c000bc0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182206d640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18080b28a0 0x7f18080b4db0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x560a6bc83780 tx=0x7f1818005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 -- 172.21.15.19:0/1661603825 shutdown_connections 2024-05-17T09:31:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8c04d0 0x560a6bc89820 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc82920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 --2- 172.21.15.19:0/1661603825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 -- 172.21.15.19:0/1661603825 >> 172.21.15.19:0/1661603825 conn(0x560a6b8d5300 msgr2=0x560a6bc7dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 -- 172.21.15.19:0/1661603825 shutdown_connections 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 -- 172.21.15.19:0/1661603825 wait complete. 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 -- start start 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.610+0000 7f182b185740 1 Processor -- start 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8e11b0 0x560a6bc93110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a6b8ca790 con 0x560a6b8b82f0 2024-05-17T09:31:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a6bba0890 con 0x560a6b8c04d0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560a6bc8a210 con 0x560a6b8e11b0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182b185740 10 client.0 _ll_register_callbacks cb 0x560a6bba3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc8bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48488/0 (socket says 172.21.15.19:48488) 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 -- 172.21.15.19:0/462878889 learned_addr learned my addr 172.21.15.19:0/462878889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182186c640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182286e640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8e11b0 0x560a6bc93110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 -- 172.21.15.19:0/462878889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8e11b0 msgr2=0x560a6bc93110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8e11b0 0x560a6bc93110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 -- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 msgr2=0x560a6bc8c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc8c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 -- 172.21.15.19:0/462878889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6b9c5620 con 0x560a6b8b82f0 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182286e640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8e11b0 0x560a6bc93110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f182206d640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc8bce0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1818009750 tx=0x7f1818004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353c86b85c7014bd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f1805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.611+0000 7f1805ffb640 1 -- 172.21.15.19:0/462878889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818009b10 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f1805ffb640 1 -- 172.21.15.19:0/462878889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1818004260 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a6bbf6b80 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f1805ffb640 1 -- 172.21.15.19:0/462878889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1818004460 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 10 client.17547 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a6bc96110 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f1805ffb640 1 -- 172.21.15.19:0/462878889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560a6bbf6b80 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f1805ffb640 1 -- 172.21.15.19:0/462878889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1818027070 con 0x560a6b8b82f0 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 10 client.17547 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 20 client.17547 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b89fd80 0x560a6bc99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 4 client.17547 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6bbf0360 con 0x560a6b89fd80 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.612+0000 7f182186c640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b89fd80 0x560a6bc99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.613+0000 7f1805ffb640 10 client.17547 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.613+0000 7f182186c640 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b89fd80 0x560a6bc99960 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e51e1030a0f3b6 server_cookie=5fe57e8aa8191e25 in_seq=0 out_seq=0 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f1805ffb640 1 -- 172.21.15.19:0/462878889 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560a6bbf0360 con 0x560a6b89fd80 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f1805ffb640 10 client.17547 handle_command_reply: tid=0 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 client.17547 shutdown 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 msgr2=0x560a6bc8bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc8bce0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1818009750 tx=0x7f1818004730 comp rx=0 tx=0).stop 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 shutdown_connections 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b89fd80 0x560a6bc99960 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8e11b0 0x560a6bc93110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc8c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 --2- 172.21.15.19:0/462878889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc8bce0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f1805ffb640 0 client.17547 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 >> 172.21.15.19:0/462878889 conn(0x560a6b8d5300 msgr2=0x560a6bc80c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 shutdown_connections 2024-05-17T09:31:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 1 -- 172.21.15.19:0/462878889 wait complete. 2024-05-17T09:31:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.614+0000 7f182b185740 20 client.17547 trim_cache size 0 max 16384 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 Processor -- start 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 -- start start 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8c04d0 0x560a6bc930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1af2c0 con 0x560a6b8b82f0 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1acb90 con 0x560a6b8e11b0 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182b185740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1aa9e0 con 0x560a6b8c04d0 2024-05-17T09:31:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48504/0 (socket says 172.21.15.19:48504) 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 -- 172.21.15.19:0/2656984524 learned_addr learned my addr 172.21.15.19:0/2656984524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182286e640 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182186c640 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8c04d0 0x560a6bc930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 -- 172.21.15.19:0/2656984524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8c04d0 msgr2=0x560a6bc930a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8c04d0 0x560a6bc930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 -- 172.21.15.19:0/2656984524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 msgr2=0x560a6bc935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 -- 172.21.15.19:0/2656984524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6bc896c0 con 0x560a6b8b82f0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.630+0000 7f182206d640 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc868f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1818005eb0 tx=0x7f1818009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b067e85da9f3c5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f1807fff640 1 -- 172.21.15.19:0/2656984524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18180035f0 con 0x560a6b8b82f0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f1807fff640 1 -- 172.21.15.19:0/2656984524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1818003790 con 0x560a6b8b82f0 2024-05-17T09:31:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f1807fff640 1 -- 172.21.15.19:0/2656984524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818029800 con 0x560a6b8b82f0 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 -- 172.21.15.19:0/2656984524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 msgr2=0x560a6bc868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc868f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1818005eb0 tx=0x7f1818009b10 comp rx=0 tx=0).stop 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 -- 172.21.15.19:0/2656984524 shutdown_connections 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8e11b0 0x560a6bc935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8c04d0 0x560a6bc930a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 --2- 172.21.15.19:0/2656984524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8b82f0 0x560a6bc868f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 -- 172.21.15.19:0/2656984524 >> 172.21.15.19:0/2656984524 conn(0x560a6b8d5300 msgr2=0x560a6bc8b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 -- 172.21.15.19:0/2656984524 shutdown_connections 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.631+0000 7f182b185740 1 -- 172.21.15.19:0/2656984524 wait complete. 2024-05-17T09:31:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 -- start start 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 Processor -- start 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc92860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 0x560a6bcfffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1ab050 con 0x560a6b8e11b0 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1ac4c0 con 0x560a6b8c04d0 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1ab720 con 0x560a6b8b82f0 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182b185740 10 client.0 _ll_register_callbacks cb 0x7f18180208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182286e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 0x560a6bcfffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182286e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 0x560a6bcfffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48520/0 (socket says 172.21.15.19:48520) 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182286e640 1 -- 172.21.15.19:0/123310731 learned_addr learned my addr 172.21.15.19:0/123310731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182186c640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.632+0000 7f182206d640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc92860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182286e640 1 -- 172.21.15.19:0/123310731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 msgr2=0x560a6bc92860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182286e640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc92860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182286e640 1 -- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 msgr2=0x560a6bc92da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182286e640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc92da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182286e640 1 -- 172.21.15.19:0/123310731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6bd00520 con 0x560a6b8e11b0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182286e640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 0x560a6bcfffe0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f18140a7190 tx=0x7f18140a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e39b5c8a4e9cca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f1805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f1805ffb640 1 -- 172.21.15.19:0/123310731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181406e060 con 0x560a6b8e11b0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f1805ffb640 1 -- 172.21.15.19:0/123310731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f18140a6510 con 0x560a6b8e11b0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1818003790 con 0x560a6b8e11b0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f1805ffb640 1 -- 172.21.15.19:0/123310731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18140c4070 con 0x560a6b8e11b0 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182b185740 10 client.17553 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a6bd017c0 con 0x560a6b8e11b0 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f1805ffb640 1 -- 172.21.15.19:0/123310731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1818003790 con 0x560a6b8e11b0 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.633+0000 7f1805ffb640 1 -- 172.21.15.19:0/123310731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f18140a9180 con 0x560a6b8e11b0 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182b185740 10 client.17553 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182b185740 10 client.17553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182b185740 20 client.17553 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182b185740 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b8751c0 0x560a6bd04160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182b185740 4 client.17553 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a6bb6a5f0 con 0x560a6b8751c0 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182206d640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b8751c0 0x560a6bd04160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f1805ffb640 10 client.17553 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.634+0000 7f182206d640 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b8751c0 0x560a6bd04160 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff9c0606ba55c9f server_cookie=d9cfbc0243451022 in_seq=0 out_seq=0 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f1805ffb640 1 -- 172.21.15.19:0/123310731 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560a6bb6a5f0 con 0x560a6b8751c0 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f1805ffb640 10 client.17553 handle_command_reply: tid=0 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 client.17553 shutdown 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 msgr2=0x560a6bcfffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 0x560a6bcfffe0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f18140a7190 tx=0x7f18140a98e0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 shutdown_connections 2024-05-17T09:31:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a6b8751c0 0x560a6bd04160 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a6b8e11b0 0x560a6bcfffe0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a6b8c04d0 0x560a6bc92da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 --2- 172.21.15.19:0/123310731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a6b8b82f0 0x560a6bc92860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f1805ffb640 0 client.17553 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 >> 172.21.15.19:0/123310731 conn(0x560a6b8d5300 msgr2=0x560a6bc8ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 shutdown_connections 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 1 -- 172.21.15.19:0/123310731 wait complete. 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f182b185740 20 client.17553 trim_cache size 0 max 16384 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:31:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:31:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18080b28a0 msgr2=0x7f18080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18080b28a0 0x7f18080b4db0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x560a6bc83780 tx=0x7f1818005f90 comp rx=0 tx=0).stop 2024-05-17T09:31:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 msgr2=0x7f181c1a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 0x7f181c1a1670 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f181400eb10 tx=0x7f181400c2d0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 shutdown_connections 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18080b28a0 0x7f18080b4db0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181c105210 0x7f181c1a1670 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f181c1023c0 0x7f181c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 --2- 172.21.15.19:0/1685601747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f181c1018c0 0x7f181c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 >> 172.21.15.19:0/1685601747 conn(0x7f181c0fd500 msgr2=0x7f181c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 shutdown_connections 2024-05-17T09:31:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:33.660+0000 7f18242d1640 1 -- 172.21.15.19:0/1685601747 wait complete. 2024-05-17T09:31:34.447 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.65 MB/sec execute 452 sec latency 16702.539 ms 2024-05-17T09:31:34.484 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:31:34.747 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:31:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:34 smithi079 ceph-mon[35502]: pgmap v792: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-05-17T09:31:34.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:34 smithi019 ceph-mon[31620]: pgmap v792: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-05-17T09:31:34.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:34 smithi069 ceph-mon[35081]: pgmap v792: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-05-17T09:31:35.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.65 MB/sec execute 453 sec latency 17702.626 ms 2024-05-17T09:31:36.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.64 MB/sec execute 454 sec latency 18702.692 ms 2024-05-17T09:31:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 -- 172.21.15.19:0/1941380798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 msgr2=0x7f7e3c110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/1941380798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c110700 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009990 tx=0x7f7e2c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 -- 172.21.15.19:0/1941380798 shutdown_connections 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/1941380798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e3c110c40 0x7f7e3c113060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/1941380798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c103090 0x7f7e3c1034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/1941380798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c110700 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 -- 172.21.15.19:0/1941380798 >> 172.21.15.19:0/1941380798 conn(0x7f7e3c0fef20 msgr2=0x7f7e3c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 -- 172.21.15.19:0/1941380798 shutdown_connections 2024-05-17T09:31:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.475+0000 7f7e43ab1700 1 -- 172.21.15.19:0/1941380798 wait complete. 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.476+0000 7f7e43ab1700 1 Processor -- start 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.476+0000 7f7e43ab1700 1 -- start start 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.476+0000 7f7e43ab1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e3c103090 0x7f7e3c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.476+0000 7f7e43ab1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e43ab1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c110c40 0x7f7e3c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e43ab1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1165f0 con 0x7f7e3c103a70 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e43ab1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c116470 con 0x7f7e3c110c40 2024-05-17T09:31:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e43ab1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c116770 con 0x7f7e3c103090 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42446/0 (socket says 172.21.15.19:42446) 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 -- 172.21.15.19:0/2832292808 learned_addr learned my addr 172.21.15.19:0/2832292808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4204e700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c110c40 0x7f7e3c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4184d700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e3c103090 0x7f7e3c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 -- 172.21.15.19:0/2832292808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e3c103090 msgr2=0x7f7e3c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e3c103090 0x7f7e3c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 -- 172.21.15.19:0/2832292808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c110c40 msgr2=0x7f7e3c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c110c40 0x7f7e3c1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 -- 172.21.15.19:0/2832292808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c1a1ef0 con 0x7f7e3c103a70 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4204e700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c110c40 0x7f7e3c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e4104c700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c19d5c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b5c0 tx=0x7f7e2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c004830 con 0x7f7e3c103a70 2024-05-17T09:31:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c1a2180 con 0x7f7e3c103a70 2024-05-17T09:31:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e2c005020 con 0x7f7e3c103a70 2024-05-17T09:31:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00f950 con 0x7f7e3c103a70 2024-05-17T09:31:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.477+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a2600 con 0x7f7e3c103a70 2024-05-17T09:31:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.478+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c0512b0 con 0x7f7e3c103a70 2024-05-17T09:31:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.481+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e2c0049d0 con 0x7f7e3c103a70 2024-05-17T09:31:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.482+0000 7f7e32ffd700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e280b2880 0x7f7e280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.482+0000 7f7e4184d700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e280b2880 0x7f7e280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.482+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7e2c0e3690 con 0x7f7e3c103a70 2024-05-17T09:31:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.482+0000 7f7e4184d700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e280b2880 0x7f7e280b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e38005950 tx=0x7f7e380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.485+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e2c013070 con 0x7f7e3c103a70 2024-05-17T09:31:36.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.753+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e3c02e5e0 con 0x7f7e3c103a70 2024-05-17T09:31:36.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:36 smithi019 ceph-mon[31620]: pgmap v793: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-05-17T09:31:36.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.754+0000 7f7e32ffd700 1 -- 172.21.15.19:0/2832292808 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f7e2c0ab030 con 0x7f7e3c103a70 2024-05-17T09:31:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:31:36.754 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e280b2880 msgr2=0x7f7e280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e280b2880 0x7f7e280b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e38005950 tx=0x7f7e380058e0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 msgr2=0x7f7e3c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c19d5c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b5c0 tx=0x7f7e2c0047b0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 shutdown_connections 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e280b2880 0x7f7e280b4d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e3c103090 0x7f7e3c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c110c40 0x7f7e3c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 --2- 172.21.15.19:0/2832292808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e3c103a70 0x7f7e3c19d5c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 >> 172.21.15.19:0/2832292808 conn(0x7f7e3c0fef20 msgr2=0x7f7e3c112880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 shutdown_connections 2024-05-17T09:31:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:36.756+0000 7f7e43ab1700 1 -- 172.21.15.19:0/2832292808 wait complete. 2024-05-17T09:31:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:31:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:36 smithi079 ceph-mon[35502]: pgmap v793: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-05-17T09:31:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:36 smithi069 ceph-mon[35081]: pgmap v793: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-05-17T09:31:37.397 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 23e7ba2f --release --await' 2024-05-17T09:31:37.397 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 23e7ba2f --release --await 2024-05-17T09:31:37.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.64 MB/sec execute 455 sec latency 19703.000 ms 2024-05-17T09:31:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:31:37.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:37 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2832292808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:37.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:37 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2832292808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:37.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:37 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2832292808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:31:38.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.64 MB/sec execute 456 sec latency 20703.069 ms 2024-05-17T09:31:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:38 smithi079 ceph-mon[35502]: pgmap v794: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-05-17T09:31:38.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:38 smithi019 ceph-mon[31620]: pgmap v794: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-05-17T09:31:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:38 smithi069 ceph-mon[35081]: pgmap v794: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-05-17T09:31:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.417+0000 7f62d8227700 1 -- 172.21.15.19:0/2281655921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 msgr2=0x7f62d0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.417+0000 7f62d8227700 1 --2- 172.21.15.19:0/2281655921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d0108200 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f62c00099c0 tx=0x7f62c0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 -- 172.21.15.19:0/2281655921 shutdown_connections 2024-05-17T09:31:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 --2- 172.21.15.19:0/2281655921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62d01034c0 0x7f62d01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 --2- 172.21.15.19:0/2281655921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d0108740 0x7f62d01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 --2- 172.21.15.19:0/2281655921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d0108200 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 -- 172.21.15.19:0/2281655921 >> 172.21.15.19:0/2281655921 conn(0x7f62d00fef10 msgr2=0x7f62d0101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 -- 172.21.15.19:0/2281655921 shutdown_connections 2024-05-17T09:31:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 -- 172.21.15.19:0/2281655921 wait complete. 2024-05-17T09:31:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 Processor -- start 2024-05-17T09:31:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.418+0000 7f62d8227700 1 -- start start 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d8227700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62d01034c0 0x7f62d00ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d8227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d00ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d8227700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d0108740 0x7f62d00ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d8227700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0113250 con 0x7f62d0107df0 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d8227700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01130d0 con 0x7f62d0108740 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d8227700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01133d0 con 0x7f62d01034c0 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d00ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d00ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42462/0 (socket says 172.21.15.19:42462) 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 -- 172.21.15.19:0/2486757949 learned_addr learned my addr 172.21.15.19:0/2486757949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d67c4700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d0108740 0x7f62d00ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62d01034c0 0x7f62d00ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 -- 172.21.15.19:0/2486757949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62d01034c0 msgr2=0x7f62d00ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62d01034c0 0x7f62d00ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 -- 172.21.15.19:0/2486757949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d0108740 msgr2=0x7f62d00ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d0108740 0x7f62d00ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.419+0000 7f62d57c2700 1 -- 172.21.15.19:0/2486757949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d01a9130 con 0x7f62d0107df0 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.420+0000 7f62d57c2700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d00ff930 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098e0 tx=0x7f62c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.420+0000 7f62c6ffd700 1 -- 172.21.15.19:0/2486757949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0027070 con 0x7f62d0107df0 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.420+0000 7f62c6ffd700 1 -- 172.21.15.19:0/2486757949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62c000fc00 con 0x7f62d0107df0 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.420+0000 7f62c6ffd700 1 -- 172.21.15.19:0/2486757949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0022ca0 con 0x7f62d0107df0 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.420+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d01a93c0 con 0x7f62d0107df0 2024-05-17T09:31:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.420+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d01a9810 con 0x7f62d0107df0 2024-05-17T09:31:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.423+0000 7f62c6ffd700 1 -- 172.21.15.19:0/2486757949 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f62c002a070 con 0x7f62d0107df0 2024-05-17T09:31:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.423+0000 7f62c6ffd700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62bc0b2880 0x7f62bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.424+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62bc0b2880 0x7f62bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.424+0000 7f62c6ffd700 1 -- 172.21.15.19:0/2486757949 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62c00e3570 con 0x7f62d0107df0 2024-05-17T09:31:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.424+0000 7f62e6c3cb80 1 Processor -- start 2024-05-17T09:31:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.424+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62bc0b2880 0x7f62bc0b4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f62cc005950 tx=0x7f62cc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.424+0000 7f62e6c3cb80 1 -- start start 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 0x556823ba5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a20f40 0x556823ba5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823b9f620 0x556823b9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556823acdd90 con 0x556823a9e4d0 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556823abd930 con 0x556823b9f620 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556823ac1970 con 0x556823a20f40 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 0x556823ba5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 0x556823ba5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42478/0 (socket says 172.21.15.19:42478) 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 -- 172.21.15.19:0/2285967749 learned_addr learned my addr 172.21.15.19:0/2285967749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d57c2700 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a20f40 0x556823ba5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d67c4700 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823b9f620 0x556823b9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 -- 172.21.15.19:0/2285967749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a20f40 msgr2=0x556823ba5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a20f40 0x556823ba5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 -- 172.21.15.19:0/2285967749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823b9f620 msgr2=0x556823b9fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823b9f620 0x556823b9fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 -- 172.21.15.19:0/2285967749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556823a0d9d0 con 0x556823a9e4d0 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 0x556823ba5560 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00fde0 tx=0x7f62cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6906059e70d5498 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.425+0000 7f62bbfff700 1 -- 172.21.15.19:0/2285967749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc016070 con 0x556823a9e4d0 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62bbfff700 1 -- 172.21.15.19:0/2285967749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62cc025470 con 0x556823a9e4d0 2024-05-17T09:31:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62bbfff700 1 -- 172.21.15.19:0/2285967749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc01b680 con 0x556823a9e4d0 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/2285967749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 msgr2=0x556823ba5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 0x556823ba5560 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00fde0 tx=0x7f62cc012000 comp rx=0 tx=0).stop 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/2285967749 shutdown_connections 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823b9f620 0x556823b9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a9e4d0 0x556823ba5560 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/2285967749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a20f40 0x556823ba5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.426+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/2285967749 >> 172.21.15.19:0/2285967749 conn(0x556823a5fd80 msgr2=0x556823b9a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.427+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/2285967749 shutdown_connections 2024-05-17T09:31:39.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.427+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/2285967749 wait complete. 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.427+0000 7f62e6c3cb80 1 -- start start 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.427+0000 7f62e6c3cb80 1 Processor -- start 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823ba9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568239c40a0 con 0x556823b9f620 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556823a5f8c0 con 0x556823a20f40 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556823bb53d0 con 0x556823a9e4d0 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62e6c3cb80 10 client.0 _ll_register_callbacks cb 0x556823a58a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42488/0 (socket says 172.21.15.19:42488) 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 -- 172.21.15.19:0/523521907 learned_addr learned my addr 172.21.15.19:0/523521907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d57c2700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823ba9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 -- 172.21.15.19:0/523521907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 msgr2=0x556823ba9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823ba9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 -- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 msgr2=0x556823ba9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 -- 172.21.15.19:0/523521907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556823a5c790 con 0x556823b9f620 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d57c2700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823ba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62d67c4700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba9ca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009f40 tx=0x7f62c8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556d1dd187ce3148 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.428+0000 7f62b97fa700 1 -- 172.21.15.19:0/523521907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8019020 con 0x556823b9f620 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62b97fa700 1 -- 172.21.15.19:0/523521907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62c8007710 con 0x556823b9f620 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62cc025470 con 0x556823b9f620 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62b97fa700 1 -- 172.21.15.19:0/523521907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62c8024050 con 0x556823b9f620 2024-05-17T09:31:39.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 10 client.17571 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556823bb6d60 con 0x556823b9f620 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62b97fa700 1 -- 172.21.15.19:0/523521907 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62cc025470 con 0x556823b9f620 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62b97fa700 1 -- 172.21.15.19:0/523521907 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f62c80158f0 con 0x556823b9f620 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 10 client.17571 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 10 client.17571 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 20 client.17571 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823bb93a0 0x556823bbb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 4 client.17571 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.429+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556823acfce0 con 0x556823bb93a0 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.430+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823bb93a0 0x556823bbb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.430+0000 7f62b97fa700 10 client.17571 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:39.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.430+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823bb93a0 0x556823bbb820 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472ab0f46ae593c6 server_cookie=915c467cbcc0b7c8 in_seq=0 out_seq=0 2024-05-17T09:31:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62b97fa700 1 -- 172.21.15.19:0/523521907 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556823acfce0 con 0x556823bb93a0 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62b97fa700 10 client.17571 handle_command_reply: tid=0 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 client.17571 shutdown 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 msgr2=0x556823ba9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba9ca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009f40 tx=0x7f62c8004320 comp rx=0 tx=0).stop 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 shutdown_connections 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823bb93a0 0x556823bbb820 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba9ca0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/523521907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823ba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62b97fa700 0 client.17571 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 >> 172.21.15.19:0/523521907 conn(0x556823a5fd80 msgr2=0x556823b94280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 shutdown_connections 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/523521907 wait complete. 2024-05-17T09:31:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.431+0000 7f62e6c3cb80 20 client.17571 trim_cache size 0 max 16384 2024-05-17T09:31:39.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181717 1.63 MB/sec execute 457 sec latency 21703.111 ms 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 Processor -- start 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 -- start start 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823b9f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b4d30 con 0x556823b9f620 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b25c0 con 0x556823a20f40 2024-05-17T09:31:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b0410 con 0x556823a9e4d0 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62d67c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62d67c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42492/0 (socket says 172.21.15.19:42492) 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62d67c4700 1 -- 172.21.15.19:0/3514182424 learned_addr learned my addr 172.21.15.19:0/3514182424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62d57c2700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823b9f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.452+0000 7f62d67c4700 1 -- 172.21.15.19:0/3514182424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 msgr2=0x556823b9f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d67c4700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823b9f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d67c4700 1 -- 172.21.15.19:0/3514182424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 msgr2=0x556823ba3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d67c4700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d67c4700 1 -- 172.21.15.19:0/3514182424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556823b9bd20 con 0x556823b9f620 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d57c2700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823b9f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:31:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62d67c4700 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba1e70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f62c8015d10 tx=0x7f62c8003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04ed9dc904a3566 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62bb7fe700 1 -- 172.21.15.19:0/3514182424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8019040 con 0x556823b9f620 2024-05-17T09:31:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62bb7fe700 1 -- 172.21.15.19:0/3514182424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62c8005e60 con 0x556823b9f620 2024-05-17T09:31:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.453+0000 7f62bb7fe700 1 -- 172.21.15.19:0/3514182424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8007970 con 0x556823b9f620 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3514182424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 msgr2=0x556823ba1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba1e70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f62c8015d10 tx=0x7f62c8003b30 comp rx=0 tx=0).stop 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3514182424 shutdown_connections 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823b9f620 0x556823ba1e70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a20f40 0x556823ba3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3514182424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823a9e4d0 0x556823b9f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3514182424 >> 172.21.15.19:0/3514182424 conn(0x556823a5fd80 msgr2=0x556823b948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3514182424 shutdown_connections 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3514182424 wait complete. 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 -- start start 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.454+0000 7f62e6c3cb80 1 Processor -- start 2024-05-17T09:31:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 0x556823bac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a9e4d0 0x556823bac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823b9f620 0x556823bacd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b0a80 con 0x556823a20f40 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b1ef0 con 0x556823a9e4d0 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b1150 con 0x556823b9f620 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62e6c3cb80 10 client.0 _ll_register_callbacks cb 0x556823a58a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 0x556823bac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 0x556823bac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42502/0 (socket says 172.21.15.19:42502) 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 -- 172.21.15.19:0/3685557778 learned_addr learned my addr 172.21.15.19:0/3685557778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d57c2700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a9e4d0 0x556823bac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d67c4700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823b9f620 0x556823bacd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 -- 172.21.15.19:0/3685557778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823b9f620 msgr2=0x556823bacd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823b9f620 0x556823bacd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 -- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a9e4d0 msgr2=0x556823bac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a9e4d0 0x556823bac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.455+0000 7f62d5fc3700 1 -- 172.21.15.19:0/3685557778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556823c23eb0 con 0x556823a20f40 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62d5fc3700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 0x556823bac2f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00fde0 tx=0x7f62cc025500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df684fe7750e0511 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62b97fa700 1 -- 172.21.15.19:0/3685557778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc016040 con 0x556823a20f40 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62b97fa700 1 -- 172.21.15.19:0/3685557778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62cc010840 con 0x556823a20f40 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c8005e60 con 0x556823a20f40 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62b97fa700 1 -- 172.21.15.19:0/3685557778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62cc03d050 con 0x556823a20f40 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62e6c3cb80 10 client.17577 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:39.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556823c24fb0 con 0x556823a20f40 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62b97fa700 1 -- 172.21.15.19:0/3685557778 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62c8005e60 con 0x556823a20f40 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62b97fa700 1 -- 172.21.15.19:0/3685557778 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f62cc005030 con 0x556823a20f40 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.456+0000 7f62e6c3cb80 10 client.17577 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62e6c3cb80 10 client.17577 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62e6c3cb80 20 client.17577 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823c25500 0x556823c27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62e6c3cb80 4 client.17577 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "23e7ba2f", "release": true, "await": true}] 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "23e7ba2f", "release": true, "await": true}) -- 0x556823ac94c0 con 0x556823c25500 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62d57c2700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823c25500 0x556823c27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62d57c2700 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823c25500 0x556823c27920 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108b44288c22bfd4 server_cookie=3a47640360d17297 in_seq=0 out_seq=0 2024-05-17T09:31:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.457+0000 7f62b97fa700 10 client.17577 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62b97fa700 1 -- 172.21.15.19:0/3685557778 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556823ac94c0 con 0x556823c25500 2024-05-17T09:31:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62b97fa700 10 client.17577 handle_command_reply: tid=0 2024-05-17T09:31:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 client.17577 shutdown 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 msgr2=0x556823bac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 0x556823bac2f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00fde0 tx=0x7f62cc025500 comp rx=0 tx=0).stop 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 shutdown_connections 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556823b9f620 0x556823bacd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556823c25500 0x556823c27920 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556823a20f40 0x556823bac2f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 --2- 172.21.15.19:0/3685557778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556823a9e4d0 0x556823bac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62b97fa700 0 client.17577 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 >> 172.21.15.19:0/3685557778 conn(0x556823a5fd80 msgr2=0x556823baa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 shutdown_connections 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 1 -- 172.21.15.19:0/3685557778 wait complete. 2024-05-17T09:31:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.461+0000 7f62e6c3cb80 20 client.17577 trim_cache size 0 max 16384 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 44, 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "23e7ba2f": { 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 44, 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:31:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.8, 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:31:39.465 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:31:39.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62bc0b2880 msgr2=0x7f62bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62bc0b2880 0x7f62bc0b4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f62cc005950 tx=0x7f62cc00a300 comp rx=0 tx=0).stop 2024-05-17T09:31:39.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 msgr2=0x7f62d00ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d00ff930 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098e0 tx=0x7f62c00047b0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 shutdown_connections 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62bc0b2880 0x7f62bc0b4d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62d01034c0 0x7f62d00ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d0108740 0x7f62d00ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 --2- 172.21.15.19:0/2486757949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62d0107df0 0x7f62d00ff930 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 >> 172.21.15.19:0/2486757949 conn(0x7f62d00fef10 msgr2=0x7f62d006ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 shutdown_connections 2024-05-17T09:31:39.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:31:39.462+0000 7f62d8227700 1 -- 172.21.15.19:0/2486757949 wait complete. 2024-05-17T09:31:40.198 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '23e7ba2f', total seconds elapsed: 28.5 2024-05-17T09:31:40.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 182174 1.63 MB/sec execute 458 sec latency 21717.450 ms 2024-05-17T09:31:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:40 smithi079 ceph-mon[35502]: pgmap v795: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-05-17T09:31:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:40 smithi069 ceph-mon[35081]: pgmap v795: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-05-17T09:31:40.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:40 smithi019 ceph-mon[31620]: pgmap v795: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-05-17T09:31:41.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 182806 1.64 MB/sec execute 459 sec latency 11.738 ms 2024-05-17T09:31:42.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 183354 1.64 MB/sec execute 460 sec latency 15.620 ms 2024-05-17T09:31:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:42 smithi079 ceph-mon[35502]: pgmap v796: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-17T09:31:42.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:42 smithi069 ceph-mon[35081]: pgmap v796: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-17T09:31:42.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:42 smithi019 ceph-mon[31620]: pgmap v796: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-17T09:31:43.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 184086 1.65 MB/sec execute 461 sec latency 16.191 ms 2024-05-17T09:31:44.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 184463 1.65 MB/sec execute 462 sec latency 16.074 ms 2024-05-17T09:31:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:44 smithi079 ceph-mon[35502]: pgmap v797: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 85 KiB/s wr, 17 op/s 2024-05-17T09:31:44.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:44 smithi069 ceph-mon[35081]: pgmap v797: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 85 KiB/s wr, 17 op/s 2024-05-17T09:31:44.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:44 smithi019 ceph-mon[31620]: pgmap v797: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 85 KiB/s wr, 17 op/s 2024-05-17T09:31:45.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 184813 1.65 MB/sec execute 463 sec latency 16.476 ms 2024-05-17T09:31:46.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 185069 1.65 MB/sec execute 464 sec latency 14.944 ms 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 -- 172.21.15.19:0/111911096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41081e0 msgr2=0x7fa8d410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 --2- 172.21.15.19:0/111911096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41081e0 0x7fa8d410a600 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc02c650 comp rx=0 tx=0).stop 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 -- 172.21.15.19:0/111911096 shutdown_connections 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 --2- 172.21.15.19:0/111911096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41081e0 0x7fa8d410a600 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 --2- 172.21.15.19:0/111911096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41034e0 0x7fa8d41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 --2- 172.21.15.19:0/111911096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41022a0 0x7fa8d4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 -- 172.21.15.19:0/111911096 >> 172.21.15.19:0/111911096 conn(0x7fa8d40fd470 msgr2=0x7fa8d40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 -- 172.21.15.19:0/111911096 shutdown_connections 2024-05-17T09:31:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.769+0000 7fa8dc461640 1 -- 172.21.15.19:0/111911096 wait complete. 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 Processor -- start 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 -- start start 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 0x7fa8d419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41034e0 0x7fa8d419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41081e0 0x7fa8d41a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d410d3e0 con 0x7fa8d41022a0 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d410d260 con 0x7fa8d41034e0 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8dc461640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d410d560 con 0x7fa8d41081e0 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 0x7fa8d419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 0x7fa8d419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36840/0 (socket says 172.21.15.19:36840) 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8da1fd640 1 -- 172.21.15.19:0/2660879665 learned_addr learned my addr 172.21.15.19:0/2660879665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41081e0 0x7fa8d41a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.770+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41034e0 0x7fa8d419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da9fe640 1 -- 172.21.15.19:0/2660879665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41034e0 msgr2=0x7fa8d419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41034e0 0x7fa8d419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da9fe640 1 -- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 msgr2=0x7fa8d419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 0x7fa8d419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da9fe640 1 -- 172.21.15.19:0/2660879665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d41a3e70 con 0x7fa8d41081e0 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 0x7fa8d419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41034e0 0x7fa8d419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41081e0 0x7fa8d41a3800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc036040 tx=0x7fa8cc032c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8cb7fe640 1 -- 172.21.15.19:0/2660879665 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc038050 con 0x7fa8d41081e0 2024-05-17T09:31:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8cb7fe640 1 -- 172.21.15.19:0/2660879665 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8cc039040 con 0x7fa8d41081e0 2024-05-17T09:31:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d41a4100 con 0x7fa8d41081e0 2024-05-17T09:31:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8cb7fe640 1 -- 172.21.15.19:0/2660879665 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0042a0 con 0x7fa8d41081e0 2024-05-17T09:31:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.771+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41a45f0 con 0x7fa8d41081e0 2024-05-17T09:31:46.775 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:46 smithi079 ceph-mon[35502]: pgmap v798: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 303 KiB/s wr, 46 op/s 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.774+0000 7fa8e33f8740 1 Processor -- start 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.774+0000 7fa8cb7fe640 1 -- 172.21.15.19:0/2660879665 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa8cc044020 con 0x7fa8d41081e0 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.774+0000 7fa8e33f8740 1 -- start start 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4841b0 0x56094f8253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f825920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f82c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56094f797c30 con 0x56094f4634d0 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56094f7877d0 con 0x56094f4841b0 2024-05-17T09:31:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56094f78b810 con 0x56094f45b2f0 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8d99fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4841b0 0x56094f8253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f825920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8d99fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4841b0 0x56094f8253a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33206/0 (socket says 172.21.15.19:33206) 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f825920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33232/0 (socket says 172.21.15.19:33232) 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8d99fc640 1 -- 172.21.15.19:0/3498189053 learned_addr learned my addr 172.21.15.19:0/3498189053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f82c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 -- 172.21.15.19:0/3498189053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4841b0 msgr2=0x56094f8253a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4841b0 0x56094f8253a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 -- 172.21.15.19:0/3498189053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 msgr2=0x56094f82c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f82c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 -- 172.21.15.19:0/3498189053 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094f49c210 con 0x56094f45b2f0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8cb7fe640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8c00b28a0 0x7fa8c00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8c00b28a0 0x7fa8c00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.775+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f825920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e520 tx=0x7fa8d000a370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd7d3f0bdda6489 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.776+0000 7fa8cb7fe640 1 -- 172.21.15.19:0/2660879665 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa8cc1025d0 con 0x7fa8d41081e0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.776+0000 7fa8c8bfe640 1 -- 172.21.15.19:0/3498189053 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004310 con 0x56094f45b2f0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.776+0000 7fa8c8bfe640 1 -- 172.21.15.19:0/3498189053 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8d00044b0 con 0x56094f45b2f0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.776+0000 7fa8c8bfe640 1 -- 172.21.15.19:0/3498189053 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000d9a0 con 0x56094f45b2f0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- 172.21.15.19:0/3498189053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 msgr2=0x56094f825920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8c00b28a0 0x7fa8c00b4db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x56094f826570 tx=0x7fa8c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f825920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e520 tx=0x7fa8d000a370 comp rx=0 tx=0).stop 2024-05-17T09:31:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- 172.21.15.19:0/3498189053 shutdown_connections 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f82c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f825920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/3498189053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4841b0 0x56094f8253a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- 172.21.15.19:0/3498189053 >> 172.21.15.19:0/3498189053 conn(0x56094f478300 msgr2=0x56094f820d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- 172.21.15.19:0/3498189053 shutdown_connections 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- 172.21.15.19:0/3498189053 wait complete. 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- start start 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 Processor -- start 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f82ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f82f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 0x56094f836180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56094f46d790 con 0x56094f4841b0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56094f743890 con 0x56094f4634d0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.777+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56094f82d270 con 0x56094f45b2f0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8e33f8740 10 client.0 _ll_register_callbacks cb 0x56094f746d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 0x56094f836180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f82ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f82ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33238/0 (socket says 172.21.15.19:33238) 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da1fd640 1 -- 172.21.15.19:0/825986732 learned_addr learned my addr 172.21.15.19:0/825986732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 0x56094f836180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36856/0 (socket says 172.21.15.19:36856) 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f82f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 -- 172.21.15.19:0/825986732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 msgr2=0x56094f82ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f82ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 -- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 msgr2=0x56094f82f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f82f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 -- 172.21.15.19:0/825986732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094f568620 con 0x56094f4841b0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 0x56094f836180 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0ffd20 tx=0x7fa8cc0c96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8142219a3484597 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8bdffb640 1 -- 172.21.15.19:0/825986732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc033ab0 con 0x56094f4841b0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8bdffb640 1 -- 172.21.15.19:0/825986732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa8cc0ec0e0 con 0x56094f4841b0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d00044b0 con 0x56094f4841b0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.778+0000 7fa8bdffb640 1 -- 172.21.15.19:0/825986732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8cc0c9f40 con 0x56094f4841b0 2024-05-17T09:31:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 10 client.17589 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56094f8390b0 con 0x56094f4841b0 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8bdffb640 1 -- 172.21.15.19:0/825986732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8d00044b0 con 0x56094f4841b0 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8bdffb640 1 -- 172.21.15.19:0/825986732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa8cc0c9aa0 con 0x56094f4841b0 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 10 client.17589 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 20 client.17589 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f442d80 0x56094f83c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 4 client.17589 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56094f799b80 con 0x56094f442d80 2024-05-17T09:31:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.779+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f442d80 0x56094f83c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.780+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f442d80 0x56094f83c810 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e460b4598068221 server_cookie=902acb479269492 in_seq=0 out_seq=0 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.780+0000 7fa8bdffb640 10 client.17589 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.781+0000 7fa8bdffb640 1 -- 172.21.15.19:0/825986732 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56094f799b80 con 0x56094f442d80 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.781+0000 7fa8bdffb640 10 client.17589 handle_command_reply: tid=0 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.781+0000 7fa8e33f8740 1 client.17589 shutdown 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 msgr2=0x56094f836180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 0x56094f836180 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0ffd20 tx=0x7fa8cc0c96d0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 shutdown_connections 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f442d80 0x56094f83c810 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4841b0 0x56094f836180 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f82f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/825986732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f45b2f0 0x56094f82ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 >> 172.21.15.19:0/825986732 conn(0x56094f478300 msgr2=0x56094f823c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8bdffb640 0 client.17589 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 shutdown_connections 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 1 -- 172.21.15.19:0/825986732 wait complete. 2024-05-17T09:31:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.782+0000 7fa8e33f8740 20 client.17589 trim_cache size 0 max 16384 2024-05-17T09:31:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 Processor -- start 2024-05-17T09:31:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 -- start start 2024-05-17T09:31:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 0x56094f820e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f8360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f823b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b13f0 con 0x56094f45b2f0 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41aecc0 con 0x56094f4634d0 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41acb10 con 0x56094f4841b0 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 0x56094f820e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 0x56094f820e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36872/0 (socket says 172.21.15.19:36872) 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8da1fd640 1 -- 172.21.15.19:0/2637218650 learned_addr learned my addr 172.21.15.19:0/2637218650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f8360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.806+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f823b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8da1fd640 1 -- 172.21.15.19:0/2637218650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 msgr2=0x56094f823b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f823b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8da1fd640 1 -- 172.21.15.19:0/2637218650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 msgr2=0x56094f8360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f8360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8da1fd640 1 -- 172.21.15.19:0/2637218650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094f82fe20 con 0x56094f45b2f0 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f8360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 0x56094f820e00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e470 tx=0x7fa8d000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5168da8e30534cb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8bffff640 1 -- 172.21.15.19:0/2637218650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d002e3a0 con 0x56094f45b2f0 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8bffff640 1 -- 172.21.15.19:0/2637218650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8d0033070 con 0x56094f45b2f0 2024-05-17T09:31:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.807+0000 7fa8bffff640 1 -- 172.21.15.19:0/2637218650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0003a80 con 0x56094f45b2f0 2024-05-17T09:31:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 -- 172.21.15.19:0/2637218650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 msgr2=0x56094f820e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 0x56094f820e00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e470 tx=0x7fa8d000a250 comp rx=0 tx=0).stop 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 -- 172.21.15.19:0/2637218650 shutdown_connections 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f823b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f4634d0 0x56094f8360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/2637218650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f45b2f0 0x56094f820e00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 -- 172.21.15.19:0/2637218650 >> 172.21.15.19:0/2637218650 conn(0x56094f478300 msgr2=0x56094f8206f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 -- 172.21.15.19:0/2637218650 shutdown_connections 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.808+0000 7fa8e33f8740 1 -- 172.21.15.19:0/2637218650 wait complete. 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 -- start start 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 Processor -- start 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 0x56094f833320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f833860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f834790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41ad180 con 0x56094f4634d0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41ae5f0 con 0x56094f45b2f0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41ad850 con 0x56094f4841b0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8e33f8740 10 client.0 _ll_register_callbacks cb 0x7fa8cc0c5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8d99fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f833860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 0x56094f833320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8da9fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f834790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8da1fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 0x56094f833320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33226/0 (socket says 172.21.15.19:33226) 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8da1fd640 1 -- 172.21.15.19:0/950826967 learned_addr learned my addr 172.21.15.19:0/950826967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.809+0000 7fa8da9fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f834790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33260/0 (socket says 172.21.15.19:33260) 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8d99fc640 1 -- 172.21.15.19:0/950826967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 msgr2=0x56094f834790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f834790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8d99fc640 1 -- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 msgr2=0x56094f833320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 0x56094f833320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8d99fc640 1 -- 172.21.15.19:0/950826967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094f834d50 con 0x56094f4634d0 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 0x56094f833320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8da9fe640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f834790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8d99fc640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f833860 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40114b0 tx=0x7fa8c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ec18999ed85c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8bdffb640 1 -- 172.21.15.19:0/950826967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4020cb0 con 0x56094f4634d0 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8bdffb640 1 -- 172.21.15.19:0/950826967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa8c401a050 con 0x56094f4634d0 2024-05-17T09:31:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d0033070 con 0x56094f4634d0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.810+0000 7fa8bdffb640 1 -- 172.21.15.19:0/950826967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8c4020e50 con 0x56094f4634d0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 10 client.17595 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56094f836080 con 0x56094f4634d0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8bdffb640 1 -- 172.21.15.19:0/950826967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8d0033070 con 0x56094f4634d0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8bdffb640 1 -- 172.21.15.19:0/950826967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa8c40047a0 con 0x56094f4634d0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 10 client.17595 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 10 client.17595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 20 client.17595 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f4181c0 0x56094f8a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 4 client.17595 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56094f793360 con 0x56094f4181c0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.811+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f4181c0 0x56094f8a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.812+0000 7fa8da1fd640 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f4181c0 0x56094f8a7430 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f8a2b981244c42 server_cookie=b7ff0f24b47c966 in_seq=0 out_seq=0 2024-05-17T09:31:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.812+0000 7fa8bdffb640 10 client.17595 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.827+0000 7fa8bdffb640 1 -- 172.21.15.19:0/950826967 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56094f793360 con 0x56094f4181c0 2024-05-17T09:31:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.827+0000 7fa8bdffb640 10 client.17595 handle_command_reply: tid=0 2024-05-17T09:31:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.827+0000 7fa8e33f8740 1 client.17595 shutdown 2024-05-17T09:31:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 msgr2=0x56094f833860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f833860 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40114b0 tx=0x7fa8c4004730 comp rx=0 tx=0).stop 2024-05-17T09:31:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 shutdown_connections 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56094f4181c0 0x56094f8a7430 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56094f4841b0 0x56094f834790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8bdffb640 0 client.17595 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56094f4634d0 0x56094f833860 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 --2- 172.21.15.19:0/950826967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56094f45b2f0 0x56094f833320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 >> 172.21.15.19:0/950826967 conn(0x56094f478300 msgr2=0x56094f828790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 shutdown_connections 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 1 -- 172.21.15.19:0/950826967 wait complete. 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8e33f8740 20 client.17595 trim_cache size 0 max 16384 2024-05-17T09:31:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:31:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:31:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:31:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8c00b28a0 msgr2=0x7fa8c00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8dc461640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8c00b28a0 0x7fa8c00b4db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x56094f826570 tx=0x7fa8c4005f90 comp rx=0 tx=0).stop 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41081e0 msgr2=0x7fa8d41a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.828+0000 7fa8dc461640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41081e0 0x7fa8d41a3800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc036040 tx=0x7fa8cc032c10 comp rx=0 tx=0).stop 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 shutdown_connections 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8d41081e0 0x7fa8d41a3800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d41034e0 0x7fa8d419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8c00b28a0 0x7fa8c00b4db0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 --2- 172.21.15.19:0/2660879665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41022a0 0x7fa8d419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 >> 172.21.15.19:0/2660879665 conn(0x7fa8d40fd470 msgr2=0x7fa8d41041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 shutdown_connections 2024-05-17T09:31:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:46.829+0000 7fa8dc461640 1 -- 172.21.15.19:0/2660879665 wait complete. 2024-05-17T09:31:46.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:46 smithi069 ceph-mon[35081]: pgmap v798: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 303 KiB/s wr, 46 op/s 2024-05-17T09:31:46.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:46 smithi019 ceph-mon[31620]: pgmap v798: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 303 KiB/s wr, 46 op/s 2024-05-17T09:31:47.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 185563 1.65 MB/sec execute 465 sec latency 17.145 ms 2024-05-17T09:31:48.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 186103 1.65 MB/sec execute 466 sec latency 15.897 ms 2024-05-17T09:31:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:48 smithi079 ceph-mon[35502]: pgmap v799: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 600 KiB/s wr, 53 op/s 2024-05-17T09:31:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:48 smithi069 ceph-mon[35081]: pgmap v799: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 600 KiB/s wr, 53 op/s 2024-05-17T09:31:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:48 smithi019 ceph-mon[31620]: pgmap v799: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 600 KiB/s wr, 53 op/s 2024-05-17T09:31:49.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 186755 1.65 MB/sec execute 467 sec latency 18.358 ms 2024-05-17T09:31:50.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 187234 1.66 MB/sec execute 468 sec latency 15.759 ms 2024-05-17T09:31:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:50 smithi079 ceph-mon[35502]: pgmap v800: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 859 KiB/s wr, 87 op/s 2024-05-17T09:31:50.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:50 smithi069 ceph-mon[35081]: pgmap v800: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 859 KiB/s wr, 87 op/s 2024-05-17T09:31:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:50 smithi019 ceph-mon[31620]: pgmap v800: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 859 KiB/s wr, 87 op/s 2024-05-17T09:31:51.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 187795 1.66 MB/sec execute 469 sec latency 13.020 ms 2024-05-17T09:31:52.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 188269 1.67 MB/sec execute 470 sec latency 28.132 ms 2024-05-17T09:31:52.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:52 smithi019 ceph-mon[31620]: pgmap v801: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 949 KiB/s wr, 103 op/s 2024-05-17T09:31:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:52 smithi079 ceph-mon[35502]: pgmap v801: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 949 KiB/s wr, 103 op/s 2024-05-17T09:31:52.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:52 smithi069 ceph-mon[35081]: pgmap v801: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 949 KiB/s wr, 103 op/s 2024-05-17T09:31:53.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 188540 1.67 MB/sec execute 471 sec latency 14.473 ms 2024-05-17T09:31:54.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 188944 1.66 MB/sec execute 472 sec latency 15.613 ms 2024-05-17T09:31:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:54 smithi079 ceph-mon[35502]: pgmap v802: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-17T09:31:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:54 smithi069 ceph-mon[35081]: pgmap v802: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-17T09:31:54.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:54 smithi019 ceph-mon[31620]: pgmap v802: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-05-17T09:31:55.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 189480 1.67 MB/sec execute 473 sec latency 16.779 ms 2024-05-17T09:31:56.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 190088 1.67 MB/sec execute 474 sec latency 15.711 ms 2024-05-17T09:31:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:56 smithi079 ceph-mon[35502]: pgmap v803: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:31:56.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:56 smithi069 ceph-mon[35081]: pgmap v803: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:31:56.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:56 smithi019 ceph-mon[31620]: pgmap v803: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:31:57.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 190592 1.67 MB/sec execute 475 sec latency 13.865 ms 2024-05-17T09:31:58.448 INFO:tasks.workunit.client.0.smithi019.stdout: 1 191309 1.68 MB/sec execute 476 sec latency 15.496 ms 2024-05-17T09:31:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:31:58 smithi079 ceph-mon[35502]: pgmap v804: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:31:58.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:31:58 smithi069 ceph-mon[35081]: pgmap v804: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:31:58.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:31:58 smithi019 ceph-mon[31620]: pgmap v804: 129 pgs: 129 active+clean; 78 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-17T09:31:59.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 191764 1.68 MB/sec execute 477 sec latency 15.467 ms 2024-05-17T09:31:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 -- 172.21.15.19:0/357029967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8108a80 msgr2=0x7f29b810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 --2- 172.21.15.19:0/357029967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8108a80 0x7f29b810aea0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ca0 tx=0x7f29b002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 -- 172.21.15.19:0/357029967 shutdown_connections 2024-05-17T09:31:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 --2- 172.21.15.19:0/357029967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8108a80 0x7f29b810aea0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 --2- 172.21.15.19:0/357029967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b8104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 --2- 172.21.15.19:0/357029967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29b8102b40 0x7f29b8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 -- 172.21.15.19:0/357029967 >> 172.21.15.19:0/357029967 conn(0x7f29b80fdd30 msgr2=0x7f29b81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 -- 172.21.15.19:0/357029967 shutdown_connections 2024-05-17T09:31:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.938+0000 7f29bffc7640 1 -- 172.21.15.19:0/357029967 wait complete. 2024-05-17T09:31:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 Processor -- start 2024-05-17T09:31:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 -- start start 2024-05-17T09:31:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 0x7f29b819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29b8108a80 0x7f29b81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e570 con 0x7f29b8102b40 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e3f0 con 0x7f29b8103d80 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bffc7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e6f0 con 0x7f29b8108a80 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bdd63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 0x7f29b819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bdd63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 0x7f29b819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52190/0 (socket says 172.21.15.19:52190) 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:38476/0 (socket says 172.21.15.19:38476) 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bdd63640 1 -- 172.21.15.19:0/3157424746 learned_addr learned my addr 172.21.15.19:0/3157424746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29be564640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29b8108a80 0x7f29b81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 -- 172.21.15.19:0/3157424746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29b8108a80 msgr2=0x7f29b81a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29b8108a80 0x7f29b81a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 -- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 msgr2=0x7f29b819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 0x7f29b819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bd562640 1 -- 172.21.15.19:0/3157424746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b81a1cc0 con 0x7f29b8103d80 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.939+0000 7f29bdd63640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 0x7f29b819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.940+0000 7f29bd562640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b819a6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9f0 tx=0x7f29a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.940+0000 7f29aeffd640 1 -- 172.21.15.19:0/3157424746 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800ccc0 con 0x7f29b8103d80 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.940+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b81a1fb0 con 0x7f29b8103d80 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.940+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b81a2470 con 0x7f29b8103d80 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.940+0000 7f29aeffd640 1 -- 172.21.15.19:0/3157424746 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29a80108c0 con 0x7f29b8103d80 2024-05-17T09:31:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.940+0000 7f29aeffd640 1 -- 172.21.15.19:0/3157424746 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8018660 con 0x7f29b8103d80 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 Processor -- start 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 -- start start 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29aeffd640 1 -- 172.21.15.19:0/3157424746 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f29a8010430 con 0x7f29b8103d80 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 0x55cf66a327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a32d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6666c4d0 0x55cf66a39c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf669a0c30 con 0x55cf6668d1b0 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cf669907d0 con 0x55cf666642f0 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cf66994810 con 0x55cf6666c4d0 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29be564640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 0x55cf66a327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29bdd63640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a32d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29be564640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 0x55cf66a327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52198/0 (socket says 172.21.15.19:52198) 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29bdd63640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a32d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:38492/0 (socket says 172.21.15.19:38492) 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.943+0000 7f29be564640 1 -- 172.21.15.19:0/1633995661 learned_addr learned my addr 172.21.15.19:0/1633995661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29bd562640 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6666c4d0 0x55cf66a39c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29be564640 1 -- 172.21.15.19:0/1633995661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6666c4d0 msgr2=0x55cf66a39c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29be564640 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6666c4d0 0x55cf66a39c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29be564640 1 -- 172.21.15.19:0/1633995661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 msgr2=0x55cf66a32d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29be564640 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a32d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29be564640 1 -- 172.21.15.19:0/1633995661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf666a5210 con 0x55cf6668d1b0 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29bdd63640 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29aeffd640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29a40b28a0 0x7f29a40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29be564640 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 0x55cf66a327b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f29b0035000 tx=0x7f29b0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99ccbc8439e0193 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29bdd63640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29a40b28a0 0x7f29a40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29aeffd640 1 -- 172.21.15.19:0/3157424746 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f29a80e2810 con 0x7f29b8103d80 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29a3fff640 1 -- 172.21.15.19:0/1633995661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0031650 con 0x55cf6668d1b0 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29a3fff640 1 -- 172.21.15.19:0/1633995661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b0036030 con 0x55cf6668d1b0 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.944+0000 7f29a3fff640 1 -- 172.21.15.19:0/1633995661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00042a0 con 0x55cf6668d1b0 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1633995661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 msgr2=0x55cf66a327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29bdd63640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29a40b28a0 0x7f29a40b4db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55cf66a33b90 tx=0x7f29b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 0x55cf66a327b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f29b0035000 tx=0x7f29b0009800 comp rx=0 tx=0).stop 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1633995661 shutdown_connections 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6666c4d0 0x55cf66a39c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1633995661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6668d1b0 0x55cf66a327b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1633995661 >> 172.21.15.19:0/1633995661 conn(0x55cf66681300 msgr2=0x55cf66a29d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1633995661 shutdown_connections 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.945+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1633995661 wait complete. 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 -- start start 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 Processor -- start 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a3bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 0x55cf66a3c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6668d1b0 0x55cf66a43370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf66676790 con 0x55cf6666c4d0 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cf6694c890 con 0x55cf666642f0 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cf66a3a4a0 con 0x55cf6668d1b0 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29c6e7a740 10 client.0 _ll_register_callbacks cb 0x55cf6694fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29bd562640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 0x55cf66a3c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29bd562640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 0x55cf66a3c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52210/0 (socket says 172.21.15.19:52210) 2024-05-17T09:31:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.946+0000 7f29bd562640 1 -- 172.21.15.19:0/2875777111 learned_addr learned my addr 172.21.15.19:0/2875777111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a3bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29be564640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6668d1b0 0x55cf66a43370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bd562640 1 -- 172.21.15.19:0/2875777111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6668d1b0 msgr2=0x55cf66a43370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bd562640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6668d1b0 0x55cf66a43370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bd562640 1 -- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 msgr2=0x55cf66a3bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bd562640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a3bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bd562640 1 -- 172.21.15.19:0/2875777111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf66771620 con 0x55cf6666c4d0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a3bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29bd562640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 0x55cf66a3c480 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f29a80a9ac0 tx=0x7f29a80a5380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ecb31cee17159d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29a17fa640 1 -- 172.21.15.19:0/2875777111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a80ade30 con 0x55cf6666c4d0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29a17fa640 1 -- 172.21.15.19:0/2875777111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f29a80a6120 con 0x55cf6666c4d0 2024-05-17T09:31:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b0036030 con 0x55cf6666c4d0 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29a17fa640 1 -- 172.21.15.19:0/2875777111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f29a80b28e0 con 0x55cf6666c4d0 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29c6e7a740 10 client.17607 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.947+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf66a46310 con 0x55cf6666c4d0 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29a17fa640 1 -- 172.21.15.19:0/2875777111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b0036030 con 0x55cf6666c4d0 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29a17fa640 1 -- 172.21.15.19:0/2875777111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f29a80a8800 con 0x55cf6666c4d0 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29c6e7a740 10 client.17607 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29c6e7a740 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29c6e7a740 20 client.17607 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6664bd80 0x55cf66a49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29c6e7a740 4 client.17607 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf669a2b80 con 0x55cf6664bd80 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6664bd80 0x55cf66a49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.948+0000 7f29a17fa640 10 client.17607 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.949+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6664bd80 0x55cf66a49a10 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a81495a3222416 server_cookie=ab6020b13a86c324 in_seq=0 out_seq=0 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29a17fa640 1 -- 172.21.15.19:0/2875777111 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cf669a2b80 con 0x55cf6664bd80 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29a17fa640 10 client.17607 handle_command_reply: tid=0 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 client.17607 shutdown 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 msgr2=0x55cf66a3c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 0x55cf66a3c480 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f29a80a9ac0 tx=0x7f29a80a5380 comp rx=0 tx=0).stop 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 shutdown_connections 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6664bd80 0x55cf66a49a10 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6668d1b0 0x55cf66a43370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6666c4d0 0x55cf66a3c480 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29a17fa640 0 client.17607 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2875777111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf666642f0 0x55cf66a3bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 >> 172.21.15.19:0/2875777111 conn(0x55cf66681300 msgr2=0x55cf66a28d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 shutdown_connections 2024-05-17T09:31:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2875777111 wait complete. 2024-05-17T09:31:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.950+0000 7f29c6e7a740 20 client.17607 trim_cache size 0 max 16384 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.969+0000 7f29c6e7a740 1 Processor -- start 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.969+0000 7f29c6e7a740 1 -- start start 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 0x55cf66a28e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf666642f0 0x55cf66a43290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a35bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f299c012460 con 0x55cf6664bd80 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f299c00fd30 con 0x55cf6666c4d0 2024-05-17T09:31:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f299c00db60 con 0x55cf666642f0 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 0x55cf66a28e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29be564640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a35bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 0x55cf66a28e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52224/0 (socket says 172.21.15.19:52224) 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 -- 172.21.15.19:0/2591150100 learned_addr learned my addr 172.21.15.19:0/2591150100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bd562640 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf666642f0 0x55cf66a43290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 -- 172.21.15.19:0/2591150100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf666642f0 msgr2=0x55cf66a43290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf666642f0 0x55cf66a43290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 -- 172.21.15.19:0/2591150100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 msgr2=0x55cf66a35bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a35bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 -- 172.21.15.19:0/2591150100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf66a329a0 con 0x55cf6664bd80 2024-05-17T09:31:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29be564640 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a35bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:31:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.970+0000 7f29bdd63640 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 0x55cf66a28e00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f29b400f930 tx=0x7f29b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b339803c3b40cc05 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.971+0000 7f29a37fe640 1 -- 172.21.15.19:0/2591150100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b401d910 con 0x55cf6664bd80 2024-05-17T09:31:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.971+0000 7f29a37fe640 1 -- 172.21.15.19:0/2591150100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b402a410 con 0x55cf6664bd80 2024-05-17T09:31:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.971+0000 7f29a37fe640 1 -- 172.21.15.19:0/2591150100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4029610 con 0x55cf6664bd80 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2591150100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 msgr2=0x55cf66a28e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 0x55cf66a28e00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f29b400f930 tx=0x7f29b4019000 comp rx=0 tx=0).stop 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2591150100 shutdown_connections 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a35bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf666642f0 0x55cf66a43290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/2591150100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf6664bd80 0x55cf66a28e00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2591150100 >> 172.21.15.19:0/2591150100 conn(0x55cf66681300 msgr2=0x55cf66a2d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2591150100 shutdown_connections 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 -- 172.21.15.19:0/2591150100 wait complete. 2024-05-17T09:31:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 -- start start 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 Processor -- start 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6664bd80 0x55cf66a40c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 0x55cf66a41170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.972+0000 7f29c6e7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f299c00e1f0 con 0x55cf666642f0 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f299c00f660 con 0x55cf6666c4d0 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f299c00e8c0 con 0x55cf6664bd80 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 10 client.0 _ll_register_callbacks cb 0x7f29a80a92a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 0x55cf66a41170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 0x55cf66a41170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52226/0 (socket says 172.21.15.19:52226) 2024-05-17T09:31:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 -- 172.21.15.19:0/1883247200 learned_addr learned my addr 172.21.15.19:0/1883247200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bdd63640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6664bd80 0x55cf66a40c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29be564640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 -- 172.21.15.19:0/1883247200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6664bd80 msgr2=0x55cf66a40c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6664bd80 0x55cf66a40c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 -- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 msgr2=0x55cf66a420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 -- 172.21.15.19:0/1883247200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf66a42660 con 0x55cf666642f0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29bd562640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 0x55cf66a41170 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9c0 tx=0x7f29a80e2ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c770ac6c4ed61c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29a17fa640 1 -- 172.21.15.19:0/1883247200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a80c3060 con 0x55cf666642f0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29a17fa640 1 -- 172.21.15.19:0/1883247200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f29a80a54a0 con 0x55cf666642f0 2024-05-17T09:31:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b402a410 con 0x55cf666642f0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29a17fa640 1 -- 172.21.15.19:0/1883247200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f29a80a9480 con 0x55cf666642f0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 10 client.17613 fetch_fsmap learned FSMap version 32 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.973+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf66ab2230 con 0x55cf666642f0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29a17fa640 1 -- 172.21.15.19:0/1883247200 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b402a410 con 0x55cf666642f0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29a17fa640 1 -- 172.21.15.19:0/1883247200 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f29a8010a70 con 0x55cf666642f0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29c6e7a740 10 client.17613 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29c6e7a740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29c6e7a740 20 client.17613 populate_metadata read hostname 'smithi019' 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6668d1b0 0x55cf66ab4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29c6e7a740 4 client.17613 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf6699c360 con 0x55cf6668d1b0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29bdd63640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6668d1b0 0x55cf66ab4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.974+0000 7f29a17fa640 10 client.17613 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.975+0000 7f29bdd63640 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6668d1b0 0x55cf66ab4c20 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e44c730920bd86 server_cookie=9d5658cb2a683a42 in_seq=0 out_seq=0 2024-05-17T09:31:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.993+0000 7f29a17fa640 1 -- 172.21.15.19:0/1883247200 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cf6699c360 con 0x55cf6668d1b0 2024-05-17T09:31:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.993+0000 7f29a17fa640 10 client.17613 handle_command_reply: tid=0 2024-05-17T09:31:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.993+0000 7f29c6e7a740 1 client.17613 shutdown 2024-05-17T09:31:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.993+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 msgr2=0x55cf66a41170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.993+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 0x55cf66a41170 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9c0 tx=0x7f29a80e2ba0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 shutdown_connections 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cf6668d1b0 0x55cf66ab4c20 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf6666c4d0 0x55cf66a420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf666642f0 0x55cf66a41170 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29a17fa640 0 client.17613 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 --2- 172.21.15.19:0/1883247200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cf6664bd80 0x55cf66a40c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 >> 172.21.15.19:0/1883247200 conn(0x55cf66681300 msgr2=0x55cf66a2d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 shutdown_connections 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 1 -- 172.21.15.19:0/1883247200 wait complete. 2024-05-17T09:31:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29c6e7a740 20 client.17613 trim_cache size 0 max 16384 2024-05-17T09:31:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:31:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:31:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:31:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29a40b28a0 msgr2=0x7f29a40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29bffc7640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29a40b28a0 0x7f29a40b4db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55cf66a33b90 tx=0x7f29b4005f90 comp rx=0 tx=0).stop 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 msgr2=0x7f29b819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29bffc7640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b819a6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9f0 tx=0x7f29a800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 shutdown_connections 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.994+0000 7f29bffc7640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29b8108a80 0x7f29b81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.995+0000 7f29bffc7640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8103d80 0x7f29b819a6c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.995+0000 7f29bffc7640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29a40b28a0 0x7f29a40b4db0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.995+0000 7f29bffc7640 1 --2- 172.21.15.19:0/3157424746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8102b40 0x7f29b819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.995+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 >> 172.21.15.19:0/3157424746 conn(0x7f29b80fdd30 msgr2=0x7f29b8109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.995+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 shutdown_connections 2024-05-17T09:31:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:31:59.995+0000 7f29bffc7640 1 -- 172.21.15.19:0/3157424746 wait complete. 2024-05-17T09:32:00.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 192004 1.68 MB/sec execute 478 sec latency 15.586 ms 2024-05-17T09:32:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:00 smithi079 ceph-mon[35502]: pgmap v805: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-05-17T09:32:00.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:00 smithi069 ceph-mon[35081]: pgmap v805: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-05-17T09:32:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:00 smithi019 ceph-mon[31620]: pgmap v805: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-05-17T09:32:01.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 192373 1.68 MB/sec execute 479 sec latency 16.268 ms 2024-05-17T09:32:02.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 192931 1.68 MB/sec execute 480 sec latency 15.795 ms 2024-05-17T09:32:02.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:02 smithi079 ceph-mon[35502]: pgmap v806: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-05-17T09:32:02.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:02 smithi069 ceph-mon[35081]: pgmap v806: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-05-17T09:32:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:02 smithi019 ceph-mon[31620]: pgmap v806: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-05-17T09:32:03.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 193473 1.68 MB/sec execute 481 sec latency 24.546 ms 2024-05-17T09:32:04.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 194003 1.68 MB/sec execute 482 sec latency 18.969 ms 2024-05-17T09:32:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:04 smithi079 ceph-mon[35502]: pgmap v807: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-17T09:32:04.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:04 smithi069 ceph-mon[35081]: pgmap v807: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-17T09:32:04.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:04 smithi019 ceph-mon[31620]: pgmap v807: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-17T09:32:05.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 194769 1.69 MB/sec execute 483 sec latency 15.020 ms 2024-05-17T09:32:06.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 195183 1.70 MB/sec execute 484 sec latency 15.770 ms 2024-05-17T09:32:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:06 smithi079 ceph-mon[35502]: pgmap v808: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-17T09:32:06.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:06 smithi069 ceph-mon[35081]: pgmap v808: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-17T09:32:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:06 smithi019 ceph-mon[31620]: pgmap v808: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-17T09:32:07.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 195494 1.70 MB/sec execute 485 sec latency 16.208 ms 2024-05-17T09:32:08.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 195818 1.69 MB/sec execute 486 sec latency 12.426 ms 2024-05-17T09:32:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:08 smithi079 ceph-mon[35502]: pgmap v809: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:32:08.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:08 smithi069 ceph-mon[35081]: pgmap v809: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:32:08.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:08 smithi019 ceph-mon[31620]: pgmap v809: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-17T09:32:09.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 196273 1.69 MB/sec execute 487 sec latency 54.233 ms 2024-05-17T09:32:10.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 196803 1.70 MB/sec execute 488 sec latency 50.238 ms 2024-05-17T09:32:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:10 smithi079 ceph-mon[35502]: pgmap v810: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:32:10.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:10 smithi069 ceph-mon[35081]: pgmap v810: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:32:10.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:10 smithi019 ceph-mon[31620]: pgmap v810: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-17T09:32:11.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 197458 1.70 MB/sec execute 489 sec latency 15.382 ms 2024-05-17T09:32:12.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 198030 1.70 MB/sec execute 490 sec latency 15.103 ms 2024-05-17T09:32:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:12 smithi079 ceph-mon[35502]: pgmap v811: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-05-17T09:32:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:12 smithi069 ceph-mon[35081]: pgmap v811: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-05-17T09:32:12.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:12 smithi019 ceph-mon[31620]: pgmap v811: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-05-17T09:32:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 -- 172.21.15.19:0/3643269749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874103c50 msgr2=0x7fa874104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3643269749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874103c50 0x7fa874104950 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa870009900 tx=0x7fa87002a410 comp rx=0 tx=0).stop 2024-05-17T09:32:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 -- 172.21.15.19:0/3643269749 shutdown_connections 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3643269749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874109b90 0x7fa87410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3643269749 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa874104e90 0x7fa8741052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3643269749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874103c50 0x7fa874104950 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 -- 172.21.15.19:0/3643269749 >> 172.21.15.19:0/3643269749 conn(0x7fa8740fee40 msgr2=0x7fa8741012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 -- 172.21.15.19:0/3643269749 shutdown_connections 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.109+0000 7fa87c15e640 1 -- 172.21.15.19:0/3643269749 wait complete. 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 Processor -- start 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 -- start start 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 0x7fa87419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa874104e90 0x7fa87419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874109b90 0x7fa8741a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa87410f600 con 0x7fa874109b90 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa87410f480 con 0x7fa874103c50 2024-05-17T09:32:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87c15e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa87410f780 con 0x7fa874104e90 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa879efa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 0x7fa87419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874109b90 0x7fa8741a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa879efa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 0x7fa87419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49236/0 (socket says 172.21.15.19:49236) 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa879efa640 1 -- 172.21.15.19:0/3420268234 learned_addr learned my addr 172.21.15.19:0/3420268234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.110+0000 7fa8796f9640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa874104e90 0x7fa87419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa879efa640 1 -- 172.21.15.19:0/3420268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa874104e90 msgr2=0x7fa87419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa879efa640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa874104e90 0x7fa87419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa879efa640 1 -- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874109b90 msgr2=0x7fa8741a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa879efa640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874109b90 0x7fa8741a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa879efa640 1 -- 172.21.15.19:0/3420268234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8741a1d40 con 0x7fa874103c50 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874109b90 0x7fa8741a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa879efa640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 0x7fa87419a200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa870005760 tx=0x7fa870005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa86affd640 1 -- 172.21.15.19:0/3420268234 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87003f030 con 0x7fa874103c50 2024-05-17T09:32:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa86affd640 1 -- 172.21.15.19:0/3420268234 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8700043d0 con 0x7fa874103c50 2024-05-17T09:32:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8741a1fd0 con 0x7fa874103c50 2024-05-17T09:32:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa86affd640 1 -- 172.21.15.19:0/3420268234 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870035070 con 0x7fa874103c50 2024-05-17T09:32:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.111+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741a23d0 con 0x7fa874103c50 2024-05-17T09:32:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.114+0000 7fa883012740 1 Processor -- start 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.114+0000 7fa86affd640 1 -- 172.21.15.19:0/3420268234 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa870042030 con 0x7fa874103c50 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.114+0000 7fa883012740 1 -- start start 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.114+0000 7fa883012740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.114+0000 7fa883012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db88f2f0 0x5570dbc59920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.114+0000 7fa883012740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8974d0 0x5570dbc60820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa883012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570dbbcbc30 con 0x5570db88f2f0 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa883012740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5570dbbbb7d0 con 0x5570db8974d0 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa883012740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5570dbbbf810 con 0x5570db8b81b0 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa879efa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db88f2f0 0x5570dbc59920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42996/0 (socket says 172.21.15.19:42996) 2024-05-17T09:32:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 -- 172.21.15.19:0/2057678550 learned_addr learned my addr 172.21.15.19:0/2057678550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa879efa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db88f2f0 0x5570dbc59920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38516/0 (socket says 172.21.15.19:38516) 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa8796f9640 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8974d0 0x5570dbc60820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 -- 172.21.15.19:0/2057678550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8974d0 msgr2=0x5570dbc60820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8974d0 0x5570dbc60820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 -- 172.21.15.19:0/2057678550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db88f2f0 msgr2=0x5570dbc59920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db88f2f0 0x5570dbc59920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 -- 172.21.15.19:0/2057678550 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570db8d0210 con 0x5570db8b81b0 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa8796f9640 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8974d0 0x5570dbc60820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa86affd640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8600b28a0 0x7fa8600b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc593a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa86c011810 tx=0x7fa86c011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1446a86b049bc661 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa879efa640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8600b28a0 0x7fa8600b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.115+0000 7fa85ffff640 1 -- 172.21.15.19:0/2057678550 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c011310 con 0x5570db8b81b0 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.116+0000 7fa86affd640 1 -- 172.21.15.19:0/3420268234 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa870102880 con 0x7fa874103c50 2024-05-17T09:32:13.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.116+0000 7fa85ffff640 1 -- 172.21.15.19:0/2057678550 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86c00f9a0 con 0x5570db8b81b0 2024-05-17T09:32:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.116+0000 7fa85ffff640 1 -- 172.21.15.19:0/2057678550 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c005660 con 0x5570db8b81b0 2024-05-17T09:32:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa879efa640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8600b28a0 0x7fa8600b4db0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x5570dbc5a780 tx=0x7fa870108550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 -- 172.21.15.19:0/2057678550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 msgr2=0x5570dbc593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc593a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa86c011810 tx=0x7fa86c011cd0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 -- 172.21.15.19:0/2057678550 shutdown_connections 2024-05-17T09:32:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8974d0 0x5570dbc60820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db88f2f0 0x5570dbc59920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 --2- 172.21.15.19:0/2057678550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc593a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 -- 172.21.15.19:0/2057678550 >> 172.21.15.19:0/2057678550 conn(0x5570db8ac300 msgr2=0x5570dbc54d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 -- 172.21.15.19:0/2057678550 shutdown_connections 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 -- 172.21.15.19:0/2057678550 wait complete. 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 -- start start 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.117+0000 7fa883012740 1 Processor -- start 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8974d0 0x5570dbc632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 0x5570dbc6a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570db8a1790 con 0x5570db8b81b0 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5570dbb77890 con 0x5570db88f2f0 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5570dbc61270 con 0x5570db8974d0 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa883012740 10 client.0 _ll_register_callbacks cb 0x5570dbb7ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 0x5570dbc6a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 0x5570dbc6a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38528/0 (socket says 172.21.15.19:38528) 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 -- 172.21.15.19:0/409107594 learned_addr learned my addr 172.21.15.19:0/409107594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa8796f9640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8974d0 0x5570dbc632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 -- 172.21.15.19:0/409107594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8974d0 msgr2=0x5570dbc632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8974d0 0x5570dbc632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 -- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 msgr2=0x5570dbc62d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc62d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa87a6fb640 1 -- 172.21.15.19:0/409107594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570db99c620 con 0x5570db8b81b0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.118+0000 7fa8796f9640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8974d0 0x5570dbc632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 0x5570dbc6a220 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa86c011620 tx=0x7fa86c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03a4f64877ec6a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa85d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa85d7fa640 1 -- 172.21.15.19:0/409107594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c01e040 con 0x5570db8b81b0 2024-05-17T09:32:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86c00f9a0 con 0x5570db8b81b0 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa85d7fa640 1 -- 172.21.15.19:0/409107594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa86c003b70 con 0x5570db8b81b0 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa85d7fa640 1 -- 172.21.15.19:0/409107594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa86c003d70 con 0x5570db8b81b0 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa883012740 10 client.17625 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5570dbc715b0 con 0x5570db8b81b0 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa85d7fa640 1 -- 172.21.15.19:0/409107594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86c00f9a0 con 0x5570db8b81b0 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.119+0000 7fa85d7fa640 1 -- 172.21.15.19:0/409107594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa86c024070 con 0x5570db8b81b0 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa883012740 10 client.17625 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa883012740 10 client.17625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa883012740 20 client.17625 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa883012740 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db876d80 0x5570dbc74af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa883012740 4 client.17625 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:32:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570dbbcdb80 con 0x5570db876d80 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa8796f9640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db876d80 0x5570dbc74af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa85d7fa640 10 client.17625 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.120+0000 7fa8796f9640 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db876d80 0x5570dbc74af0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1908051656ecdab server_cookie=f4b7e3f556d2529f in_seq=0 out_seq=0 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa85d7fa640 1 -- 172.21.15.19:0/409107594 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5570dbbcdb80 con 0x5570db876d80 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa85d7fa640 10 client.17625 handle_command_reply: tid=0 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 client.17625 shutdown 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 msgr2=0x5570dbc6a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 0x5570dbc6a220 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa86c011620 tx=0x7fa86c009b10 comp rx=0 tx=0).stop 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 shutdown_connections 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db876d80 0x5570dbc74af0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8b81b0 0x5570dbc6a220 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8974d0 0x5570dbc632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 --2- 172.21.15.19:0/409107594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc62d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 >> 172.21.15.19:0/409107594 conn(0x5570db8ac300 msgr2=0x5570dbc57c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa85d7fa640 0 client.17625 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.121+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 shutdown_connections 2024-05-17T09:32:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.122+0000 7fa883012740 1 -- 172.21.15.19:0/409107594 wait complete. 2024-05-17T09:32:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.122+0000 7fa883012740 20 client.17625 trim_cache size 0 max 16384 2024-05-17T09:32:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 Processor -- start 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 -- start start 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db88f2f0 0x5570dbc5d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8b81b0 0x5570dbc58240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741ad120 con 0x5570db8974d0 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741aa9f0 con 0x5570db8b81b0 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa883012740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741a8840 con 0x5570db88f2f0 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.137+0000 7fa8796f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc6a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38540/0 (socket says 172.21.15.19:38540) 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 -- 172.21.15.19:0/1810700041 learned_addr learned my addr 172.21.15.19:0/1810700041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8b81b0 0x5570dbc58240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db88f2f0 0x5570dbc5d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 -- 172.21.15.19:0/1810700041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db88f2f0 msgr2=0x5570dbc5d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db88f2f0 0x5570dbc5d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 -- 172.21.15.19:0/1810700041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8b81b0 msgr2=0x5570dbc58240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8b81b0 0x5570dbc58240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 -- 172.21.15.19:0/1810700041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570dbc50220 con 0x5570db8974d0 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db88f2f0 0x5570dbc5d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa8796f9640 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc6a140 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x5570dbc55ac0 tx=0x7fa864005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726a582cfdde419d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa85f7fe640 1 -- 172.21.15.19:0/1810700041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864004030 con 0x5570db8974d0 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa85f7fe640 1 -- 172.21.15.19:0/1810700041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86400f7f0 con 0x5570db8974d0 2024-05-17T09:32:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.138+0000 7fa85f7fe640 1 -- 172.21.15.19:0/1810700041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864003920 con 0x5570db8974d0 2024-05-17T09:32:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 -- 172.21.15.19:0/1810700041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 msgr2=0x5570dbc6a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc6a140 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x5570dbc55ac0 tx=0x7fa864005f90 comp rx=0 tx=0).stop 2024-05-17T09:32:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 -- 172.21.15.19:0/1810700041 shutdown_connections 2024-05-17T09:32:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db8b81b0 0x5570dbc58240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc6a140 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 --2- 172.21.15.19:0/1810700041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db88f2f0 0x5570dbc5d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 -- 172.21.15.19:0/1810700041 >> 172.21.15.19:0/1810700041 conn(0x5570db8ac300 msgr2=0x5570dbc57b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 -- 172.21.15.19:0/1810700041 shutdown_connections 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 -- 172.21.15.19:0/1810700041 wait complete. 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.139+0000 7fa883012740 1 -- start start 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 Processor -- start 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc68a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741a8eb0 con 0x5570db8974d0 2024-05-17T09:32:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741aa320 con 0x5570db88f2f0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741a9580 con 0x5570db8b81b0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa883012740 10 client.0 _ll_register_callbacks cb 0x7fa86c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa8796f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc68a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa8796f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc68f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38554/0 (socket says 172.21.15.19:38554) 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa87a6fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc68a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49256/0 (socket says 172.21.15.19:49256) 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa8796f9640 1 -- 172.21.15.19:0/3412625591 learned_addr learned my addr 172.21.15.19:0/3412625591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.140+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa8796f9640 1 -- 172.21.15.19:0/3412625591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 msgr2=0x5570dbc69490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa8796f9640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc69490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa8796f9640 1 -- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 msgr2=0x5570dbc68a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa8796f9640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc68a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa8796f9640 1 -- 172.21.15.19:0/3412625591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570dbcdb1b0 con 0x5570db8974d0 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc68a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa8796f9640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc68f50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa864027000 tx=0x7fa86400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e57bbee0a897a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa85d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa85d7fa640 1 -- 172.21.15.19:0/3412625591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86400fbd0 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86400f7f0 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa85d7fa640 1 -- 172.21.15.19:0/3412625591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa86400fd70 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa85d7fa640 1 -- 172.21.15.19:0/3412625591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa864005040 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa883012740 10 client.17631 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5570dbcdc370 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.141+0000 7fa85d7fa640 1 -- 172.21.15.19:0/3412625591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86400f7f0 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa85d7fa640 1 -- 172.21.15.19:0/3412625591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa864010430 con 0x5570db8974d0 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa883012740 10 client.17631 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa883012740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa883012740 20 client.17631 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa883012740 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db84c1c0 0x5570dbcded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa883012740 4 client.17631 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:32:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570dbbc7360 con 0x5570db84c1c0 2024-05-17T09:32:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db84c1c0 0x5570dbcded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa85d7fa640 10 client.17631 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.142+0000 7fa87a6fb640 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db84c1c0 0x5570dbcded90 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36e700721436063 server_cookie=ce62f817cd936321 in_seq=0 out_seq=0 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.162+0000 7fa85d7fa640 1 -- 172.21.15.19:0/3412625591 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5570dbbc7360 con 0x5570db84c1c0 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.162+0000 7fa85d7fa640 10 client.17631 handle_command_reply: tid=0 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 client.17631 shutdown 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 msgr2=0x5570dbc68f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc68f50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa864027000 tx=0x7fa86400fa30 comp rx=0 tx=0).stop 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 shutdown_connections 2024-05-17T09:32:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5570db84c1c0 0x5570dbcded90 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5570db8b81b0 0x5570dbc69490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570db8974d0 0x5570dbc68f50 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa85d7fa640 0 client.17631 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 --2- 172.21.15.19:0/3412625591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5570db88f2f0 0x5570dbc68a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 >> 172.21.15.19:0/3412625591 conn(0x5570db8ac300 msgr2=0x5570dbc5c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 shutdown_connections 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 1 -- 172.21.15.19:0/3412625591 wait complete. 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.163+0000 7fa883012740 20 client.17631 trim_cache size 0 max 16384 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:32:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:32:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8600b28a0 msgr2=0x7fa8600b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8600b28a0 0x7fa8600b4db0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x5570dbc5a780 tx=0x7fa870108550 comp rx=0 tx=0).stop 2024-05-17T09:32:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 msgr2=0x7fa87419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 0x7fa87419a200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa870005760 tx=0x7fa870005e20 comp rx=0 tx=0).stop 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 shutdown_connections 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa8600b28a0 0x7fa8600b4db0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa874109b90 0x7fa8741a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa874104e90 0x7fa87419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 --2- 172.21.15.19:0/3420268234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa874103c50 0x7fa87419a200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 >> 172.21.15.19:0/3420268234 conn(0x7fa8740fee40 msgr2=0x7fa874105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 shutdown_connections 2024-05-17T09:32:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:13.164+0000 7fa87c15e640 1 -- 172.21.15.19:0/3420268234 wait complete. 2024-05-17T09:32:13.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 198474 1.70 MB/sec execute 491 sec latency 21.427 ms 2024-05-17T09:32:14.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 198936 1.71 MB/sec execute 492 sec latency 13.327 ms 2024-05-17T09:32:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:14 smithi079 ceph-mon[35502]: pgmap v812: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-17T09:32:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:14 smithi069 ceph-mon[35081]: pgmap v812: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-17T09:32:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:14 smithi019 ceph-mon[31620]: pgmap v812: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-17T09:32:15.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 199186 1.71 MB/sec execute 493 sec latency 53.351 ms 2024-05-17T09:32:16.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 199617 1.71 MB/sec execute 494 sec latency 11.987 ms 2024-05-17T09:32:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:16 smithi079 ceph-mon[35502]: pgmap v813: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-17T09:32:16.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:16 smithi069 ceph-mon[35081]: pgmap v813: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-17T09:32:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:16 smithi019 ceph-mon[31620]: pgmap v813: 129 pgs: 129 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-17T09:32:17.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 200162 1.71 MB/sec execute 495 sec latency 21.442 ms 2024-05-17T09:32:18.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 200789 1.71 MB/sec execute 496 sec latency 15.032 ms 2024-05-17T09:32:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:18 smithi079 ceph-mon[35502]: pgmap v814: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:18.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:18 smithi069 ceph-mon[35081]: pgmap v814: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:18 smithi019 ceph-mon[31620]: pgmap v814: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:19.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 201257 1.71 MB/sec execute 497 sec latency 15.746 ms 2024-05-17T09:32:20.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 201952 1.72 MB/sec execute 498 sec latency 15.587 ms 2024-05-17T09:32:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:20 smithi079 ceph-mon[35502]: pgmap v815: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-05-17T09:32:20.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:20 smithi069 ceph-mon[35081]: pgmap v815: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-05-17T09:32:20.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:20 smithi019 ceph-mon[31620]: pgmap v815: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-05-17T09:32:21.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 202412 1.72 MB/sec execute 499 sec latency 16.059 ms 2024-05-17T09:32:22.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 202645 1.72 MB/sec execute 500 sec latency 14.010 ms 2024-05-17T09:32:22.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:22 smithi019 ceph-mon[31620]: pgmap v816: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-05-17T09:32:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:22 smithi079 ceph-mon[35502]: pgmap v816: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-05-17T09:32:22.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:22 smithi069 ceph-mon[35081]: pgmap v816: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-05-17T09:32:23.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 203019 1.72 MB/sec execute 501 sec latency 18.972 ms 2024-05-17T09:32:24.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 203570 1.72 MB/sec execute 502 sec latency 15.811 ms 2024-05-17T09:32:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:24 smithi079 ceph-mon[35502]: pgmap v817: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:32:24.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:24 smithi069 ceph-mon[35081]: pgmap v817: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:32:24.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:24 smithi019 ceph-mon[31620]: pgmap v817: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-17T09:32:25.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 204095 1.72 MB/sec execute 503 sec latency 26.071 ms 2024-05-17T09:32:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 -- 172.21.15.19:0/897014396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc109ba0 msgr2=0x7fdcfc10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 --2- 172.21.15.19:0/897014396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc109ba0 0x7fdcfc10bfc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009ca0 tx=0x7fdcf402a8c0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 -- 172.21.15.19:0/897014396 shutdown_connections 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 --2- 172.21.15.19:0/897014396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc109ba0 0x7fdcfc10bfc0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 --2- 172.21.15.19:0/897014396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc104ea0 0x7fdcfc1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 --2- 172.21.15.19:0/897014396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcfc103c60 0x7fdcfc104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 -- 172.21.15.19:0/897014396 >> 172.21.15.19:0/897014396 conn(0x7fdcfc0fee50 msgr2=0x7fdcfc101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 -- 172.21.15.19:0/897014396 shutdown_connections 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.274+0000 7fdd0413f640 1 -- 172.21.15.19:0/897014396 wait complete. 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 Processor -- start 2024-05-17T09:32:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 -- start start 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 0x7fdcfc19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc104ea0 0x7fdcfc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcfc109ba0 0x7fdcfc1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc10f690 con 0x7fdcfc104ea0 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc10f510 con 0x7fdcfc103c60 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd0413f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc10f810 con 0x7fdcfc109ba0 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd01edb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 0x7fdcfc19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc104ea0 0x7fdcfc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd01edb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 0x7fdcfc19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:42534/0 (socket says 172.21.15.19:42534) 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd01edb640 1 -- 172.21.15.19:0/4037900486 learned_addr learned my addr 172.21.15.19:0/4037900486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.275+0000 7fdd026dc640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcfc109ba0 0x7fdcfc1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd01edb640 1 -- 172.21.15.19:0/4037900486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcfc109ba0 msgr2=0x7fdcfc1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd01edb640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcfc109ba0 0x7fdcfc1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd01edb640 1 -- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc104ea0 msgr2=0x7fdcfc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd01edb640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc104ea0 0x7fdcfc19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd01edb640 1 -- 172.21.15.19:0/4037900486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc1a1cb0 con 0x7fdcfc103c60 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd01edb640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 0x7fdcfc19a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800e9f0 tx=0x7fdcf800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdcf2ffd640 1 -- 172.21.15.19:0/4037900486 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800ccc0 con 0x7fdcfc103c60 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcfc1a1fa0 con 0x7fdcfc103c60 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdcf2ffd640 1 -- 172.21.15.19:0/4037900486 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcf80035f0 con 0x7fdcfc103c60 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdcf2ffd640 1 -- 172.21.15.19:0/4037900486 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800ce60 con 0x7fdcfc103c60 2024-05-17T09:32:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.276+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcfc1a23b0 con 0x7fdcfc103c60 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 Processor -- start 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 -- start start 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea3e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea3ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115ea45c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56115e9acc30 con 0x56115e6702f0 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56115e99c7d0 con 0x56115e6784d0 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56115e9a0810 con 0x56115e6991b0 2024-05-17T09:32:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdcf2ffd640 1 -- 172.21.15.19:0/4037900486 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdcf8004000 con 0x7fdcfc103c60 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd026dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea3e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd016da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115ea45c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd026dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea3e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41212/0 (socket says 172.21.15.19:41212) 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd016da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115ea45c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:42536/0 (socket says 172.21.15.19:42536) 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd026dc640 1 -- 172.21.15.19:0/2107966775 learned_addr learned my addr 172.21.15.19:0/2107966775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.279+0000 7fdd01edb640 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea3ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd026dc640 1 -- 172.21.15.19:0/2107966775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 msgr2=0x56115ea45c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd026dc640 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115ea45c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd026dc640 1 -- 172.21.15.19:0/2107966775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 msgr2=0x56115ea3ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd026dc640 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea3ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd026dc640 1 -- 172.21.15.19:0/2107966775 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115e6b1210 con 0x56115e6991b0 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd026dc640 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea3e7b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4035000 tx=0x7fdcf4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184a9334aee306e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdce7fff640 1 -- 172.21.15.19:0/2107966775 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4031a80 con 0x56115e6991b0 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdce7fff640 1 -- 172.21.15.19:0/2107966775 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcf4036030 con 0x56115e6991b0 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdce7fff640 1 -- 172.21.15.19:0/2107966775 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4004340 con 0x56115e6991b0 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdcf2ffd640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdce80b28a0 0x7fdce80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdd01edb640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdce80b28a0 0x7fdce80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.280+0000 7fdcf2ffd640 1 -- 172.21.15.19:0/4037900486 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdcf80e28c0 con 0x7fdcfc103c60 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 -- 172.21.15.19:0/2107966775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 msgr2=0x56115ea3e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd01edb640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdce80b28a0 0x7fdce80b4db0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56115ea3fb90 tx=0x7fdcf80e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea3e7b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4035000 tx=0x7fdcf4009800 comp rx=0 tx=0).stop 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 -- 172.21.15.19:0/2107966775 shutdown_connections 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115ea45c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/2107966775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea3e7b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 -- 172.21.15.19:0/2107966775 >> 172.21.15.19:0/2107966775 conn(0x56115e68d300 msgr2=0x56115ea35d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 -- 172.21.15.19:0/2107966775 shutdown_connections 2024-05-17T09:32:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.281+0000 7fdd0aff4740 1 -- 172.21.15.19:0/2107966775 wait complete. 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 -- start start 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 Processor -- start 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6702f0 0x56115ea47f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 0x56115ea48480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea4f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56115e682790 con 0x56115e6784d0 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56115e958890 con 0x56115e6702f0 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56115ea464a0 con 0x56115e6991b0 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd0aff4740 10 client.0 _ll_register_callbacks cb 0x56115e95bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 0x56115ea48480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 0x56115ea48480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59294/0 (socket says 172.21.15.19:59294) 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd016da640 1 -- 172.21.15.19:0/1449740418 learned_addr learned my addr 172.21.15.19:0/1449740418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd026dc640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6702f0 0x56115ea47f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.282+0000 7fdd026dc640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea4f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd016da640 1 -- 172.21.15.19:0/1449740418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 msgr2=0x56115ea4f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd016da640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea4f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd016da640 1 -- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6702f0 msgr2=0x56115ea47f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd016da640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6702f0 0x56115ea47f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd016da640 1 -- 172.21.15.19:0/1449740418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115e77d620 con 0x56115e6784d0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd026dc640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6702f0 0x56115ea47f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd016da640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 0x56115ea48480 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdcec012720 tx=0x7fdcec012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b833142fe21b3002 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdce57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdce57fa640 1 -- 172.21.15.19:0/1449740418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004030 con 0x56115e6784d0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdce57fa640 1 -- 172.21.15.19:0/1449740418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdcec0041d0 con 0x56115e6784d0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf4036030 con 0x56115e6784d0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdce57fa640 1 -- 172.21.15.19:0/1449740418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdcec004410 con 0x56115e6784d0 2024-05-17T09:32:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd0aff4740 10 client.17643 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.283+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56115ea52370 con 0x56115e6784d0 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdce57fa640 1 -- 172.21.15.19:0/1449740418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcf4036030 con 0x56115e6784d0 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdce57fa640 1 -- 172.21.15.19:0/1449740418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdcec003ab0 con 0x56115e6784d0 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd0aff4740 10 client.17643 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd0aff4740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd0aff4740 20 client.17643 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e657d80 0x56115ea55b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd0aff4740 4 client.17643 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115e9aeb80 con 0x56115e657d80 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.284+0000 7fdd026dc640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e657d80 0x56115ea55b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdd026dc640 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e657d80 0x56115ea55b50 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d41aedb11ab8df server_cookie=9452edd14faee0b in_seq=0 out_seq=0 2024-05-17T09:32:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdce57fa640 10 client.17643 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdce57fa640 1 -- 172.21.15.19:0/1449740418 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56115e9aeb80 con 0x56115e657d80 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdce57fa640 10 client.17643 handle_command_reply: tid=0 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdd0aff4740 1 client.17643 shutdown 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 msgr2=0x56115ea48480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.285+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 0x56115ea48480 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdcec012720 tx=0x7fdcec012be0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 shutdown_connections 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e657d80 0x56115ea55b50 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6991b0 0x56115ea4f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6784d0 0x56115ea48480 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdce57fa640 0 client.17643 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/1449740418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6702f0 0x56115ea47f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 >> 172.21.15.19:0/1449740418 conn(0x56115e68d300 msgr2=0x56115ea34d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 shutdown_connections 2024-05-17T09:32:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 1 -- 172.21.15.19:0/1449740418 wait complete. 2024-05-17T09:32:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.286+0000 7fdd0aff4740 20 client.17643 trim_cache size 0 max 16384 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.302+0000 7fdd0aff4740 1 Processor -- start 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.302+0000 7fdd0aff4740 1 -- start start 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 0x56115ea42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6784d0 0x56115ea4f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1acfb0 con 0x56115e6702f0 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1aa880 con 0x56115e657d80 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1a86d0 con 0x56115e6784d0 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 0x56115ea42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea39240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59306/0 (socket says 172.21.15.19:59306) 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd016da640 1 -- 172.21.15.19:0/3581397808 learned_addr learned my addr 172.21.15.19:0/3581397808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 0x56115ea42d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:42546/0 (socket says 172.21.15.19:42546) 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6784d0 0x56115ea4f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 -- 172.21.15.19:0/3581397808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6784d0 msgr2=0x56115ea4f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6784d0 0x56115ea4f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 -- 172.21.15.19:0/3581397808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 msgr2=0x56115ea39240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea39240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd026dc640 1 -- 172.21.15.19:0/3581397808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115ea3f9b0 con 0x56115e657d80 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.303+0000 7fdd016da640 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea39240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.304+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 0x56115ea42d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4004af0 tx=0x7fdcf4031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352e3b3707aafcbf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.304+0000 7fdce77fe640 1 -- 172.21.15.19:0/3581397808 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4031ba0 con 0x56115e657d80 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.304+0000 7fdce77fe640 1 -- 172.21.15.19:0/3581397808 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcf4036070 con 0x56115e657d80 2024-05-17T09:32:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.304+0000 7fdce77fe640 1 -- 172.21.15.19:0/3581397808 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40045c0 con 0x56115e657d80 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3581397808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 msgr2=0x56115ea42d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 0x56115ea42d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4004af0 tx=0x7fdcf4031000 comp rx=0 tx=0).stop 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3581397808 shutdown_connections 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e6784d0 0x56115ea4f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea39240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3581397808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e657d80 0x56115ea42d00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3581397808 >> 172.21.15.19:0/3581397808 conn(0x56115e68d300 msgr2=0x56115ea38b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3581397808 shutdown_connections 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3581397808 wait complete. 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 -- start start 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.305+0000 7fdd0aff4740 1 Processor -- start 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e657d80 0x56115ea4ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115eabc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1a8d40 con 0x56115e6702f0 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1aa1b0 con 0x56115e6784d0 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1a9410 con 0x56115e657d80 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd0aff4740 10 client.0 _ll_register_callbacks cb 0x7fdcec009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea4cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59314/0 (socket says 172.21.15.19:59314) 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 -- 172.21.15.19:0/3614413902 learned_addr learned my addr 172.21.15.19:0/3614413902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e657d80 0x56115ea4ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115eabc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 -- 172.21.15.19:0/3614413902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e657d80 msgr2=0x56115ea4ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e657d80 0x56115ea4ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 -- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 msgr2=0x56115eabc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115eabc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.306+0000 7fdd016da640 1 -- 172.21.15.19:0/3614413902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115eabc750 con 0x56115e6702f0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdd016da640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea4cf70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0126f0 tx=0x7fdcec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4ef0f32a216046 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdce57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdce57fa640 1 -- 172.21.15.19:0/3614413902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec01c980 con 0x56115e6702f0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdce57fa640 1 -- 172.21.15.19:0/3614413902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdcec01cb20 con 0x56115e6702f0 2024-05-17T09:32:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf4036070 con 0x56115e6702f0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdce57fa640 1 -- 172.21.15.19:0/3614413902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdcec036050 con 0x56115e6702f0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdd0aff4740 10 client.17649 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56115eabd8c0 con 0x56115e6702f0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdce57fa640 1 -- 172.21.15.19:0/3614413902 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcf4036070 con 0x56115e6702f0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.307+0000 7fdce57fa640 1 -- 172.21.15.19:0/3614413902 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdcec031030 con 0x56115e6702f0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd0aff4740 10 client.17649 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd0aff4740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd0aff4740 20 client.17649 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e6991b0 0x56115eac0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd0aff4740 4 client.17649 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56115e9a8360 con 0x56115e6991b0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e6991b0 0x56115eac0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdce57fa640 10 client.17649 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.308+0000 7fdd026dc640 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e6991b0 0x56115eac0330 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5240e0d679d01c79 server_cookie=ee793c4df8eb1f4c in_seq=0 out_seq=0 2024-05-17T09:32:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.326+0000 7fdce57fa640 1 -- 172.21.15.19:0/3614413902 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56115e9a8360 con 0x56115e6991b0 2024-05-17T09:32:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.326+0000 7fdce57fa640 10 client.17649 handle_command_reply: tid=0 2024-05-17T09:32:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.326+0000 7fdd0aff4740 1 client.17649 shutdown 2024-05-17T09:32:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 msgr2=0x56115ea4cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea4cf70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0126f0 tx=0x7fdcec00f790 comp rx=0 tx=0).stop 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 shutdown_connections 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56115e6991b0 0x56115eac0330 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56115e6784d0 0x56115eabc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56115e6702f0 0x56115ea4cf70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 --2- 172.21.15.19:0/3614413902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56115e657d80 0x56115ea4ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdce57fa640 0 client.17649 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 >> 172.21.15.19:0/3614413902 conn(0x56115e68d300 msgr2=0x56115ea39560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 shutdown_connections 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 1 -- 172.21.15.19:0/3614413902 wait complete. 2024-05-17T09:32:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.327+0000 7fdd0aff4740 20 client.17649 trim_cache size 0 max 16384 2024-05-17T09:32:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:32:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:32:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:32:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdce80b28a0 msgr2=0x7fdce80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdce80b28a0 0x7fdce80b4db0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56115ea3fb90 tx=0x7fdcf80e8550 comp rx=0 tx=0).stop 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 msgr2=0x7fdcfc19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 0x7fdcfc19a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800e9f0 tx=0x7fdcf800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 shutdown_connections 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcfc109ba0 0x7fdcfc1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdce80b28a0 0x7fdce80b4db0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcfc104ea0 0x7fdcfc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 --2- 172.21.15.19:0/4037900486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcfc103c60 0x7fdcfc19a170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 >> 172.21.15.19:0/4037900486 conn(0x7fdcfc0fee50 msgr2=0x7fdcfc105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 shutdown_connections 2024-05-17T09:32:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:26.328+0000 7fdd0413f640 1 -- 172.21.15.19:0/4037900486 wait complete. 2024-05-17T09:32:26.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 204640 1.72 MB/sec execute 504 sec latency 15.302 ms 2024-05-17T09:32:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:26 smithi079 ceph-mon[35502]: pgmap v818: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-17T09:32:26.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:26 smithi069 ceph-mon[35081]: pgmap v818: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-17T09:32:26.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:26 smithi019 ceph-mon[31620]: pgmap v818: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-17T09:32:27.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 205394 1.73 MB/sec execute 505 sec latency 16.326 ms 2024-05-17T09:32:27.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:27 smithi069 ceph-mon[35081]: pgmap v819: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-17T09:32:27.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:27 smithi019 ceph-mon[31620]: pgmap v819: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-17T09:32:28.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:27 smithi079 ceph-mon[35502]: pgmap v819: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-17T09:32:28.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 205814 1.74 MB/sec execute 506 sec latency 15.636 ms 2024-05-17T09:32:29.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 206116 1.74 MB/sec execute 507 sec latency 14.631 ms 2024-05-17T09:32:30.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 206403 1.73 MB/sec execute 508 sec latency 12.930 ms 2024-05-17T09:32:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:30 smithi079 ceph-mon[35502]: pgmap v820: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:32:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:30 smithi069 ceph-mon[35081]: pgmap v820: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:32:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:30 smithi019 ceph-mon[31620]: pgmap v820: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-17T09:32:31.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 206869 1.73 MB/sec execute 509 sec latency 28.098 ms 2024-05-17T09:32:31.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:31 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:32:31.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:31 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:32:31.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:31 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:32:32.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 207391 1.74 MB/sec execute 510 sec latency 31.120 ms 2024-05-17T09:32:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:32 smithi079 ceph-mon[35502]: pgmap v821: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-17T09:32:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:32:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:32:32.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:32:32.584 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:32 smithi069 ceph-mon[35081]: pgmap v821: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-17T09:32:32.584 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:32:32.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:32:32.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:32:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:32 smithi019 ceph-mon[31620]: pgmap v821: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-17T09:32:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:32:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:32:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:32:33.449 INFO:tasks.workunit.client.0.smithi019.stdout: 1 208051 1.74 MB/sec execute 511 sec latency 15.907 ms 2024-05-17T09:32:34.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 208606 1.74 MB/sec execute 512 sec latency 15.434 ms 2024-05-17T09:32:34.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:34 smithi079 ceph-mon[35502]: pgmap v822: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-17T09:32:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:34 smithi069 ceph-mon[35081]: pgmap v822: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-17T09:32:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:34 smithi019 ceph-mon[31620]: pgmap v822: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-17T09:32:35.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 209076 1.74 MB/sec execute 513 sec latency 16.202 ms 2024-05-17T09:32:36.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 209550 1.75 MB/sec execute 514 sec latency 16.110 ms 2024-05-17T09:32:36.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:36 smithi079 ceph-mon[35502]: pgmap v823: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:32:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:36 smithi069 ceph-mon[35081]: pgmap v823: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:32:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:36 smithi019 ceph-mon[31620]: pgmap v823: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-17T09:32:37.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 209824 1.75 MB/sec execute 515 sec latency 14.947 ms 2024-05-17T09:32:38.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 210271 1.74 MB/sec execute 516 sec latency 15.220 ms 2024-05-17T09:32:38.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:38 smithi079 ceph-mon[35502]: pgmap v824: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:32:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:38 smithi069 ceph-mon[35081]: pgmap v824: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:32:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:38 smithi019 ceph-mon[31620]: pgmap v824: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 -- 172.21.15.19:0/378555706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34103d80 msgr2=0x7f6a34104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/378555706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34103d80 0x7f6a34104190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009860 tx=0x7f6a2402a340 comp rx=0 tx=0).stop 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 -- 172.21.15.19:0/378555706 shutdown_connections 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/378555706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34108a80 0x7f6a3410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/378555706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34103d80 0x7f6a34104190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/378555706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 0x7f6a34103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 -- 172.21.15.19:0/378555706 >> 172.21.15.19:0/378555706 conn(0x7f6a340fdd30 msgr2=0x7f6a341001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 -- 172.21.15.19:0/378555706 shutdown_connections 2024-05-17T09:32:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.443+0000 7f6a3c12e640 1 -- 172.21.15.19:0/378555706 wait complete. 2024-05-17T09:32:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 Processor -- start 2024-05-17T09:32:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 -- start start 2024-05-17T09:32:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 0x7f6a3406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 0x7f6a3419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34108a80 0x7f6a341a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3410e540 con 0x7f6a34103d80 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3410e3c0 con 0x7f6a34102b40 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a3c12e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3410e6c0 con 0x7f6a34108a80 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 0x7f6a3419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.444+0000 7f6a39eca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 0x7f6a3406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 0x7f6a3419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39326/0 (socket says 172.21.15.19:39326) 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a39eca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 0x7f6a3406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52766/0 (socket says 172.21.15.19:52766) 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a396c9640 1 -- 172.21.15.19:0/2959294930 learned_addr learned my addr 172.21.15.19:0/2959294930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34108a80 0x7f6a341a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 -- 172.21.15.19:0/2959294930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 msgr2=0x7f6a3406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 0x7f6a3406b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 -- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 msgr2=0x7f6a3419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 0x7f6a3419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 -- 172.21.15.19:0/2959294930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341a1cd0 con 0x7f6a34108a80 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a396c9640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 0x7f6a3419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34108a80 0x7f6a341a14b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00ef00 tx=0x7f6a2c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a2affd640 1 -- 172.21.15.19:0/2959294930 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c004b20 con 0x7f6a34108a80 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a2affd640 1 -- 172.21.15.19:0/2959294930 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a2c003e00 con 0x7f6a34108a80 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a341a1fc0 con 0x7f6a34108a80 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a2affd640 1 -- 172.21.15.19:0/2959294930 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c004820 con 0x7f6a34108a80 2024-05-17T09:32:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.445+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a2480 con 0x7f6a34108a80 2024-05-17T09:32:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.448+0000 7f6a42fd9740 1 Processor -- start 2024-05-17T09:32:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.448+0000 7f6a2affd640 1 -- 172.21.15.19:0/2959294930 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a2c01d020 con 0x7f6a34108a80 2024-05-17T09:32:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.448+0000 7f6a42fd9740 1 -- start start 2024-05-17T09:32:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb90fe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9105820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fcb9070c30 con 0x55fcb8d342f0 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcb90607d0 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fcb9064810 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fe3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:52774/0 (socket says 172.21.15.19:52774) 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 -- 172.21.15.19:0/77077824 learned_addr learned my addr 172.21.15.19:0/77077824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a39eca640 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb90fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 -- 172.21.15.19:0/77077824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 msgr2=0x55fcb9105820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9105820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 -- 172.21.15.19:0/77077824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 msgr2=0x55fcb90fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb90fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.449+0000 7f6a396c9640 1 -- 172.21.15.19:0/77077824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcb8d75210 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a2affd640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a200b2920 0x7f6a200b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a396c9640 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fe3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a240095f0 tx=0x7f6a240056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5bf75b222b6cd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:39.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 210839 1.75 MB/sec execute 517 sec latency 14.555 ms 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a39eca640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a200b2920 0x7f6a200b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a2affd640 1 -- 172.21.15.19:0/2959294930 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a2c0e2390 con 0x7f6a34108a80 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a1ffff640 1 -- 172.21.15.19:0/77077824 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24037040 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a1ffff640 1 -- 172.21.15.19:0/77077824 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a240044d0 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.450+0000 7f6a1ffff640 1 -- 172.21.15.19:0/77077824 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24032ae0 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 -- 172.21.15.19:0/77077824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 msgr2=0x55fcb90fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a39eca640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a200b2920 0x7f6a200b4e30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55fcb90ff780 tx=0x7f6a30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fe3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a240095f0 tx=0x7f6a240056d0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 -- 172.21.15.19:0/77077824 shutdown_connections 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb90fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/77077824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fe3a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 -- 172.21.15.19:0/77077824 >> 172.21.15.19:0/77077824 conn(0x55fcb8d51300 msgr2=0x55fcb90f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 -- 172.21.15.19:0/77077824 shutdown_connections 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 -- 172.21.15.19:0/77077824 wait complete. 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 -- start start 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.451+0000 7f6a42fd9740 1 Processor -- start 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb9107cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb910f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fcb8d46790 con 0x55fcb8d342f0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcb901c890 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fcb9106210 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a42fd9740 10 client.0 _ll_register_callbacks cb 0x55fcb901fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a39eca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb9107cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a39eca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb9107cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39356/0 (socket says 172.21.15.19:39356) 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a39eca640 1 -- 172.21.15.19:0/461156828 learned_addr learned my addr 172.21.15.19:0/461156828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a396c9640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb910f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a396c9640 1 -- 172.21.15.19:0/461156828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 msgr2=0x55fcb910f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a396c9640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb910f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a396c9640 1 -- 172.21.15.19:0/461156828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 msgr2=0x55fcb9107cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a396c9640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb9107cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a396c9640 1 -- 172.21.15.19:0/461156828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcb8e41620 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.452+0000 7f6a39eca640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb9107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a396c9640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9108230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a240095f0 tx=0x7f6a24030740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c333f3840347e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/461156828 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24051060 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a240044d0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/461156828 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a24032a60 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/461156828 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a24058050 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a42fd9740 10 client.27245 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.453+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fcb91120c0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/461156828 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a240044d0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/461156828 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6a24049070 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a42fd9740 10 client.27245 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a42fd9740 10 client.27245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a42fd9740 20 client.27245 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8d1bd80 0x55fcb9115420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a42fd9740 4 client.27245 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcb9072b80 con 0x55fcb8d1bd80 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.454+0000 7f6a39eca640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8d1bd80 0x55fcb9115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.455+0000 7f6a39eca640 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8d1bd80 0x55fcb9115420 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782df0c5bf7eaec1 server_cookie=7cb8db3d25a1a075 in_seq=0 out_seq=0 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.455+0000 7f6a1d7fa640 10 client.27245 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.456+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/461156828 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fcb9072b80 con 0x55fcb8d1bd80 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.456+0000 7f6a1d7fa640 10 client.27245 handle_command_reply: tid=0 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.456+0000 7f6a42fd9740 1 client.27245 shutdown 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d3c4d0 msgr2=0x55fcb9108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9108230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a240095f0 tx=0x7f6a24030740 comp rx=0 tx=0).stop 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 shutdown_connections 2024-05-17T09:32:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8d1bd80 0x55fcb9115420 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb910f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d3c4d0 0x55fcb9108230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/461156828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb9107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a1d7fa640 0 client.27245 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 >> 172.21.15.19:0/461156828 conn(0x55fcb8d51300 msgr2=0x55fcb90fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 shutdown_connections 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 1 -- 172.21.15.19:0/461156828 wait complete. 2024-05-17T09:32:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.457+0000 7f6a42fd9740 20 client.27245 trim_cache size 0 max 16384 2024-05-17T09:32:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a42fd9740 1 Processor -- start 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a42fd9740 1 -- start start 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d342f0 0x55fcb9102950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.478+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39368/0 (socket says 172.21.15.19:39368) 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a396c9640 1 -- 172.21.15.19:0/1676133189 learned_addr learned my addr 172.21.15.19:0/1676133189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a39eca640 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d342f0 0x55fcb9102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341af1a0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341aca70 con 0x55fcb8d342f0 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341aa8c0 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 -- 172.21.15.19:0/1676133189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d342f0 msgr2=0x55fcb9102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d342f0 0x55fcb9102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 -- 172.21.15.19:0/1676133189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 msgr2=0x55fcb910f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 -- 172.21.15.19:0/1676133189 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcb90fd780 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a396c9640 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fd240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a92a0 tx=0x7f6a2c0a9020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f516aed9740c284 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.479+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/1676133189 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c01a040 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/1676133189 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a2c0a95a0 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/1676133189 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0a5830 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 msgr2=0x55fcb90fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fd240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a92a0 tx=0x7f6a2c0a9020 comp rx=0 tx=0).stop 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 shutdown_connections 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d5d1b0 0x55fcb90fd240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1676133189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d342f0 0x55fcb9102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.480+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 >> 172.21.15.19:0/1676133189 conn(0x55fcb8d51300 msgr2=0x55fcb90fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.481+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 shutdown_connections 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.481+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1676133189 wait complete. 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.481+0000 7f6a42fd9740 1 -- start start 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.481+0000 7f6a42fd9740 1 Processor -- start 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb910e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb9184450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341aaf30 con 0x55fcb8d342f0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341ac3a0 con 0x55fcb8d5d1b0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341ab600 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a42fd9740 10 client.0 _ll_register_callbacks cb 0x7f6a240096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a39eca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb910e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37454/0 (socket says 172.21.15.19:37454) 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a39eca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb910e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39384/0 (socket says 172.21.15.19:39384) 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 -- 172.21.15.19:0/1389144347 learned_addr learned my addr 172.21.15.19:0/1389144347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a3a6cb640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb9184450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 -- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 msgr2=0x55fcb9184450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb9184450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 -- 172.21.15.19:0/1389144347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 msgr2=0x55fcb910e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb910e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a396c9640 1 -- 172.21.15.19:0/1389144347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcb9184a40 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.482+0000 7f6a39eca640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb910e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a396c9640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910ea30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a240095f0 tx=0x7f6a24005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aefb8d514e2229 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/1389144347 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24061060 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2c0a95a0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/1389144347 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a24004030 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/1389144347 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a24037040 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a42fd9740 10 client.27229 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fcb9185cd0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/1389144347 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a2c0a95a0 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/1389144347 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6a24049050 con 0x55fcb8d3c4d0 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.483+0000 7f6a42fd9740 10 client.27229 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a42fd9740 10 client.27229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a42fd9740 20 client.27229 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8cf11c0 0x55fcb9188690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a42fd9740 4 client.27229 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcb906c360 con 0x55fcb8cf11c0 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a39eca640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8cf11c0 0x55fcb9188690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a1d7fa640 10 client.27229 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.484+0000 7f6a39eca640 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8cf11c0 0x55fcb9188690 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220fd062f0354a35 server_cookie=84fe7661a82470a5 in_seq=0 out_seq=0 2024-05-17T09:32:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a1d7fa640 1 -- 172.21.15.19:0/1389144347 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fcb906c360 con 0x55fcb8cf11c0 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a1d7fa640 10 client.27229 handle_command_reply: tid=0 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 client.27229 shutdown 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 msgr2=0x55fcb910ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910ea30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a240095f0 tx=0x7f6a24005de0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 shutdown_connections 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fcb8cf11c0 0x55fcb9188690 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcb8d5d1b0 0x55fcb9184450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fcb8d3c4d0 0x55fcb910ea30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 --2- 172.21.15.19:0/1389144347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fcb8d342f0 0x55fcb910e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a1d7fa640 0 client.27229 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 >> 172.21.15.19:0/1389144347 conn(0x55fcb8d51300 msgr2=0x55fcb91017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 shutdown_connections 2024-05-17T09:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 1 -- 172.21.15.19:0/1389144347 wait complete. 2024-05-17T09:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.503+0000 7f6a42fd9740 20 client.27229 trim_cache size 0 max 16384 2024-05-17T09:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a200b2920 msgr2=0x7f6a200b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a200b2920 0x7f6a200b4e30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55fcb90ff780 tx=0x7f6a30005f90 comp rx=0 tx=0).stop 2024-05-17T09:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34108a80 msgr2=0x7f6a341a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34108a80 0x7f6a341a14b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00ef00 tx=0x7f6a2c00c740 comp rx=0 tx=0).stop 2024-05-17T09:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 shutdown_connections 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34108a80 0x7f6a341a14b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a200b2920 0x7f6a200b4e30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a34103d80 0x7f6a3419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 --2- 172.21.15.19:0/2959294930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34102b40 0x7f6a3406b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 >> 172.21.15.19:0/2959294930 conn(0x7f6a340fdd30 msgr2=0x7f6a34104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.504+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 shutdown_connections 2024-05-17T09:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:39.505+0000 7f6a3c12e640 1 -- 172.21.15.19:0/2959294930 wait complete. 2024-05-17T09:32:40.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 211457 1.75 MB/sec execute 518 sec latency 20.651 ms 2024-05-17T09:32:40.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:40 smithi079 ceph-mon[35502]: pgmap v825: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-17T09:32:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:40 smithi069 ceph-mon[35081]: pgmap v825: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-17T09:32:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:40 smithi019 ceph-mon[31620]: pgmap v825: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-17T09:32:41.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 211995 1.75 MB/sec execute 519 sec latency 17.753 ms 2024-05-17T09:32:42.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 212599 1.76 MB/sec execute 520 sec latency 13.940 ms 2024-05-17T09:32:42.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:42 smithi079 ceph-mon[35502]: pgmap v826: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:32:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:42 smithi069 ceph-mon[35081]: pgmap v826: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:32:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:42 smithi019 ceph-mon[31620]: pgmap v826: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:32:43.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 213062 1.76 MB/sec execute 521 sec latency 27.561 ms 2024-05-17T09:32:44.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 213298 1.76 MB/sec execute 522 sec latency 13.631 ms 2024-05-17T09:32:44.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:44 smithi079 ceph-mon[35502]: pgmap v827: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:32:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:44 smithi069 ceph-mon[35081]: pgmap v827: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:32:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:44 smithi019 ceph-mon[31620]: pgmap v827: 129 pgs: 129 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:32:45.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 213723 1.76 MB/sec execute 523 sec latency 15.975 ms 2024-05-17T09:32:46.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 214272 1.76 MB/sec execute 524 sec latency 15.434 ms 2024-05-17T09:32:46.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:46 smithi079 ceph-mon[35502]: pgmap v828: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-17T09:32:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:46 smithi069 ceph-mon[35081]: pgmap v828: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-17T09:32:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:46 smithi019 ceph-mon[31620]: pgmap v828: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-17T09:32:47.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 214864 1.76 MB/sec execute 525 sec latency 48.352 ms 2024-05-17T09:32:48.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 215376 1.76 MB/sec execute 526 sec latency 15.610 ms 2024-05-17T09:32:48.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:48 smithi079 ceph-mon[35502]: pgmap v829: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-05-17T09:32:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:48 smithi069 ceph-mon[35081]: pgmap v829: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-05-17T09:32:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:48 smithi019 ceph-mon[31620]: pgmap v829: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-05-17T09:32:49.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 216155 1.77 MB/sec execute 527 sec latency 17.942 ms 2024-05-17T09:32:50.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 216532 1.77 MB/sec execute 528 sec latency 17.840 ms 2024-05-17T09:32:50.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:50 smithi079 ceph-mon[35502]: pgmap v830: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-05-17T09:32:50.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:50 smithi069 ceph-mon[35081]: pgmap v830: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-05-17T09:32:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:50 smithi019 ceph-mon[31620]: pgmap v830: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-05-17T09:32:51.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 216830 1.77 MB/sec execute 529 sec latency 12.473 ms 2024-05-17T09:32:52.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 217169 1.77 MB/sec execute 530 sec latency 13.581 ms 2024-05-17T09:32:52.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:52 smithi079 ceph-mon[35502]: pgmap v831: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.621+0000 7fcb50429640 1 -- 172.21.15.19:0/461611805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 msgr2=0x7fcb48103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.621+0000 7fcb50429640 1 --2- 172.21.15.19:0/461611805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb48103860 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009900 tx=0x7fcb4402a410 comp rx=0 tx=0).stop 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 -- 172.21.15.19:0/461611805 shutdown_connections 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 --2- 172.21.15.19:0/461611805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb48108aa0 0x7fcb4810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 --2- 172.21.15.19:0/461611805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb48103da0 0x7fcb481041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 --2- 172.21.15.19:0/461611805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb48103860 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 -- 172.21.15.19:0/461611805 >> 172.21.15.19:0/461611805 conn(0x7fcb480fdd50 msgr2=0x7fcb48100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 -- 172.21.15.19:0/461611805 shutdown_connections 2024-05-17T09:32:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 -- 172.21.15.19:0/461611805 wait complete. 2024-05-17T09:32:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 Processor -- start 2024-05-17T09:32:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.622+0000 7fcb50429640 1 -- start start 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb50429640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb4806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb50429640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb48103da0 0x7fcb4819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb50429640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb48108aa0 0x7fcb481a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb50429640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810e4f0 con 0x7fcb48102b60 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb50429640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810e370 con 0x7fcb48103da0 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb50429640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810e670 con 0x7fcb48108aa0 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb4806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb4806bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49270/0 (socket says 172.21.15.19:49270) 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 -- 172.21.15.19:0/4101890354 learned_addr learned my addr 172.21.15.19:0/4101890354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb48103da0 0x7fcb4819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb48108aa0 0x7fcb481a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 -- 172.21.15.19:0/4101890354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb48108aa0 msgr2=0x7fcb481a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb48108aa0 0x7fcb481a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 -- 172.21.15.19:0/4101890354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb48103da0 msgr2=0x7fcb4819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb48103da0 0x7fcb4819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 -- 172.21.15.19:0/4101890354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb481a1d30 con 0x7fcb48102b60 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb4e1c5640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb4806bdb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcb44005760 tx=0x7fcb44004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.623+0000 7fcb3f7fe640 1 -- 172.21.15.19:0/4101890354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4403f030 con 0x7fcb48102b60 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.624+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb481a1fc0 con 0x7fcb48102b60 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.624+0000 7fcb3f7fe640 1 -- 172.21.15.19:0/4101890354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb44004440 con 0x7fcb48102b60 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.624+0000 7fcb3f7fe640 1 -- 172.21.15.19:0/4101890354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44035070 con 0x7fcb48102b60 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.624+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481a23d0 con 0x7fcb48102b60 2024-05-17T09:32:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 Processor -- start 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 -- start start 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb3f7fe640 1 -- 172.21.15.19:0/4101890354 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcb44042030 con 0x7fcb48102b60 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 0x55bd57a467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a46d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576804d0 0x55bd57a4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd579b4c30 con 0x55bd576782f0 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bd579a47d0 con 0x55bd576a11b0 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb573b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bd579a8810 con 0x55bd576804d0 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.627+0000 7fcb4e1c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a46d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 0x55bd57a467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 0x55bd57a467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37502/0 (socket says 172.21.15.19:37502) 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/3468503646 learned_addr learned my addr 172.21.15.19:0/3468503646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576804d0 0x55bd57a4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/3468503646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576804d0 msgr2=0x55bd57a4dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576804d0 0x55bd57a4dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/3468503646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 msgr2=0x55bd57a46d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a46d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/3468503646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd576b9210 con 0x55bd576a11b0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 0x55bd57a467b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009fd0 tx=0x7fcb40010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f9b90fcbe41184 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb3cbfe640 1 -- 172.21.15.19:0/3468503646 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40011700 con 0x55bd576a11b0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb3cbfe640 1 -- 172.21.15.19:0/3468503646 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb40011d30 con 0x55bd576a11b0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb3cbfe640 1 -- 172.21.15.19:0/3468503646 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4001cdc0 con 0x55bd576a11b0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.628+0000 7fcb3f7fe640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcb340b28a0 0x7fcb340b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.629+0000 7fcb4e1c5640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcb340b28a0 0x7fcb340b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.629+0000 7fcb3f7fe640 1 -- 172.21.15.19:0/4101890354 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcb44102a00 con 0x7fcb48102b60 2024-05-17T09:32:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 -- 172.21.15.19:0/3468503646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 msgr2=0x55bd57a467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb4e1c5640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcb340b28a0 0x7fcb340b4db0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55bd57a47b90 tx=0x7fcb44106430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 0x55bd57a467b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009fd0 tx=0x7fcb40010860 comp rx=0 tx=0).stop 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 -- 172.21.15.19:0/3468503646 shutdown_connections 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576804d0 0x55bd57a4dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a46d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3468503646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576a11b0 0x55bd57a467b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 -- 172.21.15.19:0/3468503646 >> 172.21.15.19:0/3468503646 conn(0x55bd57695300 msgr2=0x55bd57a3dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 -- 172.21.15.19:0/3468503646 shutdown_connections 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 -- 172.21.15.19:0/3468503646 wait complete. 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 -- start start 2024-05-17T09:32:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.630+0000 7fcb573b5740 1 Processor -- start 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 0x55bd57a50100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576804d0 0x55bd57a50640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd5768a790 con 0x55bd576804d0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bd57960890 con 0x55bd576782f0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bd57a4e680 con 0x55bd576a11b0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb573b5740 10 client.0 _ll_register_callbacks cb 0x55bd57963d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 0x55bd57a50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4d9c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576804d0 0x55bd57a50640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 0x55bd57a50100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37504/0 (socket says 172.21.15.19:37504) 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/1553665768 learned_addr learned my addr 172.21.15.19:0/1553665768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/1553665768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 msgr2=0x55bd57a575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 msgr2=0x55bd57a50100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 0x55bd57a50100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/1553665768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd57785620 con 0x55bd576804d0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.631+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 0x55bd57a50100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576804d0 0x55bd57a50640 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcb4806b750 tx=0x7fcb3800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dba17a1d1e52829 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb31ffb640 1 -- 172.21.15.19:0/1553665768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38003dc0 con 0x55bd576804d0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb31ffb640 1 -- 172.21.15.19:0/1553665768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcb38011440 con 0x55bd576804d0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb40011d30 con 0x55bd576804d0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb31ffb640 1 -- 172.21.15.19:0/1553665768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb38011640 con 0x55bd576804d0 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb573b5740 10 client.17679 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bd57a5a580 con 0x55bd576804d0 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.632+0000 7fcb31ffb640 1 -- 172.21.15.19:0/1553665768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb40011d30 con 0x55bd576804d0 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb31ffb640 1 -- 172.21.15.19:0/1553665768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fcb3801b7a0 con 0x55bd576804d0 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb573b5740 10 client.17679 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb573b5740 10 client.17679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb573b5740 20 client.17679 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb573b5740 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd5765fd80 0x55bd57a5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb573b5740 4 client.17679 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd579b6b80 con 0x55bd5765fd80 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd5765fd80 0x55bd57a5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb31ffb640 10 client.17679 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.633+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd5765fd80 0x55bd57a5dda0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8a7758071af180 server_cookie=740a2940be0def10 in_seq=0 out_seq=0 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb31ffb640 1 -- 172.21.15.19:0/1553665768 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bd579b6b80 con 0x55bd5765fd80 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb31ffb640 10 client.17679 handle_command_reply: tid=0 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 client.17679 shutdown 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576804d0 msgr2=0x55bd57a50640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576804d0 0x55bd57a50640 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcb4806b750 tx=0x7fcb3800bf30 comp rx=0 tx=0).stop 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 shutdown_connections 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd5765fd80 0x55bd57a5dda0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576804d0 0x55bd57a50640 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 --2- 172.21.15.19:0/1553665768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576782f0 0x55bd57a50100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb31ffb640 0 client.17679 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 >> 172.21.15.19:0/1553665768 conn(0x55bd57695300 msgr2=0x55bd57a3cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 shutdown_connections 2024-05-17T09:32:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 1 -- 172.21.15.19:0/1553665768 wait complete. 2024-05-17T09:32:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.635+0000 7fcb573b5740 20 client.17679 trim_cache size 0 max 16384 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 Processor -- start 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 -- start start 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a3d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a3d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481ad170 con 0x55bd576782f0 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481aaa40 con 0x55bd576804d0 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb573b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a8890 con 0x55bd576a11b0 2024-05-17T09:32:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a3d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a3d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49300/0 (socket says 172.21.15.19:49300) 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/3558338592 learned_addr learned my addr 172.21.15.19:0/3558338592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.651+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a3d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/3558338592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 msgr2=0x55bd57a574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/3558338592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 msgr2=0x55bd57a3d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a3d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/3558338592 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd57a3ddc0 con 0x55bd576804d0 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a3d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a3d880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb4806b750 tx=0x7fcb38003a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbef8fc169d366fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb33fff640 1 -- 172.21.15.19:0/3558338592 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38010440 con 0x55bd576804d0 2024-05-17T09:32:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb33fff640 1 -- 172.21.15.19:0/3558338592 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb3802e070 con 0x55bd576804d0 2024-05-17T09:32:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.652+0000 7fcb33fff640 1 -- 172.21.15.19:0/3558338592 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38011c20 con 0x55bd576804d0 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 -- 172.21.15.19:0/3558338592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 msgr2=0x55bd57a3d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a3d880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb4806b750 tx=0x7fcb38003a80 comp rx=0 tx=0).stop 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 -- 172.21.15.19:0/3558338592 shutdown_connections 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576a11b0 0x55bd57a574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a3d880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 --2- 172.21.15.19:0/3558338592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576782f0 0x55bd57a3d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 -- 172.21.15.19:0/3558338592 >> 172.21.15.19:0/3558338592 conn(0x55bd57695300 msgr2=0x55bd57a49c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 -- 172.21.15.19:0/3558338592 shutdown_connections 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.653+0000 7fcb573b5740 1 -- 172.21.15.19:0/3558338592 wait complete. 2024-05-17T09:32:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 -- start start 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 Processor -- start 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576782f0 0x55bd57a54e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576a11b0 0x55bd57ac4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a8f00 con 0x55bd576a11b0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481aa370 con 0x55bd576804d0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a95d0 con 0x55bd576782f0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb573b5740 10 client.0 _ll_register_callbacks cb 0x7fcb38004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576a11b0 0x55bd57ac4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4d9c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576782f0 0x55bd57a54e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4d9c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a55340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37512/0 (socket says 172.21.15.19:37512) 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4d9c4640 1 -- 172.21.15.19:0/2571602496 learned_addr learned my addr 172.21.15.19:0/2571602496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 msgr2=0x55bd57a55340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a55340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/2571602496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576a11b0 msgr2=0x55bd57ac4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576a11b0 0x55bd57ac4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.654+0000 7fcb4e9c6640 1 -- 172.21.15.19:0/2571602496 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd57ac4a20 con 0x55bd576782f0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb4e9c6640 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576782f0 0x55bd57a54e00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb400099a0 tx=0x7fcb40009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ad81d6039ad061 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb31ffb640 1 -- 172.21.15.19:0/2571602496 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40030070 con 0x55bd576782f0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb31ffb640 1 -- 172.21.15.19:0/2571602496 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcb400049e0 con 0x55bd576782f0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3802e070 con 0x55bd576782f0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb31ffb640 1 -- 172.21.15.19:0/2571602496 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb40004be0 con 0x55bd576782f0 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb573b5740 10 client.27244 fetch_fsmap learned FSMap version 32 2024-05-17T09:32:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bd57ac5c90 con 0x55bd576782f0 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.655+0000 7fcb31ffb640 1 -- 172.21.15.19:0/2571602496 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb3802e070 con 0x55bd576782f0 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb31ffb640 1 -- 172.21.15.19:0/2571602496 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fcb40005020 con 0x55bd576782f0 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb573b5740 10 client.27244 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb573b5740 10 client.27244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb573b5740 20 client.27244 populate_metadata read hostname 'smithi019' 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb573b5740 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd576351c0 0x55bd57ac8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb573b5740 4 client.27244 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd579b0360 con 0x55bd576351c0 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd576351c0 0x55bd57ac8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.656+0000 7fcb31ffb640 10 client.27244 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.657+0000 7fcb4d9c4640 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd576351c0 0x55bd57ac8450 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd44e5e89793d1c0 server_cookie=5744f66184047ccc in_seq=0 out_seq=0 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.678+0000 7fcb31ffb640 1 -- 172.21.15.19:0/2571602496 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd579b0360 con 0x55bd576351c0 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.678+0000 7fcb31ffb640 10 client.27244 handle_command_reply: tid=0 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.678+0000 7fcb573b5740 1 client.27244 shutdown 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.678+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576782f0 msgr2=0x55bd57a54e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.678+0000 7fcb573b5740 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576782f0 0x55bd57a54e00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb400099a0 tx=0x7fcb40009be0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 shutdown_connections 2024-05-17T09:32:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bd576351c0 0x55bd57ac8450 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd576a11b0 0x55bd57ac4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bd576804d0 0x55bd57a55340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 --2- 172.21.15.19:0/2571602496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd576782f0 0x55bd57a54e00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 >> 172.21.15.19:0/2571602496 conn(0x55bd57695300 msgr2=0x55bd57a4a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb31ffb640 0 client.27244 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 shutdown_connections 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 1 -- 172.21.15.19:0/2571602496 wait complete. 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb573b5740 20 client.27244 trim_cache size 0 max 16384 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:32:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:32:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:32:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcb340b28a0 msgr2=0x7fcb340b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcb340b28a0 0x7fcb340b4db0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55bd57a47b90 tx=0x7fcb44106430 comp rx=0 tx=0).stop 2024-05-17T09:32:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 msgr2=0x7fcb4806bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:32:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb4806bdb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcb44005760 tx=0x7fcb44004040 comp rx=0 tx=0).stop 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 shutdown_connections 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb48108aa0 0x7fcb481a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb48103da0 0x7fcb4819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.679+0000 7fcb50429640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcb340b28a0 0x7fcb340b4db0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.680+0000 7fcb50429640 1 --2- 172.21.15.19:0/4101890354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb48102b60 0x7fcb4806bdb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.680+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 >> 172.21.15.19:0/4101890354 conn(0x7fcb480fdd50 msgr2=0x7fcb48104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.680+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 shutdown_connections 2024-05-17T09:32:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:32:52.680+0000 7fcb50429640 1 -- 172.21.15.19:0/4101890354 wait complete. 2024-05-17T09:32:52.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:52 smithi069 ceph-mon[35081]: pgmap v831: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-17T09:32:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:52 smithi019 ceph-mon[31620]: pgmap v831: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-05-17T09:32:53.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 217640 1.77 MB/sec execute 531 sec latency 22.749 ms 2024-05-17T09:32:54.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 218248 1.77 MB/sec execute 532 sec latency 17.223 ms 2024-05-17T09:32:54.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:54 smithi079 ceph-mon[35502]: pgmap v832: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:32:54.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:54 smithi069 ceph-mon[35081]: pgmap v832: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:32:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:54 smithi019 ceph-mon[31620]: pgmap v832: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-17T09:32:55.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 218822 1.77 MB/sec execute 533 sec latency 16.140 ms 2024-05-17T09:32:56.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 219410 1.78 MB/sec execute 534 sec latency 15.734 ms 2024-05-17T09:32:56.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:56 smithi069 ceph-mon[35081]: pgmap v833: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:56 smithi019 ceph-mon[31620]: pgmap v833: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:56 smithi079 ceph-mon[35502]: pgmap v833: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:57.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 219940 1.78 MB/sec execute 535 sec latency 16.544 ms 2024-05-17T09:32:58.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 220291 1.78 MB/sec execute 536 sec latency 15.391 ms 2024-05-17T09:32:58.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:32:58 smithi069 ceph-mon[35081]: pgmap v834: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:32:58 smithi019 ceph-mon[31620]: pgmap v834: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:32:58 smithi079 ceph-mon[35502]: pgmap v834: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-17T09:32:59.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 220562 1.78 MB/sec execute 537 sec latency 13.011 ms 2024-05-17T09:33:00.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 221033 1.78 MB/sec execute 538 sec latency 12.727 ms 2024-05-17T09:33:00.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:00 smithi069 ceph-mon[35081]: pgmap v835: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:33:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:00 smithi019 ceph-mon[31620]: pgmap v835: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:33:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:00 smithi079 ceph-mon[35502]: pgmap v835: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:33:01.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 221588 1.78 MB/sec execute 539 sec latency 11.573 ms 2024-05-17T09:33:02.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 222259 1.78 MB/sec execute 540 sec latency 16.564 ms 2024-05-17T09:33:02.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:02 smithi079 ceph-mon[35502]: pgmap v836: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:33:02.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:02 smithi069 ceph-mon[35081]: pgmap v836: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:33:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:02 smithi019 ceph-mon[31620]: pgmap v836: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-05-17T09:33:03.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 222724 1.79 MB/sec execute 541 sec latency 16.718 ms 2024-05-17T09:33:04.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 223284 1.79 MB/sec execute 542 sec latency 16.274 ms 2024-05-17T09:33:04.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:04 smithi069 ceph-mon[35081]: pgmap v837: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:33:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:04 smithi019 ceph-mon[31620]: pgmap v837: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:33:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:04 smithi079 ceph-mon[35502]: pgmap v837: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-17T09:33:04.885 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:33:04.885 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.9 2024-05-17T09:33:04.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:33:05.161 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:05.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 223741 1.80 MB/sec execute 543 sec latency 15.453 ms 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 -- 172.21.15.19:0/228275509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8109b90 msgr2=0x7fa5b810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 --2- 172.21.15.19:0/228275509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8109b90 0x7fa5b810bfb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009be0 tx=0x7fa5b002a6c0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 -- 172.21.15.19:0/228275509 shutdown_connections 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 --2- 172.21.15.19:0/228275509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8109b90 0x7fa5b810bfb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 --2- 172.21.15.19:0/228275509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 0x7fa5b81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 --2- 172.21.15.19:0/228275509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8103c50 0x7fa5b8104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 -- 172.21.15.19:0/228275509 >> 172.21.15.19:0/228275509 conn(0x7fa5b80fee60 msgr2=0x7fa5b81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 -- 172.21.15.19:0/228275509 shutdown_connections 2024-05-17T09:33:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.790+0000 7fa5bebba640 1 -- 172.21.15.19:0/228275509 wait complete. 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 Processor -- start 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 -- start start 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 0x7fa5b8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 0x7fa5b8076120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8109b90 0x7fa5b8076660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810f610 con 0x7fa5b8104e90 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810f490 con 0x7fa5b8109b90 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bebba640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810f790 con 0x7fa5b8103c50 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5bc956640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 0x7fa5b8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.791+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 0x7fa5b8076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8109b90 0x7fa5b8076660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 0x7fa5b8077a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58860/0 (socket says 172.21.15.19:58860) 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 0x7fa5b8076120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57284/0 (socket says 172.21.15.19:57284) 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 -- 172.21.15.19:0/1560645748 learned_addr learned my addr 172.21.15.19:0/1560645748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 -- 172.21.15.19:0/1560645748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8109b90 msgr2=0x7fa5b8076660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8109b90 0x7fa5b8076660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 -- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 msgr2=0x7fa5b8076120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 0x7fa5b8076120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 -- 172.21.15.19:0/1560645748 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b8076d90 con 0x7fa5b8103c50 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8109b90 0x7fa5b8076660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 0x7fa5b8077a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e9f0 tx=0x7fa5ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5b5ffb640 1 -- 172.21.15.19:0/1560645748 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00ccc0 con 0x7fa5b8103c50 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5b5ffb640 1 -- 172.21.15.19:0/1560645748 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5ac0044b0 con 0x7fa5b8103c50 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b81a4240 con 0x7fa5b8103c50 2024-05-17T09:33:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.792+0000 7fa5b5ffb640 1 -- 172.21.15.19:0/1560645748 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac010430 con 0x7fa5b8103c50 2024-05-17T09:33:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.793+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b81a4730 con 0x7fa5b8103c50 2024-05-17T09:33:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.795+0000 7fa5b5ffb640 1 -- 172.21.15.19:0/1560645748 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa5ac003f90 con 0x7fa5b8103c50 2024-05-17T09:33:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.795+0000 7fa5c5a6a740 1 Processor -- start 2024-05-17T09:33:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.795+0000 7fa5c5a6a740 1 -- start start 2024-05-17T09:33:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 0x5561153603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x556115360920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x556115367820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561152d2c30 con 0x556114f9e4d0 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5561152c27d0 con 0x556114fbf1b0 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5561152c6810 con 0x556114f962f0 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 0x5561153603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x556115367820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 0x5561153603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:38370/0 (socket says 172.21.15.19:38370) 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x556115367820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57292/0 (socket says 172.21.15.19:57292) 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 -- 172.21.15.19:0/3208885574 learned_addr learned my addr 172.21.15.19:0/3208885574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bc956640 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x556115360920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5b5ffb640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa5a80b2920 0x7fa5a80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 -- 172.21.15.19:0/3208885574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 msgr2=0x556115360920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x556115360920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 -- 172.21.15.19:0/3208885574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 msgr2=0x556115367820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x556115367820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 -- 172.21.15.19:0/3208885574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556114fd7210 con 0x556114fbf1b0 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5b7fff640 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x556115367820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5b5ffb640 1 -- 172.21.15.19:0/1560645748 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa5ac0e2820 con 0x7fa5b8103c50 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.796+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa5a80b2920 0x7fa5a80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.797+0000 7fa5bd157640 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 0x5561153603a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa5b002acb0 tx=0x7fa5b00093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97856d0113949be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.797+0000 7fa5a2ffd640 1 -- 172.21.15.19:0/3208885574 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0036040 con 0x556114fbf1b0 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.797+0000 7fa5a2ffd640 1 -- 172.21.15.19:0/3208885574 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5b0037040 con 0x556114fbf1b0 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.797+0000 7fa5a2ffd640 1 -- 172.21.15.19:0/3208885574 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00498d0 con 0x556114fbf1b0 2024-05-17T09:33:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/3208885574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 msgr2=0x5561153603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 0x5561153603a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa5b002acb0 tx=0x7fa5b00093c0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa5a80b2920 0x7fa5a80b4e30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa5b004b000 tx=0x7fa5b0048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/3208885574 shutdown_connections 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x556115367820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x556115360920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/3208885574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114fbf1b0 0x5561153603a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/3208885574 >> 172.21.15.19:0/3208885574 conn(0x556114fb3300 msgr2=0x55611535bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/3208885574 shutdown_connections 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/3208885574 wait complete. 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 -- start start 2024-05-17T09:33:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.798+0000 7fa5c5a6a740 1 Processor -- start 2024-05-17T09:33:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115369c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f9e4d0 0x55611536a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 0x556115371130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556114fa8790 con 0x556114fbf1b0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55611527e890 con 0x556114f962f0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556115368270 con 0x556114f9e4d0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5c5a6a740 10 client.0 _ll_register_callbacks cb 0x556115281d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 0x556115371130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bc956640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115369c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f9e4d0 0x55611536a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 0x556115371130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57308/0 (socket says 172.21.15.19:57308) 2024-05-17T09:33:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bd157640 1 -- 172.21.15.19:0/1637344988 learned_addr learned my addr 172.21.15.19:0/1637344988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bc956640 1 -- 172.21.15.19:0/1637344988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f9e4d0 msgr2=0x55611536a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f9e4d0 0x55611536a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bc956640 1 -- 172.21.15.19:0/1637344988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 msgr2=0x556115371130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 0x556115371130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bc956640 1 -- 172.21.15.19:0/1637344988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561150a3620 con 0x556114f962f0 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 0x556115371130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.799+0000 7fa5b7fff640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f9e4d0 0x55611536a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115369c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0dffc0 tx=0x7fa5ac0a9920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321f0659dd389aba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa597fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa597fff640 1 -- 172.21.15.19:0/1637344988 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0a5220 con 0x556114f962f0 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa597fff640 1 -- 172.21.15.19:0/1637344988 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa5ac003680 con 0x556114f962f0 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b0037040 con 0x556114f962f0 2024-05-17T09:33:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa597fff640 1 -- 172.21.15.19:0/1637344988 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5ac0a95a0 con 0x556114f962f0 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 10 client.27281 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5561153740d0 con 0x556114f962f0 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa597fff640 1 -- 172.21.15.19:0/1637344988 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5b0037040 con 0x556114f962f0 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa597fff640 1 -- 172.21.15.19:0/1637344988 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa5ac0cc110 con 0x556114f962f0 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 10 client.27281 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 10 client.27281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 20 client.27281 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.800+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114f7dd80 0x556115377730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.801+0000 7fa5c5a6a740 4 client.27281 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.801+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561152d4b80 con 0x556114f7dd80 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.801+0000 7fa5b7fff640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114f7dd80 0x556115377730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.801+0000 7fa597fff640 10 client.27281 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.801+0000 7fa5b7fff640 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114f7dd80 0x556115377730 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5acff5e3aaf223a8 server_cookie=21127936035ed43 in_seq=0 out_seq=0 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa597fff640 1 -- 172.21.15.19:0/1637344988 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5561152d4b80 con 0x556114f7dd80 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa597fff640 10 client.27281 handle_command_reply: tid=0 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 client.27281 shutdown 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 msgr2=0x556115369c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115369c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0dffc0 tx=0x7fa5ac0a9920 comp rx=0 tx=0).stop 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 shutdown_connections 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114f7dd80 0x556115377730 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114fbf1b0 0x556115371130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa597fff640 0 client.27281 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f9e4d0 0x55611536a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1637344988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115369c90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 >> 172.21.15.19:0/1637344988 conn(0x556114fb3300 msgr2=0x55611535ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 shutdown_connections 2024-05-17T09:33:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1637344988 wait complete. 2024-05-17T09:33:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.802+0000 7fa5c5a6a740 20 client.27281 trim_cache size 0 max 16384 2024-05-17T09:33:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 Processor -- start 2024-05-17T09:33:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 -- start start 2024-05-17T09:33:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 0x556115364850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x5561153710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f9e4d0 0x55611535f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81b1480 con 0x556114f7dd80 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aed50 con 0x556114f9e4d0 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81acba0 con 0x556114f962f0 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bc956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 0x556115364850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f9e4d0 0x55611535f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bc956640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 0x556115364850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57318/0 (socket says 172.21.15.19:57318) 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bc956640 1 -- 172.21.15.19:0/1387509782 learned_addr learned my addr 172.21.15.19:0/1387509782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5b7fff640 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x5561153710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bd157640 1 -- 172.21.15.19:0/1387509782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 msgr2=0x5561153710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x5561153710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bd157640 1 -- 172.21.15.19:0/1387509782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 msgr2=0x556115364850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.821+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 0x556115364850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.822+0000 7fa5bd157640 1 -- 172.21.15.19:0/1387509782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611535f780 con 0x556114f9e4d0 2024-05-17T09:33:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.822+0000 7fa5bc956640 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 0x556115364850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.822+0000 7fa5bd157640 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f9e4d0 0x55611535f240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0030800 tx=0x7fa5b0033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b0d75ccd55a078 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.822+0000 7fa5a27fc640 1 -- 172.21.15.19:0/1387509782 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0049440 con 0x556114f9e4d0 2024-05-17T09:33:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.822+0000 7fa5a27fc640 1 -- 172.21.15.19:0/1387509782 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5b0004040 con 0x556114f9e4d0 2024-05-17T09:33:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.822+0000 7fa5a27fc640 1 -- 172.21.15.19:0/1387509782 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0031680 con 0x556114f9e4d0 2024-05-17T09:33:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1387509782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f9e4d0 msgr2=0x55611535f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f9e4d0 0x55611535f240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0030800 tx=0x7fa5b0033040 comp rx=0 tx=0).stop 2024-05-17T09:33:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1387509782 shutdown_connections 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f9e4d0 0x55611535f240 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f962f0 0x5561153710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/1387509782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f7dd80 0x556115364850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1387509782 >> 172.21.15.19:0/1387509782 conn(0x556114fb3300 msgr2=0x55611535eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1387509782 shutdown_connections 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/1387509782 wait complete. 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.823+0000 7fa5c5a6a740 1 -- start start 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 Processor -- start 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f7dd80 0x55611536fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115370190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x5561153dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ad210 con 0x556114f9e4d0 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ae680 con 0x556114f962f0 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ad8e0 con 0x556114f7dd80 2024-05-17T09:33:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5c5a6a740 10 client.0 _ll_register_callbacks cb 0x7fa5ac0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x5561153dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bc956640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f7dd80 0x55611536fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x5561153dddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57324/0 (socket says 172.21.15.19:57324) 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 -- 172.21.15.19:0/260414590 learned_addr learned my addr 172.21.15.19:0/260414590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115370190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 -- 172.21.15.19:0/260414590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f7dd80 msgr2=0x55611536fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f7dd80 0x55611536fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 -- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 msgr2=0x556115370190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.824+0000 7fa5bd157640 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115370190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa5bd157640 1 -- 172.21.15.19:0/260414590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561153709e0 con 0x556114f9e4d0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa5bd157640 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x5561153dddf0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0004450 tx=0x7fa5b0004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d7556ef8f82bb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa597fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa597fff640 1 -- 172.21.15.19:0/260414590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0036030 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b0004040 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa597fff640 1 -- 172.21.15.19:0/260414590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa5b0049650 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa597fff640 1 -- 172.21.15.19:0/260414590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5b0005bd0 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa5c5a6a740 10 client.17703 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5561153df350 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.825+0000 7fa597fff640 1 -- 172.21.15.19:0/260414590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5b0004040 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa597fff640 1 -- 172.21.15.19:0/260414590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa5b003c030 con 0x556114f9e4d0 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5c5a6a740 10 client.17703 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5c5a6a740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5c5a6a740 20 client.17703 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114fbf1b0 0x5561153e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5c5a6a740 4 client.17703 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:33:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561152ce360 con 0x556114fbf1b0 2024-05-17T09:33:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5bc956640 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114fbf1b0 0x5561153e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa597fff640 10 client.17703 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.826+0000 7fa5bc956640 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114fbf1b0 0x5561153e18d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83c8786badc0c32 server_cookie=a321fb020b47491a in_seq=0 out_seq=0 2024-05-17T09:33:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.848+0000 7fa597fff640 1 -- 172.21.15.19:0/260414590 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561152ce360 con 0x556114fbf1b0 2024-05-17T09:33:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.848+0000 7fa597fff640 10 client.17703 handle_command_reply: tid=0 2024-05-17T09:33:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.848+0000 7fa5c5a6a740 1 client.17703 shutdown 2024-05-17T09:33:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 msgr2=0x5561153dddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x5561153dddf0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0004450 tx=0x7fa5b0004480 comp rx=0 tx=0).stop 2024-05-17T09:33:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 shutdown_connections 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556114fbf1b0 0x5561153e18d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556114f9e4d0 0x5561153dddf0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556114f962f0 0x556115370190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa597fff640 0 client.17703 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 --2- 172.21.15.19:0/260414590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556114f7dd80 0x55611536fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 >> 172.21.15.19:0/260414590 conn(0x556114fb3300 msgr2=0x5561153637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 shutdown_connections 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 1 -- 172.21.15.19:0/260414590 wait complete. 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.849+0000 7fa5c5a6a740 20 client.17703 trim_cache size 0 max 16384 2024-05-17T09:33:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:33:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:33:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:33:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa5a80b2920 msgr2=0x7fa5a80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa5a80b2920 0x7fa5a80b4e30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa5b004b000 tx=0x7fa5b0048000 comp rx=0 tx=0).stop 2024-05-17T09:33:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 msgr2=0x7fa5b8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 0x7fa5b8077a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e9f0 tx=0x7fa5ac00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 shutdown_connections 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5b8109b90 0x7fa5b8076660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa5a80b2920 0x7fa5a80b4e30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b8104e90 0x7fa5b8076120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 --2- 172.21.15.19:0/1560645748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5b8103c50 0x7fa5b8077a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 >> 172.21.15.19:0/1560645748 conn(0x7fa5b80fee60 msgr2=0x7fa5b8105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 shutdown_connections 2024-05-17T09:33:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:05.850+0000 7fa5bebba640 1 -- 172.21.15.19:0/1560645748 wait complete. 2024-05-17T09:33:06.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 223966 1.79 MB/sec execute 544 sec latency 22.658 ms 2024-05-17T09:33:06.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:06 smithi019 ceph-mon[31620]: pgmap v838: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-17T09:33:06.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:06 smithi069 ceph-mon[35081]: pgmap v838: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-17T09:33:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:06 smithi079 ceph-mon[35502]: pgmap v838: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-17T09:33:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 -- 172.21.15.19:0/3888567070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d0108750 msgr2=0x7fb7d01100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/3888567070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d0108750 0x7fb7d01100b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b0d0 tx=0x7fb7c800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 -- 172.21.15.19:0/3888567070 shutdown_connections 2024-05-17T09:33:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/3888567070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d01034d0 0x7fb7d01038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/3888567070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d0108750 0x7fb7d01100b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/3888567070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7d0107e00 0x7fb7d0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 -- 172.21.15.19:0/3888567070 >> 172.21.15.19:0/3888567070 conn(0x7fb7d00fef20 msgr2=0x7fb7d0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 -- 172.21.15.19:0/3888567070 shutdown_connections 2024-05-17T09:33:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.142+0000 7fb7d50fd700 1 -- 172.21.15.19:0/3888567070 wait complete. 2024-05-17T09:33:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.143+0000 7fb7d50fd700 1 Processor -- start 2024-05-17T09:33:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.143+0000 7fb7d50fd700 1 -- start start 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7d50fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 0x7fb7d01a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7d50fd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7d0107e00 0x7fb7d01a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7d50fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d0108750 0x7fb7d019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7d50fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d0113250 con 0x7fb7d01034d0 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7d50fd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d01130d0 con 0x7fb7d0107e00 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7d50fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d01133d0 con 0x7fb7d0108750 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 0x7fb7d01a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 0x7fb7d01a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57348/0 (socket says 172.21.15.19:57348) 2024-05-17T09:33:07.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 -- 172.21.15.19:0/1106188306 learned_addr learned my addr 172.21.15.19:0/1106188306 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7cf59e700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d0108750 0x7fb7d019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ce59c700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7d0107e00 0x7fb7d01a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 -- 172.21.15.19:0/1106188306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d0108750 msgr2=0x7fb7d019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d0108750 0x7fb7d019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 -- 172.21.15.19:0/1106188306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7d0107e00 msgr2=0x7fb7d01a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7d0107e00 0x7fb7d01a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.144+0000 7fb7ced9d700 1 -- 172.21.15.19:0/1106188306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d019b270 con 0x7fb7d01034d0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.145+0000 7fb7ced9d700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 0x7fb7d01a09b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000e9c0 tx=0x7fb7c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.145+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000cbd0 con 0x7fb7d01034d0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.145+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7c000ee60 con 0x7fb7d01034d0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.145+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00054b0 con 0x7fb7d01034d0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.145+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d019b560 con 0x7fb7d01034d0 2024-05-17T09:33:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.145+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d00ff330 con 0x7fb7d01034d0 2024-05-17T09:33:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.146+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d0104980 con 0x7fb7d01034d0 2024-05-17T09:33:07.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.148+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb7c0004750 con 0x7fb7d01034d0 2024-05-17T09:33:07.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.149+0000 7fb7b7fff700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb7b80b2900 0x7fb7b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:07.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.149+0000 7fb7ce59c700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb7b80b2900 0x7fb7b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.149+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb7c00e2a80 con 0x7fb7d01034d0 2024-05-17T09:33:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.150+0000 7fb7ce59c700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb7b80b2900 0x7fb7b80b4e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb7d019c270 tx=0x7fb7c400f7a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:07.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.153+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7c0054df0 con 0x7fb7d01034d0 2024-05-17T09:33:07.450 INFO:tasks.workunit.client.0.smithi019.stdout: 1 224323 1.79 MB/sec execute 545 sec latency 13.551 ms 2024-05-17T09:33:07.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.469+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d019c010 con 0x7fb7d01034d0 2024-05-17T09:33:07.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.470+0000 7fb7b7fff700 1 -- 172.21.15.19:0/1106188306 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7fb7d019c010 con 0x7fb7d01034d0 2024-05-17T09:33:07.470 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:33:07.471 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:33:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb7b80b2900 msgr2=0x7fb7b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb7b80b2900 0x7fb7b80b4e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb7d019c270 tx=0x7fb7c400f7a0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 msgr2=0x7fb7d01a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 0x7fb7d01a09b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000e9c0 tx=0x7fb7c000ecd0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 shutdown_connections 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb7b80b2900 0x7fb7b80b4e10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d0108750 0x7fb7d019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d01034d0 0x7fb7d01a09b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 --2- 172.21.15.19:0/1106188306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7d0107e00 0x7fb7d01a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 >> 172.21.15.19:0/1106188306 conn(0x7fb7d00fef20 msgr2=0x7fb7d0107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 shutdown_connections 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:07.473+0000 7fb7d50fd700 1 -- 172.21.15.19:0/1106188306 wait complete. 2024-05-17T09:33:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:33:08.354 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 140.9 --await-for 45.0' 2024-05-17T09:33:08.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 140.9 --await-for 45.0 2024-05-17T09:33:08.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 224768 1.79 MB/sec execute 546 sec latency 19.948 ms 2024-05-17T09:33:08.649 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:08 smithi069 ceph-mon[35081]: pgmap v839: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-17T09:33:08.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:08 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1106188306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:08 smithi019 ceph-mon[31620]: pgmap v839: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-17T09:33:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:08 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1106188306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:08 smithi079 ceph-mon[35502]: pgmap v839: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-17T09:33:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:08 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1106188306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:09.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225324 1.79 MB/sec execute 547 sec latency 16.721 ms 2024-05-17T09:33:10.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225886 1.79 MB/sec execute 548 sec latency 23.952 ms 2024-05-17T09:33:10.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:10 smithi019 ceph-mon[31620]: pgmap v840: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-17T09:33:10.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:10 smithi069 ceph-mon[35081]: pgmap v840: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 -- 172.21.15.19:0/224540981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 msgr2=0x7fdbcc1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 --2- 172.21.15.19:0/224540981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc1100b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b0d0 tx=0x7fdbc400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 -- 172.21.15.19:0/224540981 shutdown_connections 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 --2- 172.21.15.19:0/224540981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc1100b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 --2- 172.21.15.19:0/224540981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc1034d0 0x7fdbcc1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 --2- 172.21.15.19:0/224540981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbcc107e00 0x7fdbcc108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 -- 172.21.15.19:0/224540981 >> 172.21.15.19:0/224540981 conn(0x7fdbcc0fef20 msgr2=0x7fdbcc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 -- 172.21.15.19:0/224540981 shutdown_connections 2024-05-17T09:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.739+0000 7fdbd228e700 1 -- 172.21.15.19:0/224540981 wait complete. 2024-05-17T09:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 Processor -- start 2024-05-17T09:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 -- start start 2024-05-17T09:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbcc1034d0 0x7fdbcc0ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc107e00 0x7fdbcc0ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc0ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc113280 con 0x7fdbcc108750 2024-05-17T09:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc113100 con 0x7fdbcc107e00 2024-05-17T09:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.740+0000 7fdbd228e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc113400 con 0x7fdbcc1034d0 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc0ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbcb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc107e00 0x7fdbcc0ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc0ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57360/0 (socket says 172.21.15.19:57360) 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 -- 172.21.15.19:0/1537639798 learned_addr learned my addr 172.21.15.19:0/1537639798 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbcc1034d0 0x7fdbcc0ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 -- 172.21.15.19:0/1537639798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbcc1034d0 msgr2=0x7fdbcc0ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbcc1034d0 0x7fdbcc0ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 -- 172.21.15.19:0/1537639798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc107e00 msgr2=0x7fdbcc0ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc107e00 0x7fdbcc0ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 -- 172.21.15.19:0/1537639798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc1a9100 con 0x7fdbcc108750 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc107e00 0x7fdbcc0ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbd082b700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc0ffe80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40079a0 tx=0x7fdbc40079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.741+0000 7fdbc97fa700 1 -- 172.21.15.19:0/1537639798 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc401f070 con 0x7fdbcc108750 2024-05-17T09:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.742+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc1a9390 con 0x7fdbcc108750 2024-05-17T09:33:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.742+0000 7fdbc97fa700 1 -- 172.21.15.19:0/1537639798 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc4003c00 con 0x7fdbcc108750 2024-05-17T09:33:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.742+0000 7fdbc97fa700 1 -- 172.21.15.19:0/1537639798 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc401b430 con 0x7fdbcc108750 2024-05-17T09:33:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.742+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc1a97e0 con 0x7fdbcc108750 2024-05-17T09:33:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.745+0000 7fdbc97fa700 1 -- 172.21.15.19:0/1537639798 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdbc4022070 con 0x7fdbcc108750 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.745+0000 7fdbe0ca3b80 1 Processor -- start 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbc97fa700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdbb40b2880 0x7fdbb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 -- start start 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdbb40b2880 0x7fdbb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbc97fa700 1 -- 172.21.15.19:0/1537639798 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdbc400f020 con 0x7fdbcc108750 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a683f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a67e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55914a5abf00 con 0x55914a5ea5c0 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55914a59baa0 con 0x55914a67e0b0 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55914a59fae0 con 0x55914a4fefd0 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a683f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a683f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57364/0 (socket says 172.21.15.19:57364) 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbcbfff700 1 -- 172.21.15.19:0/3995575435 learned_addr learned my addr 172.21.15.19:0/3995575435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a67e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.746+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdbb40b2880 0x7fdbb40b4d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc102550 tx=0x7fdbc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbd082b700 1 -- 172.21.15.19:0/3995575435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 msgr2=0x55914a6844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbd082b700 1 -- 172.21.15.19:0/3995575435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 msgr2=0x55914a683f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a683f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbd082b700 1 -- 172.21.15.19:0/3995575435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914a545140 con 0x55914a67e0b0 2024-05-17T09:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a67e580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4007f40 tx=0x7fdbc4006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43c098fd2434049 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbb27fc700 1 -- 172.21.15.19:0/3995575435 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40afe60 con 0x55914a67e0b0 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbb27fc700 1 -- 172.21.15.19:0/3995575435 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc40aa430 con 0x55914a67e0b0 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.747+0000 7fdbb27fc700 1 -- 172.21.15.19:0/3995575435 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40b36f0 con 0x55914a67e0b0 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3995575435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 msgr2=0x55914a67e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a67e580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4007f40 tx=0x7fdbc4006230 comp rx=0 tx=0).stop 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3995575435 shutdown_connections 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a67e580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a683f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3995575435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3995575435 >> 172.21.15.19:0/3995575435 conn(0x55914a55f350 msgr2=0x55914a678d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3995575435 shutdown_connections 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.748+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3995575435 wait complete. 2024-05-17T09:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 -- start start 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 Processor -- start 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a68d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a68dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a687b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55914a4a2130 con 0x55914a67e0b0 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55914a503f30 con 0x55914a5ea5c0 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55914a67edd0 con 0x55914a4fefd0 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.749+0000 7fdbe0ca3b80 10 client.0 _ll_register_callbacks cb 0x55914a536ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a687b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a68d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a687b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57372/0 (socket says 172.21.15.19:57372) 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a68d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58950/0 (socket says 172.21.15.19:58950) 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 -- 172.21.15.19:0/3400873666 learned_addr learned my addr 172.21.15.19:0/3400873666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a68dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 -- 172.21.15.19:0/3400873666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 msgr2=0x55914a68d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a68d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 -- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 msgr2=0x55914a68dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a68dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 -- 172.21.15.19:0/3400873666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914a4eba60 con 0x55914a67e0b0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a68d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a68dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbd082b700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a687b20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40a69a0 tx=0x7fdbc40098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a170a0bbccd11b7c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbab7fe700 1 -- 172.21.15.19:0/3400873666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40c4060 con 0x55914a67e0b0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbab7fe700 1 -- 172.21.15.19:0/3400873666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdbc40e3340 con 0x55914a67e0b0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.750+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc40aa430 con 0x55914a67e0b0 2024-05-17T09:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbab7fe700 1 -- 172.21.15.19:0/3400873666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdbc40afe60 con 0x55914a67e0b0 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 10 client.17721 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55914a5558f0 con 0x55914a67e0b0 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbab7fe700 1 -- 172.21.15.19:0/3400873666 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc40aa430 con 0x55914a67e0b0 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbab7fe700 1 -- 172.21.15.19:0/3400873666 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdbc40c9080 con 0x55914a67e0b0 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 10 client.17721 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 10 client.17721 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 20 client.17721 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a697640 0x55914a699a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 4 client.17721 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914a64a060 con 0x55914a697640 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.751+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a697640 0x55914a699a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.752+0000 7fdbab7fe700 10 client.17721 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.752+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a697640 0x55914a699a60 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26144559e15982ba server_cookie=4903633856c1a7ad in_seq=0 out_seq=0 2024-05-17T09:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.752+0000 7fdbab7fe700 1 -- 172.21.15.19:0/3400873666 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55914a64a060 con 0x55914a697640 2024-05-17T09:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbab7fe700 10 client.17721 handle_command_reply: tid=0 2024-05-17T09:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 client.17721 shutdown 2024-05-17T09:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 msgr2=0x55914a687b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a687b20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40a69a0 tx=0x7fdbc40098c0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 shutdown_connections 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a68d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a697640 0x55914a699a60 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a687b20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/3400873666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a68dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbab7fe700 0 client.17721 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 >> 172.21.15.19:0/3400873666 conn(0x55914a55f350 msgr2=0x55914a67d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 shutdown_connections 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/3400873666 wait complete. 2024-05-17T09:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.753+0000 7fdbe0ca3b80 20 client.17721 trim_cache size 0 max 16384 2024-05-17T09:33:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:10 smithi079 ceph-mon[35502]: pgmap v840: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-17T09:33:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.775+0000 7fdbe0ca3b80 1 Processor -- start 2024-05-17T09:33:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.775+0000 7fdbe0ca3b80 1 -- start start 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.775+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.775+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a69f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.775+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a69f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b4d20 con 0x55914a5ea5c0 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b25b0 con 0x55914a67e0b0 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b0400 con 0x55914a4fefd0 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a69f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a69f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57384/0 (socket says 172.21.15.19:57384) 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 -- 172.21.15.19:0/4028073424 learned_addr learned my addr 172.21.15.19:0/4028073424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbd082b700 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a69f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 -- 172.21.15.19:0/4028073424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 msgr2=0x55914a6832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 -- 172.21.15.19:0/4028073424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 msgr2=0x55914a69f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a69f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 -- 172.21.15.19:0/4028073424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914a679850 con 0x55914a5ea5c0 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a69f160 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000ec30 tx=0x7fdbc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df98d533d75a992 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.776+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/4028073424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0004030 con 0x55914a5ea5c0 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/4028073424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc001f810 con 0x55914a5ea5c0 2024-05-17T09:33:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/4028073424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0003a20 con 0x55914a5ea5c0 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/4028073424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 msgr2=0x55914a69f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a69f160 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000ec30 tx=0x7fdbc0014000 comp rx=0 tx=0).stop 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/4028073424 shutdown_connections 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a67e0b0 0x55914a69f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/4028073424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a5ea5c0 0x55914a69f160 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.777+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/4028073424 >> 172.21.15.19:0/4028073424 conn(0x55914a55f350 msgr2=0x55914a682ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.778+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/4028073424 shutdown_connections 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.778+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/4028073424 wait complete. 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.778+0000 7fdbe0ca3b80 1 -- start start 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.778+0000 7fdbe0ca3b80 1 Processor -- start 2024-05-17T09:33:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.778+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a692480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbe0ca3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a6929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b0a70 con 0x55914a67e0b0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b1ee0 con 0x55914a5ea5c0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbe0ca3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1b1140 con 0x55914a4fefd0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbe0ca3b80 10 client.0 _ll_register_callbacks cb 0x55914a536ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a6929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a6929c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57396/0 (socket says 172.21.15.19:57396) 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 -- 172.21.15.19:0/724599127 learned_addr learned my addr 172.21.15.19:0/724599127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbcb7fe700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a692480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 -- 172.21.15.19:0/724599127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 msgr2=0x55914a6940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 -- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 msgr2=0x55914a692480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a692480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 -- 172.21.15.19:0/724599127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914a693140 con 0x55914a67e0b0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbd082b700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a6929c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40aa620 tx=0x7fdbc4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02fc3075e9eff2d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.779+0000 7fdbab7fe700 1 -- 172.21.15.19:0/724599127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40c4070 con 0x55914a67e0b0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbab7fe700 1 -- 172.21.15.19:0/724599127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdbc40b3260 con 0x55914a67e0b0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc001f810 con 0x55914a67e0b0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbab7fe700 1 -- 172.21.15.19:0/724599127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdbc40aa270 con 0x55914a67e0b0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 10 client.17727 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55914a693140 con 0x55914a67e0b0 2024-05-17T09:33:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbab7fe700 1 -- 172.21.15.19:0/724599127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc40c6050 con 0x55914a67e0b0 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbab7fe700 1 -- 172.21.15.19:0/724599127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdbc40bfce0 con 0x55914a67e0b0 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 10 client.17727 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 10 client.17727 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 20 client.17727 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a7031d0 0x55914a7055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 4 client.17727 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.9, "await_for": 45.0}] 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.780+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.9, "await_for": 45.0}) -- 0x7fdbc001f810 con 0x55914a7031d0 2024-05-17T09:33:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.781+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a7031d0 0x55914a7055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:10.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.781+0000 7fdbab7fe700 10 client.17727 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:10.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.781+0000 7fdbcbfff700 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a7031d0 0x55914a7055f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895b401c6631762d server_cookie=902b640160ba0c5c in_seq=0 out_seq=0 2024-05-17T09:33:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.873+0000 7fdbab7fe700 1 -- 172.21.15.19:0/724599127 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fdbc001f810 con 0x55914a7031d0 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.873+0000 7fdbab7fe700 10 client.17727 handle_command_reply: tid=0 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 client.17727 shutdown 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 msgr2=0x55914a6929c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a6929c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40aa620 tx=0x7fdbc4009a50 comp rx=0 tx=0).stop 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 shutdown_connections 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55914a4fefd0 0x55914a6940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55914a7031d0 0x55914a7055f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55914a67e0b0 0x55914a6929c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 --2- 172.21.15.19:0/724599127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914a5ea5c0 0x55914a692480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 >> 172.21.15.19:0/724599127 conn(0x55914a55f350 msgr2=0x55914a681830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbab7fe700 0 client.17727 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 shutdown_connections 2024-05-17T09:33:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 1 -- 172.21.15.19:0/724599127 wait complete. 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.874+0000 7fdbe0ca3b80 20 client.17727 trim_cache size 0 max 16384 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 46, 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "c9ecb49d": { 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 46, 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:33:10.876 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.9, 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:10.877 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:10.878 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:10.878 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:10.878 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:10.878 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:33:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdbb40b2880 msgr2=0x7fdbb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdbb40b2880 0x7fdbb40b4d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc102550 tx=0x7fdbc0009260 comp rx=0 tx=0).stop 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 msgr2=0x7fdbcc0ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc0ffe80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40079a0 tx=0x7fdbc40079d0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 shutdown_connections 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdbb40b2880 0x7fdbb40b4d90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbcc1034d0 0x7fdbcc0ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbcc108750 0x7fdbcc0ffe80 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 --2- 172.21.15.19:0/1537639798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbcc107e00 0x7fdbcc0ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 >> 172.21.15.19:0/1537639798 conn(0x7fdbcc0fef20 msgr2=0x7fdbcc107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.875+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 shutdown_connections 2024-05-17T09:33:10.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:10.876+0000 7fdbd228e700 1 -- 172.21.15.19:0/1537639798 wait complete. 2024-05-17T09:33:11.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.79 MB/sec execute 549 sec latency 577.404 ms 2024-05-17T09:33:11.887 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c9ecb49d', quiesce took 7.0 seconds. Will release after: 13.899999999999999 2024-05-17T09:33:12.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.79 MB/sec execute 550 sec latency 1577.450 ms 2024-05-17T09:33:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:12 smithi069 ceph-mon[35081]: pgmap v841: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:33:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:12 smithi019 ceph-mon[31620]: pgmap v841: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:33:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:12 smithi079 ceph-mon[35502]: pgmap v841: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-05-17T09:33:13.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.78 MB/sec execute 551 sec latency 2577.577 ms 2024-05-17T09:33:14.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.78 MB/sec execute 552 sec latency 3577.667 ms 2024-05-17T09:33:14.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:14 smithi069 ceph-mon[35081]: pgmap v842: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:33:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:14 smithi019 ceph-mon[31620]: pgmap v842: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:33:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:14 smithi079 ceph-mon[35502]: pgmap v842: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-17T09:33:15.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.78 MB/sec execute 553 sec latency 4577.836 ms 2024-05-17T09:33:16.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.77 MB/sec execute 554 sec latency 5577.949 ms 2024-05-17T09:33:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:16 smithi069 ceph-mon[35081]: pgmap v843: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:33:16.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:16 smithi019 ceph-mon[31620]: pgmap v843: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:33:16.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:16 smithi079 ceph-mon[35502]: pgmap v843: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-17T09:33:17.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.77 MB/sec execute 555 sec latency 6578.000 ms 2024-05-17T09:33:18.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.77 MB/sec execute 556 sec latency 7578.117 ms 2024-05-17T09:33:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:18 smithi069 ceph-mon[35081]: pgmap v844: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 854 KiB/s wr, 101 op/s 2024-05-17T09:33:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:18 smithi019 ceph-mon[31620]: pgmap v844: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 854 KiB/s wr, 101 op/s 2024-05-17T09:33:18.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:18 smithi079 ceph-mon[35502]: pgmap v844: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 854 KiB/s wr, 101 op/s 2024-05-17T09:33:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.968+0000 7fb9b722b640 1 -- 172.21.15.19:0/302099761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b0106d60 msgr2=0x7fb9b0109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.968+0000 7fb9b722b640 1 --2- 172.21.15.19:0/302099761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b0106d60 0x7fb9b0109240 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb998009900 tx=0x7fb99802a410 comp rx=0 tx=0).stop 2024-05-17T09:33:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 -- 172.21.15.19:0/302099761 shutdown_connections 2024-05-17T09:33:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 --2- 172.21.15.19:0/302099761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9b01098d0 0x7fb9b010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 --2- 172.21.15.19:0/302099761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b0106d60 0x7fb9b0109240 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 --2- 172.21.15.19:0/302099761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b0104370 0x7fb9b0106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 -- 172.21.15.19:0/302099761 >> 172.21.15.19:0/302099761 conn(0x7fb9b00fdd30 msgr2=0x7fb9b01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 -- 172.21.15.19:0/302099761 shutdown_connections 2024-05-17T09:33:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.969+0000 7fb9b722b640 1 -- 172.21.15.19:0/302099761 wait complete. 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 Processor -- start 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 -- start start 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b0104370 0x7fb9b019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9b0106d60 0x7fb9b019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 0x7fb9b01a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010f5f0 con 0x7fb9b01098d0 2024-05-17T09:33:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010f470 con 0x7fb9b0104370 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b722b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010f770 con 0x7fb9b0106d60 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b57c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 0x7fb9b01a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b57c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 0x7fb9b01a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34590/0 (socket says 172.21.15.19:34590) 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b57c8640 1 -- 172.21.15.19:0/3724245362 learned_addr learned my addr 172.21.15.19:0/3724245362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.970+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b0104370 0x7fb9b019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9affff640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9b0106d60 0x7fb9b019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b57c8640 1 -- 172.21.15.19:0/3724245362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9b0106d60 msgr2=0x7fb9b019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9b0106d60 0x7fb9b019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b57c8640 1 -- 172.21.15.19:0/3724245362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b0104370 msgr2=0x7fb9b019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b0104370 0x7fb9b019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b57c8640 1 -- 172.21.15.19:0/3724245362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b01a1cb0 con 0x7fb9b01098d0 2024-05-17T09:33:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 0x7fb9b01a1640 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800eb10 tx=0x7fb9a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9adffb640 1 -- 172.21.15.19:0/3724245362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004b20 con 0x7fb9b01098d0 2024-05-17T09:33:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9adffb640 1 -- 172.21.15.19:0/3724245362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9a8004cc0 con 0x7fb9b01098d0 2024-05-17T09:33:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a1fa0 con 0x7fb9b01098d0 2024-05-17T09:33:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.971+0000 7fb9adffb640 1 -- 172.21.15.19:0/3724245362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8010690 con 0x7fb9b01098d0 2024-05-17T09:33:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.972+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a2460 con 0x7fb9b01098d0 2024-05-17T09:33:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.974+0000 7fb9be1bf740 1 Processor -- start 2024-05-17T09:33:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9adffb640 1 -- 172.21.15.19:0/3724245362 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb9a8010830 con 0x7fb9b01098d0 2024-05-17T09:33:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 -- start start 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 0x56324f38b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f38b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc94d0 0x56324f392820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56324f2fdc30 con 0x56324efea1b0 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56324f2ed7d0 con 0x56324efc12f0 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56324f2f1810 con 0x56324efc94d0 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 0x56324f38b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 0x56324f38b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34594/0 (socket says 172.21.15.19:34594) 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9affff640 1 -- 172.21.15.19:0/2212930302 learned_addr learned my addr 172.21.15.19:0/2212930302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f38b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.975+0000 7fb9affff640 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc94d0 0x56324f392820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9affff640 1 -- 172.21.15.19:0/2212930302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc94d0 msgr2=0x56324f392820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9affff640 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc94d0 0x56324f392820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9affff640 1 -- 172.21.15.19:0/2212930302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 msgr2=0x56324f38b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9affff640 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f38b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9affff640 1 -- 172.21.15.19:0/2212930302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324f002210 con 0x56324efea1b0 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f38b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9affff640 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 0x56324f38b3a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb998031460 tx=0x7fb998005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a642a9802c0436bb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9adffb640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb99c0b2920 0x7fb99c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9a6ffd640 1 -- 172.21.15.19:0/2212930302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035070 con 0x56324efea1b0 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb99c0b2920 0x7fb99c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9a6ffd640 1 -- 172.21.15.19:0/2212930302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9980044c0 con 0x56324efea1b0 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9a6ffd640 1 -- 172.21.15.19:0/2212930302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998003b80 con 0x56324efea1b0 2024-05-17T09:33:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.976+0000 7fb9adffb640 1 -- 172.21.15.19:0/3724245362 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb9a80e2ac0 con 0x7fb9b01098d0 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2212930302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 msgr2=0x56324f38b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb99c0b2920 0x7fb99c0b4e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x56324f38c780 tx=0x7fb9a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 0x56324f38b3a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb998031460 tx=0x7fb998005fd0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2212930302 shutdown_connections 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc94d0 0x56324f392820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f38b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2212930302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efea1b0 0x56324f38b3a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2212930302 >> 172.21.15.19:0/2212930302 conn(0x56324efde300 msgr2=0x56324f386d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2212930302 shutdown_connections 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.977+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2212930302 wait complete. 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 -- start start 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 Processor -- start 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f394ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f395220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f39c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56324efd3790 con 0x56324efc94d0 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56324f2a9890 con 0x56324efc12f0 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56324f393210 con 0x56324efea1b0 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.978+0000 7fb9be1bf740 10 client.0 _ll_register_callbacks cb 0x56324f2acd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f395220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9b4fc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f394ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f395220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34606/0 (socket says 172.21.15.19:34606) 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 -- 172.21.15.19:0/1109567988 learned_addr learned my addr 172.21.15.19:0/1109567988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f39c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 -- 172.21.15.19:0/1109567988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 msgr2=0x56324f39c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f39c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 -- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 msgr2=0x56324f394ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f394ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 -- 172.21.15.19:0/1109567988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324f0ce620 con 0x56324efc94d0 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f394ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9affff640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f395220 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb99803e040 tx=0x7fb998031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ee537b0b9fe0f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb98ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb98ffff640 1 -- 172.21.15.19:0/1109567988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035040 con 0x56324efc94d0 2024-05-17T09:33:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb98ffff640 1 -- 172.21.15.19:0/1109567988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb99803b870 con 0x56324efc94d0 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.979+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56324f2ffb80 con 0x56324efc94d0 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb98ffff640 1 -- 172.21.15.19:0/1109567988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb99805a050 con 0x56324efc94d0 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 10 client.17739 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56324f39f0b0 con 0x56324efc94d0 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb98ffff640 1 -- 172.21.15.19:0/1109567988 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56324f2ffb80 con 0x56324efc94d0 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb98ffff640 1 -- 172.21.15.19:0/1109567988 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb99804a540 con 0x56324efc94d0 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 10 client.17739 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 10 client.17739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 20 client.17739 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324efa8d80 0x56324f3a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 4 client.17739 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.980+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56324f2f9360 con 0x56324efa8d80 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.981+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324efa8d80 0x56324f3a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.981+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324efa8d80 0x56324f3a2410 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af209fb8d12c974e server_cookie=287d8ff7f4f83285 in_seq=0 out_seq=0 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.981+0000 7fb98ffff640 10 client.17739 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb98ffff640 1 -- 172.21.15.19:0/1109567988 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56324f2f9360 con 0x56324efa8d80 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb98ffff640 10 client.17739 handle_command_reply: tid=0 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 client.17739 shutdown 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 msgr2=0x56324f395220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f395220 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb99803e040 tx=0x7fb998031ca0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 shutdown_connections 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324efa8d80 0x56324f3a2410 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f39c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f395220 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb98ffff640 0 client.17739 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/1109567988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f394ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 >> 172.21.15.19:0/1109567988 conn(0x56324efde300 msgr2=0x56324f389c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 shutdown_connections 2024-05-17T09:33:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 1 -- 172.21.15.19:0/1109567988 wait complete. 2024-05-17T09:33:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.982+0000 7fb9be1bf740 20 client.17739 trim_cache size 0 max 16384 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 Processor -- start 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 -- start start 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f39c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f38f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f38f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01af160 con 0x56324efc94d0 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01aca30 con 0x56324efc12f0 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.998+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01aa880 con 0x56324efea1b0 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f38f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f38f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34618/0 (socket says 172.21.15.19:34618) 2024-05-17T09:33:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 -- 172.21.15.19:0/3347549311 learned_addr learned my addr 172.21.15.19:0/3347549311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f39c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f38f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 -- 172.21.15.19:0/3347549311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 msgr2=0x56324f38f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f38f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 -- 172.21.15.19:0/3347549311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 msgr2=0x56324f39c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f39c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 -- 172.21.15.19:0/3347549311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324f3958e0 con 0x56324efc94d0 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f38f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9affff640 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f38f2d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb99802a930 tx=0x7fb998005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74d0da9ca46684b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9a67fc640 1 -- 172.21.15.19:0/3347549311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035070 con 0x56324efc94d0 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9a67fc640 1 -- 172.21.15.19:0/3347549311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb998056020 con 0x56324efc94d0 2024-05-17T09:33:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:18.999+0000 7fb9a67fc640 1 -- 172.21.15.19:0/3347549311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99803b8d0 con 0x56324efc94d0 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 -- 172.21.15.19:0/3347549311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 msgr2=0x56324f38f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f38f2d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb99802a930 tx=0x7fb998005fd0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 -- 172.21.15.19:0/3347549311 shutdown_connections 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efea1b0 0x56324f38f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f38f2d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/3347549311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efc12f0 0x56324f39c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 -- 172.21.15.19:0/3347549311 >> 172.21.15.19:0/3347549311 conn(0x56324efde300 msgr2=0x56324f38a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 -- 172.21.15.19:0/3347549311 shutdown_connections 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.000+0000 7fb9be1bf740 1 -- 172.21.15.19:0/3347549311 wait complete. 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 -- start start 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 Processor -- start 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc12f0 0x56324f4059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f405ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efea1b0 0x56324f40ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01aaef0 con 0x56324efc94d0 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01ac360 con 0x56324efea1b0 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01ab5c0 con 0x56324efc12f0 2024-05-17T09:33:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.001+0000 7fb9be1bf740 10 client.0 _ll_register_callbacks cb 0x7fb998009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f405ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f405ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34622/0 (socket says 172.21.15.19:34622) 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 -- 172.21.15.19:0/2985556209 learned_addr learned my addr 172.21.15.19:0/2985556209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9b57c8640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efea1b0 0x56324f40ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc12f0 0x56324f4059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 -- 172.21.15.19:0/2985556209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc12f0 msgr2=0x56324f4059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc12f0 0x56324f4059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 -- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efea1b0 msgr2=0x56324f40ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efea1b0 0x56324f40ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 -- 172.21.15.19:0/2985556209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324f40d410 con 0x56324efc94d0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9affff640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f405ef0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb9980057c0 tx=0x7fb998032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428b3f18fd8b5315 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb98ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb98ffff640 1 -- 172.21.15.19:0/2985556209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035040 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb98ffff640 1 -- 172.21.15.19:0/2985556209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb99804a650 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.002+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb998056020 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb98ffff640 1 -- 172.21.15.19:0/2985556209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb998031780 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 10 client.17745 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56324f40e6a0 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb98ffff640 1 -- 172.21.15.19:0/2985556209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb998056020 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb98ffff640 1 -- 172.21.15.19:0/2985556209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb998065070 con 0x56324efc94d0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 10 client.17745 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 10 client.17745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 20 client.17745 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324ef7e1c0 0x56324f410ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 4 client.17745 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56324f2735f0 con 0x56324ef7e1c0 2024-05-17T09:33:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.003+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324ef7e1c0 0x56324f410ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.004+0000 7fb98ffff640 10 client.17745 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.004+0000 7fb9b4fc7640 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324ef7e1c0 0x56324f410ed0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283b2562653ae28a server_cookie=18c19c3c3aea7a09 in_seq=0 out_seq=0 2024-05-17T09:33:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb98ffff640 1 -- 172.21.15.19:0/2985556209 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56324f2735f0 con 0x56324ef7e1c0 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb98ffff640 10 client.17745 handle_command_reply: tid=0 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 client.17745 shutdown 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 msgr2=0x56324f405ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f405ef0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb9980057c0 tx=0x7fb998032f20 comp rx=0 tx=0).stop 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 shutdown_connections 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56324ef7e1c0 0x56324f410ed0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56324efea1b0 0x56324f40ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb98ffff640 0 client.17745 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56324efc94d0 0x56324f405ef0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 --2- 172.21.15.19:0/2985556209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56324efc12f0 0x56324f4059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 >> 172.21.15.19:0/2985556209 conn(0x56324efde300 msgr2=0x56324f39c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 shutdown_connections 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 1 -- 172.21.15.19:0/2985556209 wait complete. 2024-05-17T09:33:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.028+0000 7fb9be1bf740 20 client.17745 trim_cache size 0 max 16384 2024-05-17T09:33:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:33:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:33:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb99c0b2920 msgr2=0x7fb99c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb99c0b2920 0x7fb99c0b4e30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x56324f38c780 tx=0x7fb9a0005f90 comp rx=0 tx=0).stop 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 msgr2=0x7fb9b01a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 0x7fb9b01a1640 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800eb10 tx=0x7fb9a800c2d0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 shutdown_connections 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb99c0b2920 0x7fb99c0b4e30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9b01098d0 0x7fb9b01a1640 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9b0106d60 0x7fb9b019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 --2- 172.21.15.19:0/3724245362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b0104370 0x7fb9b019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 >> 172.21.15.19:0/3724245362 conn(0x7fb9b00fdd30 msgr2=0x7fb9b0106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 shutdown_connections 2024-05-17T09:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:19.029+0000 7fb9b722b640 1 -- 172.21.15.19:0/3724245362 wait complete. 2024-05-17T09:33:19.451 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.77 MB/sec execute 557 sec latency 8578.172 ms 2024-05-17T09:33:20.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.76 MB/sec execute 558 sec latency 9578.226 ms 2024-05-17T09:33:20.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:20 smithi069 ceph-mon[35081]: pgmap v845: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 632 KiB/s wr, 96 op/s 2024-05-17T09:33:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:20 smithi019 ceph-mon[31620]: pgmap v845: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 632 KiB/s wr, 96 op/s 2024-05-17T09:33:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:20 smithi079 ceph-mon[35502]: pgmap v845: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 632 KiB/s wr, 96 op/s 2024-05-17T09:33:21.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.76 MB/sec execute 559 sec latency 10578.275 ms 2024-05-17T09:33:22.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.76 MB/sec execute 560 sec latency 11578.338 ms 2024-05-17T09:33:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:22 smithi069 ceph-mon[35081]: pgmap v846: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 299 KiB/s wr, 46 op/s 2024-05-17T09:33:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:22 smithi019 ceph-mon[31620]: pgmap v846: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 299 KiB/s wr, 46 op/s 2024-05-17T09:33:22.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:22 smithi079 ceph-mon[35502]: pgmap v846: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 299 KiB/s wr, 46 op/s 2024-05-17T09:33:23.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.75 MB/sec execute 561 sec latency 12578.400 ms 2024-05-17T09:33:24.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.75 MB/sec execute 562 sec latency 13578.483 ms 2024-05-17T09:33:24.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:24 smithi069 ceph-mon[35081]: pgmap v847: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 280 KiB/s wr, 44 op/s 2024-05-17T09:33:24.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:24 smithi019 ceph-mon[31620]: pgmap v847: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 280 KiB/s wr, 44 op/s 2024-05-17T09:33:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:24 smithi079 ceph-mon[35502]: pgmap v847: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 280 KiB/s wr, 44 op/s 2024-05-17T09:33:25.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.75 MB/sec execute 563 sec latency 14578.593 ms 2024-05-17T09:33:25.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:33:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:26.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:26 smithi019 ceph-mon[31620]: pgmap v848: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 120 KiB/s wr, 37 op/s 2024-05-17T09:33:26.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.74 MB/sec execute 564 sec latency 15578.688 ms 2024-05-17T09:33:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:26 smithi069 ceph-mon[35081]: pgmap v848: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 120 KiB/s wr, 37 op/s 2024-05-17T09:33:26.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:26 smithi079 ceph-mon[35502]: pgmap v848: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 120 KiB/s wr, 37 op/s 2024-05-17T09:33:27.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.74 MB/sec execute 565 sec latency 16578.781 ms 2024-05-17T09:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/49200010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38110cb0 msgr2=0x7f6a381130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/49200010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38110cb0 0x7f6a381130d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b0a0 tx=0x7f6a3000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/49200010 shutdown_connections 2024-05-17T09:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/49200010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38103100 0x7f6a38103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/49200010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38110cb0 0x7f6a381130d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/49200010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 0x7f6a38110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/49200010 >> 172.21.15.19:0/49200010 conn(0x7f6a38078df0 msgr2=0x7f6a38079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/49200010 shutdown_connections 2024-05-17T09:33:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.883+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/49200010 wait complete. 2024-05-17T09:33:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 Processor -- start 2024-05-17T09:33:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 -- start start 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 0x7f6a3819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 0x7f6a3819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38110cb0 0x7f6a381a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38116550 con 0x7f6a38103100 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381163d0 con 0x7f6a38103a50 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.884+0000 7f6a3e3d3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381166d0 con 0x7f6a38110cb0 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 0x7f6a3819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 0x7f6a3819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57180/0 (socket says 172.21.15.19:57180) 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 -- 172.21.15.19:0/3977130343 learned_addr learned my addr 172.21.15.19:0/3977130343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a2ffff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 0x7f6a3819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a3c970700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38110cb0 0x7f6a381a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 -- 172.21.15.19:0/3977130343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38110cb0 msgr2=0x7f6a381a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38110cb0 0x7f6a381a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 -- 172.21.15.19:0/3977130343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 msgr2=0x7f6a3819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 0x7f6a3819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 -- 172.21.15.19:0/3977130343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381a1dd0 con 0x7f6a38103100 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a3c970700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38110cb0 0x7f6a381a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a2ffff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 0x7f6a3819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a37fff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 0x7f6a3819cf00 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800c920 tx=0x7f6a2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.885+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28007b00 con 0x7f6a38103100 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.886+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a28004cc0 con 0x7f6a38103100 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.886+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a381a2060 con 0x7f6a38103100 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.886+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280054f0 con 0x7f6a38103100 2024-05-17T09:33:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.886+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a99e0 con 0x7f6a38103100 2024-05-17T09:33:27.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.887+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a380512b0 con 0x7f6a38103100 2024-05-17T09:33:27.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.889+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a28007ca0 con 0x7f6a38103100 2024-05-17T09:33:27.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.889+0000 7f6a35ffb700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a240b2880 0x7f6a240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:27.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.890+0000 7f6a2ffff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a240b2880 0x7f6a240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.890+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a280e2620 con 0x7f6a38103100 2024-05-17T09:33:27.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.890+0000 7f6a2ffff700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a240b2880 0x7f6a240b4d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005e10 tx=0x7f6a20005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:27.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:27.893+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a28014030 con 0x7f6a38103100 2024-05-17T09:33:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.169+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a3802e5e0 con 0x7f6a38103100 2024-05-17T09:33:28.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.170+0000 7f6a35ffb700 1 -- 172.21.15.19:0/3977130343 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7f6a280a9f90 con 0x7f6a38103100 2024-05-17T09:33:28.170 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:33:28.170 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a240b2880 msgr2=0x7f6a240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a240b2880 0x7f6a240b4d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005e10 tx=0x7f6a20005d60 comp rx=0 tx=0).stop 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 msgr2=0x7f6a3819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 0x7f6a3819cf00 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800c920 tx=0x7f6a2800cce0 comp rx=0 tx=0).stop 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 shutdown_connections 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a240b2880 0x7f6a240b4d90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a38110cb0 0x7f6a381a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a38103100 0x7f6a3819cf00 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 --2- 172.21.15.19:0/3977130343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a38103a50 0x7f6a3819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.172+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 >> 172.21.15.19:0/3977130343 conn(0x7f6a38078df0 msgr2=0x7f6a38112190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.173+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 shutdown_connections 2024-05-17T09:33:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:28.173+0000 7f6a3e3d3700 1 -- 172.21.15.19:0/3977130343 wait complete. 2024-05-17T09:33:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:33:28.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:28 smithi019 ceph-mon[31620]: pgmap v849: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-05-17T09:33:28.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:28 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3977130343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:28.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.74 MB/sec execute 566 sec latency 17578.812 ms 2024-05-17T09:33:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:28 smithi069 ceph-mon[35081]: pgmap v849: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-05-17T09:33:28.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:28 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3977130343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:28 smithi079 ceph-mon[35502]: pgmap v849: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-05-17T09:33:28.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:28 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3977130343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:28.996 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id c9ecb49d --release --await' 2024-05-17T09:33:28.996 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id c9ecb49d --release --await 2024-05-17T09:33:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:29.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.73 MB/sec execute 567 sec latency 18578.839 ms 2024-05-17T09:33:30.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225988 1.73 MB/sec execute 568 sec latency 19578.919 ms 2024-05-17T09:33:30.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:30 smithi069 ceph-mon[35081]: pgmap v850: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-05-17T09:33:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:30 smithi019 ceph-mon[31620]: pgmap v850: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-05-17T09:33:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:30 smithi079 ceph-mon[35502]: pgmap v850: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-05-17T09:33:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 -- 172.21.15.19:0/2695315822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 msgr2=0x7f04b0109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2695315822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b0109520 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009990 tx=0x7f04a0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 -- 172.21.15.19:0/2695315822 shutdown_connections 2024-05-17T09:33:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2695315822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04b0102ef0 0x7f04b0110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2695315822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b0109520 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2695315822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b01025a0 0x7f04b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 -- 172.21.15.19:0/2695315822 >> 172.21.15.19:0/2695315822 conn(0x7f04b00783a0 msgr2=0x7f04b00787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 -- 172.21.15.19:0/2695315822 shutdown_connections 2024-05-17T09:33:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.147+0000 7f04b57a1700 1 -- 172.21.15.19:0/2695315822 wait complete. 2024-05-17T09:33:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 Processor -- start 2024-05-17T09:33:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 -- start start 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04b01025a0 0x7f04b019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b0102ef0 0x7f04b019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b01a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0114250 con 0x7f04b0109110 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01140d0 con 0x7f04b01025a0 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b57a1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01143d0 con 0x7f04b0102ef0 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b01a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b01a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57212/0 (socket says 172.21.15.19:57212) 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 -- 172.21.15.19:0/2670710771 learned_addr learned my addr 172.21.15.19:0/2670710771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04affff700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04b01025a0 0x7f04b019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b0102ef0 0x7f04b019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 -- 172.21.15.19:0/2670710771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b0102ef0 msgr2=0x7f04b019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b0102ef0 0x7f04b019d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 -- 172.21.15.19:0/2670710771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04b01025a0 msgr2=0x7f04b019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04b01025a0 0x7f04b019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.148+0000 7f04b4fa0700 1 -- 172.21.15.19:0/2670710771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b01a1cf0 con 0x7f04b0109110 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.149+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b01a1650 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f04a8007ee0 tx=0x7f04a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.149+0000 7f04adffb700 1 -- 172.21.15.19:0/2670710771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8004bb0 con 0x7f04b0109110 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.149+0000 7f04adffb700 1 -- 172.21.15.19:0/2670710771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a8004d50 con 0x7f04b0109110 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.149+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b01a1fe0 con 0x7f04b0109110 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.149+0000 7f04adffb700 1 -- 172.21.15.19:0/2670710771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a80058b0 con 0x7f04b0109110 2024-05-17T09:33:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.149+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01a9900 con 0x7f04b0109110 2024-05-17T09:33:31.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.152+0000 7f04adffb700 1 -- 172.21.15.19:0/2670710771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f04a801a020 con 0x7f04b0109110 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.153+0000 7f04adffb700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f049c0b2880 0x7f049c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.153+0000 7f04c5418b80 1 Processor -- start 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.153+0000 7f04c5418b80 1 -- start start 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.153+0000 7f04affff700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f049c0b2880 0x7f049c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.153+0000 7f04adffb700 1 -- 172.21.15.19:0/2670710771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f04a80e1d30 con 0x7f04b0109110 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4afb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4afb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4af5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04c5418b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5562e4a22e70 con 0x5562e4a61530 2024-05-17T09:33:31.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04c5418b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5562e4a12a10 con 0x5562e4975f40 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04c5418b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562e4a16a50 con 0x5562e4af5180 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4afb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4afb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57222/0 (socket says 172.21.15.19:57222) 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 -- 172.21.15.19:0/1426588265 learned_addr learned my addr 172.21.15.19:0/1426588265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04b4fa0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4af5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f049c0b2880 0x7f049c0b4d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f04a00059d0 tx=0x7f04a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04a7fff700 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4afb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 -- 172.21.15.19:0/1426588265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 msgr2=0x5562e4af5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4af5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 -- 172.21.15.19:0/1426588265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 msgr2=0x5562e4afb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4afb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04affff700 1 -- 172.21.15.19:0/1426588265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e49bc0b0 con 0x5562e4a61530 2024-05-17T09:33:31.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.154+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4af5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:31.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.155+0000 7f04affff700 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4afb040 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009990 tx=0x7f04a001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535800c09fd4153b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:31.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.155+0000 7f04a67fc700 1 -- 172.21.15.19:0/1426588265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000bbb0 con 0x5562e4a61530 2024-05-17T09:33:31.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.155+0000 7f04a67fc700 1 -- 172.21.15.19:0/1426588265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a000bd50 con 0x5562e4a61530 2024-05-17T09:33:31.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.155+0000 7f04a67fc700 1 -- 172.21.15.19:0/1426588265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000f8c0 con 0x5562e4a61530 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 -- 172.21.15.19:0/1426588265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 msgr2=0x5562e4afb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4afb040 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009990 tx=0x7f04a001e000 comp rx=0 tx=0).stop 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 -- 172.21.15.19:0/1426588265 shutdown_connections 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4af5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4afb040 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1426588265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4afb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 -- 172.21.15.19:0/1426588265 >> 172.21.15.19:0/1426588265 conn(0x5562e49d62c0 msgr2=0x5562e4aeff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 -- 172.21.15.19:0/1426588265 shutdown_connections 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 -- 172.21.15.19:0/1426588265 wait complete. 2024-05-17T09:33:31.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 -- start start 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.156+0000 7f04c5418b80 1 Processor -- start 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4afecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4aff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4aff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5562e49190a0 con 0x5562e4975f40 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5562e497aea0 con 0x5562e4af5180 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562e4af5fe0 con 0x5562e4a61530 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04c5418b80 10 client.0 _ll_register_callbacks cb 0x5562e49ada50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4afecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4afecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57224/0 (socket says 172.21.15.19:57224) 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 -- 172.21.15.19:0/2370323301 learned_addr learned my addr 172.21.15.19:0/2370323301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:31.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4aff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4aff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 -- 172.21.15.19:0/2370323301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 msgr2=0x5562e4aff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4aff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 -- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 msgr2=0x5562e4aff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4aff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 -- 172.21.15.19:0/2370323301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e49629d0 con 0x5562e4975f40 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f04affff700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4afecf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f04a000f740 tx=0x7f04a000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257be0c2b9d32978 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.157+0000 7f048f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f048f7fe700 1 -- 172.21.15.19:0/2370323301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0033030 con 0x5562e4975f40 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f048f7fe700 1 -- 172.21.15.19:0/2370323301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f04a002b050 con 0x5562e4975f40 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a000bd50 con 0x5562e4975f40 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f048f7fe700 1 -- 172.21.15.19:0/2370323301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f04a000bbb0 con 0x5562e4975f40 2024-05-17T09:33:31.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 10 client.17763 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562e49cc860 con 0x5562e4975f40 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f048f7fe700 1 -- 172.21.15.19:0/2370323301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a000bd50 con 0x5562e4975f40 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f048f7fe700 1 -- 172.21.15.19:0/2370323301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f04a0022070 con 0x5562e4975f40 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 10 client.17763 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 10 client.17763 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 20 client.17763 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b0e980 0x5562e4b10da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 4 client.17763 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.158+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e4ac0fd0 con 0x5562e4b0e980 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.159+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b0e980 0x5562e4b10da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.159+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b0e980 0x5562e4b10da0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c1b503079f9962 server_cookie=b3662719218bdd6f in_seq=0 out_seq=0 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.159+0000 7f048f7fe700 10 client.17763 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:31.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f048f7fe700 1 -- 172.21.15.19:0/2370323301 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5562e4ac0fd0 con 0x5562e4b0e980 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f048f7fe700 10 client.17763 handle_command_reply: tid=0 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 client.17763 shutdown 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 msgr2=0x5562e4afecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4afecf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f04a000f740 tx=0x7f04a000f770 comp rx=0 tx=0).stop 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 shutdown_connections 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b0e980 0x5562e4b10da0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4aff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4afecf0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f048f7fe700 0 client.17763 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2370323301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4aff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 >> 172.21.15.19:0/2370323301 conn(0x5562e49d62c0 msgr2=0x5562e4ae9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 shutdown_connections 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 1 -- 172.21.15.19:0/2370323301 wait complete. 2024-05-17T09:33:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.160+0000 7f04c5418b80 20 client.17763 trim_cache size 0 max 16384 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 Processor -- start 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 -- start start 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4af9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4b0ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4b13dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01b4bb0 con 0x5562e4a61530 2024-05-17T09:33:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01b2440 con 0x5562e4975f40 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04c5418b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01b0290 con 0x5562e4af5180 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4b0ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4b0ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57234/0 (socket says 172.21.15.19:57234) 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 -- 172.21.15.19:0/2773869750 learned_addr learned my addr 172.21.15.19:0/2773869750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04affff700 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4af9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4b13dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 -- 172.21.15.19:0/2773869750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 msgr2=0x5562e4b13dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4b13dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 -- 172.21.15.19:0/2773869750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 msgr2=0x5562e4af9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.181+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4af9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.182+0000 7f04a7fff700 1 -- 172.21.15.19:0/2773869750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e4af1820 con 0x5562e4a61530 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.182+0000 7f04affff700 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4af9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:33:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.182+0000 7f04a7fff700 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4b0ae70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0498005950 tx=0x7f049800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86003ccbdb300a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.182+0000 7f04a5ffb700 1 -- 172.21.15.19:0/2773869750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498003b60 con 0x5562e4a61530 2024-05-17T09:33:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.182+0000 7f04a5ffb700 1 -- 172.21.15.19:0/2773869750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0498003d00 con 0x5562e4a61530 2024-05-17T09:33:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.182+0000 7f04a5ffb700 1 -- 172.21.15.19:0/2773869750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498011440 con 0x5562e4a61530 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 -- 172.21.15.19:0/2773869750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 msgr2=0x5562e4b0ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4b0ae70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0498005950 tx=0x7f049800b410 comp rx=0 tx=0).stop 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 -- 172.21.15.19:0/2773869750 shutdown_connections 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4af5180 0x5562e4b13dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4975f40 0x5562e4af9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 --2- 172.21.15.19:0/2773869750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4a61530 0x5562e4b0ae70 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 -- 172.21.15.19:0/2773869750 >> 172.21.15.19:0/2773869750 conn(0x5562e49d62c0 msgr2=0x5562e4af47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 -- 172.21.15.19:0/2773869750 shutdown_connections 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 -- 172.21.15.19:0/2773869750 wait complete. 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 -- start start 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.183+0000 7f04c5418b80 1 Processor -- start 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4b14160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4b124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4b12a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01b0900 con 0x5562e4975f40 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01b1d70 con 0x5562e4af5180 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01b0fd0 con 0x5562e4a61530 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04c5418b80 10 client.0 _ll_register_callbacks cb 0x5562e49ada50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4b14160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4b14160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57248/0 (socket says 172.21.15.19:57248) 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 -- 172.21.15.19:0/1252305109 learned_addr learned my addr 172.21.15.19:0/1252305109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04a7fff700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4b124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04b4fa0700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4b12a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 -- 172.21.15.19:0/1252305109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 msgr2=0x5562e4b124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4b124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 -- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 msgr2=0x5562e4b12a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4b12a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 -- 172.21.15.19:0/1252305109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e4b12f70 con 0x5562e4975f40 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f04affff700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4b14160 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f04a0005410 tx=0x7f04a0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f9b4768d893115 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f048f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.184+0000 7f048f7fe700 1 -- 172.21.15.19:0/1252305109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0033040 con 0x5562e4975f40 2024-05-17T09:33:31.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0498003d00 con 0x5562e4975f40 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f048f7fe700 1 -- 172.21.15.19:0/1252305109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f04a002b040 con 0x5562e4975f40 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f048f7fe700 1 -- 172.21.15.19:0/1252305109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f04a004c050 con 0x5562e4975f40 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 10 client.17769 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562e4b79d30 con 0x5562e4975f40 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f048f7fe700 1 -- 172.21.15.19:0/1252305109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0498003d00 con 0x5562e4975f40 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f048f7fe700 1 -- 172.21.15.19:0/1252305109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f04a0022040 con 0x5562e4975f40 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 10 client.17769 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 10 client.17769 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 20 client.17769 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b7a3a0 0x5562e4b7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 4 client.17769 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c9ecb49d", "release": true, "await": true}] 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.185+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c9ecb49d", "release": true, "await": true}) -- 0x5562e4a24dc0 con 0x5562e4b7a3a0 2024-05-17T09:33:31.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.186+0000 7f04a7fff700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b7a3a0 0x5562e4b7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.186+0000 7f04a7fff700 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b7a3a0 0x5562e4b7c7c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c48165bd0484bba server_cookie=9127af344422e31d in_seq=0 out_seq=0 2024-05-17T09:33:31.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.186+0000 7f048f7fe700 10 client.17769 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f048f7fe700 1 -- 172.21.15.19:0/1252305109 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5562e4a24dc0 con 0x5562e4b7a3a0 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f048f7fe700 10 client.17769 handle_command_reply: tid=0 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 client.17769 shutdown 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 msgr2=0x5562e4b14160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4b14160 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f04a0005410 tx=0x7f04a0011000 comp rx=0 tx=0).stop 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 shutdown_connections 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5562e4b7a3a0 0x5562e4b7c7c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562e4af5180 0x5562e4b12a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562e4975f40 0x5562e4b14160 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 --2- 172.21.15.19:0/1252305109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562e4a61530 0x5562e4b124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 >> 172.21.15.19:0/1252305109 conn(0x5562e49d62c0 msgr2=0x5562e4b00330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f048f7fe700 0 client.17769 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 shutdown_connections 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 1 -- 172.21.15.19:0/1252305109 wait complete. 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.190+0000 7f04c5418b80 20 client.17769 trim_cache size 0 max 16384 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 48, 2024-05-17T09:33:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "c9ecb49d": { 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 48, 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.9, 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:33:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:31.195 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f049c0b2880 msgr2=0x7f049c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f049c0b2880 0x7f049c0b4d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f04a00059d0 tx=0x7f04a0005960 comp rx=0 tx=0).stop 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 msgr2=0x7f04b01a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b01a1650 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f04a8007ee0 tx=0x7f04a800d3b0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 shutdown_connections 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b0109110 0x7f04b01a1650 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04b01025a0 0x7f04b019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f049c0b2880 0x7f049c0b4d90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 --2- 172.21.15.19:0/2670710771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b0102ef0 0x7f04b019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 >> 172.21.15.19:0/2670710771 conn(0x7f04b00783a0 msgr2=0x7f04b00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 shutdown_connections 2024-05-17T09:33:31.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:31.191+0000 7f04b57a1700 1 -- 172.21.15.19:0/2670710771 wait complete. 2024-05-17T09:33:31.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 226103 1.73 MB/sec execute 569 sec latency 20318.570 ms 2024-05-17T09:33:31.945 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c9ecb49d', total seconds elapsed: 27.1 2024-05-17T09:33:31.946 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.9 2024-05-17T09:33:31.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 -- 172.21.15.19:0/1618529392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d01051c0 msgr2=0x7f87d01055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 --2- 172.21.15.19:0/1618529392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d01051c0 0x7f87d01055d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402a410 comp rx=0 tx=0).stop 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 -- 172.21.15.19:0/1618529392 shutdown_connections 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 --2- 172.21.15.19:0/1618529392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d0102370 0x7f87d010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 --2- 172.21.15.19:0/1618529392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0101870 0x7f87d0101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 --2- 172.21.15.19:0/1618529392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d01051c0 0x7f87d01055d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 -- 172.21.15.19:0/1618529392 >> 172.21.15.19:0/1618529392 conn(0x7f87d00fd470 msgr2=0x7f87d00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 -- 172.21.15.19:0/1618529392 shutdown_connections 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.144+0000 7f87d7186640 1 -- 172.21.15.19:0/1618529392 wait complete. 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 Processor -- start 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 -- start start 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 0x7f87d019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0102370 0x7f87d019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d01051c0 0x7f87d01a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87d010f5e0 con 0x7f87d0101870 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87d010f460 con 0x7f87d0102370 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d7186640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87d010f760 con 0x7f87d01051c0 2024-05-17T09:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.145+0000 7f87d4f22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 0x7f87d019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0102370 0x7f87d019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87d4f22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 0x7f87d019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57260/0 (socket says 172.21.15.19:57260) 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87d4f22640 1 -- 172.21.15.19:0/3033605617 learned_addr learned my addr 172.21.15.19:0/3033605617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87d5723640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d01051c0 0x7f87d01a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 -- 172.21.15.19:0/3033605617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d01051c0 msgr2=0x7f87d01a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d01051c0 0x7f87d01a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 -- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 msgr2=0x7f87d019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 0x7f87d019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 -- 172.21.15.19:0/3033605617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d01a1d10 con 0x7f87d0102370 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87d5723640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d01051c0 0x7f87d01a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87d4f22640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 0x7f87d019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cffff640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0102370 0x7f87d019a780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00e9f0 tx=0x7f87bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cdffb640 1 -- 172.21.15.19:0/3033605617 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00cd80 con 0x7f87d0102370 2024-05-17T09:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cdffb640 1 -- 172.21.15.19:0/3033605617 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87bc0035f0 con 0x7f87d0102370 2024-05-17T09:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87cdffb640 1 -- 172.21.15.19:0/3033605617 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc010670 con 0x7f87d0102370 2024-05-17T09:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.146+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d01a1fa0 con 0x7f87d0102370 2024-05-17T09:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.147+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d01a24b0 con 0x7f87d0102370 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.150+0000 7f87cdffb640 1 -- 172.21.15.19:0/3033605617 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f87bc004020 con 0x7f87d0102370 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.150+0000 7f87de043740 1 Processor -- start 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.150+0000 7f87de043740 1 -- start start 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87cdffb640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87c00b2920 0x7f87c00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87de043740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7f71b0 0x55d50cb983a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87de043740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 0x55d50cb98920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87cdffb640 1 -- 172.21.15.19:0/3033605617 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f87bc0e2890 con 0x7f87d0102370 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87de043740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cb9f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87d5723640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87c00b2920 0x7f87c00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 0x55d50cb98920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87d4f22640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7f71b0 0x55d50cb983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 0x55d50cb98920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57264/0 (socket says 172.21.15.19:57264) 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87cffff640 1 -- 172.21.15.19:0/3979191089 learned_addr learned my addr 172.21.15.19:0/3979191089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d50cb0ac30 con 0x55d50c7ce2f0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d50cafa7d0 con 0x55d50c7d64d0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.151+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d50cafe810 con 0x55d50c7f71b0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d5723640 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cb9f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d4f22640 1 -- 172.21.15.19:0/3979191089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 msgr2=0x55d50cb9f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d4f22640 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cb9f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d4f22640 1 -- 172.21.15.19:0/3979191089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 msgr2=0x55d50cb98920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d5723640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87c00b2920 0x7f87c00b4e30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009fd0 tx=0x7f87c8014cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d4f22640 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 0x55d50cb98920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d4f22640 1 -- 172.21.15.19:0/3979191089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50c80f210 con 0x55d50c7f71b0 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87cffff640 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 0x55d50cb98920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87d4f22640 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7f71b0 0x55d50cb983a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87c40057c0 tx=0x7f87c4005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd01e9a2bd1f133f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.152+0000 7f87baffd640 1 -- 172.21.15.19:0/3979191089 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035030 con 0x55d50c7f71b0 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87baffd640 1 -- 172.21.15.19:0/3979191089 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c4040070 con 0x55d50c7f71b0 2024-05-17T09:33:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87baffd640 1 -- 172.21.15.19:0/3979191089 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4003ac0 con 0x55d50c7f71b0 2024-05-17T09:33:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7f71b0 msgr2=0x55d50cb983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7f71b0 0x55d50cb983a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87c40057c0 tx=0x7f87c4005e50 comp rx=0 tx=0).stop 2024-05-17T09:33:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 shutdown_connections 2024-05-17T09:33:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cb9f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7ce2f0 0x55d50cb98920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 --2- 172.21.15.19:0/3979191089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7f71b0 0x55d50cb983a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.153+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 >> 172.21.15.19:0/3979191089 conn(0x55d50c7eb300 msgr2=0x55d50cb93d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.154+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 shutdown_connections 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.154+0000 7f87de043740 1 -- 172.21.15.19:0/3979191089 wait complete. 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.154+0000 7f87de043740 1 -- start start 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.154+0000 7f87de043740 1 Processor -- start 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cba21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7f71b0 0x55d50cba9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d50c7e0790 con 0x55d50c7f71b0 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d50cab6890 con 0x55d50c7d64d0 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d50cba0210 con 0x55d50c7ce2f0 2024-05-17T09:33:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87de043740 10 client.0 _ll_register_callbacks cb 0x55d50cab9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cba21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87d5723640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7f71b0 0x55d50cba9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cba21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:32846/0 (socket says 172.21.15.19:32846) 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 -- 172.21.15.19:0/4063641478 learned_addr learned my addr 172.21.15.19:0/4063641478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87d4f22640 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 -- 172.21.15.19:0/4063641478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 msgr2=0x55d50cba1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 -- 172.21.15.19:0/4063641478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7f71b0 msgr2=0x55d50cba9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7f71b0 0x55d50cba9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 -- 172.21.15.19:0/4063641478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50c8db620 con 0x55d50c7d64d0 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87cffff640 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cba21d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0a5b30 tx=0x7f87bc0a94d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c882efb4f6aa2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.155+0000 7f87affff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87affff640 1 -- 172.21.15.19:0/4063641478 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0c3060 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c4040070 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87affff640 1 -- 172.21.15.19:0/4063641478 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f87bc009310 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87affff640 1 -- 172.21.15.19:0/4063641478 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f87bc0aa200 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 10 client.27362 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d50cbac130 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87affff640 1 -- 172.21.15.19:0/4063641478 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c4040070 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87affff640 1 -- 172.21.15.19:0/4063641478 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f87bc004310 con 0x55d50c7d64d0 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 10 client.27362 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 10 client.27362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 20 client.27362 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7b5d80 0x55d50cbaf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 4 client.27362 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.156+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50cb0cb80 con 0x55d50c7b5d80 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.157+0000 7f87d4f22640 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7b5d80 0x55d50cbaf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.157+0000 7f87d4f22640 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7b5d80 0x55d50cbaf900 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=834793731326ca72 server_cookie=e313349b777c8282 in_seq=0 out_seq=0 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.157+0000 7f87affff640 10 client.27362 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.158+0000 7f87affff640 1 -- 172.21.15.19:0/4063641478 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d50cb0cb80 con 0x55d50c7b5d80 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.158+0000 7f87affff640 10 client.27362 handle_command_reply: tid=0 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.158+0000 7f87de043740 1 client.27362 shutdown 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 msgr2=0x55d50cba21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cba21d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0a5b30 tx=0x7f87bc0a94d0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 shutdown_connections 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7b5d80 0x55d50cbaf900 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87affff640 0 client.27362 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7f71b0 0x55d50cba9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7d64d0 0x55d50cba21d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 --2- 172.21.15.19:0/4063641478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 >> 172.21.15.19:0/4063641478 conn(0x55d50c7eb300 msgr2=0x55d50cb92d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 shutdown_connections 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 1 -- 172.21.15.19:0/4063641478 wait complete. 2024-05-17T09:33:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.159+0000 7f87de043740 20 client.27362 trim_cache size 0 max 16384 2024-05-17T09:33:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 Processor -- start 2024-05-17T09:33:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 -- start start 2024-05-17T09:33:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cb9c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cb97240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01af340 con 0x55d50c7d64d0 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01acc10 con 0x55d50c7b5d80 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.185+0000 7f87de043740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01aaa60 con 0x55d50c7ce2f0 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cb9c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cb9c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:32852/0 (socket says 172.21.15.19:32852) 2024-05-17T09:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d5723640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cb97240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 -- 172.21.15.19:0/2677632381 learned_addr learned my addr 172.21.15.19:0/2677632381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87cffff640 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 -- 172.21.15.19:0/2677632381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 msgr2=0x55d50cba9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 -- 172.21.15.19:0/2677632381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 msgr2=0x55d50cb97240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cb97240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 -- 172.21.15.19:0/2677632381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50cbad5a0 con 0x55d50c7b5d80 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87cffff640 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.186+0000 7f87d4f22640 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cb9c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000c00 tx=0x7f87c402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b03cd91a14c341a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.187+0000 7f87ba7fc640 1 -- 172.21.15.19:0/2677632381 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035040 con 0x55d50c7b5d80 2024-05-17T09:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.187+0000 7f87ba7fc640 1 -- 172.21.15.19:0/2677632381 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c4040040 con 0x55d50c7b5d80 2024-05-17T09:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.187+0000 7f87ba7fc640 1 -- 172.21.15.19:0/2677632381 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4003e00 con 0x55d50c7b5d80 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 -- 172.21.15.19:0/2677632381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 msgr2=0x55d50cb9c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cb9c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000c00 tx=0x7f87c402a410 comp rx=0 tx=0).stop 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 -- 172.21.15.19:0/2677632381 shutdown_connections 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cb97240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 --2- 172.21.15.19:0/2677632381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cb9c8f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 -- 172.21.15.19:0/2677632381 >> 172.21.15.19:0/2677632381 conn(0x55d50c7eb300 msgr2=0x55d50cb96b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 -- 172.21.15.19:0/2677632381 shutdown_connections 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 -- 172.21.15.19:0/2677632381 wait complete. 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 -- start start 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.188+0000 7f87de043740 1 Processor -- start 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cba72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cba8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ab0d0 con 0x55d50c7d64d0 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ac540 con 0x55d50c7b5d80 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ab7a0 con 0x55d50c7ce2f0 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87de043740 10 client.0 _ll_register_callbacks cb 0x7f87bc0a5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87d5723640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cba8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87d5723640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cba8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57308/0 (socket says 172.21.15.19:57308) 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87d5723640 1 -- 172.21.15.19:0/3599443967 learned_addr learned my addr 172.21.15.19:0/3599443967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87cffff640 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.189+0000 7f87d4f22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cba72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87d5723640 1 -- 172.21.15.19:0/3599443967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 msgr2=0x55d50cba7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87d5723640 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87d5723640 1 -- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 msgr2=0x55d50cba72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87d5723640 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cba72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87d5723640 1 -- 172.21.15.19:0/3599443967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50cba8cf0 con 0x55d50c7d64d0 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87d5723640 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cba8730 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f87c8010cd0 tx=0x7f87c801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10419ddec6c111ca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87affff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87affff640 1 -- 172.21.15.19:0/3599443967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c802f060 con 0x55d50c7d64d0 2024-05-17T09:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c4040040 con 0x55d50c7d64d0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87affff640 1 -- 172.21.15.19:0/3599443967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f87c8011440 con 0x55d50c7d64d0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87affff640 1 -- 172.21.15.19:0/3599443967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f87c8036050 con 0x55d50c7d64d0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87de043740 10 client.17787 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.190+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d50cc2d3b0 con 0x55d50c7d64d0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87affff640 1 -- 172.21.15.19:0/3599443967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c4040040 con 0x55d50c7d64d0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87affff640 1 -- 172.21.15.19:0/3599443967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f87c8029070 con 0x55d50c7d64d0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87de043740 10 client.17787 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87de043740 10 client.17787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87de043740 20 client.17787 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87de043740 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7f71b0 0x55d50cc2fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87de043740 4 client.17787 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d50cb06360 con 0x55d50c7f71b0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.191+0000 7f87d4f22640 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7f71b0 0x55d50cc2fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.192+0000 7f87affff640 10 client.17787 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.192+0000 7f87d4f22640 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7f71b0 0x55d50cc2fcf0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4de709c1bef27cf server_cookie=d4f080ac9f601d6 in_seq=0 out_seq=0 2024-05-17T09:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87affff640 1 -- 172.21.15.19:0/3599443967 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d50cb06360 con 0x55d50c7f71b0 2024-05-17T09:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87affff640 10 client.17787 handle_command_reply: tid=0 2024-05-17T09:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 client.17787 shutdown 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 msgr2=0x55d50cba8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cba8730 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f87c8010cd0 tx=0x7f87c801a000 comp rx=0 tx=0).stop 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 shutdown_connections 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d50c7f71b0 0x55d50cc2fcf0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d50c7d64d0 0x55d50cba8730 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d50c7ce2f0 0x55d50cba7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87affff640 0 client.17787 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 --2- 172.21.15.19:0/3599443967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d50c7b5d80 0x55d50cba72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 >> 172.21.15.19:0/3599443967 conn(0x55d50c7eb300 msgr2=0x55d50cb9c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 shutdown_connections 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 1 -- 172.21.15.19:0/3599443967 wait complete. 2024-05-17T09:33:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.218+0000 7f87de043740 20 client.17787 trim_cache size 0 max 16384 2024-05-17T09:33:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:33:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:33:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87c00b2920 msgr2=0x7f87c00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87c00b2920 0x7f87c00b4e30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009fd0 tx=0x7f87c8014cb0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0102370 msgr2=0x7f87d019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0102370 0x7f87d019a780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00e9f0 tx=0x7f87bc00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 shutdown_connections 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87d01051c0 0x7f87d01a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d0102370 0x7f87d019a780 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f87c00b2920 0x7f87c00b4e30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 --2- 172.21.15.19:0/3033605617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d0101870 0x7f87d019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 >> 172.21.15.19:0/3033605617 conn(0x7f87d00fd470 msgr2=0x7f87d0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 shutdown_connections 2024-05-17T09:33:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:32.219+0000 7f87d7186640 1 -- 172.21.15.19:0/3033605617 wait complete. 2024-05-17T09:33:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:32.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 226796 1.74 MB/sec execute 570 sec latency 15.642 ms 2024-05-17T09:33:32.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:32 smithi019 ceph-mon[31620]: pgmap v851: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-05-17T09:33:32.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:32 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:33:32.682 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:32 smithi069 ceph-mon[35081]: pgmap v851: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-05-17T09:33:32.683 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:32 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:33:32.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:32 smithi079 ceph-mon[35502]: pgmap v851: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-05-17T09:33:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:32 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:33:33.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 227173 1.74 MB/sec execute 571 sec latency 15.437 ms 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 -- 172.21.15.19:0/2308632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 msgr2=0x7fe7a4110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/2308632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a4110090 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00b0d0 tx=0x7fe79c00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 -- 172.21.15.19:0/2308632433 shutdown_connections 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/2308632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a4110090 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/2308632433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41034b0 0x7fe7a41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/2308632433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a4107de0 0x7fe7a41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 -- 172.21.15.19:0/2308632433 >> 172.21.15.19:0/2308632433 conn(0x7fe7a40fedb0 msgr2=0x7fe7a41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 -- 172.21.15.19:0/2308632433 shutdown_connections 2024-05-17T09:33:34.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.131+0000 7fe7a8d46700 1 -- 172.21.15.19:0/2308632433 wait complete. 2024-05-17T09:33:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.132+0000 7fe7a8d46700 1 Processor -- start 2024-05-17T09:33:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 -- start start 2024-05-17T09:33:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 0x7fe7a419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a4107de0 0x7fe7a419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:34.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4113210 con 0x7fe7a4108730 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4113090 con 0x7fe7a4107de0 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a8d46700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4113390 con 0x7fe7a41034b0 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 0x7fe7a419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a4107de0 0x7fe7a419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 0x7fe7a419bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:49840/0 (socket says 172.21.15.19:49840) 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a419c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55470/0 (socket says 172.21.15.19:55470) 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 -- 172.21.15.19:0/1789361824 learned_addr learned my addr 172.21.15.19:0/1789361824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 -- 172.21.15.19:0/1789361824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a4107de0 msgr2=0x7fe7a419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a4107de0 0x7fe7a419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 -- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 msgr2=0x7fe7a419c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:34.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a419c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.133+0000 7fe7a259c700 1 -- 172.21.15.19:0/1789361824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a41a6e60 con 0x7fe7a41034b0 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7a2d9d700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a419c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7a1d9b700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a4107de0 0x7fe7a419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7a259c700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 0x7fe7a419bb10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe79400d950 tx=0x7fe79400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794016070 con 0x7fe7a41034b0 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe79400fd20 con 0x7fe7a41034b0 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a4199960 con 0x7fe7a41034b0 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79400e3b0 con 0x7fe7a41034b0 2024-05-17T09:33:34.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.134+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a4199ea0 con 0x7fe7a41034b0 2024-05-17T09:33:34.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.135+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a40512b0 con 0x7fe7a41034b0 2024-05-17T09:33:34.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.137+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe794019070 con 0x7fe7a41034b0 2024-05-17T09:33:34.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.138+0000 7fe7937fe700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe78c0b2880 0x7fe78c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:34.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.138+0000 7fe7a1d9b700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe78c0b2880 0x7fe78c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:34.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.138+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe7940e20f0 con 0x7fe7a41034b0 2024-05-17T09:33:34.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.138+0000 7fe7a1d9b700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe78c0b2880 0x7fe78c0b4d90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4106820 tx=0x7fe7980073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:34.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.141+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7940a9a00 con 0x7fe7a41034b0 2024-05-17T09:33:34.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.432+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a402e5e0 con 0x7fe7a41034b0 2024-05-17T09:33:34.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.432+0000 7fe7937fe700 1 -- 172.21.15.19:0/1789361824 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7fe7940a9300 con 0x7fe7a41034b0 2024-05-17T09:33:34.433 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:33:34.433 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:33:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe78c0b2880 msgr2=0x7fe78c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe78c0b2880 0x7fe78c0b4d90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4106820 tx=0x7fe7980073d0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 msgr2=0x7fe7a419bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 0x7fe7a419bb10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe79400d950 tx=0x7fe79400dc60 comp rx=0 tx=0).stop 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 shutdown_connections 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe78c0b2880 0x7fe78c0b4d90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe7a41034b0 0x7fe7a419bb10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4108730 0x7fe7a419c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 --2- 172.21.15.19:0/1789361824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a4107de0 0x7fe7a419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 >> 172.21.15.19:0/1789361824 conn(0x7fe7a40fedb0 msgr2=0x7fe7a4109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 shutdown_connections 2024-05-17T09:33:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:34.435+0000 7fe7a8d46700 1 -- 172.21.15.19:0/1789361824 wait complete. 2024-05-17T09:33:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:33:34.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 227431 1.74 MB/sec execute 572 sec latency 16.597 ms 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: pgmap v852: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 68 KiB/s wr, 9 op/s 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:33:34.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:33:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: pgmap v852: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 68 KiB/s wr, 9 op/s 2024-05-17T09:33:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:33:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:33:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: pgmap v852: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 68 KiB/s wr, 9 op/s 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:33:34.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:33:35.148 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 140.9 --await-for 45.0' 2024-05-17T09:33:35.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 140.9 --await-for 45.0 2024-05-17T09:33:35.420 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:35.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 227665 1.74 MB/sec execute 573 sec latency 15.026 ms 2024-05-17T09:33:35.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:35 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:33:35.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:35 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:33:35.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:35 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1789361824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:35.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:35 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:33:35.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:35 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:33:35.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:35 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1789361824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:35 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:33:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:35 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:33:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:35 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1789361824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:36.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228126 1.74 MB/sec execute 574 sec latency 15.405 ms 2024-05-17T09:33:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:36 smithi069 ceph-mon[35081]: pgmap v853: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 167 KiB/s wr, 28 op/s 2024-05-17T09:33:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:36 smithi019 ceph-mon[31620]: pgmap v853: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 167 KiB/s wr, 28 op/s 2024-05-17T09:33:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:36 smithi079 ceph-mon[35502]: pgmap v853: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 167 KiB/s wr, 28 op/s 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 -- 172.21.15.19:0/290978998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 msgr2=0x7f7b2c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 --2- 172.21.15.19:0/290978998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c1038c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0099c0 tx=0x7f7b1c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 -- 172.21.15.19:0/290978998 shutdown_connections 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 --2- 172.21.15.19:0/290978998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c108730 0x7f7b2c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 --2- 172.21.15.19:0/290978998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c1038c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 --2- 172.21.15.19:0/290978998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c107de0 0x7f7b2c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 -- 172.21.15.19:0/290978998 >> 172.21.15.19:0/290978998 conn(0x7f7b2c0fedb0 msgr2=0x7f7b2c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 -- 172.21.15.19:0/290978998 shutdown_connections 2024-05-17T09:33:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.345+0000 7f7b32999700 1 -- 172.21.15.19:0/290978998 wait complete. 2024-05-17T09:33:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.346+0000 7f7b32999700 1 Processor -- start 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.346+0000 7f7b32999700 1 -- start start 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b32999700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b32999700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 0x7f7b2c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b32999700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c108730 0x7f7b2c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b32999700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c113260 con 0x7f7b2c1034b0 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b32999700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1130e0 con 0x7f7b2c108730 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b32999700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1133e0 con 0x7f7b2c107de0 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 0x7f7b2c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b30f36700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c108730 0x7f7b2c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c19bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55492/0 (socket says 172.21.15.19:55492) 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b2bfff700 1 -- 172.21.15.19:0/4203277091 learned_addr learned my addr 172.21.15.19:0/4203277091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 0x7f7b2c19c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:49860/0 (socket says 172.21.15.19:49860) 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 -- 172.21.15.19:0/4203277091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c108730 msgr2=0x7f7b2c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c108730 0x7f7b2c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 -- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 msgr2=0x7f7b2c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c19bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b23fff700 1 -- 172.21.15.19:0/4203277091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c1a6f40 con 0x7f7b2c107de0 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b30f36700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c108730 0x7f7b2c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.347+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.348+0000 7f7b23fff700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 0x7f7b2c19c160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400e9c0 tx=0x7f7b1400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.348+0000 7f7b29ffb700 1 -- 172.21.15.19:0/4203277091 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400c9c0 con 0x7f7b2c107de0 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.348+0000 7f7b29ffb700 1 -- 172.21.15.19:0/4203277091 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b14010430 con 0x7f7b2c107de0 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.348+0000 7f7b29ffb700 1 -- 172.21.15.19:0/4203277091 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14018680 con 0x7f7b2c107de0 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.348+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c1a7230 con 0x7f7b2c107de0 2024-05-17T09:33:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.348+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1a7720 con 0x7f7b2c107de0 2024-05-17T09:33:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.351+0000 7f7b29ffb700 1 -- 172.21.15.19:0/4203277091 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7b140108e0 con 0x7f7b2c107de0 2024-05-17T09:33:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.351+0000 7f7b29ffb700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7b180b2900 0x7f7b180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b29ffb700 1 -- 172.21.15.19:0/4203277091 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7b140e2650 con 0x7f7b2c107de0 2024-05-17T09:33:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7b180b2900 0x7f7b180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 Processor -- start 2024-05-17T09:33:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 -- start start 2024-05-17T09:33:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x56346762a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7b180b2900 0x7f7b180b4e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c000c00 tx=0x7f7b1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634674a6fd0 0x56346762ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x56346762f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563467553f00 con 0x56346762ede0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563467543aa0 con 0x5634674a6fd0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.352+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563467547ae0 con 0x5634675925c0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b30f36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x56346762f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x56346762a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b30f36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x56346762f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55502/0 (socket says 172.21.15.19:55502) 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x56346762a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:49870/0 (socket says 172.21.15.19:49870) 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b30f36700 1 -- 172.21.15.19:0/3133372073 learned_addr learned my addr 172.21.15.19:0/3133372073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b23fff700 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634674a6fd0 0x56346762ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 -- 172.21.15.19:0/3133372073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634674a6fd0 msgr2=0x56346762ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634674a6fd0 0x56346762ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 -- 172.21.15.19:0/3133372073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 msgr2=0x56346762f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x56346762f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 -- 172.21.15.19:0/3133372073 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634674ed140 con 0x5634675925c0 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b23fff700 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634674a6fd0 0x56346762ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x56346762a760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00b910 tx=0x7f7b1c022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ae7c63841aac61 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b227fc700 1 -- 172.21.15.19:0/3133372073 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0037c0 con 0x5634675925c0 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b227fc700 1 -- 172.21.15.19:0/3133372073 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b1c003df0 con 0x5634675925c0 2024-05-17T09:33:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.353+0000 7f7b227fc700 1 -- 172.21.15.19:0/3133372073 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c030cf0 con 0x5634675925c0 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3133372073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 msgr2=0x56346762a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x56346762a760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00b910 tx=0x7f7b1c022000 comp rx=0 tx=0).stop 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3133372073 shutdown_connections 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x56346762a760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x56346762f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3133372073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634674a6fd0 0x56346762ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3133372073 >> 172.21.15.19:0/3133372073 conn(0x563467507350 msgr2=0x563467620d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.354+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3133372073 shutdown_connections 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.355+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3133372073 wait complete. 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.355+0000 7f7b413aeb80 1 -- start start 2024-05-17T09:33:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.355+0000 7f7b413aeb80 1 Processor -- start 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634674a6fd0 0x563467634070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x563467634af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56346744a130 con 0x56346762ede0 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634674abf30 con 0x5634675925c0 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56346762fb30 con 0x5634674a6fd0 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b413aeb80 10 client.0 _ll_register_callbacks cb 0x5634674deae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b30f36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x563467634af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b30f36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x563467634af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55518/0 (socket says 172.21.15.19:55518) 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b30f36700 1 -- 172.21.15.19:0/3144830102 learned_addr learned my addr 172.21.15.19:0/3144830102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634674a6fd0 0x563467634070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 -- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 msgr2=0x5634676345b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:33:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676345b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 -- 172.21.15.19:0/3144830102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 msgr2=0x563467634af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x563467634af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 -- 172.21.15.19:0/3144830102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563467493a60 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b30f36700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x563467634af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634674a6fd0 0x563467634070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c003760 tx=0x7f7b1c0307e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da4be19dee668214 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.356+0000 7f7b0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/3144830102 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c03e060 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/3144830102 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7b1c030d40 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b1c003df0 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/3144830102 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b1c0037c0 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 10 client.27355 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634674fd8f0 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/3144830102 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b1c003df0 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/3144830102 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7b1c038030 con 0x5634674a6fd0 2024-05-17T09:33:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 10 client.27355 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 10 client.27355 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 20 client.27355 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.357+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563467643bd0 0x563467645ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.358+0000 7f7b413aeb80 4 client.27355 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.358+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634675f2060 con 0x563467643bd0 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.358+0000 7f7b23fff700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563467643bd0 0x563467645ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.358+0000 7f7b0b7fe700 10 client.27355 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.358+0000 7f7b23fff700 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563467643bd0 0x563467645ff0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e2ff4bc8419f713 server_cookie=8b50944d01309fb4 in_seq=0 out_seq=0 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/3144830102 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5634675f2060 con 0x563467643bd0 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b0b7fe700 10 client.27355 handle_command_reply: tid=0 2024-05-17T09:33:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 client.27355 shutdown 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634674a6fd0 msgr2=0x563467634070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634674a6fd0 0x563467634070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c003760 tx=0x7f7b1c0307e0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 shutdown_connections 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634674a6fd0 0x563467634070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563467643bd0 0x563467645ff0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56346762ede0 0x563467634af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/3144830102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b0b7fe700 0 client.27355 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 >> 172.21.15.19:0/3144830102 conn(0x563467507350 msgr2=0x5634676250a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.359+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 shutdown_connections 2024-05-17T09:33:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.360+0000 7f7b413aeb80 1 -- 172.21.15.19:0/3144830102 wait complete. 2024-05-17T09:33:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.360+0000 7f7b413aeb80 20 client.27355 trim_cache size 0 max 16384 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 Processor -- start 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 -- start start 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x56346761b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56346762ede0 0x563467633280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1b09c0 con 0x5634674a6fd0 2024-05-17T09:33:37.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1ae250 con 0x5634675925c0 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1ac0a0 con 0x56346762ede0 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x56346761b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b23fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x56346761b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55522/0 (socket says 172.21.15.19:55522) 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b2bfff700 1 -- 172.21.15.19:0/2488269259 learned_addr learned my addr 172.21.15.19:0/2488269259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.381+0000 7f7b30f36700 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56346762ede0 0x563467633280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b23fff700 1 -- 172.21.15.19:0/2488269259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56346762ede0 msgr2=0x563467633280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b23fff700 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56346762ede0 0x563467633280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b23fff700 1 -- 172.21.15.19:0/2488269259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 msgr2=0x56346761b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b23fff700 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x56346761b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b23fff700 1 -- 172.21.15.19:0/2488269259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563467634e60 con 0x5634675925c0 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x56346761b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b30f36700 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56346762ede0 0x563467633280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b23fff700 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676400a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b140a6b30 tx=0x7f7b140cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efef65185bf28ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b21ffb700 1 -- 172.21.15.19:0/2488269259 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140a98c0 con 0x5634675925c0 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b21ffb700 1 -- 172.21.15.19:0/2488269259 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b140adc70 con 0x5634675925c0 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.382+0000 7f7b21ffb700 1 -- 172.21.15.19:0/2488269259 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140a9b20 con 0x5634675925c0 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 -- 172.21.15.19:0/2488269259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 msgr2=0x5634676400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676400a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b140a6b30 tx=0x7f7b140cc040 comp rx=0 tx=0).stop 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 -- 172.21.15.19:0/2488269259 shutdown_connections 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56346762ede0 0x563467633280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x56346761b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/2488269259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634675925c0 0x5634676400a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 -- 172.21.15.19:0/2488269259 >> 172.21.15.19:0/2488269259 conn(0x563467507350 msgr2=0x563467625710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 -- 172.21.15.19:0/2488269259 shutdown_connections 2024-05-17T09:33:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.383+0000 7f7b413aeb80 1 -- 172.21.15.19:0/2488269259 wait complete. 2024-05-17T09:33:37.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 -- start start 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 Processor -- start 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x563467649070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x5634676495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56346762ede0 0x5634676475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1ac710 con 0x5634674a6fd0 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1adb80 con 0x56346762ede0 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c1acde0 con 0x5634675925c0 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b413aeb80 10 client.0 _ll_register_callbacks cb 0x7f7b24003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x563467649070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x563467649070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55536/0 (socket says 172.21.15.19:55536) 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b2bfff700 1 -- 172.21.15.19:0/1858549019 learned_addr learned my addr 172.21.15.19:0/1858549019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:37.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b23fff700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x5634676495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.384+0000 7f7b30f36700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56346762ede0 0x5634676475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b23fff700 1 -- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56346762ede0 msgr2=0x5634676475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b23fff700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56346762ede0 0x5634676475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b23fff700 1 -- 172.21.15.19:0/1858549019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 msgr2=0x563467649070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b23fff700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x563467649070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b23fff700 1 -- 172.21.15.19:0/1858549019 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563467647ca0 con 0x5634675925c0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b23fff700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x5634676495b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b140a9a80 tx=0x7f7b140b2190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a5fc2e1406826cd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/1858549019 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140e2340 con 0x5634675925c0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/1858549019 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7b140a5b50 con 0x5634675925c0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b140adc70 con 0x5634675925c0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/1858549019 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b140a9b20 con 0x5634675925c0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b413aeb80 10 client.27361 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.385+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634676aed90 con 0x5634675925c0 2024-05-17T09:33:37.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/1858549019 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b140adc70 con 0x5634675925c0 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/1858549019 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7b1400cb60 con 0x5634675925c0 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b413aeb80 10 client.27361 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b413aeb80 10 client.27361 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b413aeb80 20 client.27361 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5634676af240 0x5634676b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b413aeb80 4 client.27361 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.9, "await_for": 45.0}] 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.9, "await_for": 45.0}) -- 0x563467555e50 con 0x5634676af240 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.386+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5634676af240 0x5634676b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.387+0000 7f7b0b7fe700 10 client.27361 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.387+0000 7f7b2bfff700 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5634676af240 0x5634676b1660 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44bb2e59a709117 server_cookie=8d621e7993f75830 in_seq=0 out_seq=0 2024-05-17T09:33:37.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.74 MB/sec execute 575 sec latency 60.754 ms 2024-05-17T09:33:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.459+0000 7f7b0b7fe700 1 -- 172.21.15.19:0/1858549019 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563467555e50 con 0x5634676af240 2024-05-17T09:33:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.459+0000 7f7b0b7fe700 10 client.27361 handle_command_reply: tid=0 2024-05-17T09:33:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.459+0000 7f7b413aeb80 1 client.27361 shutdown 2024-05-17T09:33:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 msgr2=0x5634676495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x5634676495b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b140a9a80 tx=0x7f7b140b2190 comp rx=0 tx=0).stop 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 shutdown_connections 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5634676af240 0x5634676b1660 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56346762ede0 0x5634676475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634674a6fd0 0x563467649070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b0b7fe700 0 client.27361 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 --2- 172.21.15.19:0/1858549019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5634675925c0 0x5634676495b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 >> 172.21.15.19:0/1858549019 conn(0x563467507350 msgr2=0x563467632850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 shutdown_connections 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 1 -- 172.21.15.19:0/1858549019 wait complete. 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b413aeb80 20 client.27361 trim_cache size 0 max 16384 2024-05-17T09:33:37.461 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 50, 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "388fcb62": { 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 50, 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:33:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.9, 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:37.463 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7b180b2900 msgr2=0x7f7b180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b32999700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7b180b2900 0x7f7b180b4e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c000c00 tx=0x7f7b1c01a040 comp rx=0 tx=0).stop 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 msgr2=0x7f7b2c19c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.460+0000 7f7b32999700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 0x7f7b2c19c160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400e9c0 tx=0x7f7b1400ecd0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 shutdown_connections 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b2c108730 0x7f7b2c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b2c1034b0 0x7f7b2c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7b180b2900 0x7f7b180b4e10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 --2- 172.21.15.19:0/4203277091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b2c107de0 0x7f7b2c19c160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 >> 172.21.15.19:0/4203277091 conn(0x7f7b2c0fedb0 msgr2=0x7f7b2c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 shutdown_connections 2024-05-17T09:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:37.461+0000 7f7b32999700 1 -- 172.21.15.19:0/4203277091 wait complete. 2024-05-17T09:33:38.252 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '388fcb62', quiesce took 6.3 seconds. Will release after: 14.599999999999998 2024-05-17T09:33:38.452 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.74 MB/sec execute 576 sec latency 1060.780 ms 2024-05-17T09:33:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:38 smithi069 ceph-mon[35081]: pgmap v854: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 414 KiB/s wr, 32 op/s 2024-05-17T09:33:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:38 smithi019 ceph-mon[31620]: pgmap v854: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 414 KiB/s wr, 32 op/s 2024-05-17T09:33:38.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:38 smithi079 ceph-mon[35502]: pgmap v854: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 414 KiB/s wr, 32 op/s 2024-05-17T09:33:39.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.73 MB/sec execute 577 sec latency 2060.812 ms 2024-05-17T09:33:40.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.73 MB/sec execute 578 sec latency 3060.906 ms 2024-05-17T09:33:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:40 smithi069 ceph-mon[35081]: pgmap v855: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 616 KiB/s wr, 76 op/s 2024-05-17T09:33:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:40 smithi019 ceph-mon[31620]: pgmap v855: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 616 KiB/s wr, 76 op/s 2024-05-17T09:33:40.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:40 smithi079 ceph-mon[35502]: pgmap v855: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 616 KiB/s wr, 76 op/s 2024-05-17T09:33:41.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.73 MB/sec execute 579 sec latency 4060.970 ms 2024-05-17T09:33:42.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.72 MB/sec execute 580 sec latency 5061.009 ms 2024-05-17T09:33:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:42 smithi069 ceph-mon[35081]: pgmap v856: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 616 KiB/s wr, 75 op/s 2024-05-17T09:33:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:42 smithi019 ceph-mon[31620]: pgmap v856: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 616 KiB/s wr, 75 op/s 2024-05-17T09:33:42.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:42 smithi079 ceph-mon[35502]: pgmap v856: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 616 KiB/s wr, 75 op/s 2024-05-17T09:33:43.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.72 MB/sec execute 581 sec latency 6061.086 ms 2024-05-17T09:33:44.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.72 MB/sec execute 582 sec latency 7061.178 ms 2024-05-17T09:33:44.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:44 smithi069 ceph-mon[35081]: pgmap v857: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-05-17T09:33:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:44 smithi019 ceph-mon[31620]: pgmap v857: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-05-17T09:33:44.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:44 smithi079 ceph-mon[35502]: pgmap v857: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 -- 172.21.15.19:0/1893153132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff681018c0 msgr2=0x7eff68101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 --2- 172.21.15.19:0/1893153132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff681018c0 0x7eff68101d30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff5002a410 comp rx=0 tx=0).stop 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 -- 172.21.15.19:0/1893153132 shutdown_connections 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 --2- 172.21.15.19:0/1893153132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff681023c0 0x7eff6810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 --2- 172.21.15.19:0/1893153132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff681018c0 0x7eff68101d30 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 --2- 172.21.15.19:0/1893153132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff68105210 0x7eff68105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 -- 172.21.15.19:0/1893153132 >> 172.21.15.19:0/1893153132 conn(0x7eff680fd500 msgr2=0x7eff680ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 -- 172.21.15.19:0/1893153132 shutdown_connections 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.338+0000 7eff6e57d640 1 -- 172.21.15.19:0/1893153132 wait complete. 2024-05-17T09:33:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 Processor -- start 2024-05-17T09:33:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 -- start start 2024-05-17T09:33:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff681018c0 0x7eff6819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff681023c0 0x7eff6819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 0x7eff681a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810f6a0 con 0x7eff68105210 2024-05-17T09:33:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810f520 con 0x7eff681018c0 2024-05-17T09:33:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6e57d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810f820 con 0x7eff681023c0 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6cb1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 0x7eff681a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6cb1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 0x7eff681a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48614/0 (socket says 172.21.15.19:48614) 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff6cb1a640 1 -- 172.21.15.19:0/2153024819 learned_addr learned my addr 172.21.15.19:0/2153024819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff67fff640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff681018c0 0x7eff6819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.339+0000 7eff5ffff640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff681023c0 0x7eff6819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6cb1a640 1 -- 172.21.15.19:0/2153024819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff681023c0 msgr2=0x7eff6819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff681023c0 0x7eff6819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6cb1a640 1 -- 172.21.15.19:0/2153024819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff681018c0 msgr2=0x7eff6819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff681018c0 0x7eff6819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6cb1a640 1 -- 172.21.15.19:0/2153024819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff681a1cf0 con 0x7eff68105210 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 0x7eff681a1680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff6000eb10 tx=0x7eff6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff65ffb640 1 -- 172.21.15.19:0/2153024819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60004b20 con 0x7eff68105210 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff65ffb640 1 -- 172.21.15.19:0/2153024819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff60004cc0 con 0x7eff68105210 2024-05-17T09:33:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff681a1fe0 con 0x7eff68105210 2024-05-17T09:33:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff65ffb640 1 -- 172.21.15.19:0/2153024819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60010690 con 0x7eff68105210 2024-05-17T09:33:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.340+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a2440 con 0x7eff68105210 2024-05-17T09:33:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 Processor -- start 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff65ffb640 1 -- 172.21.15.19:0/2153024819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7eff60005020 con 0x7eff68105210 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 -- start start 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab56d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab574820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5600ab4dfc30 con 0x5600ab1a32f0 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5600ab4cf7d0 con 0x5600ab1ab4d0 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff75432740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5600ab4d3810 con 0x5600ab1cc1b0 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48630/0 (socket says 172.21.15.19:48630) 2024-05-17T09:33:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.343+0000 7eff67fff640 1 -- 172.21.15.19:0/1390124178 learned_addr learned my addr 172.21.15.19:0/1390124178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5ffff640 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab574820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5ffff640 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab56d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff67fff640 1 -- 172.21.15.19:0/1390124178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 msgr2=0x5600ab56d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff67fff640 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab56d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff67fff640 1 -- 172.21.15.19:0/1390124178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 msgr2=0x5600ab574820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff67fff640 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab574820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff67fff640 1 -- 172.21.15.19:0/1390124178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ab1e4210 con 0x5600ab1a32f0 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5ffff640 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab574820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff67fff640 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56d920 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff58009c60 tx=0x7eff58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d726bcc203cd4342 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff65ffb640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff540b28a0 0x7eff540b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5e7fc640 1 -- 172.21.15.19:0/1390124178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004220 con 0x5600ab1a32f0 2024-05-17T09:33:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5ffff640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff540b28a0 0x7eff540b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5e7fc640 1 -- 172.21.15.19:0/1390124178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff580043c0 con 0x5600ab1a32f0 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff5e7fc640 1 -- 172.21.15.19:0/1390124178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004c20 con 0x5600ab1a32f0 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.344+0000 7eff65ffb640 1 -- 172.21.15.19:0/2153024819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7eff600e2b20 con 0x7eff68105210 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 -- 172.21.15.19:0/1390124178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 msgr2=0x5600ab56d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff5ffff640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff540b28a0 0x7eff540b4db0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7eff50005760 tx=0x7eff5003f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56d920 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff58009c60 tx=0x7eff58011810 comp rx=0 tx=0).stop 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 -- 172.21.15.19:0/1390124178 shutdown_connections 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab574820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56d920 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 --2- 172.21.15.19:0/1390124178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab56d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 -- 172.21.15.19:0/1390124178 >> 172.21.15.19:0/1390124178 conn(0x5600ab1c0300 msgr2=0x5600ab568d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 -- 172.21.15.19:0/1390124178 shutdown_connections 2024-05-17T09:33:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.345+0000 7eff75432740 1 -- 172.21.15.19:0/1390124178 wait complete. 2024-05-17T09:33:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 -- start start 2024-05-17T09:33:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 Processor -- start 2024-05-17T09:33:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab57b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab57b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5600ab1b5790 con 0x5600ab1a32f0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5600ab48b890 con 0x5600ab1ab4d0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.346+0000 7eff75432740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5600ab5795c0 con 0x5600ab1cc1b0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff75432740 10 client.0 _ll_register_callbacks cb 0x5600ab48ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab57b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab57b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48644/0 (socket says 172.21.15.19:48644) 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 -- 172.21.15.19:0/1927549279 learned_addr learned my addr 172.21.15.19:0/1927549279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff5ffff640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab57b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 -- 172.21.15.19:0/1927549279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 msgr2=0x5600ab582520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 -- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 msgr2=0x5600ab57b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab57b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 -- 172.21.15.19:0/1927549279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ab2b0620 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff67fff640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab57b080 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7eff58009c30 tx=0x7eff58019c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2caf04d87a3e37d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff477fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff477fe640 1 -- 172.21.15.19:0/1927549279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58027030 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff477fe640 1 -- 172.21.15.19:0/1927549279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7eff5802c410 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.347+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff580043c0 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff477fe640 1 -- 172.21.15.19:0/1927549279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff58034050 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 10 client.17823 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600ab585450 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff477fe640 1 -- 172.21.15.19:0/1927549279 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff580043c0 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff477fe640 1 -- 172.21.15.19:0/1927549279 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7eff58023a60 con 0x5600ab1a32f0 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 10 client.17823 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 20 client.17823 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab18ad80 0x5600ab588990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 4 client.17823 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ab4e1b80 con 0x5600ab18ad80 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.348+0000 7eff5ffff640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab18ad80 0x5600ab588990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.349+0000 7eff477fe640 10 client.17823 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.349+0000 7eff5ffff640 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab18ad80 0x5600ab588990 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de3773909c44a07 server_cookie=c4dee0e7f587157e in_seq=0 out_seq=0 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff477fe640 1 -- 172.21.15.19:0/1927549279 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5600ab4e1b80 con 0x5600ab18ad80 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff477fe640 10 client.17823 handle_command_reply: tid=0 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 client.17823 shutdown 2024-05-17T09:33:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 msgr2=0x5600ab57b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab57b080 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7eff58009c30 tx=0x7eff58019c90 comp rx=0 tx=0).stop 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 shutdown_connections 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab18ad80 0x5600ab588990 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab57b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff477fe640 0 client.17823 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 --2- 172.21.15.19:0/1927549279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab57b080 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 >> 172.21.15.19:0/1927549279 conn(0x5600ab1c0300 msgr2=0x5600ab56bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 shutdown_connections 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 1 -- 172.21.15.19:0/1927549279 wait complete. 2024-05-17T09:33:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.350+0000 7eff75432740 20 client.17823 trim_cache size 0 max 16384 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 Processor -- start 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 -- start start 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab582440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff681ad050 con 0x5600ab1a32f0 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff681aa920 con 0x5600ab1ab4d0 2024-05-17T09:33:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff75432740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a8770 con 0x5600ab1cc1b0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48660/0 (socket says 172.21.15.19:48660) 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 -- 172.21.15.19:0/2473998205 learned_addr learned my addr 172.21.15.19:0/2473998205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff5ffff640 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab582440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 -- 172.21.15.19:0/2473998205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 msgr2=0x5600ab582980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 -- 172.21.15.19:0/2473998205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 msgr2=0x5600ab582440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab582440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff67fff640 1 -- 172.21.15.19:0/2473998205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ab579740 con 0x5600ab1a32f0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.366+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.367+0000 7eff67fff640 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56c600 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7eff58009380 tx=0x7eff580099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2401e54d611079a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.367+0000 7eff5dffb640 1 -- 172.21.15.19:0/2473998205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58027070 con 0x5600ab1a32f0 2024-05-17T09:33:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.367+0000 7eff5dffb640 1 -- 172.21.15.19:0/2473998205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff5802e960 con 0x5600ab1a32f0 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.367+0000 7eff5dffb640 1 -- 172.21.15.19:0/2473998205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5801d040 con 0x5600ab1a32f0 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 -- 172.21.15.19:0/2473998205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 msgr2=0x5600ab56c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56c600 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7eff58009380 tx=0x7eff580099b0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 -- 172.21.15.19:0/2473998205 shutdown_connections 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab582980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1ab4d0 0x5600ab582440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 --2- 172.21.15.19:0/2473998205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1a32f0 0x5600ab56c600 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 -- 172.21.15.19:0/2473998205 >> 172.21.15.19:0/2473998205 conn(0x5600ab1c0300 msgr2=0x5600ab5712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 -- 172.21.15.19:0/2473998205 shutdown_connections 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 -- 172.21.15.19:0/2473998205 wait complete. 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 -- start start 2024-05-17T09:33:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.368+0000 7eff75432740 1 Processor -- start 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1a32f0 0x5600ab581c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 0x5600ab5821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab5ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a8de0 con 0x5600ab1ab4d0 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff681aa250 con 0x5600ab1a32f0 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a94b0 con 0x5600ab1cc1b0 2024-05-17T09:33:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff75432740 10 client.0 _ll_register_callbacks cb 0x7eff580198a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 0x5600ab5821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 0x5600ab5821d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48676/0 (socket says 172.21.15.19:48676) 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 -- 172.21.15.19:0/3145803776 learned_addr learned my addr 172.21.15.19:0/3145803776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff67fff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1a32f0 0x5600ab581c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff6cb1a640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab5ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 -- 172.21.15.19:0/3145803776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 msgr2=0x5600ab5ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab5ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 -- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1a32f0 msgr2=0x5600ab581c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1a32f0 0x5600ab581c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 -- 172.21.15.19:0/3145803776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ab5ef550 con 0x5600ab1ab4d0 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff67fff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1a32f0 0x5600ab581c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff5ffff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 0x5600ab5821d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7eff50005de0 tx=0x7eff50005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af319aaf86814620 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff477fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.369+0000 7eff477fe640 1 -- 172.21.15.19:0/3145803776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50043030 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff477fe640 1 -- 172.21.15.19:0/3145803776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7eff50003be0 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5802e960 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff477fe640 1 -- 172.21.15.19:0/3145803776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff50063050 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 10 client.17829 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600ab5f0730 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff477fe640 1 -- 172.21.15.19:0/3145803776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff5802e960 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff477fe640 1 -- 172.21.15.19:0/3145803776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7eff50054250 con 0x5600ab1ab4d0 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 10 client.17829 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 20 client.17829 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab1601c0 0x5600ab5f3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 4 client.17829 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.370+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600ab4db360 con 0x5600ab1601c0 2024-05-17T09:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.371+0000 7eff67fff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab1601c0 0x5600ab5f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.371+0000 7eff477fe640 10 client.17829 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.371+0000 7eff67fff640 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab1601c0 0x5600ab5f3150 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850c64904d0d15a4 server_cookie=1e388e15ad0eb503 in_seq=0 out_seq=0 2024-05-17T09:33:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff477fe640 1 -- 172.21.15.19:0/3145803776 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5600ab4db360 con 0x5600ab1601c0 2024-05-17T09:33:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff477fe640 10 client.17829 handle_command_reply: tid=0 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 client.17829 shutdown 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 msgr2=0x5600ab5821d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 0x5600ab5821d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7eff50005de0 tx=0x7eff50005d70 comp rx=0 tx=0).stop 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 shutdown_connections 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5600ab1601c0 0x5600ab5f3150 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600ab1cc1b0 0x5600ab5ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff477fe640 0 client.17829 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600ab1ab4d0 0x5600ab5821d0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 --2- 172.21.15.19:0/3145803776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600ab1a32f0 0x5600ab581c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 >> 172.21.15.19:0/3145803776 conn(0x5600ab1c0300 msgr2=0x5600ab58d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 shutdown_connections 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 1 -- 172.21.15.19:0/3145803776 wait complete. 2024-05-17T09:33:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.397+0000 7eff75432740 20 client.17829 trim_cache size 0 max 16384 2024-05-17T09:33:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:33:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:33:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff540b28a0 msgr2=0x7eff540b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff540b28a0 0x7eff540b4db0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7eff50005760 tx=0x7eff5003f040 comp rx=0 tx=0).stop 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 msgr2=0x7eff681a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 0x7eff681a1680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff6000eb10 tx=0x7eff6000c2d0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 shutdown_connections 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7eff540b28a0 0x7eff540b4db0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff68105210 0x7eff681a1680 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff681023c0 0x7eff6819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 --2- 172.21.15.19:0/2153024819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff681018c0 0x7eff6819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 >> 172.21.15.19:0/2153024819 conn(0x7eff680fd500 msgr2=0x7eff68105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 shutdown_connections 2024-05-17T09:33:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:45.398+0000 7eff6e57d640 1 -- 172.21.15.19:0/2153024819 wait complete. 2024-05-17T09:33:45.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.71 MB/sec execute 583 sec latency 8061.294 ms 2024-05-17T09:33:46.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.71 MB/sec execute 584 sec latency 9061.363 ms 2024-05-17T09:33:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:46 smithi069 ceph-mon[35081]: pgmap v858: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 567 KiB/s wr, 83 op/s 2024-05-17T09:33:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:46 smithi019 ceph-mon[31620]: pgmap v858: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 567 KiB/s wr, 83 op/s 2024-05-17T09:33:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:46 smithi079 ceph-mon[35502]: pgmap v858: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 567 KiB/s wr, 83 op/s 2024-05-17T09:33:47.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.71 MB/sec execute 585 sec latency 10061.465 ms 2024-05-17T09:33:48.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.71 MB/sec execute 586 sec latency 11061.537 ms 2024-05-17T09:33:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:48 smithi069 ceph-mon[35081]: pgmap v859: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 467 KiB/s wr, 63 op/s 2024-05-17T09:33:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:48 smithi019 ceph-mon[31620]: pgmap v859: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 467 KiB/s wr, 63 op/s 2024-05-17T09:33:48.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:48 smithi079 ceph-mon[35502]: pgmap v859: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 467 KiB/s wr, 63 op/s 2024-05-17T09:33:49.453 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.70 MB/sec execute 587 sec latency 12061.631 ms 2024-05-17T09:33:50.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.70 MB/sec execute 588 sec latency 13061.738 ms 2024-05-17T09:33:50.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:50 smithi069 ceph-mon[35081]: pgmap v860: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 221 KiB/s wr, 61 op/s 2024-05-17T09:33:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:50 smithi019 ceph-mon[31620]: pgmap v860: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 221 KiB/s wr, 61 op/s 2024-05-17T09:33:50.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:50 smithi079 ceph-mon[35502]: pgmap v860: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 221 KiB/s wr, 61 op/s 2024-05-17T09:33:51.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.70 MB/sec execute 589 sec latency 14061.840 ms 2024-05-17T09:33:52.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.69 MB/sec execute 590 sec latency 15061.891 ms 2024-05-17T09:33:52.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:52 smithi069 ceph-mon[35081]: pgmap v861: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-05-17T09:33:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:52 smithi019 ceph-mon[31620]: pgmap v861: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-05-17T09:33:52.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:52 smithi079 ceph-mon[35502]: pgmap v861: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-05-17T09:33:52.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:33:53.118 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:53.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.69 MB/sec execute 591 sec latency 16061.922 ms 2024-05-17T09:33:54.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.69 MB/sec execute 592 sec latency 17061.956 ms 2024-05-17T09:33:54.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:54 smithi019 ceph-mon[31620]: pgmap v862: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-05-17T09:33:54.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:54 smithi079 ceph-mon[35502]: pgmap v862: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-05-17T09:33:54.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:54 smithi069 ceph-mon[35081]: pgmap v862: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 -- 172.21.15.19:0/1813944577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 msgr2=0x7fc960110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 --2- 172.21.15.19:0/1813944577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc960110090 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fc9580099c0 tx=0x7fc958009cd0 comp rx=0 tx=0).stop 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 -- 172.21.15.19:0/1813944577 shutdown_connections 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 --2- 172.21.15.19:0/1813944577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9601034b0 0x7fc9601038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 --2- 172.21.15.19:0/1813944577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc960110090 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 --2- 172.21.15.19:0/1813944577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc960107de0 0x7fc9601081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 -- 172.21.15.19:0/1813944577 >> 172.21.15.19:0/1813944577 conn(0x7fc9600fedb0 msgr2=0x7fc9601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 -- 172.21.15.19:0/1813944577 shutdown_connections 2024-05-17T09:33:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.997+0000 7fc967d36700 1 -- 172.21.15.19:0/1813944577 wait complete. 2024-05-17T09:33:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 Processor -- start 2024-05-17T09:33:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 -- start start 2024-05-17T09:33:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9601034b0 0x7fc9601a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc960107de0 0x7fc9601a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc96019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc960113260 con 0x7fc960108730 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601130e0 con 0x7fc960107de0 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc967d36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601133e0 con 0x7fc9601034b0 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc9662d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc96019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc9662d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc96019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38708/0 (socket says 172.21.15.19:38708) 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc9662d3700 1 -- 172.21.15.19:0/89209471 learned_addr learned my addr 172.21.15.19:0/89209471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc9652d1700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc960107de0 0x7fc9601a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.998+0000 7fc965ad2700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9601034b0 0x7fc9601a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc9662d3700 1 -- 172.21.15.19:0/89209471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9601034b0 msgr2=0x7fc9601a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc9662d3700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9601034b0 0x7fc9601a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc9662d3700 1 -- 172.21.15.19:0/89209471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc960107de0 msgr2=0x7fc9601a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc9662d3700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc960107de0 0x7fc9601a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc9662d3700 1 -- 172.21.15.19:0/89209471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96019b1d0 con 0x7fc960108730 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc9662d3700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc96019aaf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc95800b580 tx=0x7fc95800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958027070 con 0x7fc960108730 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96019b460 con 0x7fc960108730 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc95800fc70 con 0x7fc960108730 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958022d20 con 0x7fc960108730 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:54.999+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9601a7760 con 0x7fc960108730 2024-05-17T09:33:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.000+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc960104960 con 0x7fc960108730 2024-05-17T09:33:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.002+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc95802a070 con 0x7fc960108730 2024-05-17T09:33:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.002+0000 7fc956ffd700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc94c0b2880 0x7fc94c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.002+0000 7fc965ad2700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc94c0b2880 0x7fc94c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.002+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc9580e3360 con 0x7fc960108730 2024-05-17T09:33:55.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.003+0000 7fc965ad2700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc94c0b2880 0x7fc94c0b4d90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009550 tx=0x7fc95c0091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.006+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc958013070 con 0x7fc960108730 2024-05-17T09:33:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.275+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc96019bfe0 con 0x7fc960108730 2024-05-17T09:33:55.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.276+0000 7fc956ffd700 1 -- 172.21.15.19:0/89209471 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11853 (secure 0 0 0) 0x7fc9580aad60 con 0x7fc960108730 2024-05-17T09:33:55.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:33:55.275 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:25:07.624364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:33:55.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc94c0b2880 msgr2=0x7fc94c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:55.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc94c0b2880 0x7fc94c0b4d90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009550 tx=0x7fc95c0091f0 comp rx=0 tx=0).stop 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 msgr2=0x7fc96019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc96019aaf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc95800b580 tx=0x7fc95800f460 comp rx=0 tx=0).stop 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 shutdown_connections 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc94c0b2880 0x7fc94c0b4d90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9601034b0 0x7fc9601a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc960108730 0x7fc96019aaf0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 --2- 172.21.15.19:0/89209471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc960107de0 0x7fc9601a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 >> 172.21.15.19:0/89209471 conn(0x7fc9600fedb0 msgr2=0x7fc96006b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 shutdown_connections 2024-05-17T09:33:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:55.278+0000 7fc967d36700 1 -- 172.21.15.19:0/89209471 wait complete. 2024-05-17T09:33:55.279 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-17T09:33:55.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.69 MB/sec execute 593 sec latency 18062.070 ms 2024-05-17T09:33:55.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:55 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/89209471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:55.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:55 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/89209471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:55.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:55 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/89209471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:33:56.095 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 388fcb62 --release --await' 2024-05-17T09:33:56.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 388fcb62 --release --await 2024-05-17T09:33:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:33:56.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.68 MB/sec execute 594 sec latency 19062.128 ms 2024-05-17T09:33:56.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:56 smithi019 ceph-mon[31620]: pgmap v863: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-17T09:33:56.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:56 smithi079 ceph-mon[35502]: pgmap v863: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-17T09:33:56.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:56 smithi069 ceph-mon[35081]: pgmap v863: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-17T09:33:57.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228614 1.68 MB/sec execute 595 sec latency 20062.203 ms 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 -- 172.21.15.19:0/3585315185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8108740 msgr2=0x7fbcb81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/3585315185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8108740 0x7fbcb81100a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000b0d0 tx=0x7fbcb000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 -- 172.21.15.19:0/3585315185 shutdown_connections 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/3585315185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8108740 0x7fbcb81100a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/3585315185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb8103510 0x7fbcb8103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/3585315185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb8107df0 0x7fbcb8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 -- 172.21.15.19:0/3585315185 >> 172.21.15.19:0/3585315185 conn(0x7fbcb80fef80 msgr2=0x7fbcb81013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 -- 172.21.15.19:0/3585315185 shutdown_connections 2024-05-17T09:33:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 -- 172.21.15.19:0/3585315185 wait complete. 2024-05-17T09:33:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.112+0000 7fbcbd06f700 1 Processor -- start 2024-05-17T09:33:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 -- start start 2024-05-17T09:33:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb8103510 0x7fbcb819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb8108740 0x7fbcb819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8113220 con 0x7fbcb8107df0 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81130a0 con 0x7fbcb8103510 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcbd06f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81133a0 con 0x7fbcb8108740 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38726/0 (socket says 172.21.15.19:38726) 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcb659c700 1 -- 172.21.15.19:0/46083970 learned_addr learned my addr 172.21.15.19:0/46083970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcb759e700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb8108740 0x7fbcb819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.113+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb8103510 0x7fbcb819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcb659c700 1 -- 172.21.15.19:0/46083970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb8108740 msgr2=0x7fbcb819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcb659c700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb8108740 0x7fbcb819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcb659c700 1 -- 172.21.15.19:0/46083970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb8103510 msgr2=0x7fbcb819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcb659c700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb8103510 0x7fbcb819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcb659c700 1 -- 172.21.15.19:0/46083970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb81a6e90 con 0x7fbcb8107df0 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcb659c700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c080 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00d980 tx=0x7fbcac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbc9ffff700 1 -- 172.21.15.19:0/46083970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac017070 con 0x7fbcb8107df0 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb81a7180 con 0x7fbcb8107df0 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbc9ffff700 1 -- 172.21.15.19:0/46083970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcac010460 con 0x7fbcb8107df0 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbc9ffff700 1 -- 172.21.15.19:0/46083970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00de60 con 0x7fbcb8107df0 2024-05-17T09:33:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.114+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81a76c0 con 0x7fbcb8107df0 2024-05-17T09:33:58.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.117+0000 7fbc9ffff700 1 -- 172.21.15.19:0/46083970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbcac00f620 con 0x7fbcb8107df0 2024-05-17T09:33:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.118+0000 7fbc9ffff700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbca00b2880 0x7fbca00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.118+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbca00b2880 0x7fbca00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.118+0000 7fbc9ffff700 1 -- 172.21.15.19:0/46083970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbcac0e2100 con 0x7fbcb8107df0 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.118+0000 7fbccba84b80 1 Processor -- start 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 -- start start 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbca00b2880 0x7fbca00b4d90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005fd0 tx=0x7fbca800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c6e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0c68620 0x55ffd0c68a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ffd0b96d90 con 0x55ffd0ae9f40 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ffd0b86930 con 0x55ffd0b674d0 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbccba84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ffd0b8a970 con 0x55ffd0c68620 2024-05-17T09:33:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38730/0 (socket says 172.21.15.19:38730) 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 -- 172.21.15.19:0/3296077166 learned_addr learned my addr 172.21.15.19:0/3296077166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c6e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb759e700 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0c68620 0x55ffd0c68a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 -- 172.21.15.19:0/3296077166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0c68620 msgr2=0x55ffd0c68a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0c68620 0x55ffd0c68a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 -- 172.21.15.19:0/3296077166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 msgr2=0x55ffd0c6e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c6e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.119+0000 7fbcb659c700 1 -- 172.21.15.19:0/3296077166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd0ad69d0 con 0x55ffd0ae9f40 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.120+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6eaa0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0a9350 tx=0x7fbcac0a8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59651ee5960bc122 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.120+0000 7fbc9d7fa700 1 -- 172.21.15.19:0/3296077166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac020030 con 0x55ffd0ae9f40 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.120+0000 7fbc9d7fa700 1 -- 172.21.15.19:0/3296077166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcac0a9a50 con 0x55ffd0ae9f40 2024-05-17T09:33:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.120+0000 7fbc9d7fa700 1 -- 172.21.15.19:0/3296077166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0a8880 con 0x55ffd0ae9f40 2024-05-17T09:33:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 -- 172.21.15.19:0/3296077166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 msgr2=0x55ffd0c6eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6eaa0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0a9350 tx=0x7fbcac0a8c00 comp rx=0 tx=0).stop 2024-05-17T09:33:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 -- 172.21.15.19:0/3296077166 shutdown_connections 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0c68620 0x55ffd0c68a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c6e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3296077166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6eaa0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 -- 172.21.15.19:0/3296077166 >> 172.21.15.19:0/3296077166 conn(0x55ffd0b28d80 msgr2=0x55ffd0c63520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 -- 172.21.15.19:0/3296077166 shutdown_connections 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 -- 172.21.15.19:0/3296077166 wait complete. 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 -- start start 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.121+0000 7fbccba84b80 1 Processor -- start 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0c72730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0c68620 0x55ffd0c72c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ffd0a8d0a0 con 0x55ffd0b674d0 2024-05-17T09:33:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ffd0b288c0 con 0x55ffd0c68620 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ffd0c69440 con 0x55ffd0ae9f40 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbccba84b80 10 client.0 _ll_register_callbacks cb 0x55ffd0b21a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0c72730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0c72730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38744/0 (socket says 172.21.15.19:38744) 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 -- 172.21.15.19:0/3663359641 learned_addr learned my addr 172.21.15.19:0/3663359641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb759e700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0c68620 0x55ffd0c72c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 -- 172.21.15.19:0/3663359641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 msgr2=0x55ffd0c721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 -- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0c68620 msgr2=0x55ffd0c72c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0c68620 0x55ffd0c72c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 -- 172.21.15.19:0/3663359641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd0b25790 con 0x55ffd0b674d0 2024-05-17T09:33:58.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb759e700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0c68620 0x55ffd0c72c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0c72730 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbcac010750 tx=0x7fbcac0b2080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12513027571c765c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.122+0000 7fbc9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbc9aff5700 1 -- 172.21.15.19:0/3663359641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0c2070 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbc9aff5700 1 -- 172.21.15.19:0/3663359641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbcac009d50 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcac0a9a50 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbc9aff5700 1 -- 172.21.15.19:0/3663359641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcac020070 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 10 client.17847 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ffd0c7fb10 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbc9aff5700 1 -- 172.21.15.19:0/3663359641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcac0a9a50 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbc9aff5700 1 -- 172.21.15.19:0/3663359641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbcac0a8090 con 0x55ffd0b674d0 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 10 client.17847 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 10 client.17847 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 20 client.17847 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c81fe0 0x55ffd0c84400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 4 client.17847 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.123+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd0b98ce0 con 0x55ffd0c81fe0 2024-05-17T09:33:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.124+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c81fe0 0x55ffd0c84400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.124+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c81fe0 0x55ffd0c84400 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6e1ab3a580f38b server_cookie=6d64d49137ed5d5e in_seq=0 out_seq=0 2024-05-17T09:33:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.124+0000 7fbc9aff5700 10 client.17847 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbc9aff5700 1 -- 172.21.15.19:0/3663359641 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ffd0b98ce0 con 0x55ffd0c81fe0 2024-05-17T09:33:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbc9aff5700 10 client.17847 handle_command_reply: tid=0 2024-05-17T09:33:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 client.17847 shutdown 2024-05-17T09:33:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 msgr2=0x55ffd0c72730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0c72730 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbcac010750 tx=0x7fbcac0b2080 comp rx=0 tx=0).stop 2024-05-17T09:33:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 shutdown_connections 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c81fe0 0x55ffd0c84400 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0c68620 0x55ffd0c72c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3663359641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0c72730 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbc9aff5700 0 client.17847 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 >> 172.21.15.19:0/3663359641 conn(0x55ffd0b28d80 msgr2=0x55ffd0c5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 shutdown_connections 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 1 -- 172.21.15.19:0/3663359641 wait complete. 2024-05-17T09:33:58.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.125+0000 7fbccba84b80 20 client.17847 trim_cache size 0 max 16384 2024-05-17T09:33:58.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 Processor -- start 2024-05-17T09:33:58.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 -- start start 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 0x55ffd0c7e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81b2c90 con 0x55ffd0b28d80 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81b0520 con 0x55ffd0b674d0 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbccba84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81ae370 con 0x55ffd0ae9f40 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 0x55ffd0c7e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 0x55ffd0c7e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38756/0 (socket says 172.21.15.19:38756) 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 -- 172.21.15.19:0/3885321222 learned_addr learned my addr 172.21.15.19:0/3885321222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb659c700 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb759e700 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 -- 172.21.15.19:0/3885321222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 msgr2=0x55ffd0c6dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 -- 172.21.15.19:0/3885321222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 msgr2=0x55ffd0c87430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c87430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.146+0000 7fbcb6d9d700 1 -- 172.21.15.19:0/3885321222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd0c732b0 con 0x55ffd0b28d80 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 0x55ffd0c7e300 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbca800a4a0 tx=0x7fbca8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba3f7d8053ae318 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbc9cff9700 1 -- 172.21.15.19:0/3885321222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8010d30 con 0x55ffd0b28d80 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbc9cff9700 1 -- 172.21.15.19:0/3885321222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbca8004cc0 con 0x55ffd0b28d80 2024-05-17T09:33:58.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbc9cff9700 1 -- 172.21.15.19:0/3885321222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca801e430 con 0x55ffd0b28d80 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbccba84b80 1 -- 172.21.15.19:0/3885321222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 msgr2=0x55ffd0c7e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 0x55ffd0c7e300 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbca800a4a0 tx=0x7fbca8014000 comp rx=0 tx=0).stop 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbccba84b80 1 -- 172.21.15.19:0/3885321222 shutdown_connections 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.147+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b674d0 0x55ffd0c87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b28d80 0x55ffd0c7e300 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 --2- 172.21.15.19:0/3885321222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0c6dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- 172.21.15.19:0/3885321222 >> 172.21.15.19:0/3885321222 conn(0x55ffd0c637b0 msgr2=0x55ffd0c5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- 172.21.15.19:0/3885321222 shutdown_connections 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- 172.21.15.19:0/3885321222 wait complete. 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- start start 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 Processor -- start 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0ce6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b28d80 0x55ffd0ce6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0ce0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81ae9e0 con 0x55ffd0b674d0 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81afe50 con 0x55ffd0b28d80 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81af0b0 con 0x55ffd0ae9f40 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbccba84b80 10 client.0 _ll_register_callbacks cb 0x55ffd0b21a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.148+0000 7fbcb759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0ce0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0ce0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38772/0 (socket says 172.21.15.19:38772) 2024-05-17T09:33:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 -- 172.21.15.19:0/1006851315 learned_addr learned my addr 172.21.15.19:0/1006851315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0ce6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb659c700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b28d80 0x55ffd0ce6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 -- 172.21.15.19:0/1006851315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 msgr2=0x55ffd0ce6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0ce6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 -- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b28d80 msgr2=0x55ffd0ce6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b28d80 0x55ffd0ce6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 -- 172.21.15.19:0/1006851315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd0ce1390 con 0x55ffd0b674d0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbcb759e700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0ce0d20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000b0d0 tx=0x7fbcb0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a656510a40aee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbc9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbc9aff5700 1 -- 172.21.15.19:0/1006851315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00077c0 con 0x55ffd0b674d0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbc9aff5700 1 -- 172.21.15.19:0/1006851315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbcb0007960 con 0x55ffd0b674d0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca8004cc0 con 0x55ffd0b674d0 2024-05-17T09:33:58.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbc9aff5700 1 -- 172.21.15.19:0/1006851315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcb002f050 con 0x55ffd0b674d0 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbccba84b80 10 client.17853 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.149+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ffd0cee030 con 0x55ffd0b674d0 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbc9aff5700 1 -- 172.21.15.19:0/1006851315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbca8004cc0 con 0x55ffd0b674d0 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbc9aff5700 1 -- 172.21.15.19:0/1006851315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbcb0011070 con 0x55ffd0b674d0 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbccba84b80 10 client.17853 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbccba84b80 10 client.17853 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbccba84b80 20 client.17853 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbccba84b80 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c68620 0x55ffd0cf06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbccba84b80 4 client.17853 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "388fcb62", "release": true, "await": true}] 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "388fcb62", "release": true, "await": true}) -- 0x55ffd0b924c0 con 0x55ffd0c68620 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.150+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c68620 0x55ffd0cf06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.151+0000 7fbcb6d9d700 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c68620 0x55ffd0cf06f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45da55946ae908ad server_cookie=51646454e65dd9a8 in_seq=0 out_seq=0 2024-05-17T09:33:58.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.151+0000 7fbc9aff5700 10 client.17853 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbc9aff5700 1 -- 172.21.15.19:0/1006851315 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ffd0b924c0 con 0x55ffd0c68620 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbc9aff5700 10 client.17853 handle_command_reply: tid=0 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 client.17853 shutdown 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 msgr2=0x55ffd0ce0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0ce0d20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000b0d0 tx=0x7fbcb0004c80 comp rx=0 tx=0).stop 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 shutdown_connections 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ffd0ae9f40 0x55ffd0ce6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ffd0c68620 0x55ffd0cf06f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ffd0b674d0 0x55ffd0ce0d20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 --2- 172.21.15.19:0/1006851315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffd0b28d80 0x55ffd0ce6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 >> 172.21.15.19:0/1006851315 conn(0x55ffd0c637b0 msgr2=0x55ffd0c73680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbc9aff5700 0 client.17853 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 shutdown_connections 2024-05-17T09:33:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.155+0000 7fbccba84b80 1 -- 172.21.15.19:0/1006851315 wait complete. 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbccba84b80 20 client.17853 trim_cache size 0 max 16384 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 52, 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "388fcb62": { 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 52, 2024-05-17T09:33:58.156 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.9, 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:33:58.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:58.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:58.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:58.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:58.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:33:58.158 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbca00b2880 msgr2=0x7fbca00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbca00b2880 0x7fbca00b4d90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005fd0 tx=0x7fbca800a560 comp rx=0 tx=0).stop 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 msgr2=0x7fbcb819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c080 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00d980 tx=0x7fbcac00dd40 comp rx=0 tx=0).stop 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 shutdown_connections 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbca00b2880 0x7fbca00b4d90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbcb8108740 0x7fbcb819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb8103510 0x7fbcb819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 --2- 172.21.15.19:0/46083970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb8107df0 0x7fbcb819c080 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 >> 172.21.15.19:0/46083970 conn(0x7fbcb80fef80 msgr2=0x7fbcb8107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 shutdown_connections 2024-05-17T09:33:58.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:33:58.156+0000 7fbcbd06f700 1 -- 172.21.15.19:0/46083970 wait complete. 2024-05-17T09:33:58.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228753 1.68 MB/sec execute 596 sec latency 20766.309 ms 2024-05-17T09:33:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 -- 172.21.15.19:0/985681268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c103da0 msgr2=0x7f7c2c1041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 --2- 172.21.15.19:0/985681268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1041b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009900 tx=0x7f7c1402a410 comp rx=0 tx=0).stop 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 -- 172.21.15.19:0/985681268 shutdown_connections 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 --2- 172.21.15.19:0/985681268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 --2- 172.21.15.19:0/985681268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1041b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 --2- 172.21.15.19:0/985681268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c102b60 0x7f7c2c103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 -- 172.21.15.19:0/985681268 >> 172.21.15.19:0/985681268 conn(0x7f7c2c0fdd50 msgr2=0x7f7c2c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 -- 172.21.15.19:0/985681268 shutdown_connections 2024-05-17T09:33:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.509+0000 7f7c30815640 1 -- 172.21.15.19:0/985681268 wait complete. 2024-05-17T09:33:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 Processor -- start 2024-05-17T09:33:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 -- start start 2024-05-17T09:33:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c2c102b60 0x7f7c2c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c1a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10e530 con 0x7f7c2c108aa0 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10e3b0 con 0x7f7c2c102b60 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.510+0000 7f7c30815640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10e6b0 con 0x7f7c2c103da0 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c1a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c1a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38796/0 (socket says 172.21.15.19:38796) 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3387737601 learned_addr learned my addr 172.21.15.19:0/3387737601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c2c102b60 0x7f7c2c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c23fff640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3387737601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c103da0 msgr2=0x7f7c2c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3387737601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c2c102b60 msgr2=0x7f7c2c06bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c2c102b60 0x7f7c2c06bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3387737601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c1a3ea0 con 0x7f7c2c108aa0 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c23fff640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c1a3910 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400cce0 tx=0x7f7c2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.511+0000 7f7c237fe640 1 -- 172.21.15.19:0/3387737601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400f450 con 0x7f7c2c108aa0 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.512+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c1a4190 con 0x7f7c2c108aa0 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.512+0000 7f7c237fe640 1 -- 172.21.15.19:0/3387737601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c2400fa80 con 0x7f7c2c108aa0 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.512+0000 7f7c237fe640 1 -- 172.21.15.19:0/3387737601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24018ad0 con 0x7f7c2c108aa0 2024-05-17T09:33:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.512+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c1a46e0 con 0x7f7c2c108aa0 2024-05-17T09:33:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.515+0000 7f7c377a4740 1 Processor -- start 2024-05-17T09:33:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.515+0000 7f7c237fe640 1 -- 172.21.15.19:0/3387737601 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7c2400f5f0 con 0x7f7c2c108aa0 2024-05-17T09:33:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 -- start start 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d295bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d296130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cecb4d0 0x56043d29d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56043d1ffc30 con 0x56043ceec1b0 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56043d1ef7d0 con 0x56043cec32f0 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c377a4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56043d1f3810 con 0x56043cecb4d0 2024-05-17T09:33:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d295bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d295bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38808/0 (socket says 172.21.15.19:38808) 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c23fff640 1 -- 172.21.15.19:0/2309373415 learned_addr learned my addr 172.21.15.19:0/2309373415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c2a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d296130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.516+0000 7f7c23fff640 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cecb4d0 0x56043d29d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c2a59c640 1 -- 172.21.15.19:0/2309373415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cecb4d0 msgr2=0x56043d29d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cecb4d0 0x56043d29d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c2a59c640 1 -- 172.21.15.19:0/2309373415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 msgr2=0x56043d295bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d295bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c2a59c640 1 -- 172.21.15.19:0/2309373415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043cf04210 con 0x56043cec32f0 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c237fe640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c180b28a0 0x7f7c180b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d296130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c103760 tx=0x7f7c1c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2949f082c5c239 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c23fff640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c180b28a0 0x7f7c180b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c237fe640 1 -- 172.21.15.19:0/3387737601 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7c240e2ce0 con 0x7f7c2c108aa0 2024-05-17T09:33:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c20ff9640 1 -- 172.21.15.19:0/2309373415 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c01c480 con 0x56043cec32f0 2024-05-17T09:33:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c20ff9640 1 -- 172.21.15.19:0/2309373415 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c1c0118d0 con 0x56043cec32f0 2024-05-17T09:33:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.517+0000 7f7c20ff9640 1 -- 172.21.15.19:0/2309373415 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c01a600 con 0x56043cec32f0 2024-05-17T09:33:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 -- 172.21.15.19:0/2309373415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 msgr2=0x56043d296130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c23fff640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c180b28a0 0x7f7c180b4db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7c140319f0 tx=0x7f7c14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d296130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c103760 tx=0x7f7c1c0107a0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 -- 172.21.15.19:0/2309373415 shutdown_connections 2024-05-17T09:33:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cecb4d0 0x56043d29d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d296130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 --2- 172.21.15.19:0/2309373415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d295bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 -- 172.21.15.19:0/2309373415 >> 172.21.15.19:0/2309373415 conn(0x56043cee0300 msgr2=0x56043d291540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 -- 172.21.15.19:0/2309373415 shutdown_connections 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.518+0000 7f7c377a4740 1 -- 172.21.15.19:0/2309373415 wait complete. 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 -- start start 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 Processor -- start 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d29f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cecb4d0 0x56043d29f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d2a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56043ced5790 con 0x56043ceec1b0 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56043d1ab890 con 0x56043cecb4d0 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56043d29da80 con 0x56043cec32f0 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c377a4740 10 client.0 _ll_register_callbacks cb 0x56043d1aed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d2a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d2a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38822/0 (socket says 172.21.15.19:38822) 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.519+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3234612413 learned_addr learned my addr 172.21.15.19:0/3234612413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c23fff640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cecb4d0 0x56043d29f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d29f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3234612413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 msgr2=0x56043d29f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d29f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cecb4d0 msgr2=0x56043d29f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cecb4d0 0x56043d29f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/3234612413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043cfd0620 con 0x56043ceec1b0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d2a6940 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7c240a9c50 tx=0x7f7c2400f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02dc55c593d2abd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c11ffb640 1 -- 172.21.15.19:0/3234612413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240a9f50 con 0x56043ceec1b0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1c0118d0 con 0x56043ceec1b0 2024-05-17T09:33:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c11ffb640 1 -- 172.21.15.19:0/3234612413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7c240b2920 con 0x56043ceec1b0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c11ffb640 1 -- 172.21.15.19:0/3234612413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7c240ae300 con 0x56043ceec1b0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c377a4740 10 client.17865 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.520+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56043d2a98e0 con 0x56043ceec1b0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c11ffb640 1 -- 172.21.15.19:0/3234612413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c1c0118d0 con 0x56043ceec1b0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c11ffb640 1 -- 172.21.15.19:0/3234612413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7c240a8c20 con 0x56043ceec1b0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c377a4740 10 client.17865 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c377a4740 10 client.17865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c377a4740 20 client.17865 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c377a4740 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ceaad80 0x56043d2ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c377a4740 4 client.17865 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56043d201b80 con 0x56043ceaad80 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.521+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ceaad80 0x56043d2ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.522+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ceaad80 0x56043d2ad010 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc30d64723c7f985 server_cookie=94b917963495391a in_seq=0 out_seq=0 2024-05-17T09:33:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.522+0000 7f7c11ffb640 10 client.17865 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c11ffb640 1 -- 172.21.15.19:0/3234612413 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56043d201b80 con 0x56043ceaad80 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c11ffb640 10 client.17865 handle_command_reply: tid=0 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 client.17865 shutdown 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 msgr2=0x56043d2a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d2a6940 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7c240a9c50 tx=0x7f7c2400f8e0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 shutdown_connections 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ceaad80 0x56043d2ad010 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043ceec1b0 0x56043d2a6940 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c11ffb640 0 client.17865 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cecb4d0 0x56043d29f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 --2- 172.21.15.19:0/3234612413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d29f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 >> 172.21.15.19:0/3234612413 conn(0x56043cee0300 msgr2=0x56043d294450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 shutdown_connections 2024-05-17T09:33:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 1 -- 172.21.15.19:0/3234612413 wait complete. 2024-05-17T09:33:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.523+0000 7f7c377a4740 20 client.17865 trim_cache size 0 max 16384 2024-05-17T09:33:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.540+0000 7f7c377a4740 1 Processor -- start 2024-05-17T09:33:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.540+0000 7f7c377a4740 1 -- start start 2024-05-17T09:33:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.540+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d287d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043ceec1b0 0x56043d299fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c377a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1ace80 con 0x56043cecb4d0 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c377a4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1aa750 con 0x56043cec32f0 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c377a4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1a85a0 con 0x56043ceec1b0 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38828/0 (socket says 172.21.15.19:38828) 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c23fff640 1 -- 172.21.15.19:0/883117717 learned_addr learned my addr 172.21.15.19:0/883117717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043ceec1b0 0x56043d299fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d287d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/883117717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 msgr2=0x56043d287d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d287d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/883117717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 msgr2=0x56043d2a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 -- 172.21.15.19:0/883117717 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043d2a05e0 con 0x56043ceec1b0 2024-05-17T09:33:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c23fff640 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043ceec1b0 0x56043d299fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c240a71c0 tx=0x7f7c2400fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62036ace14dc123 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c13fff640 1 -- 172.21.15.19:0/883117717 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240a9f50 con 0x56043ceec1b0 2024-05-17T09:33:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c13fff640 1 -- 172.21.15.19:0/883117717 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c240b2db0 con 0x56043ceec1b0 2024-05-17T09:33:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.541+0000 7f7c13fff640 1 -- 172.21.15.19:0/883117717 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240af7e0 con 0x56043ceec1b0 2024-05-17T09:33:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 -- 172.21.15.19:0/883117717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043ceec1b0 msgr2=0x56043d299fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043ceec1b0 0x56043d299fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c240a71c0 tx=0x7f7c2400fa10 comp rx=0 tx=0).stop 2024-05-17T09:33:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 -- 172.21.15.19:0/883117717 shutdown_connections 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043ceec1b0 0x56043d299fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 --2- 172.21.15.19:0/883117717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043cec32f0 0x56043d287d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 -- 172.21.15.19:0/883117717 >> 172.21.15.19:0/883117717 conn(0x56043cee0300 msgr2=0x56043d294340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.542+0000 7f7c377a4740 1 -- 172.21.15.19:0/883117717 shutdown_connections 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 -- 172.21.15.19:0/883117717 wait complete. 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 -- start start 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 Processor -- start 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d2a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043ceec1b0 0x56043d2a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1a8c10 con 0x56043cecb4d0 2024-05-17T09:33:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1aa080 con 0x56043ceec1b0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1a92e0 con 0x56043cec32f0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c377a4740 10 client.0 _ll_register_callbacks cb 0x7f7c240a98a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38838/0 (socket says 172.21.15.19:38838) 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c23fff640 1 -- 172.21.15.19:0/1146485592 learned_addr learned my addr 172.21.15.19:0/1146485592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d2a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.543+0000 7f7c2ad9d640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043ceec1b0 0x56043d2a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c23fff640 1 -- 172.21.15.19:0/1146485592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 msgr2=0x56043d2a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c23fff640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d2a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c23fff640 1 -- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043ceec1b0 msgr2=0x56043d2a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c23fff640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043ceec1b0 0x56043d2a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c23fff640 1 -- 172.21.15.19:0/1146485592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043d2a60f0 con 0x56043cecb4d0 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d2a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:33:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c23fff640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a4c00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7c140057c0 tx=0x7f7c1402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86442ae631aa6dc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c11ffb640 1 -- 172.21.15.19:0/1146485592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14035040 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c11ffb640 1 -- 172.21.15.19:0/1146485592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7c14046070 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c240b2db0 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c11ffb640 1 -- 172.21.15.19:0/1146485592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7c1405e050 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c377a4740 10 client.17871 fetch_fsmap learned FSMap version 32 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.544+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56043d315c30 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c11ffb640 1 -- 172.21.15.19:0/1146485592 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c240b2db0 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c11ffb640 1 -- 172.21.15.19:0/1146485592 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7c140584c0 con 0x56043cecb4d0 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c377a4740 10 client.17871 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c377a4740 10 client.17871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c377a4740 20 client.17871 populate_metadata read hostname 'smithi019' 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c377a4740 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ce801c0 0x56043d318650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:33:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c377a4740 4 client.17871 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:33:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56043d1fb360 con 0x56043ce801c0 2024-05-17T09:33:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ce801c0 0x56043d318650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:33:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c11ffb640 10 client.17871 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.545+0000 7f7c2a59c640 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ce801c0 0x56043d318650 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb50c77f9b734a65 server_cookie=bc36b4ad6647d0e7 in_seq=0 out_seq=0 2024-05-17T09:33:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c11ffb640 1 -- 172.21.15.19:0/1146485592 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56043d1fb360 con 0x56043ce801c0 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c11ffb640 10 client.17871 handle_command_reply: tid=0 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 client.17871 shutdown 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 msgr2=0x56043d2a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a4c00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7c140057c0 tx=0x7f7c1402a410 comp rx=0 tx=0).stop 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 shutdown_connections 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56043ce801c0 0x56043d318650 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56043ceec1b0 0x56043d2a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043cecb4d0 0x56043d2a4c00 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c11ffb640 0 client.17871 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 --2- 172.21.15.19:0/1146485592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56043cec32f0 0x56043d2a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 >> 172.21.15.19:0/1146485592 conn(0x56043cee0300 msgr2=0x56043d294d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.571+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 shutdown_connections 2024-05-17T09:33:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c377a4740 1 -- 172.21.15.19:0/1146485592 wait complete. 2024-05-17T09:33:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c377a4740 20 client.17871 trim_cache size 0 max 16384 2024-05-17T09:33:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:33:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:33:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c180b28a0 msgr2=0x7f7c180b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c180b28a0 0x7f7c180b4db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7c140319f0 tx=0x7f7c14005fd0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 msgr2=0x7f7c2c1a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c1a3910 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400cce0 tx=0x7f7c2400a620 comp rx=0 tx=0).stop 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 shutdown_connections 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7c180b28a0 0x7f7c180b4db0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c108aa0 0x7f7c2c1a3910 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2c103da0 0x7f7c2c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 --2- 172.21.15.19:0/3387737601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c2c102b60 0x7f7c2c06bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:33:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 >> 172.21.15.19:0/3387737601 conn(0x7f7c2c0fdd50 msgr2=0x7f7c2c104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:33:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 shutdown_connections 2024-05-17T09:33:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:33:58.572+0000 7f7c30815640 1 -- 172.21.15.19:0/3387737601 wait complete. 2024-05-17T09:33:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:33:58 smithi019 ceph-mon[31620]: pgmap v864: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-05-17T09:33:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:33:58 smithi079 ceph-mon[35502]: pgmap v864: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-05-17T09:33:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:33:58 smithi069 ceph-mon[35081]: pgmap v864: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-05-17T09:33:59.049 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '388fcb62', total seconds elapsed: 27.1 2024-05-17T09:33:59.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 229390 1.68 MB/sec execute 597 sec latency 16.063 ms 2024-05-17T09:34:00.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 229933 1.68 MB/sec execute 598 sec latency 15.458 ms 2024-05-17T09:34:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:00 smithi079 ceph-mon[35502]: pgmap v865: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-05-17T09:34:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:00 smithi069 ceph-mon[35081]: pgmap v865: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-05-17T09:34:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:00 smithi019 ceph-mon[31620]: pgmap v865: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-05-17T09:34:01.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 230388 1.68 MB/sec execute 599 sec latency 16.341 ms 2024-05-17T09:34:02.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 cleanup 600 sec 2024-05-17T09:34:02.732 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:02 smithi069 ceph-mon[35081]: pgmap v866: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-17T09:34:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:02 smithi079 ceph-mon[35502]: pgmap v866: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-17T09:34:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:02 smithi019 ceph-mon[31620]: pgmap v866: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-17T09:34:03.454 INFO:tasks.workunit.client.0.smithi019.stdout: 1 cleanup 601 sec 2024-05-17T09:34:03.458 INFO:tasks.workunit.client.0.smithi019.stdout: 0 cleanup 601 sec 2024-05-17T09:34:03.458 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-17T09:34:03.458 INFO:tasks.workunit.client.0.smithi019.stdout: Operation Count AvgLat MaxLat 2024-05-17T09:34:03.458 INFO:tasks.workunit.client.0.smithi019.stdout: ---------------------------------------- 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: NTCreateX 32534 3.834 84.212 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Close 23886 0.054 2.523 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Rename 1378 7.495 59.865 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Unlink 6579 23.994 26815.554 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Qpathinfo 29497 3.131 81.891 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Qfileinfo 5141 0.091 54.928 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Qfsinfo 5407 0.336 5.635 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Sfileinfo 2650 15.851 20318.565 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: Find 11395 10.785 96.200 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: WriteX 16064 2.958 22904.349 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: ReadX 51018 0.004 0.618 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: LockX 106 0.699 1.111 2024-05-17T09:34:03.459 INFO:tasks.workunit.client.0.smithi019.stdout: UnlockX 106 0.586 0.991 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout: Flush 2279 3.492 125.812 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout:Throughput 1.68466 MB/sec 1 clients 1 procs max_latency=26815.563 ms 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stderr:+ dbench 10 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-17T09:34:03.460 INFO:tasks.workunit.client.0.smithi019.stdout:8 of 10 processes prepared for launch 0 sec 2024-05-17T09:34:03.560 INFO:tasks.workunit.client.0.smithi019.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-17T09:34:03.560 INFO:tasks.workunit.client.0.smithi019.stdout:releasing clients 2024-05-17T09:34:04.560 INFO:tasks.workunit.client.0.smithi019.stdout: 10 254 96.32 MB/sec warmup 1 sec latency 59.496 ms 2024-05-17T09:34:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:04 smithi079 ceph-mon[35502]: pgmap v867: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 288 KiB/s wr, 18 op/s 2024-05-17T09:34:04.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:04 smithi069 ceph-mon[35081]: pgmap v867: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 288 KiB/s wr, 18 op/s 2024-05-17T09:34:04.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:04 smithi019 ceph-mon[31620]: pgmap v867: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 288 KiB/s wr, 18 op/s 2024-05-17T09:34:05.560 INFO:tasks.workunit.client.0.smithi019.stdout: 10 462 80.96 MB/sec warmup 2 sec latency 77.471 ms 2024-05-17T09:34:06.560 INFO:tasks.workunit.client.0.smithi019.stdout: 10 666 70.07 MB/sec warmup 3 sec latency 107.310 ms 2024-05-17T09:34:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:06 smithi079 ceph-mon[35502]: pgmap v868: 129 pgs: 129 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-05-17T09:34:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:06 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:06.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:06 smithi069 ceph-mon[35081]: pgmap v868: 129 pgs: 129 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-05-17T09:34:06.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:06 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:06 smithi019 ceph-mon[31620]: pgmap v868: 129 pgs: 129 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-05-17T09:34:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:06 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:07.560 INFO:tasks.workunit.client.0.smithi019.stdout: 10 800 53.63 MB/sec warmup 4 sec latency 129.114 ms 2024-05-17T09:34:08.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 915 43.49 MB/sec warmup 5 sec latency 318.423 ms 2024-05-17T09:34:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:08 smithi079 ceph-mon[35502]: pgmap v869: 129 pgs: 129 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 103 op/s 2024-05-17T09:34:08.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:08 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:08 smithi069 ceph-mon[35081]: pgmap v869: 129 pgs: 129 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 103 op/s 2024-05-17T09:34:09.009 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:08 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:09.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:08 smithi019 ceph-mon[31620]: pgmap v869: 129 pgs: 129 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 103 op/s 2024-05-17T09:34:09.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:08 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:09.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1015 37.94 MB/sec warmup 6 sec latency 170.200 ms 2024-05-17T09:34:10.560 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1103 32.65 MB/sec warmup 7 sec latency 201.727 ms 2024-05-17T09:34:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:10 smithi079 ceph-mon[35502]: pgmap v870: 129 pgs: 129 active+clean; 265 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 16 MiB/s wr, 173 op/s 2024-05-17T09:34:10.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:10 smithi069 ceph-mon[35081]: pgmap v870: 129 pgs: 129 active+clean; 265 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 16 MiB/s wr, 173 op/s 2024-05-17T09:34:10.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:10 smithi019 ceph-mon[31620]: pgmap v870: 129 pgs: 129 active+clean; 265 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 16 MiB/s wr, 173 op/s 2024-05-17T09:34:11.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1207 28.82 MB/sec warmup 8 sec latency 166.006 ms 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.697+0000 7f226ebc7640 1 -- 172.21.15.19:0/1722668328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268103c80 msgr2=0x7f2268104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.697+0000 7f226ebc7640 1 --2- 172.21.15.19:0/1722668328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268103c80 0x7f2268104980 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f225c009900 tx=0x7f225c02a410 comp rx=0 tx=0).stop 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 -- 172.21.15.19:0/1722668328 shutdown_connections 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 --2- 172.21.15.19:0/1722668328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268109bc0 0x7f226810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 --2- 172.21.15.19:0/1722668328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2268104ec0 0x7f22681052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 --2- 172.21.15.19:0/1722668328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268103c80 0x7f2268104980 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 -- 172.21.15.19:0/1722668328 >> 172.21.15.19:0/1722668328 conn(0x7f22680fee90 msgr2=0x7f2268101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 -- 172.21.15.19:0/1722668328 shutdown_connections 2024-05-17T09:34:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 -- 172.21.15.19:0/1722668328 wait complete. 2024-05-17T09:34:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 Processor -- start 2024-05-17T09:34:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.698+0000 7f226ebc7640 1 -- start start 2024-05-17T09:34:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226ebc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 0x7f226819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226ebc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2268104ec0 0x7f226819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226ebc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268109bc0 0x7f22681a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226ebc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f226810f610 con 0x7f2268109bc0 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226ebc7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f226810f490 con 0x7f2268103c80 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226ebc7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f226810f790 con 0x7f2268104ec0 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 0x7f226819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226d164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268109bc0 0x7f22681a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 0x7f226819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51780/0 (socket says 172.21.15.19:51780) 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 -- 172.21.15.19:0/2653414841 learned_addr learned my addr 172.21.15.19:0/2653414841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f2267fff640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2268104ec0 0x7f226819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 -- 172.21.15.19:0/2653414841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2268104ec0 msgr2=0x7f226819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2268104ec0 0x7f226819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 -- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268109bc0 msgr2=0x7f22681a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268109bc0 0x7f22681a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 -- 172.21.15.19:0/2653414841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22681a1db0 con 0x7f2268103c80 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226d164640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268109bc0 0x7f22681a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.699+0000 7f226c963640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 0x7f226819a270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f225c005ab0 tx=0x7f225c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.700+0000 7f2265ffb640 1 -- 172.21.15.19:0/2653414841 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c03f030 con 0x7f2268103c80 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.700+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22681a2040 con 0x7f2268103c80 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.700+0000 7f2265ffb640 1 -- 172.21.15.19:0/2653414841 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f225c032e10 con 0x7f2268103c80 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.700+0000 7f2265ffb640 1 -- 172.21.15.19:0/2653414841 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c035070 con 0x7f2268103c80 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.700+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681a2580 con 0x7f2268103c80 2024-05-17T09:34:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.703+0000 7f2265ffb640 1 -- 172.21.15.19:0/2653414841 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f225c042030 con 0x7f2268103c80 2024-05-17T09:34:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.703+0000 7f2275a79740 1 Processor -- start 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.703+0000 7f2275a79740 1 -- start start 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2275a79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4cd1b0 0x5647cc86e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2275a79740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4a42f0 0x5647cc86e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2275a79740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc875820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2275a79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5647cc7e0c30 con 0x5647cc4cd1b0 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2275a79740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5647cc7d07d0 con 0x5647cc4a42f0 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2275a79740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5647cc7d4810 con 0x5647cc4ac4d0 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f226d164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4cd1b0 0x5647cc86e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2265ffb640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22580b28a0 0x7f22580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f226c963640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4a42f0 0x5647cc86e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f226d164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4cd1b0 0x5647cc86e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47186/0 (socket says 172.21.15.19:47186) 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f226d164640 1 -- 172.21.15.19:0/3578245442 learned_addr learned my addr 172.21.15.19:0/3578245442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2267fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc875820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f226c963640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4a42f0 0x5647cc86e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51784/0 (socket says 172.21.15.19:51784) 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f226d164640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22580b28a0 0x7f22580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2265ffb640 1 -- 172.21.15.19:0/2653414841 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f225c102920 con 0x7f2268103c80 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2267fff640 1 -- 172.21.15.19:0/3578245442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4a42f0 msgr2=0x5647cc86e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2267fff640 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4a42f0 0x5647cc86e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2267fff640 1 -- 172.21.15.19:0/3578245442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4cd1b0 msgr2=0x5647cc86e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2267fff640 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4cd1b0 0x5647cc86e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.704+0000 7f2267fff640 1 -- 172.21.15.19:0/3578245442 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cc4e5210 con 0x5647cc4ac4d0 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.705+0000 7f226d164640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22580b28a0 0x7f22580b4db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f22600099e0 tx=0x7f2260004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.705+0000 7f2267fff640 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc875820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2254009bf0 tx=0x7f2254011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb7ee3fe21abb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.705+0000 7f2252ffd640 1 -- 172.21.15.19:0/3578245442 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254009da0 con 0x5647cc4ac4d0 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.705+0000 7f2252ffd640 1 -- 172.21.15.19:0/3578245442 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2254005020 con 0x5647cc4ac4d0 2024-05-17T09:34:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.705+0000 7f2252ffd640 1 -- 172.21.15.19:0/3578245442 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540044c0 con 0x5647cc4ac4d0 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 -- 172.21.15.19:0/3578245442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 msgr2=0x5647cc875820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc875820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2254009bf0 tx=0x7f2254011810 comp rx=0 tx=0).stop 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 -- 172.21.15.19:0/3578245442 shutdown_connections 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc875820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4a42f0 0x5647cc86e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 --2- 172.21.15.19:0/3578245442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4cd1b0 0x5647cc86e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 -- 172.21.15.19:0/3578245442 >> 172.21.15.19:0/3578245442 conn(0x5647cc4c1300 msgr2=0x5647cc869d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 -- 172.21.15.19:0/3578245442 shutdown_connections 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 -- 172.21.15.19:0/3578245442 wait complete. 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.706+0000 7f2275a79740 1 -- start start 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 Processor -- start 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 0x5647cc880490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc8809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4cd1b0 0x5647cc887930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5647cc4b6790 con 0x5647cc4a42f0 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5647cc78c890 con 0x5647cc4cd1b0 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f226c963640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 0x5647cc880490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5647cc876270 con 0x5647cc4ac4d0 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f226c963640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 0x5647cc880490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47202/0 (socket says 172.21.15.19:47202) 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f226c963640 1 -- 172.21.15.19:0/1109915262 learned_addr learned my addr 172.21.15.19:0/1109915262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2275a79740 10 client.0 _ll_register_callbacks cb 0x5647cc78fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f2267fff640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc8809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.707+0000 7f226d164640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4cd1b0 0x5647cc887930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226c963640 1 -- 172.21.15.19:0/1109915262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 msgr2=0x5647cc8809d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226c963640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc8809d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226c963640 1 -- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4cd1b0 msgr2=0x5647cc887930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226c963640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4cd1b0 0x5647cc887930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226c963640 1 -- 172.21.15.19:0/1109915262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cc5b1620 con 0x5647cc4a42f0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226d164640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4cd1b0 0x5647cc887930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f226c963640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 0x5647cc880490 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c9bd0 tx=0x7f225c0c9ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f754651a2e2598ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2247fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2247fff640 1 -- 172.21.15.19:0/1109915262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c049040 con 0x5647cc4a42f0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2247fff640 1 -- 172.21.15.19:0/1109915262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f225c0c9ea0 con 0x5647cc4a42f0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2254005020 con 0x5647cc4a42f0 2024-05-17T09:34:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2247fff640 1 -- 172.21.15.19:0/1109915262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f225c0e9050 con 0x5647cc4a42f0 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2275a79740 10 client.17883 fetch_fsmap learned FSMap version 34 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.708+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5647cc88a860 con 0x5647cc4a42f0 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2247fff640 1 -- 172.21.15.19:0/1109915262 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2254005020 con 0x5647cc4a42f0 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2247fff640 1 -- 172.21.15.19:0/1109915262 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f225c0d2560 con 0x5647cc4a42f0 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2275a79740 10 client.17883 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2275a79740 10 client.17883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2275a79740 20 client.17883 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2275a79740 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc48bd80 0x5647cc88dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2275a79740 4 client.17883 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647cc7e2b80 con 0x5647cc48bd80 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.709+0000 7f2267fff640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc48bd80 0x5647cc88dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.710+0000 7f2247fff640 10 client.17883 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.710+0000 7f2267fff640 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc48bd80 0x5647cc88dfa0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5692f27c394574 server_cookie=54e00857ad08cf19 in_seq=0 out_seq=0 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2247fff640 1 -- 172.21.15.19:0/1109915262 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5647cc7e2b80 con 0x5647cc48bd80 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2247fff640 10 client.17883 handle_command_reply: tid=0 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 client.17883 shutdown 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 msgr2=0x5647cc880490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 0x5647cc880490 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c9bd0 tx=0x7f225c0c9ad0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 shutdown_connections 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc48bd80 0x5647cc88dfa0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4cd1b0 0x5647cc887930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4ac4d0 0x5647cc8809d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 --2- 172.21.15.19:0/1109915262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4a42f0 0x5647cc880490 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 >> 172.21.15.19:0/1109915262 conn(0x5647cc4c1300 msgr2=0x5647cc868d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.714+0000 7f2247fff640 0 client.17883 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.715+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 shutdown_connections 2024-05-17T09:34:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.715+0000 7f2275a79740 1 -- 172.21.15.19:0/1109915262 wait complete. 2024-05-17T09:34:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.715+0000 7f2275a79740 20 client.17883 trim_cache size 0 max 16384 2024-05-17T09:34:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.734+0000 7f2275a79740 1 Processor -- start 2024-05-17T09:34:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.734+0000 7f2275a79740 1 -- start start 2024-05-17T09:34:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.734+0000 7f2275a79740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc868a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.734+0000 7f2275a79740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc8878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.734+0000 7f2275a79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc871740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.734+0000 7f2275a79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22681ad0d0 con 0x5647cc4611c0 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2275a79740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22681aa940 con 0x5647cc4ac4d0 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2275a79740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22681a8790 con 0x5647cc4cd1b0 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f226c963640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc868a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f226d164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc871740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc8878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f226c963640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc868a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51790/0 (socket says 172.21.15.19:51790) 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f226c963640 1 -- 172.21.15.19:0/852742994 learned_addr learned my addr 172.21.15.19:0/852742994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 -- 172.21.15.19:0/852742994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 msgr2=0x5647cc868a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc868a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 -- 172.21.15.19:0/852742994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 msgr2=0x5647cc871740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc871740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 -- 172.21.15.19:0/852742994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cc86e550 con 0x5647cc4cd1b0 2024-05-17T09:34:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f2267fff640 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc8878c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f225400a010 tx=0x7f2254009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d8a256cdb2cdba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.735+0000 7f22527fc640 1 -- 172.21.15.19:0/852742994 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254009da0 con 0x5647cc4cd1b0 2024-05-17T09:34:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.736+0000 7f22527fc640 1 -- 172.21.15.19:0/852742994 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2254003a80 con 0x5647cc4cd1b0 2024-05-17T09:34:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.736+0000 7f22527fc640 1 -- 172.21.15.19:0/852742994 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254026420 con 0x5647cc4cd1b0 2024-05-17T09:34:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.736+0000 7f2275a79740 1 -- 172.21.15.19:0/852742994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 msgr2=0x5647cc8878c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.736+0000 7f2275a79740 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc8878c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f225400a010 tx=0x7f2254009850 comp rx=0 tx=0).stop 2024-05-17T09:34:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 -- 172.21.15.19:0/852742994 shutdown_connections 2024-05-17T09:34:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc871740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc8878c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 --2- 172.21.15.19:0/852742994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc868a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 -- 172.21.15.19:0/852742994 >> 172.21.15.19:0/852742994 conn(0x5647cc4c1300 msgr2=0x5647cc868330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 -- 172.21.15.19:0/852742994 shutdown_connections 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 -- 172.21.15.19:0/852742994 wait complete. 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 -- start start 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.737+0000 7f2275a79740 1 Processor -- start 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc884aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc884fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc885f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22681a8e00 con 0x5647cc4611c0 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22681aa270 con 0x5647cc4ac4d0 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22681a94d0 con 0x5647cc4cd1b0 2024-05-17T09:34:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2275a79740 10 client.0 _ll_register_callbacks cb 0x7f225c009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f226c963640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc884aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc884fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc884fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51794/0 (socket says 172.21.15.19:51794) 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 -- 172.21.15.19:0/3096005775 learned_addr learned my addr 172.21.15.19:0/3096005775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f226d164640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc885f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 -- 172.21.15.19:0/3096005775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 msgr2=0x5647cc885f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc885f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 -- 172.21.15.19:0/3096005775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 msgr2=0x5647cc884aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc884aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f2267fff640 1 -- 172.21.15.19:0/3096005775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cc8864d0 con 0x5647cc4ac4d0 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.738+0000 7f226d164640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc885f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2267fff640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc884fe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2254011e10 tx=0x7f2254005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb872caee10680d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2247fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2247fff640 1 -- 172.21.15.19:0/3096005775 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540035f0 con 0x5647cc4ac4d0 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2247fff640 1 -- 172.21.15.19:0/3096005775 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2254003790 con 0x5647cc4ac4d0 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2254003a80 con 0x5647cc4ac4d0 2024-05-17T09:34:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2247fff640 1 -- 172.21.15.19:0/3096005775 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f225402f440 con 0x5647cc4ac4d0 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2275a79740 10 client.27455 fetch_fsmap learned FSMap version 34 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5647cc887760 con 0x5647cc4ac4d0 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.739+0000 7f2247fff640 1 -- 172.21.15.19:0/3096005775 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2254003a80 con 0x5647cc4ac4d0 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2247fff640 1 -- 172.21.15.19:0/3096005775 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f225402d070 con 0x5647cc4ac4d0 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2275a79740 10 client.27455 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2275a79740 10 client.27455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2275a79740 20 client.27455 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2275a79740 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc8f67a0 0x5647cc8f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2275a79740 4 client.27455 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647cc7dc360 con 0x5647cc8f67a0 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f226c963640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc8f67a0 0x5647cc8f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.740+0000 7f2247fff640 10 client.27455 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:11.741+0000 7f226c963640 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc8f67a0 0x5647cc8f8bc0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c103112e5d3ff0 server_cookie=6ffb5d737f7cd8ff in_seq=0 out_seq=0 2024-05-17T09:34:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2247fff640 1 -- 172.21.15.19:0/3096005775 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5647cc7dc360 con 0x5647cc8f67a0 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2247fff640 10 client.27455 handle_command_reply: tid=0 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 client.27455 shutdown 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 msgr2=0x5647cc884fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc884fe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2254011e10 tx=0x7f2254005c20 comp rx=0 tx=0).stop 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 shutdown_connections 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5647cc8f67a0 0x5647cc8f8bc0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5647cc4cd1b0 0x5647cc885f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5647cc4ac4d0 0x5647cc884fe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 --2- 172.21.15.19:0/3096005775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647cc4611c0 0x5647cc884aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 >> 172.21.15.19:0/3096005775 conn(0x5647cc4c1300 msgr2=0x5647cc8807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2247fff640 0 client.27455 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 shutdown_connections 2024-05-17T09:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 1 -- 172.21.15.19:0/3096005775 wait complete. 2024-05-17T09:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.250+0000 7f2275a79740 20 client.27455 trim_cache size 0 max 16384 2024-05-17T09:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22580b28a0 msgr2=0x7f22580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22580b28a0 0x7f22580b4db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f22600099e0 tx=0x7f2260004ab0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 msgr2=0x7f226819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 0x7f226819a270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f225c005ab0 tx=0x7f225c0042f0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 shutdown_connections 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22580b28a0 0x7f22580b4db0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2268109bc0 0x7f22681a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2268104ec0 0x7f226819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 --2- 172.21.15.19:0/2653414841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2268103c80 0x7f226819a270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 >> 172.21.15.19:0/2653414841 conn(0x7f22680fee90 msgr2=0x7f2268105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 shutdown_connections 2024-05-17T09:34:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:12.251+0000 7f226ebc7640 1 -- 172.21.15.19:0/2653414841 wait complete. 2024-05-17T09:34:12.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1269 25.70 MB/sec warmup 9 sec latency 374.640 ms 2024-05-17T09:34:12.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:12 smithi079 ceph-mon[35502]: pgmap v871: 129 pgs: 129 active+clean; 278 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 18 MiB/s wr, 176 op/s 2024-05-17T09:34:12.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:12 smithi069 ceph-mon[35081]: pgmap v871: 129 pgs: 129 active+clean; 278 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 18 MiB/s wr, 176 op/s 2024-05-17T09:34:12.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:12 smithi019 ceph-mon[31620]: pgmap v871: 129 pgs: 129 active+clean; 278 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 18 MiB/s wr, 176 op/s 2024-05-17T09:34:13.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1423 25.73 MB/sec warmup 10 sec latency 146.434 ms 2024-05-17T09:34:14.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1567 23.75 MB/sec warmup 11 sec latency 149.394 ms 2024-05-17T09:34:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:14 smithi079 ceph-mon[35502]: pgmap v872: 129 pgs: 129 active+clean; 280 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 18 MiB/s wr, 236 op/s 2024-05-17T09:34:14.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:14 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:14 smithi069 ceph-mon[35081]: pgmap v872: 129 pgs: 129 active+clean; 280 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 18 MiB/s wr, 236 op/s 2024-05-17T09:34:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:14 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:14 smithi019 ceph-mon[31620]: pgmap v872: 129 pgs: 129 active+clean; 280 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 18 MiB/s wr, 236 op/s 2024-05-17T09:34:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:14 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:34:15.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1729 21.88 MB/sec warmup 12 sec latency 76.425 ms 2024-05-17T09:34:16.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 1901 20.32 MB/sec warmup 13 sec latency 78.869 ms 2024-05-17T09:34:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:16 smithi079 ceph-mon[35502]: pgmap v873: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 19 MiB/s wr, 339 op/s 2024-05-17T09:34:16.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:16 smithi069 ceph-mon[35081]: pgmap v873: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 19 MiB/s wr, 339 op/s 2024-05-17T09:34:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:16 smithi019 ceph-mon[31620]: pgmap v873: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 19 MiB/s wr, 339 op/s 2024-05-17T09:34:17.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 2076 19.02 MB/sec warmup 14 sec latency 91.643 ms 2024-05-17T09:34:18.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 2236 18.06 MB/sec warmup 15 sec latency 111.440 ms 2024-05-17T09:34:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:18 smithi079 ceph-mon[35502]: pgmap v874: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 18 MiB/s wr, 406 op/s 2024-05-17T09:34:18.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:18 smithi069 ceph-mon[35081]: pgmap v874: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 18 MiB/s wr, 406 op/s 2024-05-17T09:34:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:18 smithi019 ceph-mon[31620]: pgmap v874: 129 pgs: 129 active+clean; 274 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 18 MiB/s wr, 406 op/s 2024-05-17T09:34:19.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 2392 17.58 MB/sec warmup 16 sec latency 122.149 ms 2024-05-17T09:34:20.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 2567 17.32 MB/sec warmup 17 sec latency 94.049 ms 2024-05-17T09:34:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:20 smithi079 ceph-mon[35502]: pgmap v875: 129 pgs: 129 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 7.5 MiB/s wr, 398 op/s 2024-05-17T09:34:20.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:20 smithi069 ceph-mon[35081]: pgmap v875: 129 pgs: 129 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 7.5 MiB/s wr, 398 op/s 2024-05-17T09:34:20.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:20 smithi019 ceph-mon[31620]: pgmap v875: 129 pgs: 129 active+clean; 275 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 7.5 MiB/s wr, 398 op/s 2024-05-17T09:34:21.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 2798 17.39 MB/sec warmup 18 sec latency 69.627 ms 2024-05-17T09:34:22.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3031 17.71 MB/sec warmup 19 sec latency 75.802 ms 2024-05-17T09:34:22.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:22 smithi019 ceph-mon[31620]: pgmap v876: 129 pgs: 129 active+clean; 277 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-05-17T09:34:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:22 smithi079 ceph-mon[35502]: pgmap v876: 129 pgs: 129 active+clean; 277 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-05-17T09:34:22.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:22 smithi069 ceph-mon[35081]: pgmap v876: 129 pgs: 129 active+clean; 277 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-05-17T09:34:23.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3166 17.23 MB/sec warmup 20 sec latency 87.012 ms 2024-05-17T09:34:24.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3427 17.57 MB/sec warmup 21 sec latency 95.981 ms 2024-05-17T09:34:24.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:24 smithi079 ceph-mon[35502]: pgmap v877: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-05-17T09:34:24.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:24 smithi069 ceph-mon[35081]: pgmap v877: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-05-17T09:34:24.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:24 smithi019 ceph-mon[31620]: pgmap v877: 129 pgs: 129 active+clean; 286 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-05-17T09:34:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 -- 172.21.15.19:0/880032671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff298104e90 msgr2=0x7ff2981072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 --2- 172.21.15.19:0/880032671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff298104e90 0x7ff2981072b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff288009900 tx=0x7ff28802a3f0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 -- 172.21.15.19:0/880032671 shutdown_connections 2024-05-17T09:34:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 --2- 172.21.15.19:0/880032671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2981077f0 0x7ff298107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 --2- 172.21.15.19:0/880032671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff298104e90 0x7ff2981072b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 --2- 172.21.15.19:0/880032671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff298104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.392+0000 7ff29f613640 1 -- 172.21.15.19:0/880032671 >> 172.21.15.19:0/880032671 conn(0x7ff2980fdd40 msgr2=0x7ff2981001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.393+0000 7ff29f613640 1 -- 172.21.15.19:0/880032671 shutdown_connections 2024-05-17T09:34:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.393+0000 7ff29f613640 1 -- 172.21.15.19:0/880032671 wait complete. 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 Processor -- start 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 -- start start 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff2981a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff298104e90 0x7ff2981a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2981077f0 0x7ff2981a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff298115ca0 con 0x7ff298102530 2024-05-17T09:34:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff298115b20 con 0x7ff2981077f0 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.394+0000 7ff29f613640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff298115e20 con 0x7ff298104e90 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff2981a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff2981a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59936/0 (socket says 172.21.15.19:59936) 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 -- 172.21.15.19:0/2914258700 learned_addr learned my addr 172.21.15.19:0/2914258700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29cbae640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff298104e90 0x7ff2981a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2981077f0 0x7ff2981a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 -- 172.21.15.19:0/2914258700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff298104e90 msgr2=0x7ff2981a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff298104e90 0x7ff2981a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 -- 172.21.15.19:0/2914258700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2981077f0 msgr2=0x7ff2981a7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2981077f0 0x7ff2981a7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.395+0000 7ff29d3af640 1 -- 172.21.15.19:0/2914258700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2981a8380 con 0x7ff298102530 2024-05-17T09:34:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.396+0000 7ff29d3af640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff2981a0840 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d770 tx=0x7ff29400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.396+0000 7ff28e7fc640 1 -- 172.21.15.19:0/2914258700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294017070 con 0x7ff298102530 2024-05-17T09:34:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.396+0000 7ff28e7fc640 1 -- 172.21.15.19:0/2914258700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff294009c30 con 0x7ff298102530 2024-05-17T09:34:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.396+0000 7ff28e7fc640 1 -- 172.21.15.19:0/2914258700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29400f620 con 0x7ff298102530 2024-05-17T09:34:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.397+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2981a8670 con 0x7ff298102530 2024-05-17T09:34:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.397+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2981a8bf0 con 0x7ff298102530 2024-05-17T09:34:25.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.401+0000 7ff2a659f740 1 Processor -- start 2024-05-17T09:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 -- start start 2024-05-17T09:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b71d03bc30 con 0x55b71cd074d0 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b71d02b7d0 con 0x55b71ccff2f0 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.402+0000 7ff2a659f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b71d02f810 con 0x55b71cd281b0 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29dbb0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff28e7fc640 1 -- 172.21.15.19:0/2914258700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff29401a070 con 0x7ff298102530 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29dbb0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:53222/0 (socket says 172.21.15.19:53222) 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29dbb0640 1 -- 172.21.15.19:0/2701295556 learned_addr learned my addr 172.21.15.19:0/2701295556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29cbae640 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29cbae640 1 -- 172.21.15.19:0/2701295556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 msgr2=0x55b71d0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29cbae640 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29cbae640 1 -- 172.21.15.19:0/2701295556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 msgr2=0x55b71d0c9920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29cbae640 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0c9920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29cbae640 1 -- 172.21.15.19:0/2701295556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71cd40210 con 0x55b71cd074d0 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.403+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.404+0000 7ff29cbae640 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d0820 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff2981a18e0 tx=0x7ff288031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320541912612dd85 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.404+0000 7ff28e7fc640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2840b28a0 0x7ff2840b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.405+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2840b28a0 0x7ff2840b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.405+0000 7ff2837fe640 1 -- 172.21.15.19:0/2701295556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288035030 con 0x55b71cd074d0 2024-05-17T09:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.405+0000 7ff2837fe640 1 -- 172.21.15.19:0/2701295556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff288031860 con 0x55b71cd074d0 2024-05-17T09:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.405+0000 7ff2837fe640 1 -- 172.21.15.19:0/2701295556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288003b50 con 0x55b71cd074d0 2024-05-17T09:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.405+0000 7ff28e7fc640 1 -- 172.21.15.19:0/2914258700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff2940a59b0 con 0x7ff298102530 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 -- 172.21.15.19:0/2701295556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 msgr2=0x55b71d0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d0820 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff2981a18e0 tx=0x7ff288031670 comp rx=0 tx=0).stop 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2840b28a0 0x7ff2840b4db0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff290009fd0 tx=0x7ff29000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 -- 172.21.15.19:0/2701295556 shutdown_connections 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d0820 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 --2- 172.21.15.19:0/2701295556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 -- 172.21.15.19:0/2701295556 >> 172.21.15.19:0/2701295556 conn(0x55b71cd1c300 msgr2=0x55b71d0c4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 -- 172.21.15.19:0/2701295556 shutdown_connections 2024-05-17T09:34:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.406+0000 7ff2a659f740 1 -- 172.21.15.19:0/2701295556 wait complete. 2024-05-17T09:34:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.407+0000 7ff2a659f740 1 -- start start 2024-05-17T09:34:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.407+0000 7ff2a659f740 1 Processor -- start 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.407+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.407+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff2a659f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b71cd11790 con 0x55b71cd074d0 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff2a659f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b71cfe7890 con 0x55b71ccff2f0 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff2a659f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b71d0d1210 con 0x55b71cd281b0 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff2a659f740 10 client.0 _ll_register_callbacks cb 0x55b71cfead60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29cbae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29cbae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59954/0 (socket says 172.21.15.19:59954) 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29cbae640 1 -- 172.21.15.19:0/1435754864 learned_addr learned my addr 172.21.15.19:0/1435754864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29d3af640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 -- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 msgr2=0x55b71d0d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 -- 172.21.15.19:0/1435754864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 msgr2=0x55b71d0d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 -- 172.21.15.19:0/1435754864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b71ce0c620 con 0x55b71cd281b0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29cbae640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0da1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55b71d0ca570 tx=0x7ff290004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a118c2c83cfb3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.408+0000 7ff280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff280ff9640 1 -- 172.21.15.19:0/1435754864 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290028060 con 0x55b71cd281b0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b71d03db80 con 0x55b71cd281b0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff280ff9640 1 -- 172.21.15.19:0/1435754864 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff29001a4f0 con 0x55b71cd281b0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff280ff9640 1 -- 172.21.15.19:0/1435754864 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff29001a6f0 con 0x55b71cd281b0 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff2a659f740 10 client.27445 fetch_fsmap learned FSMap version 35 2024-05-17T09:34:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b71d0dd240 con 0x55b71cd281b0 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff280ff9640 1 -- 172.21.15.19:0/1435754864 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b71d03db80 con 0x55b71cd281b0 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.409+0000 7ff280ff9640 1 -- 172.21.15.19:0/1435754864 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff2900236f0 con 0x55b71cd281b0 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff2a659f740 10 client.27445 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff2a659f740 10 client.27445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff2a659f740 20 client.27445 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff2a659f740 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71cce6d80 0x55b71d0e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff2a659f740 4 client.27445 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b71d037360 con 0x55b71cce6d80 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff29d3af640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71cce6d80 0x55b71d0e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff280ff9640 10 client.27445 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.410+0000 7ff29d3af640 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71cce6d80 0x55b71d0e09c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d258a7902d1b18 server_cookie=f2d4b7af0b7f70ce in_seq=0 out_seq=0 2024-05-17T09:34:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.412+0000 7ff280ff9640 1 -- 172.21.15.19:0/1435754864 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b71d037360 con 0x55b71cce6d80 2024-05-17T09:34:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.412+0000 7ff280ff9640 10 client.27445 handle_command_reply: tid=0 2024-05-17T09:34:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 client.27445 shutdown 2024-05-17T09:34:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 msgr2=0x55b71d0da1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0da1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55b71d0ca570 tx=0x7ff290004a40 comp rx=0 tx=0).stop 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 shutdown_connections 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71cce6d80 0x55b71d0e09c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d0da1f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 --2- 172.21.15.19:0/1435754864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 >> 172.21.15.19:0/1435754864 conn(0x55b71cd1c300 msgr2=0x55b71d0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff280ff9640 0 client.27445 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 shutdown_connections 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.413+0000 7ff2a659f740 1 -- 172.21.15.19:0/1435754864 wait complete. 2024-05-17T09:34:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.414+0000 7ff2a659f740 20 client.27445 trim_cache size 0 max 16384 2024-05-17T09:34:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.441+0000 7ff2a659f740 1 Processor -- start 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.441+0000 7ff2a659f740 1 -- start start 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71ccff2f0 0x55b71d0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71cd074d0 0x55b71d0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 0x55b71d0da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff2a659f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b5ad0 con 0x55b71cd281b0 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff2a659f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b3340 con 0x55b71cd074d0 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff2a659f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b1190 con 0x55b71ccff2f0 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 0x55b71d0da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 0x55b71d0da110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59960/0 (socket says 172.21.15.19:59960) 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29d3af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71ccff2f0 0x55b71d0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 -- 172.21.15.19:0/679826458 learned_addr learned my addr 172.21.15.19:0/679826458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29cbae640 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71cd074d0 0x55b71d0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 -- 172.21.15.19:0/679826458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71ccff2f0 msgr2=0x55b71d0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71ccff2f0 0x55b71d0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 -- 172.21.15.19:0/679826458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71cd074d0 msgr2=0x55b71d0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71cd074d0 0x55b71d0c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29dbb0640 1 -- 172.21.15.19:0/679826458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71d0ca5a0 con 0x55b71cd281b0 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.442+0000 7ff29d3af640 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71ccff2f0 0x55b71d0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.443+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 0x55b71d0da110 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff29000cfd0 tx=0x7ff290009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f27c4c55b6e622 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.443+0000 7ff282ffd640 1 -- 172.21.15.19:0/679826458 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290038070 con 0x55b71cd281b0 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.443+0000 7ff282ffd640 1 -- 172.21.15.19:0/679826458 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff290023870 con 0x55b71cd281b0 2024-05-17T09:34:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.443+0000 7ff282ffd640 1 -- 172.21.15.19:0/679826458 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29001acc0 con 0x55b71cd281b0 2024-05-17T09:34:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 -- 172.21.15.19:0/679826458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 msgr2=0x55b71d0da110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 0x55b71d0da110 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff29000cfd0 tx=0x7ff290009840 comp rx=0 tx=0).stop 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 -- 172.21.15.19:0/679826458 shutdown_connections 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd281b0 0x55b71d0da110 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71cd074d0 0x55b71d0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 --2- 172.21.15.19:0/679826458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71ccff2f0 0x55b71d0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 -- 172.21.15.19:0/679826458 >> 172.21.15.19:0/679826458 conn(0x55b71cd1c300 msgr2=0x55b71d0c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 -- 172.21.15.19:0/679826458 shutdown_connections 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 -- 172.21.15.19:0/679826458 wait complete. 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.444+0000 7ff2a659f740 1 -- start start 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 Processor -- start 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d147040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff29cbae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b1800 con 0x55b71cd074d0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff29d3af640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff29cbae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59974/0 (socket says 172.21.15.19:59974) 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff29cbae640 1 -- 172.21.15.19:0/548303504 learned_addr learned my addr 172.21.15.19:0/548303504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b2c70 con 0x55b71ccff2f0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981b1ed0 con 0x55b71cd281b0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff2a659f740 10 client.0 _ll_register_callbacks cb 0x7ff2900077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.445+0000 7ff29dbb0640 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff29d3af640 1 -- 172.21.15.19:0/548303504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 msgr2=0x55b71d147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff29d3af640 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff29d3af640 1 -- 172.21.15.19:0/548303504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 msgr2=0x55b71d0d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff29d3af640 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff29d3af640 1 -- 172.21.15.19:0/548303504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b71d0dac60 con 0x55b71ccff2f0 2024-05-17T09:34:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff29d3af640 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d7760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d740 tx=0x7ff2940a9020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87b6f0c7d844989 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff280ff9640 1 -- 172.21.15.19:0/548303504 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940c2070 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff280ff9640 1 -- 172.21.15.19:0/548303504 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff2940a8820 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff290023870 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff280ff9640 1 -- 172.21.15.19:0/548303504 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff294021040 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff2a659f740 10 client.27470 fetch_fsmap learned FSMap version 35 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.446+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b71d1485c0 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff280ff9640 1 -- 172.21.15.19:0/548303504 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2940ad770 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff280ff9640 1 -- 172.21.15.19:0/548303504 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff2940a4d30 con 0x55b71ccff2f0 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff2a659f740 10 client.27470 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff2a659f740 10 client.27470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff2a659f740 20 client.27470 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff2a659f740 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71ccbc1c0 0x55b71d14b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff2a659f740 4 client.27470 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:34:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff290023870 con 0x55b71ccbc1c0 2024-05-17T09:34:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.447+0000 7ff29cbae640 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71ccbc1c0 0x55b71d14b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.448+0000 7ff280ff9640 10 client.27470 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.448+0000 7ff29cbae640 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71ccbc1c0 0x55b71d14b030 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc0147ad0024144 server_cookie=3b407bcf9ffffa34 in_seq=0 out_seq=0 2024-05-17T09:34:25.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3549 17.07 MB/sec warmup 22 sec latency 137.141 ms 2024-05-17T09:34:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.580+0000 7ff280ff9640 1 -- 172.21.15.19:0/548303504 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7ff290023870 con 0x55b71ccbc1c0 2024-05-17T09:34:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.580+0000 7ff280ff9640 10 client.27470 handle_command_reply: tid=0 2024-05-17T09:34:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 client.27470 shutdown 2024-05-17T09:34:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 msgr2=0x55b71d0d7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d7760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d740 tx=0x7ff2940a9020 comp rx=0 tx=0).stop 2024-05-17T09:34:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 shutdown_connections 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b71ccbc1c0 0x55b71d14b030 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b71cd281b0 0x55b71d147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b71cd074d0 0x55b71d0d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff280ff9640 0 client.27470 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 --2- 172.21.15.19:0/548303504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b71ccff2f0 0x55b71d0d7760 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 >> 172.21.15.19:0/548303504 conn(0x55b71cd1c300 msgr2=0x55b71d0c8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 shutdown_connections 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 1 -- 172.21.15.19:0/548303504 wait complete. 2024-05-17T09:34:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.581+0000 7ff2a659f740 20 client.27470 trim_cache size 0 max 16384 2024-05-17T09:34:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:34:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:34:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:34:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2840b28a0 msgr2=0x7ff2840b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2840b28a0 0x7ff2840b4db0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff290009fd0 tx=0x7ff29000e040 comp rx=0 tx=0).stop 2024-05-17T09:34:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 msgr2=0x7ff2981a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff2981a0840 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d770 tx=0x7ff29400dc30 comp rx=0 tx=0).stop 2024-05-17T09:34:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 shutdown_connections 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2981077f0 0x7ff2981a7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff298104e90 0x7ff2981a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff2840b28a0 0x7ff2840b4db0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 --2- 172.21.15.19:0/2914258700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff298102530 0x7ff2981a0840 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.586+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 >> 172.21.15.19:0/2914258700 conn(0x7ff2980fdd40 msgr2=0x7ff298103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.587+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 shutdown_connections 2024-05-17T09:34:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:25.587+0000 7ff29f613640 1 -- 172.21.15.19:0/2914258700 wait complete. 2024-05-17T09:34:26.561 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3633 16.40 MB/sec warmup 23 sec latency 169.484 ms 2024-05-17T09:34:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:26 smithi079 ceph-mon[35502]: pgmap v878: 129 pgs: 129 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.5 MiB/s wr, 361 op/s 2024-05-17T09:34:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:26 smithi069 ceph-mon[35081]: pgmap v878: 129 pgs: 129 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.5 MiB/s wr, 361 op/s 2024-05-17T09:34:26.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:26 smithi019 ceph-mon[31620]: pgmap v878: 129 pgs: 129 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.5 MiB/s wr, 361 op/s 2024-05-17T09:34:27.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3725 15.74 MB/sec warmup 24 sec latency 83.075 ms 2024-05-17T09:34:28.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 3935 15.77 MB/sec warmup 25 sec latency 91.273 ms 2024-05-17T09:34:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:28 smithi079 ceph-mon[35502]: pgmap v879: 129 pgs: 129 active+clean; 303 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 4.1 MiB/s wr, 306 op/s 2024-05-17T09:34:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:28 smithi069 ceph-mon[35081]: pgmap v879: 129 pgs: 129 active+clean; 303 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 4.1 MiB/s wr, 306 op/s 2024-05-17T09:34:28.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:28 smithi019 ceph-mon[31620]: pgmap v879: 129 pgs: 129 active+clean; 303 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 4.1 MiB/s wr, 306 op/s 2024-05-17T09:34:29.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4103 15.47 MB/sec warmup 26 sec latency 117.439 ms 2024-05-17T09:34:30.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4235 15.07 MB/sec warmup 27 sec latency 88.763 ms 2024-05-17T09:34:30.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:30 smithi079 ceph-mon[35502]: pgmap v880: 129 pgs: 129 active+clean; 307 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-05-17T09:34:30.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:30 smithi069 ceph-mon[35081]: pgmap v880: 129 pgs: 129 active+clean; 307 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-05-17T09:34:30.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:30 smithi019 ceph-mon[31620]: pgmap v880: 129 pgs: 129 active+clean; 307 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-05-17T09:34:31.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4378 14.79 MB/sec warmup 28 sec latency 79.961 ms 2024-05-17T09:34:32.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4535 14.69 MB/sec warmup 29 sec latency 72.088 ms 2024-05-17T09:34:32.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:32 smithi079 ceph-mon[35502]: pgmap v881: 129 pgs: 129 active+clean; 302 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 4.5 MiB/s wr, 218 op/s 2024-05-17T09:34:32.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:32 smithi069 ceph-mon[35081]: pgmap v881: 129 pgs: 129 active+clean; 302 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 4.5 MiB/s wr, 218 op/s 2024-05-17T09:34:32.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:32 smithi019 ceph-mon[31620]: pgmap v881: 129 pgs: 129 active+clean; 302 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 4.5 MiB/s wr, 218 op/s 2024-05-17T09:34:33.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4678 14.23 MB/sec warmup 30 sec latency 120.179 ms 2024-05-17T09:34:34.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4839 13.85 MB/sec warmup 31 sec latency 80.240 ms 2024-05-17T09:34:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:34 smithi079 ceph-mon[35502]: pgmap v882: 129 pgs: 129 active+clean; 299 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 5.6 MiB/s wr, 259 op/s 2024-05-17T09:34:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:34 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:34:34.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:34 smithi069 ceph-mon[35081]: pgmap v882: 129 pgs: 129 active+clean; 299 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 5.6 MiB/s wr, 259 op/s 2024-05-17T09:34:34.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:34 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:34:34.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:34 smithi019 ceph-mon[31620]: pgmap v882: 129 pgs: 129 active+clean; 299 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 5.6 MiB/s wr, 259 op/s 2024-05-17T09:34:34.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:34 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:34:35.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 4990 13.49 MB/sec warmup 32 sec latency 81.870 ms 2024-05-17T09:34:35.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:34:35.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:34:35.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:34:35.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:34:35.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:34:35.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:34:36.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 5150 13.22 MB/sec warmup 33 sec latency 117.249 ms 2024-05-17T09:34:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:36 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:34:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:36 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:34:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:36 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:34:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:36 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:34:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:36 smithi079 ceph-mon[35502]: pgmap v883: 129 pgs: 129 active+clean; 296 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-17T09:34:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:36 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:34:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:36 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:34:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:36 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:34:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:36 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:34:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:36 smithi069 ceph-mon[35081]: pgmap v883: 129 pgs: 129 active+clean; 296 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-17T09:34:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:36 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:34:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:36 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:34:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:36 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:34:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:36 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:34:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:36 smithi019 ceph-mon[31620]: pgmap v883: 129 pgs: 129 active+clean; 296 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-17T09:34:37.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 5320 13.12 MB/sec warmup 34 sec latency 96.255 ms 2024-05-17T09:34:38.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 5469 12.93 MB/sec warmup 35 sec latency 69.205 ms 2024-05-17T09:34:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 -- 172.21.15.19:0/2372234957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220102b40 msgr2=0x7fa220103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2372234957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220102b40 0x7fa220103840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa210009900 tx=0x7fa21002a410 comp rx=0 tx=0).stop 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 -- 172.21.15.19:0/2372234957 shutdown_connections 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2372234957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa22010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2372234957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220103d80 0x7fa220104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2372234957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220102b40 0x7fa220103840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 -- 172.21.15.19:0/2372234957 >> 172.21.15.19:0/2372234957 conn(0x7fa2200fdd30 msgr2=0x7fa2201001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 -- 172.21.15.19:0/2372234957 shutdown_connections 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.743+0000 7fa2259b7640 1 -- 172.21.15.19:0/2372234957 wait complete. 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 Processor -- start 2024-05-17T09:34:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 -- start start 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220102b40 0x7fa22019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220103d80 0x7fa22019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa2201a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa22006b510 con 0x7fa220108a80 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa22006b390 con 0x7fa220102b40 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa2259b7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa22010f840 con 0x7fa220103d80 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa21effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220102b40 0x7fa22019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa2201a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa2201a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49948/0 (socket says 172.21.15.19:49948) 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa21f7fe640 1 -- 172.21.15.19:0/2008250120 learned_addr learned my addr 172.21.15.19:0/2008250120 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.744+0000 7fa217fff640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220103d80 0x7fa22019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21f7fe640 1 -- 172.21.15.19:0/2008250120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220103d80 msgr2=0x7fa22019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21f7fe640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220103d80 0x7fa22019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21f7fe640 1 -- 172.21.15.19:0/2008250120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220102b40 msgr2=0x7fa22019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21f7fe640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220102b40 0x7fa22019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21f7fe640 1 -- 172.21.15.19:0/2008250120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2201a3f20 con 0x7fa220108a80 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21effd640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220102b40 0x7fa22019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa21f7fe640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa2201a38b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa21800eb10 tx=0x7fa21800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa2249b5640 1 -- 172.21.15.19:0/2008250120 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218004b20 con 0x7fa220108a80 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa2249b5640 1 -- 172.21.15.19:0/2008250120 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa218004cc0 con 0x7fa220108a80 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.745+0000 7fa2249b5640 1 -- 172.21.15.19:0/2008250120 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218010690 con 0x7fa220108a80 2024-05-17T09:34:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.746+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2201a4210 con 0x7fa220108a80 2024-05-17T09:34:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.746+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a4710 con 0x7fa220108a80 2024-05-17T09:34:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.749+0000 7fa2249b5640 1 -- 172.21.15.19:0/2008250120 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa218005020 con 0x7fa220108a80 2024-05-17T09:34:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.749+0000 7fa22c86b740 1 Processor -- start 2024-05-17T09:34:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 -- start start 2024-05-17T09:34:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5064d0 0x55b93c8cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b93c83ac30 con 0x55b93c5064d0 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b93c82a7d0 con 0x55b93c5271b0 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa22c86b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b93c82e810 con 0x55b93c4fe2f0 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa217fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa217fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37542/0 (socket says 172.21.15.19:37542) 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa217fff640 1 -- 172.21.15.19:0/2079570793 learned_addr learned my addr 172.21.15.19:0/2079570793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa217fff640 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5064d0 0x55b93c8cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.750+0000 7fa21effd640 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa217fff640 1 -- 172.21.15.19:0/2079570793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c4fe2f0 msgr2=0x55b93c8c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa217fff640 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa217fff640 1 -- 172.21.15.19:0/2079570793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5064d0 msgr2=0x55b93c8cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa217fff640 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5064d0 0x55b93c8cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa217fff640 1 -- 172.21.15.19:0/2079570793 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93c53f210 con 0x55b93c5271b0 2024-05-17T09:34:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa2249b5640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa20c0b2920 0x7fa20c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa217fff640 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8c83a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa208011810 tx=0x7fa2080097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa9e7e306c783cd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa21effd640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa20c0b2920 0x7fa20c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa2249b5640 1 -- 172.21.15.19:0/2008250120 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa2180e2af0 con 0x7fa220108a80 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa215ffb640 1 -- 172.21.15.19:0/2079570793 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208004030 con 0x55b93c5271b0 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa215ffb640 1 -- 172.21.15.19:0/2079570793 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa208011e50 con 0x55b93c5271b0 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.751+0000 7fa215ffb640 1 -- 172.21.15.19:0/2079570793 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2080035f0 con 0x55b93c5271b0 2024-05-17T09:34:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.752+0000 7fa22c86b740 1 -- 172.21.15.19:0/2079570793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 msgr2=0x55b93c8c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.752+0000 7fa21effd640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa20c0b2920 0x7fa20c0b4e30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa210005760 tx=0x7fa210005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.752+0000 7fa22c86b740 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8c83a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa208011810 tx=0x7fa2080097d0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 -- 172.21.15.19:0/2079570793 shutdown_connections 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5064d0 0x55b93c8cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 --2- 172.21.15.19:0/2079570793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8c83a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 -- 172.21.15.19:0/2079570793 >> 172.21.15.19:0/2079570793 conn(0x55b93c51b300 msgr2=0x55b93c8c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 -- 172.21.15.19:0/2079570793 shutdown_connections 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 -- 172.21.15.19:0/2079570793 wait complete. 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.753+0000 7fa22c86b740 1 -- start start 2024-05-17T09:34:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 Processor -- start 2024-05-17T09:34:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa21effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa21effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37556/0 (socket says 172.21.15.19:37556) 2024-05-17T09:34:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa21effd640 1 -- 172.21.15.19:0/3303933267 learned_addr learned my addr 172.21.15.19:0/3303933267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa217fff640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b93c510790 con 0x55b93c5271b0 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b93c7e6890 con 0x55b93c4fe2f0 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b93c8d0270 con 0x55b93c5064d0 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.754+0000 7fa21f7fe640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa22c86b740 10 client.0 _ll_register_callbacks cb 0x55b93c7e9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21effd640 1 -- 172.21.15.19:0/3303933267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 msgr2=0x55b93c8d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21effd640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21effd640 1 -- 172.21.15.19:0/3303933267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 msgr2=0x55b93c8d9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21effd640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21effd640 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93c60b620 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21f7fe640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa21effd640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d1cc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa210005ab0 tx=0x7fa210009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3d11e3ddd23bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa1feffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa1feffd640 1 -- 172.21.15.19:0/3303933267 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210035040 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.755+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa208011e50 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.756+0000 7fa1feffd640 1 -- 172.21.15.19:0/3303933267 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa2100035f0 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.756+0000 7fa1feffd640 1 -- 172.21.15.19:0/3303933267 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa2100037f0 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.756+0000 7fa22c86b740 10 client.27482 fetch_fsmap learned FSMap version 35 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.756+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b93c8e4960 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.756+0000 7fa1feffd640 1 -- 172.21.15.19:0/3303933267 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa208011e50 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.756+0000 7fa1feffd640 1 -- 172.21.15.19:0/3303933267 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa210044070 con 0x55b93c4fe2f0 2024-05-17T09:34:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa22c86b740 10 client.27482 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa22c86b740 10 client.27482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa22c86b740 20 client.27482 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4e5d80 0x55b93c8e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa22c86b740 4 client.27482 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93c83cb80 con 0x55b93c4e5d80 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa217fff640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4e5d80 0x55b93c8e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa1feffd640 10 client.27482 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.757+0000 7fa217fff640 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4e5d80 0x55b93c8e80f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de43e3ca7d33ec83 server_cookie=16f4fec981331bf2 in_seq=0 out_seq=0 2024-05-17T09:34:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa1feffd640 1 -- 172.21.15.19:0/3303933267 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b93c83cb80 con 0x55b93c4e5d80 2024-05-17T09:34:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa1feffd640 10 client.27482 handle_command_reply: tid=0 2024-05-17T09:34:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 client.27482 shutdown 2024-05-17T09:34:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 msgr2=0x55b93c8d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d1cc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa210005ab0 tx=0x7fa210009260 comp rx=0 tx=0).stop 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 shutdown_connections 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4e5d80 0x55b93c8e80f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa1feffd640 0 client.27482 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3303933267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d1cc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 >> 172.21.15.19:0/3303933267 conn(0x55b93c51b300 msgr2=0x55b93c8c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 shutdown_connections 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.759+0000 7fa22c86b740 1 -- 172.21.15.19:0/3303933267 wait complete. 2024-05-17T09:34:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.760+0000 7fa22c86b740 20 client.27482 trim_cache size 0 max 16384 2024-05-17T09:34:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:38 smithi079 ceph-mon[35502]: pgmap v884: 129 pgs: 129 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 Processor -- start 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 -- start start 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ad100 con 0x55b93c4fe2f0 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201aa970 con 0x55b93c5271b0 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.789+0000 7fa22c86b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201a87e0 con 0x55b93c5064d0 2024-05-17T09:34:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa217fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49972/0 (socket says 172.21.15.19:49972) 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 -- 172.21.15.19:0/3077744549 learned_addr learned my addr 172.21.15.19:0/3077744549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa217fff640 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:38766/0 (socket says 172.21.15.19:38766) 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 -- 172.21.15.19:0/3077744549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 msgr2=0x55b93c8d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 -- 172.21.15.19:0/3077744549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 msgr2=0x55b93c8d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8d9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 -- 172.21.15.19:0/3077744549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93c8bdfd0 con 0x55b93c4fe2f0 2024-05-17T09:34:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.790+0000 7fa21effd640 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c2330 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa210009230 tx=0x7fa210003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b594cf4d231b1ab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.791+0000 7fa2157fa640 1 -- 172.21.15.19:0/3077744549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210035070 con 0x55b93c4fe2f0 2024-05-17T09:34:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.791+0000 7fa2157fa640 1 -- 172.21.15.19:0/3077744549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa210004450 con 0x55b93c4fe2f0 2024-05-17T09:34:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.791+0000 7fa2157fa640 1 -- 172.21.15.19:0/3077744549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210046740 con 0x55b93c4fe2f0 2024-05-17T09:34:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 -- 172.21.15.19:0/3077744549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 msgr2=0x55b93c8c2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c2330 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa210009230 tx=0x7fa210003780 comp rx=0 tx=0).stop 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 -- 172.21.15.19:0/3077744549 shutdown_connections 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c5271b0 0x55b93c8d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 --2- 172.21.15.19:0/3077744549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c4fe2f0 0x55b93c8c2330 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 -- 172.21.15.19:0/3077744549 >> 172.21.15.19:0/3077744549 conn(0x55b93c51b300 msgr2=0x55b93c8cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 -- 172.21.15.19:0/3077744549 shutdown_connections 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 -- 172.21.15.19:0/3077744549 wait complete. 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.792+0000 7fa22c86b740 1 -- start start 2024-05-17T09:34:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 Processor -- start 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201a8e30 con 0x55b93c5271b0 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201aa2a0 con 0x55b93c4fe2f0 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201a9500 con 0x55b93c5064d0 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa22c86b740 10 client.0 _ll_register_callbacks cb 0x7fa210032d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa21effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa217fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa21effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37582/0 (socket says 172.21.15.19:37582) 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa21effd640 1 -- 172.21.15.19:0/218446390 learned_addr learned my addr 172.21.15.19:0/218446390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.793+0000 7fa21f7fe640 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa217fff640 1 -- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 msgr2=0x55b93c8d6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa217fff640 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa217fff640 1 -- 172.21.15.19:0/218446390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 msgr2=0x55b93c8d7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa217fff640 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa217fff640 1 -- 172.21.15.19:0/218446390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93c8d82c0 con 0x55b93c5064d0 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa217fff640 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d6dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2080117e0 tx=0x7fa208005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a824b10861353f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa1feffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa1feffd640 1 -- 172.21.15.19:0/218446390 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20800fbf0 con 0x55b93c5064d0 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa210004450 con 0x55b93c5064d0 2024-05-17T09:34:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.794+0000 7fa1feffd640 1 -- 172.21.15.19:0/218446390 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa20800f790 con 0x55b93c5064d0 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa1feffd640 1 -- 172.21.15.19:0/218446390 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa208011020 con 0x55b93c5064d0 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 10 client.27469 fetch_fsmap learned FSMap version 35 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b93c8d95a0 con 0x55b93c5064d0 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa1feffd640 1 -- 172.21.15.19:0/218446390 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa210004450 con 0x55b93c5064d0 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa1feffd640 1 -- 172.21.15.19:0/218446390 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa208003680 con 0x55b93c5064d0 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 10 client.27469 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 10 client.27469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 20 client.27469 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4bb1c0 0x55b93c952a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 4 client.27469 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.795+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b93c836360 con 0x55b93c4bb1c0 2024-05-17T09:34:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.796+0000 7fa21effd640 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4bb1c0 0x55b93c952a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.796+0000 7fa1feffd640 10 client.27469 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.796+0000 7fa21effd640 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4bb1c0 0x55b93c952a50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47537f1a5868d32 server_cookie=bb11818c885d2530 in_seq=0 out_seq=0 2024-05-17T09:34:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.881+0000 7fa1feffd640 1 -- 172.21.15.19:0/218446390 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b93c836360 con 0x55b93c4bb1c0 2024-05-17T09:34:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.881+0000 7fa1feffd640 10 client.27469 handle_command_reply: tid=0 2024-05-17T09:34:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.881+0000 7fa22c86b740 1 client.27469 shutdown 2024-05-17T09:34:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 msgr2=0x55b93c8d6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d6dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2080117e0 tx=0x7fa208005d40 comp rx=0 tx=0).stop 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 shutdown_connections 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b93c4bb1c0 0x55b93c952a50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b93c5271b0 0x55b93c8d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b93c5064d0 0x55b93c8d6dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa1feffd640 0 client.27469 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 --2- 172.21.15.19:0/218446390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b93c4fe2f0 0x55b93c8d6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 >> 172.21.15.19:0/218446390 conn(0x55b93c51b300 msgr2=0x55b93c8cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 shutdown_connections 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 1 -- 172.21.15.19:0/218446390 wait complete. 2024-05-17T09:34:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.882+0000 7fa22c86b740 20 client.27469 trim_cache size 0 max 16384 2024-05-17T09:34:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:34:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:34:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa20c0b2920 msgr2=0x7fa20c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa20c0b2920 0x7fa20c0b4e30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa210005760 tx=0x7fa210005fd0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 msgr2=0x7fa2201a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa2201a38b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa21800eb10 tx=0x7fa21800c2d0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 shutdown_connections 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa20c0b2920 0x7fa20c0b4e30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa220108a80 0x7fa2201a38b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa220103d80 0x7fa22019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 --2- 172.21.15.19:0/2008250120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa220102b40 0x7fa22019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 >> 172.21.15.19:0/2008250120 conn(0x7fa2200fdd30 msgr2=0x7fa2200fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 shutdown_connections 2024-05-17T09:34:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:38.883+0000 7fa2259b7640 1 -- 172.21.15.19:0/2008250120 wait complete. 2024-05-17T09:34:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:38 smithi069 ceph-mon[35081]: pgmap v884: 129 pgs: 129 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-05-17T09:34:38.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:38 smithi019 ceph-mon[31620]: pgmap v884: 129 pgs: 129 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-05-17T09:34:39.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 5666 13.12 MB/sec warmup 36 sec latency 139.294 ms 2024-05-17T09:34:40.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 5903 13.29 MB/sec warmup 37 sec latency 67.986 ms 2024-05-17T09:34:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:40 smithi079 ceph-mon[35502]: pgmap v885: 129 pgs: 129 active+clean; 288 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-05-17T09:34:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:40 smithi069 ceph-mon[35081]: pgmap v885: 129 pgs: 129 active+clean; 288 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-05-17T09:34:40.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:40 smithi019 ceph-mon[31620]: pgmap v885: 129 pgs: 129 active+clean; 288 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-05-17T09:34:41.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 6130 13.14 MB/sec warmup 38 sec latency 64.467 ms 2024-05-17T09:34:42.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 6242 12.86 MB/sec warmup 39 sec latency 67.544 ms 2024-05-17T09:34:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:42 smithi079 ceph-mon[35502]: pgmap v886: 129 pgs: 129 active+clean; 289 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-05-17T09:34:42.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:42 smithi069 ceph-mon[35081]: pgmap v886: 129 pgs: 129 active+clean; 289 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-05-17T09:34:42.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:42 smithi019 ceph-mon[31620]: pgmap v886: 129 pgs: 129 active+clean; 289 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-05-17T09:34:43.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 6368 12.68 MB/sec warmup 40 sec latency 75.543 ms 2024-05-17T09:34:44.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 6573 12.82 MB/sec warmup 41 sec latency 89.395 ms 2024-05-17T09:34:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:44 smithi079 ceph-mon[35502]: pgmap v887: 129 pgs: 129 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-05-17T09:34:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:44 smithi069 ceph-mon[35081]: pgmap v887: 129 pgs: 129 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-05-17T09:34:44.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:44 smithi019 ceph-mon[31620]: pgmap v887: 129 pgs: 129 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-05-17T09:34:45.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 6734 12.76 MB/sec warmup 42 sec latency 63.127 ms 2024-05-17T09:34:46.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 6879 12.62 MB/sec warmup 43 sec latency 65.015 ms 2024-05-17T09:34:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:46 smithi079 ceph-mon[35502]: pgmap v888: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-05-17T09:34:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:46 smithi069 ceph-mon[35081]: pgmap v888: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-05-17T09:34:46.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:46 smithi019 ceph-mon[31620]: pgmap v888: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-05-17T09:34:47.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7035 12.61 MB/sec warmup 44 sec latency 61.553 ms 2024-05-17T09:34:48.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7193 12.48 MB/sec warmup 45 sec latency 98.188 ms 2024-05-17T09:34:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:48 smithi079 ceph-mon[35502]: pgmap v889: 129 pgs: 129 active+clean; 304 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 4.3 MiB/s wr, 281 op/s 2024-05-17T09:34:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:48 smithi069 ceph-mon[35081]: pgmap v889: 129 pgs: 129 active+clean; 304 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 4.3 MiB/s wr, 281 op/s 2024-05-17T09:34:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:48 smithi019 ceph-mon[31620]: pgmap v889: 129 pgs: 129 active+clean; 304 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 4.3 MiB/s wr, 281 op/s 2024-05-17T09:34:49.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7404 12.48 MB/sec warmup 46 sec latency 175.350 ms 2024-05-17T09:34:50.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7520 12.29 MB/sec warmup 47 sec latency 168.510 ms 2024-05-17T09:34:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:50 smithi079 ceph-mon[35502]: pgmap v890: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 4.5 MiB/s wr, 227 op/s 2024-05-17T09:34:50.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:50 smithi069 ceph-mon[35081]: pgmap v890: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 4.5 MiB/s wr, 227 op/s 2024-05-17T09:34:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:50 smithi019 ceph-mon[31620]: pgmap v890: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 4.5 MiB/s wr, 227 op/s 2024-05-17T09:34:51.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7644 12.21 MB/sec warmup 48 sec latency 80.267 ms 2024-05-17T09:34:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 -- 172.21.15.19:0/48843830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8102b40 msgr2=0x7f2fb8103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 --2- 172.21.15.19:0/48843830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8102b40 0x7f2fb8103840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009900 tx=0x7f2fb402a410 comp rx=0 tx=0).stop 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 -- 172.21.15.19:0/48843830 shutdown_connections 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 --2- 172.21.15.19:0/48843830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8108a80 0x7f2fb810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 --2- 172.21.15.19:0/48843830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb8103d80 0x7f2fb8104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 --2- 172.21.15.19:0/48843830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8102b40 0x7f2fb8103840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 -- 172.21.15.19:0/48843830 >> 172.21.15.19:0/48843830 conn(0x7f2fb80fdd30 msgr2=0x7f2fb81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.036+0000 7f2fbf822640 1 -- 172.21.15.19:0/48843830 shutdown_connections 2024-05-17T09:34:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.037+0000 7f2fbf822640 1 -- 172.21.15.19:0/48843830 wait complete. 2024-05-17T09:34:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.037+0000 7f2fbf822640 1 Processor -- start 2024-05-17T09:34:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.037+0000 7f2fbf822640 1 -- start start 2024-05-17T09:34:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbf822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb8102b40 0x7f2fb806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbf822640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8103d80 0x7f2fb819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbf822640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 0x7f2fb81a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbf822640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb810e540 con 0x7f2fb8103d80 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbf822640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb810e3c0 con 0x7f2fb8102b40 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbf822640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb810e6c0 con 0x7f2fb8108a80 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbddbf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 0x7f2fb81a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbddbf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 0x7f2fb81a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42980/0 (socket says 172.21.15.19:42980) 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbd5be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb8102b40 0x7f2fb806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbddbf640 1 -- 172.21.15.19:0/796195676 learned_addr learned my addr 172.21.15.19:0/796195676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8103d80 0x7f2fb819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbddbf640 1 -- 172.21.15.19:0/796195676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb8102b40 msgr2=0x7f2fb806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.038+0000 7f2fbddbf640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb8102b40 0x7f2fb806b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbddbf640 1 -- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8103d80 msgr2=0x7f2fb819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbddbf640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8103d80 0x7f2fb819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbddbf640 1 -- 172.21.15.19:0/796195676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb8198fd0 con 0x7f2fb8108a80 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8103d80 0x7f2fb819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbddbf640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 0x7f2fb81a1510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000efd0 tx=0x7f2fb000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fae7fc640 1 -- 172.21.15.19:0/796195676 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0003b70 con 0x7f2fb8108a80 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fae7fc640 1 -- 172.21.15.19:0/796195676 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fb0003d10 con 0x7f2fb8108a80 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb81992c0 con 0x7f2fb8108a80 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fae7fc640 1 -- 172.21.15.19:0/796195676 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0010850 con 0x7f2fb8108a80 2024-05-17T09:34:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.039+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb8199810 con 0x7f2fb8108a80 2024-05-17T09:34:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.043+0000 7f2fae7fc640 1 -- 172.21.15.19:0/796195676 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2fb0004020 con 0x7f2fb8108a80 2024-05-17T09:34:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 Processor -- start 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 -- start start 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f92e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f92e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f935820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d32f8a0c30 con 0x55d32f56c4d0 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d32f8907d0 con 0x55d32f58d1b0 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d32f894810 con 0x55d32f5642f0 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f92e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f92e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55326/0 (socket says 172.21.15.19:55326) 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/4012438349 learned_addr learned my addr 172.21.15.19:0/4012438349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fae7fc640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2fa40b2920 0x7f2fa40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f92e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.044+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f935820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbddbf640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2fa40b2920 0x7f2fa40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fae7fc640 1 -- 172.21.15.19:0/796195676 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2fb00e2a30 con 0x7f2fb8108a80 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbd5be640 1 -- 172.21.15.19:0/4012438349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 msgr2=0x55d32f92e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f92e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbd5be640 1 -- 172.21.15.19:0/4012438349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 msgr2=0x55d32f935820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f935820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbd5be640 1 -- 172.21.15.19:0/4012438349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32f5a5210 con 0x55d32f5642f0 2024-05-17T09:34:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbddbf640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2fa40b2920 0x7f2fa40b4e30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00a9c30 tx=0x7f2fb0004810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.045+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f92e920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fb402aa00 tx=0x7f2fb4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d643b94f176a6a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.046+0000 7f2fa37fe640 1 -- 172.21.15.19:0/4012438349 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4035030 con 0x55d32f5642f0 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.046+0000 7f2fa37fe640 1 -- 172.21.15.19:0/4012438349 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fb4040070 con 0x55d32f5642f0 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.046+0000 7f2fa37fe640 1 -- 172.21.15.19:0/4012438349 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4031760 con 0x55d32f5642f0 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 -- 172.21.15.19:0/4012438349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 msgr2=0x55d32f92e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f92e920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fb402aa00 tx=0x7f2fb4005e80 comp rx=0 tx=0).stop 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 -- 172.21.15.19:0/4012438349 shutdown_connections 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f935820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f92e920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/4012438349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f92e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 -- 172.21.15.19:0/4012438349 >> 172.21.15.19:0/4012438349 conn(0x55d32f581300 msgr2=0x55d32f929d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 -- 172.21.15.19:0/4012438349 shutdown_connections 2024-05-17T09:34:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.047+0000 7f2fc67ae740 1 -- 172.21.15.19:0/4012438349 wait complete. 2024-05-17T09:34:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fc67ae740 1 -- start start 2024-05-17T09:34:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fc67ae740 1 Processor -- start 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f5642f0 0x55d32f937cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f9381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f93f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d32f576790 con 0x55d32f56c4d0 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.048+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f5642f0 0x55d32f937cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d32f84c890 con 0x55d32f5642f0 2024-05-17T09:34:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d32f936210 con 0x55d32f58d1b0 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f5642f0 0x55d32f937cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55328/0 (socket says 172.21.15.19:55328) 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/942765948 learned_addr learned my addr 172.21.15.19:0/942765948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f9381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fc67ae740 10 client.0 _ll_register_callbacks cb 0x55d32f84fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f93f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbd5be640 1 -- 172.21.15.19:0/942765948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 msgr2=0x55d32f93f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f93f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbd5be640 1 -- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f5642f0 msgr2=0x55d32f937cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f5642f0 0x55d32f937cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.049+0000 7f2fbd5be640 1 -- 172.21.15.19:0/942765948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32f671620 con 0x55d32f56c4d0 2024-05-17T09:34:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f93f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f9381f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x55d32f935f40 tx=0x7f2fb4003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79c9b8b56e0746b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/942765948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4035070 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb4040070 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/942765948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2fb4003bb0 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/942765948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2fb4009520 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fc67ae740 10 client.17934 fetch_fsmap learned FSMap version 35 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.050+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d32f941e50 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/942765948 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fb4031990 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/942765948 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2fb4045890 con 0x55d32f56c4d0 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fc67ae740 10 client.17934 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fc67ae740 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fc67ae740 20 client.17934 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f54bd80 0x55d32f9451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fc67ae740 4 client.17934 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:34:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2fb4040070 con 0x55d32f54bd80 2024-05-17T09:34:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.051+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f54bd80 0x55d32f9451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.052+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f54bd80 0x55d32f9451b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f73065f73831804 server_cookie=3088fa4c9fae34b2 in_seq=0 out_seq=0 2024-05-17T09:34:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.052+0000 7f2fa0ff9640 10 client.17934 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/942765948 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f2fb4040070 con 0x55d32f54bd80 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fa0ff9640 10 client.17934 handle_command_reply: tid=0 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 client.17934 shutdown 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 msgr2=0x55d32f9381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f9381f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x55d32f935f40 tx=0x7f2fb4003fd0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 shutdown_connections 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f54bd80 0x55d32f9451b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f93f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f9381f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/942765948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f5642f0 0x55d32f937cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fa0ff9640 0 client.17934 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 >> 172.21.15.19:0/942765948 conn(0x55d32f581300 msgr2=0x55d32f92cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 shutdown_connections 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.053+0000 7f2fc67ae740 1 -- 172.21.15.19:0/942765948 wait complete. 2024-05-17T09:34:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.054+0000 7f2fc67ae740 20 client.17934 trim_cache size 0 max 16384 2024-05-17T09:34:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.085+0000 7f2fc67ae740 1 Processor -- start 2024-05-17T09:34:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.085+0000 7f2fc67ae740 1 -- start start 2024-05-17T09:34:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.085+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f9375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.085+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f93f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.085+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f949880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fbd5be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f93f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f9375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f9375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:43006/0 (socket says 172.21.15.19:43006) 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/723686363 learned_addr learned my addr 172.21.15.19:0/723686363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b5900 con 0x55d32f56c4d0 2024-05-17T09:34:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b3170 con 0x55d32f58d1b0 2024-05-17T09:34:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b0fc0 con 0x55d32f5642f0 2024-05-17T09:34:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.086+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f949880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/723686363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 msgr2=0x55d32f949880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f949880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/723686363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 msgr2=0x55d32f93f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f93f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/723686363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32f923fd0 con 0x55d32f5642f0 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f9375c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8004700 tx=0x7f2fa8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0360df098e8802 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fa2ffd640 1 -- 172.21.15.19:0/723686363 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004050 con 0x55d32f5642f0 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fa2ffd640 1 -- 172.21.15.19:0/723686363 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fa80128e0 con 0x55d32f5642f0 2024-05-17T09:34:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.087+0000 7f2fa2ffd640 1 -- 172.21.15.19:0/723686363 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8005410 con 0x55d32f5642f0 2024-05-17T09:34:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 msgr2=0x55d32f9375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f9375c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8004700 tx=0x7f2fa8005f90 comp rx=0 tx=0).stop 2024-05-17T09:34:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 shutdown_connections 2024-05-17T09:34:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f58d1b0 0x55d32f949880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f56c4d0 0x55d32f93f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/723686363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f5642f0 0x55d32f9375c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 >> 172.21.15.19:0/723686363 conn(0x55d32f581300 msgr2=0x55d32f932310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 shutdown_connections 2024-05-17T09:34:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 -- 172.21.15.19:0/723686363 wait complete. 2024-05-17T09:34:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 -- start start 2024-05-17T09:34:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.089+0000 7f2fc67ae740 1 Processor -- start 2024-05-17T09:34:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f5642f0 0x55d32f93e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f56c4d0 0x55d32f93e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f9ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b1630 con 0x55d32f5642f0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b2aa0 con 0x55d32f56c4d0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b1d00 con 0x55d32f58d1b0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fc67ae740 10 client.0 _ll_register_callbacks cb 0x7f2fb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f5642f0 0x55d32f93e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fbcdbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f5642f0 0x55d32f93e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34424/0 (socket says 172.21.15.19:34424) 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fbcdbd640 1 -- 172.21.15.19:0/3612752284 learned_addr learned my addr 172.21.15.19:0/3612752284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f56c4d0 0x55d32f93e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.090+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f9ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbd5be640 1 -- 172.21.15.19:0/3612752284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 msgr2=0x55d32f9ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f9ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbd5be640 1 -- 172.21.15.19:0/3612752284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f5642f0 msgr2=0x55d32f93e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f5642f0 0x55d32f93e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbd5be640 1 -- 172.21.15.19:0/3612752284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32f9abdd0 con 0x55d32f56c4d0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f9ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fbd5be640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f56c4d0 0x55d32f93e7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fb402a930 tx=0x7f2fb402aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd49c345f6663c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/3612752284 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4035040 con 0x55d32f56c4d0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/3612752284 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2fb4045670 con 0x55d32f56c4d0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.091+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa80128e0 con 0x55d32f56c4d0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.092+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/3612752284 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2fb4045890 con 0x55d32f56c4d0 2024-05-17T09:34:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.092+0000 7f2fc67ae740 10 client.27506 fetch_fsmap learned FSMap version 35 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.092+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d32f9ad060 con 0x55d32f56c4d0 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.092+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/3612752284 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fa80128e0 con 0x55d32f56c4d0 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/3612752284 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2fb4067470 con 0x55d32f56c4d0 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fc67ae740 10 client.27506 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fc67ae740 10 client.27506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fc67ae740 20 client.27506 populate_metadata read hostname 'smithi019' 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f5211c0 0x55d32f9afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fc67ae740 4 client.27506 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d32f8a2b80 con 0x55d32f5211c0 2024-05-17T09:34:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f5211c0 0x55d32f9afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:34:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.093+0000 7f2fa0ff9640 10 client.27506 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.094+0000 7f2fbcdbd640 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f5211c0 0x55d32f9afab0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f83783922a0967 server_cookie=4e5b1ec9471959b8 in_seq=0 out_seq=0 2024-05-17T09:34:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fa0ff9640 1 -- 172.21.15.19:0/3612752284 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d32f8a2b80 con 0x55d32f5211c0 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fa0ff9640 10 client.27506 handle_command_reply: tid=0 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 client.27506 shutdown 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f56c4d0 msgr2=0x55d32f93e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f56c4d0 0x55d32f93e7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fb402a930 tx=0x7f2fb402aa90 comp rx=0 tx=0).stop 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 shutdown_connections 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d32f5211c0 0x55d32f9afab0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d32f58d1b0 0x55d32f9ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32f56c4d0 0x55d32f93e7b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fa0ff9640 0 client.27506 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 --2- 172.21.15.19:0/3612752284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32f5642f0 0x55d32f93e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 >> 172.21.15.19:0/3612752284 conn(0x55d32f581300 msgr2=0x55d32f931850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 shutdown_connections 2024-05-17T09:34:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.180+0000 7f2fc67ae740 1 -- 172.21.15.19:0/3612752284 wait complete. 2024-05-17T09:34:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fc67ae740 20 client.27506 trim_cache size 0 max 16384 2024-05-17T09:34:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:34:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:34:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2fa40b2920 msgr2=0x7f2fa40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2fa40b2920 0x7f2fa40b4e30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00a9c30 tx=0x7f2fb0004810 comp rx=0 tx=0).stop 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 msgr2=0x7f2fb81a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 0x7f2fb81a1510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000efd0 tx=0x7f2fb000c6a0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 shutdown_connections 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2fb8108a80 0x7f2fb81a1510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2fa40b2920 0x7f2fa40b4e30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb8103d80 0x7f2fb819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 --2- 172.21.15.19:0/796195676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb8102b40 0x7f2fb806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.181+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 >> 172.21.15.19:0/796195676 conn(0x7f2fb80fdd30 msgr2=0x7f2fb8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.182+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 shutdown_connections 2024-05-17T09:34:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:34:52.182+0000 7f2fbf822640 1 -- 172.21.15.19:0/796195676 wait complete. 2024-05-17T09:34:52.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7788 12.17 MB/sec warmup 49 sec latency 140.092 ms 2024-05-17T09:34:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:52 smithi079 ceph-mon[35502]: pgmap v891: 129 pgs: 129 active+clean; 306 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 4.0 MiB/s wr, 183 op/s 2024-05-17T09:34:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:52 smithi069 ceph-mon[35081]: pgmap v891: 129 pgs: 129 active+clean; 306 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 4.0 MiB/s wr, 183 op/s 2024-05-17T09:34:52.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:52 smithi019 ceph-mon[31620]: pgmap v891: 129 pgs: 129 active+clean; 306 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 4.0 MiB/s wr, 183 op/s 2024-05-17T09:34:53.562 INFO:tasks.workunit.client.0.smithi019.stdout: 10 7985 12.26 MB/sec warmup 50 sec latency 63.886 ms 2024-05-17T09:34:54.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 8121 12.09 MB/sec warmup 51 sec latency 80.761 ms 2024-05-17T09:34:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:54 smithi019 ceph-mon[31620]: pgmap v892: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 5.0 MiB/s wr, 260 op/s 2024-05-17T09:34:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:54 smithi079 ceph-mon[35502]: pgmap v892: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 5.0 MiB/s wr, 260 op/s 2024-05-17T09:34:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:54 smithi069 ceph-mon[35081]: pgmap v892: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 5.0 MiB/s wr, 260 op/s 2024-05-17T09:34:55.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 8253 11.93 MB/sec warmup 52 sec latency 89.788 ms 2024-05-17T09:34:56.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 8394 11.75 MB/sec warmup 53 sec latency 75.677 ms 2024-05-17T09:34:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:56 smithi079 ceph-mon[35502]: pgmap v893: 129 pgs: 129 active+clean; 305 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-05-17T09:34:56.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:56 smithi069 ceph-mon[35081]: pgmap v893: 129 pgs: 129 active+clean; 305 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-05-17T09:34:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:56 smithi019 ceph-mon[31620]: pgmap v893: 129 pgs: 129 active+clean; 305 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-05-17T09:34:57.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 8548 11.63 MB/sec warmup 54 sec latency 74.843 ms 2024-05-17T09:34:58.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 8704 11.52 MB/sec warmup 55 sec latency 75.948 ms 2024-05-17T09:34:58.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:34:58 smithi069 ceph-mon[35081]: pgmap v894: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.8 MiB/s wr, 262 op/s 2024-05-17T09:34:58.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:34:58 smithi019 ceph-mon[31620]: pgmap v894: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.8 MiB/s wr, 262 op/s 2024-05-17T09:34:59.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:34:58 smithi079 ceph-mon[35502]: pgmap v894: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.8 MiB/s wr, 262 op/s 2024-05-17T09:34:59.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 8869 11.45 MB/sec warmup 56 sec latency 73.341 ms 2024-05-17T09:35:00.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 9054 11.44 MB/sec warmup 57 sec latency 76.502 ms 2024-05-17T09:35:00.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:00 smithi069 ceph-mon[35081]: pgmap v895: 129 pgs: 129 active+clean; 307 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-05-17T09:35:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:00 smithi019 ceph-mon[31620]: pgmap v895: 129 pgs: 129 active+clean; 307 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-05-17T09:35:01.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:00 smithi079 ceph-mon[35502]: pgmap v895: 129 pgs: 129 active+clean; 307 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-05-17T09:35:01.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 9268 11.52 MB/sec warmup 58 sec latency 58.201 ms 2024-05-17T09:35:02.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 9483 11.61 MB/sec warmup 59 sec latency 149.043 ms 2024-05-17T09:35:02.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:02 smithi079 ceph-mon[35502]: pgmap v896: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-05-17T09:35:02.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:02 smithi069 ceph-mon[35081]: pgmap v896: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-05-17T09:35:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:02 smithi019 ceph-mon[31620]: pgmap v896: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-05-17T09:35:03.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 9604 11.50 MB/sec warmup 60 sec latency 60.074 ms 2024-05-17T09:35:04.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 9750 11.48 MB/sec warmup 61 sec latency 61.630 ms 2024-05-17T09:35:04.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:04 smithi019 ceph-mon[31620]: pgmap v897: 129 pgs: 129 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 4.4 MiB/s wr, 290 op/s 2024-05-17T09:35:04.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:04 smithi069 ceph-mon[35081]: pgmap v897: 129 pgs: 129 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 4.4 MiB/s wr, 290 op/s 2024-05-17T09:35:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:04 smithi079 ceph-mon[35502]: pgmap v897: 129 pgs: 129 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 4.4 MiB/s wr, 290 op/s 2024-05-17T09:35:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 -- 172.21.15.19:0/4067551854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c102b40 msgr2=0x7f6e4c103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 --2- 172.21.15.19:0/4067551854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c102b40 0x7f6e4c103840 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802a410 comp rx=0 tx=0).stop 2024-05-17T09:35:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 -- 172.21.15.19:0/4067551854 shutdown_connections 2024-05-17T09:35:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 --2- 172.21.15.19:0/4067551854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c108a80 0x7f6e4c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 --2- 172.21.15.19:0/4067551854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c103d80 0x7f6e4c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 --2- 172.21.15.19:0/4067551854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c102b40 0x7f6e4c103840 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 -- 172.21.15.19:0/4067551854 >> 172.21.15.19:0/4067551854 conn(0x7f6e4c0fdd30 msgr2=0x7f6e4c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 -- 172.21.15.19:0/4067551854 shutdown_connections 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.325+0000 7f6e53451640 1 -- 172.21.15.19:0/4067551854 wait complete. 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 Processor -- start 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 -- start start 2024-05-17T09:35:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 0x7f6e4c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c103d80 0x7f6e4c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c108a80 0x7f6e4c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10f5d0 con 0x7f6e4c103d80 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10f450 con 0x7f6e4c108a80 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.326+0000 7f6e53451640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10f750 con 0x7f6e4c102b40 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 0x7f6e4c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e509ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c103d80 0x7f6e4c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 0x7f6e4c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:51960/0 (socket says 172.21.15.19:51960) 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 -- 172.21.15.19:0/3549882426 learned_addr learned my addr 172.21.15.19:0/3549882426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e519ee640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c108a80 0x7f6e4c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 -- 172.21.15.19:0/3549882426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c108a80 msgr2=0x7f6e4c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c108a80 0x7f6e4c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 -- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c103d80 msgr2=0x7f6e4c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c103d80 0x7f6e4c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 -- 172.21.15.19:0/3549882426 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c1a1d40 con 0x7f6e4c102b40 2024-05-17T09:35:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e519ee640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c108a80 0x7f6e4c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.327+0000 7f6e511ed640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 0x7f6e4c19a200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005760 tx=0x7f6e48004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.328+0000 7f6e427fc640 1 -- 172.21.15.19:0/3549882426 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4803f030 con 0x7f6e4c102b40 2024-05-17T09:35:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.328+0000 7f6e427fc640 1 -- 172.21.15.19:0/3549882426 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e48004440 con 0x7f6e4c102b40 2024-05-17T09:35:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.328+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c1a1fd0 con 0x7f6e4c102b40 2024-05-17T09:35:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.328+0000 7f6e427fc640 1 -- 172.21.15.19:0/3549882426 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48035070 con 0x7f6e4c102b40 2024-05-17T09:35:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.328+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c1a2430 con 0x7f6e4c102b40 2024-05-17T09:35:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.331+0000 7f6e5a3e8740 1 Processor -- start 2024-05-17T09:35:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.331+0000 7f6e5a3e8740 1 -- start start 2024-05-17T09:35:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.331+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc66b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562dc661ac30 con 0x562dc62de2f0 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562dc660a7d0 con 0x562dc63071b0 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562dc660e810 con 0x562dc62e64d0 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc66b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc66b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:51970/0 (socket says 172.21.15.19:51970) 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 -- 172.21.15.19:0/2773063650 learned_addr learned my addr 172.21.15.19:0/2773063650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e427fc640 1 -- 172.21.15.19:0/3549882426 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6e48042030 con 0x7f6e4c102b40 2024-05-17T09:35:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e519ee640 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e511ed640 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 -- 172.21.15.19:0/2773063650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 msgr2=0x562dc66b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 -- 172.21.15.19:0/2773063650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 msgr2=0x562dc66b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.332+0000 7f6e509ec640 1 -- 172.21.15.19:0/2773063650 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc631f210 con 0x562dc62e64d0 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.333+0000 7f6e511ed640 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:35:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.333+0000 7f6e509ec640 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc66b7f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009c90 tx=0x7f6e3c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c612acf3cfe9f48c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.333+0000 7f6e377fe640 1 -- 172.21.15.19:0/2773063650 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c004230 con 0x562dc62e64d0 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.333+0000 7f6e427fc640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e380b29a0 0x7f6e380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.333+0000 7f6e377fe640 1 -- 172.21.15.19:0/2773063650 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e3c0043d0 con 0x562dc62e64d0 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.334+0000 7f6e427fc640 1 -- 172.21.15.19:0/3549882426 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6e481028e0 con 0x7f6e4c102b40 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.334+0000 7f6e519ee640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e380b29a0 0x7f6e380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2773063650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 msgr2=0x562dc66b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc66b7f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009c90 tx=0x7f6e3c011810 comp rx=0 tx=0).stop 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e519ee640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e380b29a0 0x7f6e380b4eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x562dc66a74d0 tx=0x7f6e440096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2773063650 shutdown_connections 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc66b7f40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2773063650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2773063650 >> 172.21.15.19:0/2773063650 conn(0x562dc62fb300 msgr2=0x562dc66a3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2773063650 shutdown_connections 2024-05-17T09:35:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.335+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2773063650 wait complete. 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.336+0000 7f6e5a3e8740 1 -- start start 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.336+0000 7f6e5a3e8740 1 Processor -- start 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62e64d0 0x562dc66ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562dc62f0790 con 0x562dc62e64d0 2024-05-17T09:35:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562dc65c6890 con 0x562dc63071b0 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562dc66b8ab0 con 0x562dc62de2f0 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e5a3e8740 10 client.0 _ll_register_callbacks cb 0x562dc65c9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66ba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:51986/0 (socket says 172.21.15.19:51986) 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e519ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e511ed640 1 -- 172.21.15.19:0/2463400712 learned_addr learned my addr 172.21.15.19:0/2463400712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e509ec640 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62e64d0 0x562dc66ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e519ee640 1 -- 172.21.15.19:0/2463400712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 msgr2=0x562dc66ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.337+0000 7f6e519ee640 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e519ee640 1 -- 172.21.15.19:0/2463400712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62e64d0 msgr2=0x562dc66ba990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e519ee640 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62e64d0 0x562dc66ba990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e519ee640 1 -- 172.21.15.19:0/2463400712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc63eb620 con 0x562dc63071b0 2024-05-17T09:35:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e519ee640 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66c18f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e44010b40 tx=0x7f6e44004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8292a842ce2359 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e34ff9640 1 -- 172.21.15.19:0/2463400712 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4401fbc0 con 0x562dc63071b0 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3c0043d0 con 0x562dc63071b0 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e34ff9640 1 -- 172.21.15.19:0/2463400712 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6e44017030 con 0x562dc63071b0 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e34ff9640 1 -- 172.21.15.19:0/2463400712 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6e44031050 con 0x562dc63071b0 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e5a3e8740 10 client.27518 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.338+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562dc66c48b0 con 0x562dc63071b0 2024-05-17T09:35:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e34ff9640 1 -- 172.21.15.19:0/2463400712 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e3c0043d0 con 0x562dc63071b0 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e34ff9640 1 -- 172.21.15.19:0/2463400712 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f6e44004470 con 0x562dc63071b0 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e5a3e8740 10 client.27518 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e5a3e8740 10 client.27518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e5a3e8740 20 client.27518 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc62c5d80 0x562dc66c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e5a3e8740 4 client.27518 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dc661cb80 con 0x562dc62c5d80 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.339+0000 7f6e511ed640 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc62c5d80 0x562dc66c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.340+0000 7f6e34ff9640 10 client.27518 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.340+0000 7f6e511ed640 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc62c5d80 0x562dc66c80f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90810823cc79262d server_cookie=f29b1ba67ffffc9e in_seq=0 out_seq=0 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e34ff9640 1 -- 172.21.15.19:0/2463400712 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562dc661cb80 con 0x562dc62c5d80 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e34ff9640 10 client.27518 handle_command_reply: tid=0 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 client.27518 shutdown 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 msgr2=0x562dc66c18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66c18f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e44010b40 tx=0x7f6e44004ab0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 shutdown_connections 2024-05-17T09:35:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc62c5d80 0x562dc66c80f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc63071b0 0x562dc66c18f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e34ff9640 0 client.27518 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62e64d0 0x562dc66ba990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/2463400712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.341+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 >> 172.21.15.19:0/2463400712 conn(0x562dc62fb300 msgr2=0x562dc66a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.342+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 shutdown_connections 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.342+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/2463400712 wait complete. 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.342+0000 7f6e5a3e8740 20 client.27518 trim_cache size 0 max 16384 2024-05-17T09:35:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.365+0000 7f6e5a3e8740 1 Processor -- start 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.365+0000 7f6e5a3e8740 1 -- start start 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 0x562dc66cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62e64d0 0x562dc66c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1ad0e0 con 0x562dc62c5d80 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1aa950 con 0x562dc62e64d0 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1a87a0 con 0x562dc62de2f0 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 0x562dc66cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e509ec640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 0x562dc66cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52902/0 (socket says 172.21.15.19:52902) 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e511ed640 1 -- 172.21.15.19:0/804395218 learned_addr learned my addr 172.21.15.19:0/804395218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e519ee640 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62e64d0 0x562dc66c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e509ec640 1 -- 172.21.15.19:0/804395218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62e64d0 msgr2=0x562dc66c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e509ec640 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62e64d0 0x562dc66c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e509ec640 1 -- 172.21.15.19:0/804395218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 msgr2=0x562dc66cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e509ec640 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 0x562dc66cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e509ec640 1 -- 172.21.15.19:0/804395218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc66a7050 con 0x562dc62de2f0 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.366+0000 7f6e511ed640 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 0x562dc66cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.367+0000 7f6e509ec640 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66c1810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0058c0 tx=0x7f6e3c0095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd49fdb63475d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.367+0000 7f6e36ffd640 1 -- 172.21.15.19:0/804395218 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c01f040 con 0x562dc62de2f0 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.367+0000 7f6e36ffd640 1 -- 172.21.15.19:0/804395218 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e3c0054d0 con 0x562dc62de2f0 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.367+0000 7f6e36ffd640 1 -- 172.21.15.19:0/804395218 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0035f0 con 0x562dc62de2f0 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/804395218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 msgr2=0x562dc66c1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66c1810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0058c0 tx=0x7f6e3c0095b0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/804395218 shutdown_connections 2024-05-17T09:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62e64d0 0x562dc66c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62de2f0 0x562dc66c1810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/804395218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62c5d80 0x562dc66cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/804395218 >> 172.21.15.19:0/804395218 conn(0x562dc62fb300 msgr2=0x562dc66a3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/804395218 shutdown_connections 2024-05-17T09:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.368+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/804395218 wait complete. 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 -- start start 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 Processor -- start 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62c5d80 0x562dc66c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc672e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1a8e10 con 0x562dc62de2f0 2024-05-17T09:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1aa280 con 0x562dc62c5d80 2024-05-17T09:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e5a3e8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1a94e0 con 0x562dc62e64d0 2024-05-17T09:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e509ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.369+0000 7f6e511ed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62c5d80 0x562dc66c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e5a3e8740 10 client.0 _ll_register_callbacks cb 0x7f6e44011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e509ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66c1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52912/0 (socket says 172.21.15.19:52912) 2024-05-17T09:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e509ec640 1 -- 172.21.15.19:0/195436870 learned_addr learned my addr 172.21.15.19:0/195436870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc672e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 -- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62c5d80 msgr2=0x562dc66c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62c5d80 0x562dc66c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 -- 172.21.15.19:0/195436870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 msgr2=0x562dc66c1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66c1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 -- 172.21.15.19:0/195436870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc672ecb0 con 0x562dc62e64d0 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e519ee640 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc672e770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e440274b0 tx=0x7f6e44016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8acf9c64dd0d3e20 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e34ff9640 1 -- 172.21.15.19:0/195436870 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44027a00 con 0x562dc62e64d0 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e34ff9640 1 -- 172.21.15.19:0/195436870 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6e44027ba0 con 0x562dc62e64d0 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.370+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3c0054d0 con 0x562dc62e64d0 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e34ff9640 1 -- 172.21.15.19:0/195436870 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6e44027da0 con 0x562dc62e64d0 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 10 client.27502 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562dc672fed0 con 0x562dc62e64d0 2024-05-17T09:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e34ff9640 1 -- 172.21.15.19:0/195436870 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e3c0054d0 con 0x562dc62e64d0 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e34ff9640 1 -- 172.21.15.19:0/195436870 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f6e44034070 con 0x562dc62e64d0 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 10 client.27502 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 10 client.27502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 20 client.27502 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc63071b0 0x562dc6732910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 4 client.27502 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562dc6616360 con 0x562dc63071b0 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.371+0000 7f6e511ed640 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc63071b0 0x562dc6732910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.372+0000 7f6e34ff9640 10 client.27502 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.372+0000 7f6e511ed640 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc63071b0 0x562dc6732910 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c43302bcd2b4cd server_cookie=1b567aeaa7e626a0 in_seq=0 out_seq=0 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e34ff9640 1 -- 172.21.15.19:0/195436870 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562dc6616360 con 0x562dc63071b0 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e34ff9640 10 client.27502 handle_command_reply: tid=0 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 client.27502 shutdown 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 msgr2=0x562dc672e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc672e770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e440274b0 tx=0x7f6e44016000 comp rx=0 tx=0).stop 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 shutdown_connections 2024-05-17T09:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562dc63071b0 0x562dc6732910 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562dc62e64d0 0x562dc672e770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e34ff9640 0 client.27502 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc62de2f0 0x562dc66c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 --2- 172.21.15.19:0/195436870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dc62c5d80 0x562dc66c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 >> 172.21.15.19:0/195436870 conn(0x562dc62fb300 msgr2=0x562dc66a2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.449+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 shutdown_connections 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e5a3e8740 1 -- 172.21.15.19:0/195436870 wait complete. 2024-05-17T09:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e5a3e8740 20 client.27502 trim_cache size 0 max 16384 2024-05-17T09:35:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:35:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:35:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:35:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e380b29a0 msgr2=0x7f6e380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e53451640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e380b29a0 0x7f6e380b4eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x562dc66a74d0 tx=0x7f6e440096a0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 msgr2=0x7f6e4c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e53451640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 0x7f6e4c19a200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005760 tx=0x7f6e48004040 comp rx=0 tx=0).stop 2024-05-17T09:35:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.450+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 shutdown_connections 2024-05-17T09:35:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c108a80 0x7f6e4c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6e380b29a0 0x7f6e380b4eb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e4c103d80 0x7f6e4c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 --2- 172.21.15.19:0/3549882426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e4c102b40 0x7f6e4c19a200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 >> 172.21.15.19:0/3549882426 conn(0x7f6e4c0fdd30 msgr2=0x7f6e4c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 shutdown_connections 2024-05-17T09:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:05.451+0000 7f6e53451640 1 -- 172.21.15.19:0/3549882426 wait complete. 2024-05-17T09:35:05.576 INFO:tasks.workunit.client.0.smithi019.stdout: 10 9946 11.52 MB/sec warmup 62 sec latency 118.211 ms 2024-05-17T09:35:06.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 10106 11.47 MB/sec warmup 63 sec latency 60.691 ms 2024-05-17T09:35:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:06 smithi069 ceph-mon[35081]: pgmap v898: 129 pgs: 129 active+clean; 314 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-17T09:35:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:06 smithi019 ceph-mon[31620]: pgmap v898: 129 pgs: 129 active+clean; 314 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-17T09:35:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:06 smithi079 ceph-mon[35502]: pgmap v898: 129 pgs: 129 active+clean; 314 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-17T09:35:07.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 10235 11.34 MB/sec warmup 64 sec latency 75.532 ms 2024-05-17T09:35:08.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 10371 11.25 MB/sec warmup 65 sec latency 83.534 ms 2024-05-17T09:35:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:08 smithi069 ceph-mon[35081]: pgmap v899: 129 pgs: 129 active+clean; 311 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 4.5 MiB/s wr, 280 op/s 2024-05-17T09:35:08.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:08 smithi019 ceph-mon[31620]: pgmap v899: 129 pgs: 129 active+clean; 311 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 4.5 MiB/s wr, 280 op/s 2024-05-17T09:35:09.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:08 smithi079 ceph-mon[35502]: pgmap v899: 129 pgs: 129 active+clean; 311 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 4.5 MiB/s wr, 280 op/s 2024-05-17T09:35:09.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 10525 11.16 MB/sec warmup 66 sec latency 69.058 ms 2024-05-17T09:35:09.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:09 smithi069 ceph-mon[35081]: pgmap v900: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-05-17T09:35:09.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:09 smithi019 ceph-mon[31620]: pgmap v900: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-05-17T09:35:10.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:09 smithi079 ceph-mon[35502]: pgmap v900: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-05-17T09:35:10.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 10702 11.14 MB/sec warmup 67 sec latency 82.168 ms 2024-05-17T09:35:11.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 10888 11.12 MB/sec warmup 68 sec latency 75.965 ms 2024-05-17T09:35:12.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 11044 11.07 MB/sec warmup 69 sec latency 96.540 ms 2024-05-17T09:35:12.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:12 smithi069 ceph-mon[35081]: pgmap v901: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T09:35:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:12 smithi019 ceph-mon[31620]: pgmap v901: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T09:35:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:12 smithi079 ceph-mon[35502]: pgmap v901: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T09:35:13.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 11200 11.02 MB/sec warmup 70 sec latency 81.042 ms 2024-05-17T09:35:14.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 11376 11.07 MB/sec warmup 71 sec latency 88.166 ms 2024-05-17T09:35:14.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:14 smithi069 ceph-mon[35081]: pgmap v902: 129 pgs: 129 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-05-17T09:35:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:14 smithi019 ceph-mon[31620]: pgmap v902: 129 pgs: 129 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-05-17T09:35:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:14 smithi079 ceph-mon[35502]: pgmap v902: 129 pgs: 129 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-05-17T09:35:15.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 11570 11.08 MB/sec warmup 72 sec latency 86.705 ms 2024-05-17T09:35:16.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 11733 10.99 MB/sec warmup 73 sec latency 71.848 ms 2024-05-17T09:35:16.679 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:35:16.679 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.6 2024-05-17T09:35:16.680 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:35:16.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:16 smithi069 ceph-mon[35081]: pgmap v903: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 4.1 MiB/s wr, 261 op/s 2024-05-17T09:35:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:16 smithi019 ceph-mon[31620]: pgmap v903: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 4.1 MiB/s wr, 261 op/s 2024-05-17T09:35:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:16 smithi079 ceph-mon[35502]: pgmap v903: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 4.1 MiB/s wr, 261 op/s 2024-05-17T09:35:17.058 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:35:17.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 11880 10.92 MB/sec warmup 74 sec latency 71.590 ms 2024-05-17T09:35:18.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12030 10.90 MB/sec warmup 75 sec latency 157.240 ms 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 -- 172.21.15.19:0/112473521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88104ec0 msgr2=0x7f7a881052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 --2- 172.21.15.19:0/112473521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88104ec0 0x7f7a881052d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009860 tx=0x7f7a7c02a340 comp rx=0 tx=0).stop 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 -- 172.21.15.19:0/112473521 shutdown_connections 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 --2- 172.21.15.19:0/112473521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 0x7f7a8810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 --2- 172.21.15.19:0/112473521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88104ec0 0x7f7a881052d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 --2- 172.21.15.19:0/112473521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88103c80 0x7f7a88104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 -- 172.21.15.19:0/112473521 >> 172.21.15.19:0/112473521 conn(0x7f7a880fee90 msgr2=0x7f7a88101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 -- 172.21.15.19:0/112473521 shutdown_connections 2024-05-17T09:35:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.577+0000 7f7a8d661640 1 -- 172.21.15.19:0/112473521 wait complete. 2024-05-17T09:35:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 Processor -- start 2024-05-17T09:35:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 -- start start 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 0x7f7a8819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88104ec0 0x7f7a8819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 0x7f7a881a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810f630 con 0x7f7a88104ec0 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810f4b0 con 0x7f7a88103c80 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a8d661640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810f7b0 con 0x7f7a88109bc0 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 0x7f7a8819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88104ec0 0x7f7a8819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 0x7f7a8819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36160/0 (socket says 172.21.15.19:36160) 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a86ffd640 1 -- 172.21.15.19:0/3307088108 learned_addr learned my addr 172.21.15.19:0/3307088108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.578+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 0x7f7a881a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a86ffd640 1 -- 172.21.15.19:0/3307088108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 msgr2=0x7f7a881a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a86ffd640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 0x7f7a881a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a86ffd640 1 -- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88104ec0 msgr2=0x7f7a8819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a86ffd640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88104ec0 0x7f7a8819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a86ffd640 1 -- 172.21.15.19:0/3307088108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a881a1d40 con 0x7f7a88103c80 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 0x7f7a881a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a86ffd640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 0x7f7a8819a200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800e9c0 tx=0x7f7a7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a6ffff640 1 -- 172.21.15.19:0/3307088108 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800ccc0 con 0x7f7a88103c80 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a88074f50 con 0x7f7a88103c80 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a6ffff640 1 -- 172.21.15.19:0/3307088108 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a780035f0 con 0x7f7a88103c80 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.579+0000 7f7a6ffff640 1 -- 172.21.15.19:0/3307088108 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800ce60 con 0x7f7a88103c80 2024-05-17T09:35:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.580+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a880754c0 con 0x7f7a88103c80 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 Processor -- start 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 -- start start 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 0x565360aee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653607242f0 0x565360aee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a6ffff640 1 -- 172.21.15.19:0/3307088108 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7a78004000 con 0x7f7a88103c80 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565360a60c30 con 0x56536072c4d0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565360a507d0 con 0x5653607242f0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.583+0000 7f7a945ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565360a54810 con 0x56536074d1b0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 0x565360aee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 0x565360aee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:54098/0 (socket says 172.21.15.19:54098) 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 -- 172.21.15.19:0/168813850 learned_addr learned my addr 172.21.15.19:0/168813850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a86ffd640 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653607242f0 0x565360aee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a867fc640 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 -- 172.21.15.19:0/168813850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653607242f0 msgr2=0x565360aee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653607242f0 0x565360aee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 -- 172.21.15.19:0/168813850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 msgr2=0x565360af5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 -- 172.21.15.19:0/168813850 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565360765210 con 0x56536074d1b0 2024-05-17T09:35:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a877fe640 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 0x565360aee3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7a800117e0 tx=0x7f7a80011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935e527a5d7ae99e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a6d7fa640 1 -- 172.21.15.19:0/168813850 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80011310 con 0x56536074d1b0 2024-05-17T09:35:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a6d7fa640 1 -- 172.21.15.19:0/168813850 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a80003e00 con 0x56536074d1b0 2024-05-17T09:35:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.584+0000 7f7a6d7fa640 1 -- 172.21.15.19:0/168813850 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80004450 con 0x56536074d1b0 2024-05-17T09:35:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.585+0000 7f7a6ffff640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a700b28a0 0x7f7a700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.585+0000 7f7a86ffd640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a700b28a0 0x7f7a700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.585+0000 7f7a6ffff640 1 -- 172.21.15.19:0/3307088108 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7a780e27c0 con 0x7f7a88103c80 2024-05-17T09:35:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 -- 172.21.15.19:0/168813850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 msgr2=0x565360aee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 0x565360aee3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7a800117e0 tx=0x7f7a80011ca0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a86ffd640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a700b28a0 0x7f7a700b4db0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x565360aef780 tx=0x7f7a780e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 -- 172.21.15.19:0/168813850 shutdown_connections 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653607242f0 0x565360aee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 --2- 172.21.15.19:0/168813850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56536074d1b0 0x565360aee3a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 -- 172.21.15.19:0/168813850 >> 172.21.15.19:0/168813850 conn(0x565360741300 msgr2=0x565360ae9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 -- 172.21.15.19:0/168813850 shutdown_connections 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 -- 172.21.15.19:0/168813850 wait complete. 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 -- start start 2024-05-17T09:35:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.586+0000 7f7a945ef740 1 Processor -- start 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 0x565360af8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536074d1b0 0x565360aff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565360736790 con 0x56536074d1b0 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565360a0c890 con 0x56536072c4d0 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565360af6270 con 0x5653607242f0 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a945ef740 10 client.0 _ll_register_callbacks cb 0x565360a0fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 0x565360af8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 0x565360af8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36182/0 (socket says 172.21.15.19:36182) 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 -- 172.21.15.19:0/3645063209 learned_addr learned my addr 172.21.15.19:0/3645063209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536074d1b0 0x565360aff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 -- 172.21.15.19:0/3645063209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 msgr2=0x565360af7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 -- 172.21.15.19:0/3645063209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536074d1b0 msgr2=0x565360aff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536074d1b0 0x565360aff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a867fc640 1 -- 172.21.15.19:0/3645063209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565360831620 con 0x56536072c4d0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.587+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536074d1b0 0x565360aff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a867fc640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 0x565360af8230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02a930 tx=0x7f7a7c030ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d2faaf6746a761 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3645063209 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c048060 con 0x56536072c4d0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3645063209 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7a7c004450 con 0x56536072c4d0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a80003e00 con 0x56536072c4d0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3645063209 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7a7c04f050 con 0x56536072c4d0 2024-05-17T09:35:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a945ef740 10 client.27536 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x565360b02130 con 0x56536072c4d0 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.588+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3645063209 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a80003e00 con 0x56536072c4d0 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3645063209 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f7a7c004b60 con 0x56536072c4d0 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a945ef740 10 client.27536 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a945ef740 10 client.27536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a945ef740 20 client.27536 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56536070bd80 0x565360b058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a945ef740 4 client.27536 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565360a62b80 con 0x56536070bd80 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.589+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56536070bd80 0x565360b058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56536070bd80 0x565360b058d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ccca86b0d614c2 server_cookie=49e5025c0c6914fd in_seq=0 out_seq=0 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a6aff5640 10 client.27536 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3645063209 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565360a62b80 con 0x56536070bd80 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a6aff5640 10 client.27536 handle_command_reply: tid=0 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a945ef740 1 client.27536 shutdown 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 msgr2=0x565360af8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 0x565360af8230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02a930 tx=0x7f7a7c030ce0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 shutdown_connections 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.590+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56536070bd80 0x565360b058d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536074d1b0 0x565360aff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536072c4d0 0x565360af8230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a6aff5640 0 client.27536 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3645063209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 >> 172.21.15.19:0/3645063209 conn(0x565360741300 msgr2=0x565360aecc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 shutdown_connections 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 1 -- 172.21.15.19:0/3645063209 wait complete. 2024-05-17T09:35:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.591+0000 7f7a945ef740 20 client.27536 trim_cache size 0 max 16384 2024-05-17T09:35:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 Processor -- start 2024-05-17T09:35:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 -- start start 2024-05-17T09:35:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360aff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881ad170 con 0x56536072c4d0 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881aa9e0 con 0x56536074d1b0 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.616+0000 7f7a945ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881a8830 con 0x5653607242f0 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360aff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360aff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36188/0 (socket says 172.21.15.19:36188) 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a877fe640 1 -- 172.21.15.19:0/2985953347 learned_addr learned my addr 172.21.15.19:0/2985953347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a877fe640 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 -- 172.21.15.19:0/2985953347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 msgr2=0x565360af21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 -- 172.21.15.19:0/2985953347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 msgr2=0x565360aff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360aff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 -- 172.21.15.19:0/2985953347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565360af8e30 con 0x56536072c4d0 2024-05-17T09:35:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a877fe640 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360aff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.617+0000 7f7a867fc640 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af2730 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02a860 tx=0x7f7a7c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff081f4ce06523c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.618+0000 7f7a6cff9640 1 -- 172.21.15.19:0/2985953347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c048040 con 0x56536072c4d0 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.618+0000 7f7a6cff9640 1 -- 172.21.15.19:0/2985953347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a7c005850 con 0x56536072c4d0 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.618+0000 7f7a6cff9640 1 -- 172.21.15.19:0/2985953347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c005260 con 0x56536072c4d0 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 -- 172.21.15.19:0/2985953347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 msgr2=0x565360af2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af2730 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02a860 tx=0x7f7a7c02a340 comp rx=0 tx=0).stop 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 -- 172.21.15.19:0/2985953347 shutdown_connections 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360aff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360af2730 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 --2- 172.21.15.19:0/2985953347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360af21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 -- 172.21.15.19:0/2985953347 >> 172.21.15.19:0/2985953347 conn(0x565360741300 msgr2=0x565360aed1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 -- 172.21.15.19:0/2985953347 shutdown_connections 2024-05-17T09:35:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 -- 172.21.15.19:0/2985953347 wait complete. 2024-05-17T09:35:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.619+0000 7f7a945ef740 1 -- start start 2024-05-17T09:35:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 Processor -- start 2024-05-17T09:35:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360afee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360aff360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360b6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881a8ea0 con 0x56536072c4d0 2024-05-17T09:35:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881aa310 con 0x56536074d1b0 2024-05-17T09:35:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881a9570 con 0x5653607242f0 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a945ef740 10 client.0 _ll_register_callbacks cb 0x7f7a7c030900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360aff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a867fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360aff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40046/0 (socket says 172.21.15.19:40046) 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360b6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a867fc640 1 -- 172.21.15.19:0/3714261300 learned_addr learned my addr 172.21.15.19:0/3714261300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360afee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a877fe640 1 -- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 msgr2=0x565360b6c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.620+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360b6c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a877fe640 1 -- 172.21.15.19:0/3714261300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 msgr2=0x565360aff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360aff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a877fe640 1 -- 172.21.15.19:0/3714261300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565360b6c860 con 0x5653607242f0 2024-05-17T09:35:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a867fc640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360aff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a877fe640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360afee20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000f790 tx=0x7f7a80009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c36b58c66b927c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3714261300 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80023040 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3714261300 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7a8000fbc0 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a7c005850 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3714261300 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7a8000fdc0 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a945ef740 10 client.27520 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.621+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x565360b6daf0 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3714261300 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a7c005850 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3714261300 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f7a80018020 con 0x5653607242f0 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a945ef740 10 client.27520 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a945ef740 10 client.27520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a945ef740 20 client.27520 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5653606e11c0 0x565360b701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a945ef740 4 client.27520 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:35:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565360a5c360 con 0x5653606e11c0 2024-05-17T09:35:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.622+0000 7f7a867fc640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5653606e11c0 0x565360b701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.623+0000 7f7a6aff5640 10 client.27520 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.623+0000 7f7a867fc640 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5653606e11c0 0x565360b701a0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4fe91611605630 server_cookie=69bdf4a4451a581d in_seq=0 out_seq=0 2024-05-17T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:18 smithi069 ceph-mon[35081]: pgmap v904: 129 pgs: 129 active+clean; 313 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-05-17T09:35:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:18 smithi019 ceph-mon[31620]: pgmap v904: 129 pgs: 129 active+clean; 313 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-05-17T09:35:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a6aff5640 1 -- 172.21.15.19:0/3714261300 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565360a5c360 con 0x5653606e11c0 2024-05-17T09:35:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a6aff5640 10 client.27520 handle_command_reply: tid=0 2024-05-17T09:35:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 client.27520 shutdown 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 msgr2=0x565360afee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360afee20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000f790 tx=0x7f7a80009500 comp rx=0 tx=0).stop 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 shutdown_connections 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5653606e11c0 0x565360b701a0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56536074d1b0 0x565360b6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a6aff5640 0 client.27520 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56536072c4d0 0x565360aff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 --2- 172.21.15.19:0/3714261300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653607242f0 0x565360afee20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 >> 172.21.15.19:0/3714261300 conn(0x565360741300 msgr2=0x565360af21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 shutdown_connections 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 1 -- 172.21.15.19:0/3714261300 wait complete. 2024-05-17T09:35:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.728+0000 7f7a945ef740 20 client.27520 trim_cache size 0 max 16384 2024-05-17T09:35:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:35:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:35:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a700b28a0 msgr2=0x7f7a700b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a700b28a0 0x7f7a700b4db0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x565360aef780 tx=0x7f7a780e8550 comp rx=0 tx=0).stop 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 msgr2=0x7f7a8819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 0x7f7a8819a200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800e9c0 tx=0x7f7a7800ee80 comp rx=0 tx=0).stop 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 shutdown_connections 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88109bc0 0x7f7a881a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7a700b28a0 0x7f7a700b4db0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88104ec0 0x7f7a8819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 --2- 172.21.15.19:0/3307088108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a88103c80 0x7f7a8819a200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.729+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 >> 172.21.15.19:0/3307088108 conn(0x7f7a880fee90 msgr2=0x7f7a88105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.730+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 shutdown_connections 2024-05-17T09:35:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:18.730+0000 7f7a8d661640 1 -- 172.21.15.19:0/3307088108 wait complete. 2024-05-17T09:35:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:18 smithi079 ceph-mon[35502]: pgmap v904: 129 pgs: 129 active+clean; 313 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-05-17T09:35:19.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12181 10.84 MB/sec warmup 76 sec latency 123.447 ms 2024-05-17T09:35:20.563 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12261 10.74 MB/sec warmup 77 sec latency 222.775 ms 2024-05-17T09:35:20.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:20 smithi069 ceph-mon[35081]: pgmap v905: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.0 MiB/s wr, 276 op/s 2024-05-17T09:35:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:20 smithi019 ceph-mon[31620]: pgmap v905: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.0 MiB/s wr, 276 op/s 2024-05-17T09:35:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:20 smithi079 ceph-mon[35502]: pgmap v905: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.0 MiB/s wr, 276 op/s 2024-05-17T09:35:21.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12387 10.69 MB/sec warmup 78 sec latency 139.232 ms 2024-05-17T09:35:22.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12561 10.69 MB/sec warmup 79 sec latency 73.285 ms 2024-05-17T09:35:22.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:22 smithi069 ceph-mon[35081]: pgmap v906: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-05-17T09:35:22.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:22 smithi019 ceph-mon[31620]: pgmap v906: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-05-17T09:35:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:22 smithi079 ceph-mon[35502]: pgmap v906: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-05-17T09:35:23.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12732 10.77 MB/sec warmup 80 sec latency 168.967 ms 2024-05-17T09:35:24.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12881 10.73 MB/sec warmup 81 sec latency 188.332 ms 2024-05-17T09:35:24.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:24 smithi019 ceph-mon[31620]: pgmap v907: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-05-17T09:35:24.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:24 smithi069 ceph-mon[35081]: pgmap v907: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-05-17T09:35:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:24 smithi079 ceph-mon[35502]: pgmap v907: 129 pgs: 129 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-05-17T09:35:25.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 12996 10.71 MB/sec warmup 82 sec latency 158.752 ms 2024-05-17T09:35:26.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13109 10.65 MB/sec warmup 83 sec latency 141.623 ms 2024-05-17T09:35:26.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:26 smithi069 ceph-mon[35081]: pgmap v908: 129 pgs: 129 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-17T09:35:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:26 smithi019 ceph-mon[31620]: pgmap v908: 129 pgs: 129 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-17T09:35:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:26 smithi079 ceph-mon[35502]: pgmap v908: 129 pgs: 129 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-17T09:35:27.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13203 10.59 MB/sec warmup 84 sec latency 142.090 ms 2024-05-17T09:35:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.984+0000 7fa4117fc700 1 -- 172.21.15.19:0/2468041323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 msgr2=0x7fa40c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.984+0000 7fa4117fc700 1 --2- 172.21.15.19:0/2468041323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c1081f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa4000099c0 tx=0x7fa400009cd0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.986+0000 7fa4117fc700 1 -- 172.21.15.19:0/2468041323 shutdown_connections 2024-05-17T09:35:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.986+0000 7fa4117fc700 1 --2- 172.21.15.19:0/2468041323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa40c108730 0x7fa40c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.986+0000 7fa4117fc700 1 --2- 172.21.15.19:0/2468041323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.986+0000 7fa4117fc700 1 --2- 172.21.15.19:0/2468041323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c1081f0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.986+0000 7fa4117fc700 1 -- 172.21.15.19:0/2468041323 >> 172.21.15.19:0/2468041323 conn(0x7fa40c0fedb0 msgr2=0x7fa40c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.988+0000 7fa4117fc700 1 -- 172.21.15.19:0/2468041323 shutdown_connections 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.989+0000 7fa4117fc700 1 -- 172.21.15.19:0/2468041323 wait complete. 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.990+0000 7fa4117fc700 1 Processor -- start 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.990+0000 7fa4117fc700 1 -- start start 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.990+0000 7fa4117fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.990+0000 7fa4117fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c10a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.990+0000 7fa4117fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa40c108730 0x7fa40c10a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa4117fc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c113210 con 0x7fa40c107de0 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa4117fc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c113090 con 0x7fa40c1034b0 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa4117fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c113390 con 0x7fa40c108730 2024-05-17T09:35:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c10bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c10bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47664/0 (socket says 172.21.15.19:47664) 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 -- 172.21.15.19:0/1598613736 learned_addr learned my addr 172.21.15.19:0/1598613736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40b7fe700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa40c108730 0x7fa40c10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40a7fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c10a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 -- 172.21.15.19:0/1598613736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa40c108730 msgr2=0x7fa40c10a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa40c108730 0x7fa40c10a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 -- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 msgr2=0x7fa40c10a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c10a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40affd700 1 -- 172.21.15.19:0/1598613736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c10af40 con 0x7fa40c1034b0 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.991+0000 7fa40a7fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.992+0000 7fa40affd700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c10bcd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9a0 tx=0x7fa3fc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.992+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00cbd0 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.992+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3fc00cd70 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.992+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1ad7c0 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.992+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc018680 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.992+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1add20 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.994+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c104960 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.997+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa3fc010430 con 0x7fa40c1034b0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.998+0000 7fa3f3fff700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3f40b2900 0x7fa3f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.998+0000 7fa40a7fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3f40b2900 0x7fa3f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.998+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa3fc0e2b00 con 0x7fa40c1034b0 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:34:12.645154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:26.998+0000 7fa40a7fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3f40b2900 0x7fa3f40b4e10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa400000c00 tx=0x7fa40001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.002+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3fc0aa470 con 0x7fa40c1034b0 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.391+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c0512b0 con 0x7fa40c1034b0 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.392+0000 7fa3f3fff700 1 -- 172.21.15.19:0/1598613736 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11858 (secure 0 0 0) 0x7fa3fc0a9d70 con 0x7fa40c1034b0 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3f40b2900 msgr2=0x7fa3f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3f40b2900 0x7fa3f40b4e10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa400000c00 tx=0x7fa40001a040 comp rx=0 tx=0).stop 2024-05-17T09:35:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 msgr2=0x7fa40c10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c10bcd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9a0 tx=0x7fa3fc00ecb0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 shutdown_connections 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa3f40b2900 0x7fa3f40b4e10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa40c108730 0x7fa40c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa40c1034b0 0x7fa40c10bcd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 --2- 172.21.15.19:0/1598613736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa40c107de0 0x7fa40c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 >> 172.21.15.19:0/1598613736 conn(0x7fa40c0fedb0 msgr2=0x7fa40c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 shutdown_connections 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:27.395+0000 7fa4117fc700 1 -- 172.21.15.19:0/1598613736 wait complete. 2024-05-17T09:35:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-05-17T09:35:28.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13359 10.59 MB/sec warmup 85 sec latency 77.835 ms 2024-05-17T09:35:28.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:28 smithi019 ceph-mon[31620]: pgmap v909: 129 pgs: 129 active+clean; 325 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-17T09:35:28.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:28 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1598613736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:28.605 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 147.6 --await-for 45.0' 2024-05-17T09:35:28.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 147.6 --await-for 45.0 2024-05-17T09:35:28.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:28 smithi069 ceph-mon[35081]: pgmap v909: 129 pgs: 129 active+clean; 325 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-17T09:35:28.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:28 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1598613736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:28 smithi079 ceph-mon[35502]: pgmap v909: 129 pgs: 129 active+clean; 325 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-17T09:35:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:28 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1598613736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:29.012 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:35:29.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13512 10.57 MB/sec warmup 86 sec latency 80.734 ms 2024-05-17T09:35:30.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:30 smithi019 ceph-mon[31620]: pgmap v910: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-17T09:35:30.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13643 10.47 MB/sec warmup 87 sec latency 69.225 ms 2024-05-17T09:35:30.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:30 smithi069 ceph-mon[35081]: pgmap v910: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-17T09:35:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:30 smithi079 ceph-mon[35502]: pgmap v910: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-17T09:35:31.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 -- 172.21.15.19:0/2323264275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654107de0 msgr2=0x7f86541081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 --2- 172.21.15.19:0/2323264275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654107de0 0x7f86541081f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8644009990 tx=0x7f8644009ca0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 -- 172.21.15.19:0/2323264275 shutdown_connections 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 --2- 172.21.15.19:0/2323264275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8654108730 0x7f8654110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 --2- 172.21.15.19:0/2323264275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 --2- 172.21.15.19:0/2323264275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654107de0 0x7f86541081f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.325+0000 7f865c3dc700 1 -- 172.21.15.19:0/2323264275 >> 172.21.15.19:0/2323264275 conn(0x7f86540fedb0 msgr2=0x7f86541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.326+0000 7f865c3dc700 1 -- 172.21.15.19:0/2323264275 shutdown_connections 2024-05-17T09:35:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.326+0000 7f865c3dc700 1 -- 172.21.15.19:0/2323264275 wait complete. 2024-05-17T09:35:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.327+0000 7f865c3dc700 1 Processor -- start 2024-05-17T09:35:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.327+0000 7f865c3dc700 1 -- start start 2024-05-17T09:35:31.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865c3dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865c3dc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8654107de0 0x7f86541a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865c3dc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654108730 0x7f86541a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865c3dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86541121c0 con 0x7f86541034b0 2024-05-17T09:35:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865c3dc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8654112040 con 0x7f8654108730 2024-05-17T09:35:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865c3dc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8654112340 con 0x7f8654107de0 2024-05-17T09:35:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55728/0 (socket says 172.21.15.19:55728) 2024-05-17T09:35:31.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 -- 172.21.15.19:0/3745054149 learned_addr learned my addr 172.21.15.19:0/3745054149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f8659977700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8654107de0 0x7f86541a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 -- 172.21.15.19:0/3745054149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8654107de0 msgr2=0x7f86541a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8654107de0 0x7f86541a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 -- 172.21.15.19:0/3745054149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654108730 msgr2=0x7f86541a5bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654108730 0x7f86541a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.328+0000 7f865a178700 1 -- 172.21.15.19:0/3745054149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86541a6260 con 0x7f86541034b0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.329+0000 7f865a178700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541a13e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f865000bed0 tx=0x7f86500091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.329+0000 7f864b7fe700 1 -- 172.21.15.19:0/3745054149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500099f0 con 0x7f86541034b0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.329+0000 7f864b7fe700 1 -- 172.21.15.19:0/3745054149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8650009b90 con 0x7f86541034b0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.329+0000 7f864b7fe700 1 -- 172.21.15.19:0/3745054149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650016e40 con 0x7f86541034b0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.329+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86541a6550 con 0x7f86541034b0 2024-05-17T09:35:31.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.329+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86541a69e0 con 0x7f86541034b0 2024-05-17T09:35:31.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.333+0000 7f864b7fe700 1 -- 172.21.15.19:0/3745054149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f865001f420 con 0x7f86541034b0 2024-05-17T09:35:31.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f864b7fe700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86400b2a50 0x7f86400b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 Processor -- start 2024-05-17T09:35:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f8659977700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86400b2a50 0x7f86400b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 -- start start 2024-05-17T09:35:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec21bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec216580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5652ec143f00 con 0x5652ec2160b0 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652ec133aa0 con 0x5652ec096fd0 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.335+0000 7f866adf1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5652ec137ae0 con 0x5652ec1825c0 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a979700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec216580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec21bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a979700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec216580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55730/0 (socket says 172.21.15.19:55730) 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a979700 1 -- 172.21.15.19:0/1492837487 learned_addr learned my addr 172.21.15.19:0/1492837487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec21bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41416/0 (socket says 172.21.15.19:41416) 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f864b7fe700 1 -- 172.21.15.19:0/3745054149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86500e2420 con 0x7f86541034b0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f8659977700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86400b2a50 0x7f86400b4f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f864400b2e0 tx=0x7f864400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f8659977700 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 -- 172.21.15.19:0/1492837487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 msgr2=0x5652ec21c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 -- 172.21.15.19:0/1492837487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 msgr2=0x5652ec216580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec216580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 -- 172.21.15.19:0/1492837487 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ec0dd140 con 0x5652ec1825c0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f8659977700 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.336+0000 7f865a178700 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec21bf20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f86500dfab0 tx=0x7f8650011d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b8f96cb11d52f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.337+0000 7f8648d47700 1 -- 172.21.15.19:0/1492837487 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865007c7b0 con 0x5652ec1825c0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.337+0000 7f8648d47700 1 -- 172.21.15.19:0/1492837487 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86500a9140 con 0x5652ec1825c0 2024-05-17T09:35:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.337+0000 7f8648d47700 1 -- 172.21.15.19:0/1492837487 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500071c0 con 0x5652ec1825c0 2024-05-17T09:35:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 -- 172.21.15.19:0/1492837487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 msgr2=0x5652ec21bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec21bf20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f86500dfab0 tx=0x7f8650011d20 comp rx=0 tx=0).stop 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 -- 172.21.15.19:0/1492837487 shutdown_connections 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec21bf20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec216580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1492837487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 -- 172.21.15.19:0/1492837487 >> 172.21.15.19:0/1492837487 conn(0x5652ec0f7350 msgr2=0x5652ec210d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 -- 172.21.15.19:0/1492837487 shutdown_connections 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.338+0000 7f866adf1b80 1 -- 172.21.15.19:0/1492837487 wait complete. 2024-05-17T09:35:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.339+0000 7f866adf1b80 1 -- start start 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.339+0000 7f866adf1b80 1 Processor -- start 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec220110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec220650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f865a178700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f865a178700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47700/0 (socket says 172.21.15.19:47700) 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f8659977700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec220110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f8659977700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec220110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41432/0 (socket says 172.21.15.19:41432) 2024-05-17T09:35:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f865a178700 1 -- 172.21.15.19:0/2180509577 learned_addr learned my addr 172.21.15.19:0/2180509577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5652ec03a130 con 0x5652ec2160b0 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652ec09bf30 con 0x5652ec096fd0 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5652ec222bb0 con 0x5652ec1825c0 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.340+0000 7f866adf1b80 10 client.0 _ll_register_callbacks cb 0x5652ec0ceae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec220650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 -- 172.21.15.19:0/2180509577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 msgr2=0x5652ec220110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec220110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 -- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 msgr2=0x5652ec21fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ec083a60 con 0x5652ec2160b0 2024-05-17T09:35:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f8659977700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec220110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.341+0000 7f865a979700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec220650 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f864c00bc30 tx=0x7f864c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbed98ce280c92b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f863dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f863dffb700 1 -- 172.21.15.19:0/2180509577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c01a070 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f863dffb700 1 -- 172.21.15.19:0/2180509577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f864c004db0 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86500a9140 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f863dffb700 1 -- 172.21.15.19:0/2180509577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f864c022050 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f866adf1b80 10 client.17991 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.342+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5652ec0ed8f0 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f863dffb700 1 -- 172.21.15.19:0/2180509577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f864c003a00 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f863dffb700 1 -- 172.21.15.19:0/2180509577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f864c01d030 con 0x5652ec2160b0 2024-05-17T09:35:31.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f866adf1b80 10 client.17991 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f866adf1b80 10 client.17991 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f866adf1b80 20 client.17991 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f866adf1b80 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec22fe90 0x5652ec2322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f866adf1b80 4 client.17991 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86500a9140 con 0x5652ec22fe90 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f865a178700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec22fe90 0x5652ec2322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.343+0000 7f863dffb700 10 client.17991 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.344+0000 7f865a178700 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec22fe90 0x5652ec2322e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf666552d9762839 server_cookie=654ff2fb32745db8 in_seq=0 out_seq=0 2024-05-17T09:35:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f863dffb700 1 -- 172.21.15.19:0/2180509577 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f86500a9140 con 0x5652ec22fe90 2024-05-17T09:35:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f863dffb700 10 client.17991 handle_command_reply: tid=0 2024-05-17T09:35:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 client.17991 shutdown 2024-05-17T09:35:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 msgr2=0x5652ec220650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec220650 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f864c00bc30 tx=0x7f864c009d60 comp rx=0 tx=0).stop 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 shutdown_connections 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec22fe90 0x5652ec2322e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec2160b0 0x5652ec220650 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec21fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 --2- 172.21.15.19:0/2180509577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec1825c0 0x5652ec220110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 >> 172.21.15.19:0/2180509577 conn(0x5652ec0f7350 msgr2=0x5652ec2150a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f863dffb700 0 client.17991 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 shutdown_connections 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 1 -- 172.21.15.19:0/2180509577 wait complete. 2024-05-17T09:35:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.345+0000 7f866adf1b80 20 client.17991 trim_cache size 0 max 16384 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.377+0000 7f866adf1b80 1 Processor -- start 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.377+0000 7f866adf1b80 1 -- start start 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.377+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 0x5652ec222070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec1825c0 0x5652ec20b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 0x5652ec222070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec22ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f866adf1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86541b9300 con 0x5652ec096fd0 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f866adf1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86541b6b30 con 0x5652ec1825c0 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f866adf1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86541b4980 con 0x5652ec2160b0 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 0x5652ec222070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55750/0 (socket says 172.21.15.19:55750) 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 -- 172.21.15.19:0/1631436933 learned_addr learned my addr 172.21.15.19:0/1631436933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f8659977700 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec1825c0 0x5652ec20b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a979700 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec22ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 -- 172.21.15.19:0/1631436933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 msgr2=0x5652ec22ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec22ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 -- 172.21.15.19:0/1631436933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec1825c0 msgr2=0x5652ec20b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec1825c0 0x5652ec20b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 -- 172.21.15.19:0/1631436933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ec22d420 con 0x5652ec096fd0 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f8659977700 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec1825c0 0x5652ec20b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f865a178700 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 0x5652ec222070 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f86500dfae0 tx=0x7f86500cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ac8ec530f2cb13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.378+0000 7f863ffff700 1 -- 172.21.15.19:0/1631436933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500c3070 con 0x5652ec096fd0 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.379+0000 7f863ffff700 1 -- 172.21.15.19:0/1631436933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86500a9140 con 0x5652ec096fd0 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.379+0000 7f863ffff700 1 -- 172.21.15.19:0/1631436933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650083070 con 0x5652ec096fd0 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 -- 172.21.15.19:0/1631436933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 msgr2=0x5652ec222070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 0x5652ec222070 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f86500dfae0 tx=0x7f86500cc040 comp rx=0 tx=0).stop 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 -- 172.21.15.19:0/1631436933 shutdown_connections 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec22ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec096fd0 0x5652ec222070 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1631436933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec1825c0 0x5652ec20b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 -- 172.21.15.19:0/1631436933 >> 172.21.15.19:0/1631436933 conn(0x5652ec0f7350 msgr2=0x5652ec20ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 -- 172.21.15.19:0/1631436933 shutdown_connections 2024-05-17T09:35:31.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 -- 172.21.15.19:0/1631436933 wait complete. 2024-05-17T09:35:31.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.380+0000 7f866adf1b80 1 -- start start 2024-05-17T09:35:31.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 Processor -- start 2024-05-17T09:35:31.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec227b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 0x5652ec2280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec224b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86541b4ff0 con 0x5652ec1825c0 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86541b6460 con 0x5652ec096fd0 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86541b56c0 con 0x5652ec2160b0 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f8659977700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 0x5652ec2280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f8659977700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 0x5652ec2280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55754/0 (socket says 172.21.15.19:55754) 2024-05-17T09:35:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f866adf1b80 10 client.0 _ll_register_callbacks cb 0x5652ec0ceae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f8659977700 1 -- 172.21.15.19:0/1861245349 learned_addr learned my addr 172.21.15.19:0/1861245349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.381+0000 7f865a178700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec227b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f865a979700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec224b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f8659977700 1 -- 172.21.15.19:0/1861245349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 msgr2=0x5652ec224b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f8659977700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec224b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f8659977700 1 -- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 msgr2=0x5652ec227b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f8659977700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec227b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f8659977700 1 -- 172.21.15.19:0/1861245349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ec228620 con 0x5652ec1825c0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f865a178700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec227b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f8659977700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 0x5652ec2280b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8644009990 tx=0x7f8644009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888161b0aa65e615 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f863dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f863dffb700 1 -- 172.21.15.19:0/1861245349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864400fe60 con 0x5652ec1825c0 2024-05-17T09:35:31.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f863dffb700 1 -- 172.21.15.19:0/1861245349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f864401d030 con 0x5652ec1825c0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.382+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86500a9140 con 0x5652ec1825c0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f863dffb700 1 -- 172.21.15.19:0/1861245349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f864403b050 con 0x5652ec1825c0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 10 client.17997 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5652ec29ad90 con 0x5652ec1825c0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f863dffb700 1 -- 172.21.15.19:0/1861245349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86500a9140 con 0x5652ec1825c0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f863dffb700 1 -- 172.21.15.19:0/1861245349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f86440224c0 con 0x5652ec1825c0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 10 client.17997 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 10 client.17997 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 20 client.17997 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec29b620 0x5652ec29da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 4 client.17997 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.6, "await_for": 45.0}] 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.6, "await_for": 45.0}) -- 0x5652ec1e2060 con 0x5652ec29b620 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.383+0000 7f865a178700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec29b620 0x5652ec29da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.384+0000 7f863dffb700 10 client.17997 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.384+0000 7f865a178700 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec29b620 0x5652ec29da40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19635586ab7b29b8 server_cookie=27cb223b733d03d1 in_seq=0 out_seq=0 2024-05-17T09:35:31.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13759 10.43 MB/sec warmup 88 sec latency 179.741 ms 2024-05-17T09:35:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.776+0000 7f863dffb700 1 -- 172.21.15.19:0/1861245349 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5652ec1e2060 con 0x5652ec29b620 2024-05-17T09:35:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f863dffb700 10 client.17997 handle_command_reply: tid=0 2024-05-17T09:35:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 client.17997 shutdown 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 msgr2=0x5652ec2280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 0x5652ec2280b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8644009990 tx=0x7f8644009ca0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 shutdown_connections 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5652ec2160b0 0x5652ec224b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5652ec29b620 0x5652ec29da40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652ec096fd0 0x5652ec227b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 --2- 172.21.15.19:0/1861245349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5652ec1825c0 0x5652ec2280b0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 >> 172.21.15.19:0/1861245349 conn(0x5652ec0f7350 msgr2=0x5652ec2150a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f863dffb700 0 client.17997 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 shutdown_connections 2024-05-17T09:35:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.777+0000 7f866adf1b80 1 -- 172.21.15.19:0/1861245349 wait complete. 2024-05-17T09:35:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.778+0000 7f866adf1b80 20 client.17997 trim_cache size 0 max 16384 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 35, 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 54, 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "9d531f9d": { 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 54, 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:35:31.779 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 147.6, 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:35:31.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:31.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:31.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:31.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:31.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:31.781 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86400b2a50 msgr2=0x7f86400b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86400b2a50 0x7f86400b4f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f864400b2e0 tx=0x7f864400b610 comp rx=0 tx=0).stop 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 msgr2=0x7f86541a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541a13e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f865000bed0 tx=0x7f86500091e0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 shutdown_connections 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8654108730 0x7f86541a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86541034b0 0x7f86541a13e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86400b2a50 0x7f86400b4f60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 --2- 172.21.15.19:0/3745054149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8654107de0 0x7f86541a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 >> 172.21.15.19:0/3745054149 conn(0x7f86540fedb0 msgr2=0x7f8654101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 shutdown_connections 2024-05-17T09:35:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:31.779+0000 7f865c3dc700 1 -- 172.21.15.19:0/3745054149 wait complete. 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 -- 172.21.15.19:0/3765909905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 msgr2=0x7f17f410bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 --2- 172.21.15.19:0/3765909905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 0x7f17f410bfa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009ca0 tx=0x7f17ec02a8c0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 -- 172.21.15.19:0/3765909905 shutdown_connections 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 --2- 172.21.15.19:0/3765909905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 0x7f17f410bfa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 --2- 172.21.15.19:0/3765909905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 0x7f17f4105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 --2- 172.21.15.19:0/3765909905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f4104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 -- 172.21.15.19:0/3765909905 >> 172.21.15.19:0/3765909905 conn(0x7f17f40fee50 msgr2=0x7f17f41012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 -- 172.21.15.19:0/3765909905 shutdown_connections 2024-05-17T09:35:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.857+0000 7f17fbe61640 1 -- 172.21.15.19:0/3765909905 wait complete. 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 Processor -- start 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 -- start start 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 0x7f17f419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 0x7f17f41a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410f670 con 0x7f17f4103c40 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410f4f0 con 0x7f17f4109b80 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.858+0000 7f17fbe61640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410f7f0 con 0x7f17f4104e80 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55764/0 (socket says 172.21.15.19:55764) 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 -- 172.21.15.19:0/360426660 learned_addr learned my addr 172.21.15.19:0/360426660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f93fc640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 0x7f17f419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 0x7f17f41a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 -- 172.21.15.19:0/360426660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 msgr2=0x7f17f419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 0x7f17f419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 -- 172.21.15.19:0/360426660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 msgr2=0x7f17f41a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 0x7f17f41a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 -- 172.21.15.19:0/360426660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f41a1d20 con 0x7f17f4103c40 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f93fc640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 0x7f17f419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f419a1e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e980 tx=0x7f17f000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17eaffd640 1 -- 172.21.15.19:0/360426660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000cd80 con 0x7f17f4103c40 2024-05-17T09:35:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17eaffd640 1 -- 172.21.15.19:0/360426660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17f00108c0 con 0x7f17f4103c40 2024-05-17T09:35:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f41a1fb0 con 0x7f17f4103c40 2024-05-17T09:35:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17eaffd640 1 -- 172.21.15.19:0/360426660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0018840 con 0x7f17f4103c40 2024-05-17T09:35:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.859+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f41a2470 con 0x7f17f4103c40 2024-05-17T09:35:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.864+0000 7f1802dee740 1 Processor -- start 2024-05-17T09:35:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.864+0000 7f1802dee740 1 -- start start 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.864+0000 7f1802dee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f1802dee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfeb0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f1802dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f1802dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2dfe22c30 con 0x55d2dfaee4d0 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f1802dee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2dfe127d0 con 0x55d2dfae62f0 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f1802dee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d2dfe16810 con 0x55d2dfb0f1b0 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f17fa3fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f17fa3fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41464/0 (socket says 172.21.15.19:41464) 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1341209028 learned_addr learned my addr 172.21.15.19:0/1341209028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.865+0000 7f17f93fc640 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.866+0000 7f17f93fc640 1 -- 172.21.15.19:0/1341209028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 msgr2=0x55d2dfeb03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.866+0000 7f17f93fc640 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.866+0000 7f17f93fc640 1 -- 172.21.15.19:0/1341209028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 msgr2=0x55d2dfeb0920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:35:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.866+0000 7f17f93fc640 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfeb0920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.866+0000 7f17f93fc640 1 -- 172.21.15.19:0/1341209028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dfb27210 con 0x55d2dfaee4d0 2024-05-17T09:35:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.866+0000 7f17f93fc640 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb7820 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009fa0 tx=0x7f17e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ca9a24e1723c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.867+0000 7f17dffff640 1 -- 172.21.15.19:0/1341209028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40040f0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.867+0000 7f17dffff640 1 -- 172.21.15.19:0/1341209028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17e4004290 con 0x55d2dfaee4d0 2024-05-17T09:35:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.867+0000 7f17dffff640 1 -- 172.21.15.19:0/1341209028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400f700 con 0x55d2dfaee4d0 2024-05-17T09:35:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 -- 172.21.15.19:0/1341209028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 msgr2=0x55d2dfeb7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb7820 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009fa0 tx=0x7f17e40118d0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f17eaffd640 1 -- 172.21.15.19:0/360426660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17f001f020 con 0x7f17f4103c40 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 -- 172.21.15.19:0/1341209028 shutdown_connections 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb7820 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfeb0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 --2- 172.21.15.19:0/1341209028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 -- 172.21.15.19:0/1341209028 >> 172.21.15.19:0/1341209028 conn(0x55d2dfb03300 msgr2=0x55d2dfeabd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 -- 172.21.15.19:0/1341209028 shutdown_connections 2024-05-17T09:35:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.868+0000 7f1802dee740 1 -- 172.21.15.19:0/1341209028 wait complete. 2024-05-17T09:35:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.870+0000 7f17eaffd640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17e00b2830 0x7f17e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.870+0000 7f17eaffd640 1 -- 172.21.15.19:0/360426660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f17f00e26c0 con 0x7f17f4103c40 2024-05-17T09:35:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.870+0000 7f1802dee740 1 -- start start 2024-05-17T09:35:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.870+0000 7f1802dee740 1 Processor -- start 2024-05-17T09:35:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.871+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17e00b2830 0x7f17e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.871+0000 7f1802dee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfebb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.871+0000 7f1802dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfebbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f1802dee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f1802dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2dfaf8790 con 0x55d2dfaee4d0 2024-05-17T09:35:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f17f93fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfebbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f17f93fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfebbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55782/0 (socket says 172.21.15.19:55782) 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f17f93fc640 1 -- 172.21.15.19:0/3380753965 learned_addr learned my addr 172.21.15.19:0/3380753965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfebb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2dfdce890 con 0x55d2dfae62f0 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d2dfebc6d0 con 0x55d2dfb0f1b0 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.872+0000 7f1802dee740 10 client.0 _ll_register_callbacks cb 0x55d2dfdd1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17f93fc640 1 -- 172.21.15.19:0/3380753965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 msgr2=0x55d2dfeb8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17f93fc640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17f93fc640 1 -- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 msgr2=0x55d2dfebb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17f93fc640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfebb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17f93fc640 1 -- 172.21.15.19:0/3380753965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dfbf3620 con 0x55d2dfaee4d0 2024-05-17T09:35:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:35:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.873+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17e00b2830 0x7f17e00b4d40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f17ec02aeb0 tx=0x7f17ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.876+0000 7f17f93fc640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfebbc20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f17e401b040 tx=0x7f17e4009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1833a91f717cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.876+0000 7f17dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.876+0000 7f17dd7fa640 1 -- 172.21.15.19:0/3380753965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4028020 con 0x55d2dfaee4d0 2024-05-17T09:35:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.876+0000 7f17dd7fa640 1 -- 172.21.15.19:0/3380753965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f17e4016040 con 0x55d2dfaee4d0 2024-05-17T09:35:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.876+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e4004290 con 0x55d2dfaee4d0 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.877+0000 7f17dd7fa640 1 -- 172.21.15.19:0/3380753965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f17e40035f0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.877+0000 7f1802dee740 10 client.18009 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.877+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d2dfecc9f0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f17dd7fa640 1 -- 172.21.15.19:0/3380753965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17e4004290 con 0x55d2dfaee4d0 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f17dd7fa640 1 -- 172.21.15.19:0/3380753965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f17e400f4e0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f1802dee740 10 client.18009 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f1802dee740 10 client.18009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f1802dee740 20 client.18009 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f1802dee740 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfacdd80 0x55d2dfecfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f1802dee740 4 client.18009 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2dfe24b80 con 0x55d2dfacdd80 2024-05-17T09:35:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.878+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfacdd80 0x55d2dfecfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.880+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfacdd80 0x55d2dfecfff0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2859854679144691 server_cookie=fd819f159be41488 in_seq=0 out_seq=0 2024-05-17T09:35:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.880+0000 7f17dd7fa640 10 client.18009 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.881+0000 7f17dd7fa640 1 -- 172.21.15.19:0/3380753965 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d2dfe24b80 con 0x55d2dfacdd80 2024-05-17T09:35:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.881+0000 7f17dd7fa640 10 client.18009 handle_command_reply: tid=0 2024-05-17T09:35:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.881+0000 7f1802dee740 1 client.18009 shutdown 2024-05-17T09:35:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.882+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 msgr2=0x55d2dfebbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.882+0000 7f1802dee740 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfebbc20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f17e401b040 tx=0x7f17e4009be0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.882+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 shutdown_connections 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f1802dee740 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfacdd80 0x55d2dfecfff0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f1802dee740 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfb0f1b0 0x55d2dfeb8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f1802dee740 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfebbc20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f1802dee740 1 --2- 172.21.15.19:0/3380753965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfae62f0 0x55d2dfebb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f17dd7fa640 0 client.18009 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 >> 172.21.15.19:0/3380753965 conn(0x55d2dfb03300 msgr2=0x55d2dfeaad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.883+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 shutdown_connections 2024-05-17T09:35:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.884+0000 7f1802dee740 1 -- 172.21.15.19:0/3380753965 wait complete. 2024-05-17T09:35:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.884+0000 7f1802dee740 20 client.18009 trim_cache size 0 max 16384 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 Processor -- start 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 -- start start 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfeaf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfeaf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41ad1e0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41aaa50 con 0x55d2dfacdd80 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f17fa3fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f1802dee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41a88a0 con 0x55d2dfae62f0 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f17fa3fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55790/0 (socket says 172.21.15.19:55790) 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.906+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1496203039 learned_addr learned my addr 172.21.15.19:0/1496203039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17f93fc640 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfeaf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfeaf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1496203039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 msgr2=0x55d2dfeaf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfeaf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1496203039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 msgr2=0x55d2dfeaf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfeaf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1496203039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dfebc030 con 0x55d2dfaee4d0 2024-05-17T09:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfeaf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb37c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f17ec035000 tx=0x7f17ec041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70045f209e2dd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17df7fe640 1 -- 172.21.15.19:0/1496203039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec003a30 con 0x55d2dfaee4d0 2024-05-17T09:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17df7fe640 1 -- 172.21.15.19:0/1496203039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17ec036050 con 0x55d2dfaee4d0 2024-05-17T09:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.907+0000 7f17df7fe640 1 -- 172.21.15.19:0/1496203039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0305c0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 -- 172.21.15.19:0/1496203039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 msgr2=0x55d2dfeb37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb37c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f17ec035000 tx=0x7f17ec041000 comp rx=0 tx=0).stop 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 -- 172.21.15.19:0/1496203039 shutdown_connections 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfeb37c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfeaf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 --2- 172.21.15.19:0/1496203039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfeaf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 -- 172.21.15.19:0/1496203039 >> 172.21.15.19:0/1496203039 conn(0x55d2dfb03300 msgr2=0x55d2dfeaeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 -- 172.21.15.19:0/1496203039 shutdown_connections 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.908+0000 7f1802dee740 1 -- 172.21.15.19:0/1496203039 wait complete. 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 -- start start 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 Processor -- start 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfedf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfedfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfee0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41a8f10 con 0x55d2dfaee4d0 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41aa380 con 0x55d2dfacdd80 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17f41a95e0 con 0x55d2dfae62f0 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f17fa3fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfee0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f17f9bfd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfedf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f17fa3fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfee0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55802/0 (socket says 172.21.15.19:55802) 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1071849173 learned_addr learned my addr 172.21.15.19:0/1071849173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f17f9bfd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfedf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47756/0 (socket says 172.21.15.19:47756) 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.909+0000 7f1802dee740 10 client.0 _ll_register_callbacks cb 0x7f17e4005450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17f93fc640 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfedfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1071849173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 msgr2=0x55d2dfedfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfedfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 msgr2=0x55d2dfedf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfedf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17fa3fe640 1 -- 172.21.15.19:0/1071849173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dfee1090 con 0x55d2dfaee4d0 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17fa3fe640 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfee0ad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f17ec054040 tx=0x7f17ec003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fadb58b0c4646c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17dd7fa640 1 -- 172.21.15.19:0/1071849173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec05f060 con 0x55d2dfaee4d0 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17dd7fa640 1 -- 172.21.15.19:0/1071849173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f17ec042040 con 0x55d2dfaee4d0 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ec036050 con 0x55d2dfaee4d0 2024-05-17T09:35:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f17dd7fa640 1 -- 172.21.15.19:0/1071849173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f17ec004000 con 0x55d2dfaee4d0 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f1802dee740 10 client.18015 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.910+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d2dff385c0 con 0x55d2dfaee4d0 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f17dd7fa640 1 -- 172.21.15.19:0/1071849173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17ec036050 con 0x55d2dfaee4d0 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f17dd7fa640 1 -- 172.21.15.19:0/1071849173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f17ec047420 con 0x55d2dfaee4d0 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f1802dee740 10 client.18015 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f1802dee740 10 client.18015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f1802dee740 20 client.18015 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f1802dee740 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfb0f1b0 0x55d2dff3b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f1802dee740 4 client.18015 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2dfe1e360 con 0x55d2dfb0f1b0 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.911+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfb0f1b0 0x55d2dff3b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.912+0000 7f17dd7fa640 10 client.18015 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.912+0000 7f17f9bfd640 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfb0f1b0 0x55d2dff3b040 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf1c12a09bc1b85 server_cookie=85fbd54698f10edb in_seq=0 out_seq=0 2024-05-17T09:35:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f17dd7fa640 1 -- 172.21.15.19:0/1071849173 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d2dfe1e360 con 0x55d2dfb0f1b0 2024-05-17T09:35:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f17dd7fa640 10 client.18015 handle_command_reply: tid=0 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 client.18015 shutdown 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 msgr2=0x55d2dfee0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfee0ad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f17ec054040 tx=0x7f17ec003f00 comp rx=0 tx=0).stop 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 shutdown_connections 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d2dfb0f1b0 0x55d2dff3b040 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2dfaee4d0 0x55d2dfee0ad0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d2dfae62f0 0x55d2dfedfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f17dd7fa640 0 client.18015 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 --2- 172.21.15.19:0/1071849173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2dfacdd80 0x55d2dfedf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 >> 172.21.15.19:0/1071849173 conn(0x55d2dfb03300 msgr2=0x55d2dfeaf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 shutdown_connections 2024-05-17T09:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 1 -- 172.21.15.19:0/1071849173 wait complete. 2024-05-17T09:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.981+0000 7f1802dee740 20 client.18015 trim_cache size 0 max 16384 2024-05-17T09:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17e00b2830 msgr2=0x7f17e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17e00b2830 0x7f17e00b4d40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f17ec02aeb0 tx=0x7f17ec005fd0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 msgr2=0x7f17f419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f419a1e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e980 tx=0x7f17f000ee40 comp rx=0 tx=0).stop 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 shutdown_connections 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17f4109b80 0x7f17f41a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17f4104e80 0x7f17f419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f17e00b2830 0x7f17e00b4d40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 --2- 172.21.15.19:0/360426660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f4103c40 0x7f17f419a1e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 >> 172.21.15.19:0/360426660 conn(0x7f17f40fee50 msgr2=0x7f17f4105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 shutdown_connections 2024-05-17T09:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:31.982+0000 7f17fbe61640 1 -- 172.21.15.19:0/360426660 wait complete. 2024-05-17T09:35:32.438 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:32 smithi019 ceph-mon[31620]: pgmap v911: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 3.2 MiB/s wr, 187 op/s 2024-05-17T09:35:32.465 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9d531f9d', quiesce took 15.8 seconds. Will release after: 11.8 2024-05-17T09:35:32.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 10.31 MB/sec warmup 89 sec latency 969.029 ms 2024-05-17T09:35:32.574 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:32 smithi079 ceph-mon[35502]: pgmap v911: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 3.2 MiB/s wr, 187 op/s 2024-05-17T09:35:32.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:32 smithi069 ceph-mon[35081]: pgmap v911: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 3.2 MiB/s wr, 187 op/s 2024-05-17T09:35:33.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 10.20 MB/sec warmup 90 sec latency 1969.120 ms 2024-05-17T09:35:34.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 10.08 MB/sec warmup 91 sec latency 2969.219 ms 2024-05-17T09:35:34.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:34 smithi069 ceph-mon[35081]: pgmap v912: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-05-17T09:35:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:34 smithi019 ceph-mon[31620]: pgmap v912: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-05-17T09:35:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:34 smithi079 ceph-mon[35502]: pgmap v912: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-05-17T09:35:35.564 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.97 MB/sec warmup 92 sec latency 3969.276 ms 2024-05-17T09:35:35.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:35 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:35:35.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:35 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:35:35.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:35 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:35:36.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.87 MB/sec warmup 93 sec latency 4969.357 ms 2024-05-17T09:35:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:36 smithi069 ceph-mon[35081]: pgmap v913: 129 pgs: 129 active+clean; 325 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-05-17T09:35:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:36 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:35:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:36 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:35:36.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:36 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:35:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:36 smithi019 ceph-mon[31620]: pgmap v913: 129 pgs: 129 active+clean; 325 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-05-17T09:35:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:35:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:35:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:35:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:36 smithi079 ceph-mon[35502]: pgmap v913: 129 pgs: 129 active+clean; 325 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-05-17T09:35:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:35:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:35:36.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:35:37.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.76 MB/sec warmup 94 sec latency 5969.455 ms 2024-05-17T09:35:38.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.66 MB/sec warmup 95 sec latency 6969.584 ms 2024-05-17T09:35:38.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:38 smithi069 ceph-mon[35081]: pgmap v914: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-05-17T09:35:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:38 smithi019 ceph-mon[31620]: pgmap v914: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-05-17T09:35:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:38 smithi079 ceph-mon[35502]: pgmap v914: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-05-17T09:35:39.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.56 MB/sec warmup 96 sec latency 7969.696 ms 2024-05-17T09:35:40.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.46 MB/sec warmup 97 sec latency 8969.798 ms 2024-05-17T09:35:40.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:40 smithi069 ceph-mon[35081]: pgmap v915: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-05-17T09:35:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:40 smithi019 ceph-mon[31620]: pgmap v915: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-05-17T09:35:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:40 smithi079 ceph-mon[35502]: pgmap v915: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-05-17T09:35:41.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.36 MB/sec warmup 98 sec latency 9969.922 ms 2024-05-17T09:35:42.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.27 MB/sec warmup 99 sec latency 10970.048 ms 2024-05-17T09:35:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:42 smithi069 ceph-mon[35081]: pgmap v916: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-17T09:35:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:42 smithi019 ceph-mon[31620]: pgmap v916: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-17T09:35:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:42 smithi079 ceph-mon[35502]: pgmap v916: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-17T09:35:43.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.18 MB/sec warmup 100 sec latency 11970.179 ms 2024-05-17T09:35:44.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:35:44.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:35:44.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.08 MB/sec warmup 101 sec latency 12970.242 ms 2024-05-17T09:35:44.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:44 smithi019 ceph-mon[31620]: pgmap v917: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-17T09:35:44.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:44 smithi069 ceph-mon[35081]: pgmap v917: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-17T09:35:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:44 smithi079 ceph-mon[35502]: pgmap v917: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-17T09:35:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.091+0000 7ff45b684640 1 -- 172.21.15.19:0/1814929763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102e50 msgr2=0x7ff454103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.091+0000 7ff45b684640 1 --2- 172.21.15.19:0/1814929763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102e50 0x7ff454103260 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff444009930 tx=0x7ff44402a420 comp rx=0 tx=0).stop 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 -- 172.21.15.19:0/1814929763 shutdown_connections 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 --2- 172.21.15.19:0/1814929763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 0x7ff454103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 --2- 172.21.15.19:0/1814929763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102e50 0x7ff454103260 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 --2- 172.21.15.19:0/1814929763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102500 0x7ff454102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 -- 172.21.15.19:0/1814929763 >> 172.21.15.19:0/1814929763 conn(0x7ff4540fdd10 msgr2=0x7ff4541001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 -- 172.21.15.19:0/1814929763 shutdown_connections 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 -- 172.21.15.19:0/1814929763 wait complete. 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 Processor -- start 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.092+0000 7ff45b684640 1 -- start start 2024-05-17T09:35:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 0x7ff45419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102e50 0x7ff45419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 0x7ff4541a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff454111810 con 0x7ff454102500 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff454111690 con 0x7ff4541037a0 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff454111990 con 0x7ff454102e50 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 0x7ff45419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 0x7ff45419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51202/0 (socket says 172.21.15.19:51202) 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 -- 172.21.15.19:0/713676048 learned_addr learned my addr 172.21.15.19:0/713676048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff458c1f640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102e50 0x7ff45419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459c21640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 0x7ff4541a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 -- 172.21.15.19:0/713676048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102e50 msgr2=0x7ff45419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102e50 0x7ff45419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 -- 172.21.15.19:0/713676048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 msgr2=0x7ff4541a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 0x7ff4541a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 -- 172.21.15.19:0/713676048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4541a3f80 con 0x7ff454102500 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff458c1f640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102e50 0x7ff45419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459c21640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 0x7ff4541a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff459420640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 0x7ff45419c440 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff45000e9f0 tx=0x7ff45000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff44a7fc640 1 -- 172.21.15.19:0/713676048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45000ccc0 con 0x7ff454102500 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff44a7fc640 1 -- 172.21.15.19:0/713676048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4500054b0 con 0x7ff454102500 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4541a4270 con 0x7ff454102500 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff44a7fc640 1 -- 172.21.15.19:0/713676048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450010430 con 0x7ff454102500 2024-05-17T09:35:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.093+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541a47c0 con 0x7ff454102500 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 Processor -- start 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 -- start start 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff44a7fc640 1 -- 172.21.15.19:0/713676048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff4500046c0 con 0x7ff454102500 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x5591911003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191100920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559191072c30 con 0x559190d5f1b0 2024-05-17T09:35:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591910627d0 con 0x559190d3e4d0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.096+0000 7ff462608740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559191066810 con 0x559190d362f0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x5591911003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x5591911003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51212/0 (socket says 172.21.15.19:51212) 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 -- 172.21.15.19:0/657627710 learned_addr learned my addr 172.21.15.19:0/657627710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff458c1f640 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459420640 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 -- 172.21.15.19:0/657627710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 msgr2=0x559191100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 -- 172.21.15.19:0/657627710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 msgr2=0x559191107820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191107820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 -- 172.21.15.19:0/657627710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559190d77210 con 0x559190d5f1b0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459c21640 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x5591911003a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009f60 tx=0x7ff44c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1981a720b945fb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff44a7fc640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4400b2920 0x7ff4400b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff459420640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4400b2920 0x7ff4400b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff43f7fe640 1 -- 172.21.15.19:0/657627710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c011640 con 0x559190d5f1b0 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff43f7fe640 1 -- 172.21.15.19:0/657627710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff44c011c70 con 0x559190d5f1b0 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff43f7fe640 1 -- 172.21.15.19:0/657627710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c01bc10 con 0x559190d5f1b0 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.097+0000 7ff44a7fc640 1 -- 172.21.15.19:0/713676048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff4500e29d0 con 0x7ff454102500 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 -- 172.21.15.19:0/657627710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 msgr2=0x5591911003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff459420640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4400b2920 0x7ff4400b4e30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x559191101780 tx=0x7ff45004f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x5591911003a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009f60 tx=0x7ff44c009f90 comp rx=0 tx=0).stop 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 -- 172.21.15.19:0/657627710 shutdown_connections 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 --2- 172.21.15.19:0/657627710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x5591911003a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 -- 172.21.15.19:0/657627710 >> 172.21.15.19:0/657627710 conn(0x559190d53300 msgr2=0x5591910fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 -- 172.21.15.19:0/657627710 shutdown_connections 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.098+0000 7ff462608740 1 -- 172.21.15.19:0/657627710 wait complete. 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 -- start start 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 Processor -- start 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 0x55919110a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d5f1b0 0x5591911111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559190d48790 con 0x559190d3e4d0 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55919101e890 con 0x559190d5f1b0 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559191108270 con 0x559190d362f0 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff462608740 10 client.0 _ll_register_callbacks cb 0x559191021d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff458c1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 0x55919110a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.099+0000 7ff458c1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 0x55919110a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51216/0 (socket says 172.21.15.19:51216) 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 -- 172.21.15.19:0/3518770034 learned_addr learned my addr 172.21.15.19:0/3518770034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff459c21640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d5f1b0 0x5591911111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff459c21640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 -- 172.21.15.19:0/3518770034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 msgr2=0x559191109d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191109d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 -- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d5f1b0 msgr2=0x5591911111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d5f1b0 0x5591911111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 -- 172.21.15.19:0/3518770034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559190e43620 con 0x559190d3e4d0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff458c1f640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 0x55919110a2b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff444005ab0 tx=0x7ff444031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7229dac86b391fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff43cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff43cff9640 1 -- 172.21.15.19:0/3518770034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444035040 con 0x559190d3e4d0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff43cff9640 1 -- 172.21.15.19:0/3518770034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff444004240 con 0x559190d3e4d0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44c011c70 con 0x559190d3e4d0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff43cff9640 1 -- 172.21.15.19:0/3518770034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff444004440 con 0x559190d3e4d0 2024-05-17T09:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff462608740 10 client.18027 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.100+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559191114140 con 0x559190d3e4d0 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff43cff9640 1 -- 172.21.15.19:0/3518770034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff44c011c70 con 0x559190d3e4d0 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff43cff9640 1 -- 172.21.15.19:0/3518770034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff44403d6e0 con 0x559190d3e4d0 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff462608740 10 client.18027 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff462608740 10 client.18027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff462608740 20 client.18027 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff462608740 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190d1dd80 0x559191117920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff462608740 4 client.18027 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559191074b80 con 0x559190d1dd80 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.101+0000 7ff459c21640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190d1dd80 0x559191117920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff459c21640 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190d1dd80 0x559191117920 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1366f913cd399156 server_cookie=e2e8a6a3a48990a6 in_seq=0 out_seq=0 2024-05-17T09:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff43cff9640 10 client.18027 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff43cff9640 1 -- 172.21.15.19:0/3518770034 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559191074b80 con 0x559190d1dd80 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff43cff9640 10 client.18027 handle_command_reply: tid=0 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff462608740 1 client.18027 shutdown 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 msgr2=0x55919110a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.102+0000 7ff462608740 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 0x55919110a2b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff444005ab0 tx=0x7ff444031820 comp rx=0 tx=0).stop 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 shutdown_connections 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190d1dd80 0x559191117920 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d5f1b0 0x5591911111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d3e4d0 0x55919110a2b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff43cff9640 0 client.18027 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 --2- 172.21.15.19:0/3518770034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 >> 172.21.15.19:0/3518770034 conn(0x559190d53300 msgr2=0x5591910fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 shutdown_connections 2024-05-17T09:35:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 1 -- 172.21.15.19:0/3518770034 wait complete. 2024-05-17T09:35:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.103+0000 7ff462608740 20 client.18027 trim_cache size 0 max 16384 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 Processor -- start 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 -- start start 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 0x55919111d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x5591911110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d5f1b0 0x55919111d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541b1590 con 0x559190d362f0 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541aee00 con 0x559190d3e4d0 2024-05-17T09:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff462608740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541acc50 con 0x559190d5f1b0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff459c21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 0x55919111d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff459c21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 0x55919111d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51232/0 (socket says 172.21.15.19:51232) 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.118+0000 7ff459c21640 1 -- 172.21.15.19:0/1631201489 learned_addr learned my addr 172.21.15.19:0/1631201489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff458c1f640 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x5591911110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d5f1b0 0x55919111d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 -- 172.21.15.19:0/1631201489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d5f1b0 msgr2=0x55919111d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d5f1b0 0x55919111d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 -- 172.21.15.19:0/1631201489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 msgr2=0x5591911110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x5591911110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 -- 172.21.15.19:0/1631201489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919110ae10 con 0x559190d362f0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff459c21640 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 0x55919111d120 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009f30 tx=0x7ff44c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1489d786794f15f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff43effd640 1 -- 172.21.15.19:0/1631201489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c01eb80 con 0x559190d362f0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff43effd640 1 -- 172.21.15.19:0/1631201489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff44c01ed20 con 0x559190d362f0 2024-05-17T09:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.119+0000 7ff43effd640 1 -- 172.21.15.19:0/1631201489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c01e170 con 0x559190d362f0 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 -- 172.21.15.19:0/1631201489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 msgr2=0x55919111d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 0x55919111d120 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009f30 tx=0x7ff44c0092a0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 -- 172.21.15.19:0/1631201489 shutdown_connections 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d5f1b0 0x55919111d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x5591911110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 --2- 172.21.15.19:0/1631201489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d362f0 0x55919111d120 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 -- 172.21.15.19:0/1631201489 >> 172.21.15.19:0/1631201489 conn(0x559190d53300 msgr2=0x5591911038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 -- 172.21.15.19:0/1631201489 shutdown_connections 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 -- 172.21.15.19:0/1631201489 wait complete. 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 -- start start 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.120+0000 7ff462608740 1 Processor -- start 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191110310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191110850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x55919117e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541ad2c0 con 0x559190d5f1b0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541ae730 con 0x559190d3e4d0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541ad990 con 0x559190d362f0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff462608740 10 client.0 _ll_register_callbacks cb 0x7ff444005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x55919117e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x55919117e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51246/0 (socket says 172.21.15.19:51246) 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 -- 172.21.15.19:0/176927824 learned_addr learned my addr 172.21.15.19:0/176927824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191110310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff458c1f640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 -- 172.21.15.19:0/176927824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 msgr2=0x559191110310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191110310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 -- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 msgr2=0x559191110850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191110850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 -- 172.21.15.19:0/176927824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919117e6d0 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff459c21640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x55919117e0e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x559191178040 tx=0x7ff44c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83862e322e6f42e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.121+0000 7ff43cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff43cff9640 1 -- 172.21.15.19:0/176927824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c01e170 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff43cff9640 1 -- 172.21.15.19:0/176927824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff44c01e310 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44c01ed20 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff43cff9640 1 -- 172.21.15.19:0/176927824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff44c01e550 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 10 client.18033 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55919117f960 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff43cff9640 1 -- 172.21.15.19:0/176927824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff44c01ed20 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff43cff9640 1 -- 172.21.15.19:0/176927824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff44c032070 con 0x559190d5f1b0 2024-05-17T09:35:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 10 client.18033 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 10 client.18033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 20 client.18033 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190cf31c0 0x559191182130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 4 client.18033 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559191109b40 con 0x559190cf31c0 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.122+0000 7ff458c1f640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190cf31c0 0x559191182130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.123+0000 7ff458c1f640 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190cf31c0 0x559191182130 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320f04ad27bc97a7 server_cookie=5571e945fd4d9e7a in_seq=0 out_seq=0 2024-05-17T09:35:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.123+0000 7ff43cff9640 10 client.18033 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.157+0000 7ff43cff9640 1 -- 172.21.15.19:0/176927824 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559191109b40 con 0x559190cf31c0 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.157+0000 7ff43cff9640 10 client.18033 handle_command_reply: tid=0 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.157+0000 7ff462608740 1 client.18033 shutdown 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.157+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 msgr2=0x55919117e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.157+0000 7ff462608740 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x55919117e0e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x559191178040 tx=0x7ff44c005cf0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 shutdown_connections 2024-05-17T09:35:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559190cf31c0 0x559191182130 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559190d5f1b0 0x55919117e0e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff43cff9640 0 client.18033 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559190d3e4d0 0x559191110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 --2- 172.21.15.19:0/176927824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559190d362f0 0x559191110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 >> 172.21.15.19:0/176927824 conn(0x559190d53300 msgr2=0x5591911041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 shutdown_connections 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 1 -- 172.21.15.19:0/176927824 wait complete. 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff462608740 20 client.18033 trim_cache size 0 max 16384 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:35:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4400b2920 msgr2=0x7ff4400b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4400b2920 0x7ff4400b4e30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x559191101780 tx=0x7ff45004f040 comp rx=0 tx=0).stop 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 msgr2=0x7ff45419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 0x7ff45419c440 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff45000e9f0 tx=0x7ff45000eeb0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 shutdown_connections 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4541037a0 0x7ff4541a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.158+0000 7ff45b684640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff454102e50 0x7ff45419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.159+0000 7ff45b684640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff4400b2920 0x7ff4400b4e30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.159+0000 7ff45b684640 1 --2- 172.21.15.19:0/713676048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff454102500 0x7ff45419c440 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.159+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 >> 172.21.15.19:0/713676048 conn(0x7ff4540fdd10 msgr2=0x7ff45410a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.159+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 shutdown_connections 2024-05-17T09:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:45.159+0000 7ff45b684640 1 -- 172.21.15.19:0/713676048 wait complete. 2024-05-17T09:35:45.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 9.00 MB/sec warmup 102 sec latency 13970.278 ms 2024-05-17T09:35:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 -- 172.21.15.19:0/1351092896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 msgr2=0x7f023c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 --2- 172.21.15.19:0/1351092896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c1111b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f023400b0d0 tx=0x7f023400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 -- 172.21.15.19:0/1351092896 shutdown_connections 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 --2- 172.21.15.19:0/1351092896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f023c1045d0 0x7f023c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 --2- 172.21.15.19:0/1351092896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c1111b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 --2- 172.21.15.19:0/1351092896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 0x7f023c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 -- 172.21.15.19:0/1351092896 >> 172.21.15.19:0/1351092896 conn(0x7f023c0ffec0 msgr2=0x7f023c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 -- 172.21.15.19:0/1351092896 shutdown_connections 2024-05-17T09:35:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.401+0000 7f02440b8700 1 -- 172.21.15.19:0/1351092896 wait complete. 2024-05-17T09:35:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.402+0000 7f02440b8700 1 Processor -- start 2024-05-17T09:35:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.402+0000 7f02440b8700 1 -- start start 2024-05-17T09:35:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.402+0000 7f02440b8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f023c1045d0 0x7f023c076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.402+0000 7f02440b8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 0x7f023c077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.402+0000 7f02440b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.402+0000 7f02440b8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f023c114330 con 0x7f023c109850 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f02440b8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f023c1141b0 con 0x7f023c108f00 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f02440b8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f023c1144b0 con 0x7f023c1045d0 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c077640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51266/0 (socket says 172.21.15.19:51266) 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 -- 172.21.15.19:0/335780589 learned_addr learned my addr 172.21.15.19:0/335780589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0241e54700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f023c1045d0 0x7f023c076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0241653700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 0x7f023c077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 -- 172.21.15.19:0/335780589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f023c1045d0 msgr2=0x7f023c076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f023c1045d0 0x7f023c076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 -- 172.21.15.19:0/335780589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 msgr2=0x7f023c077100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 0x7f023c077100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 -- 172.21.15.19:0/335780589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c1a90a0 con 0x7f023c109850 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0241653700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 0x7f023c077100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0242655700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c077640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0234009850 tx=0x7f0234007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023401f070 con 0x7f023c109850 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0234005500 con 0x7f023c109850 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c1a9330 con 0x7f023c109850 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023401b9a0 con 0x7f023c109850 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.403+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c1a97e0 con 0x7f023c109850 2024-05-17T09:35:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.404+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c0512b0 con 0x7f023c109850 2024-05-17T09:35:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.406+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0234022070 con 0x7f023c109850 2024-05-17T09:35:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.407+0000 7f0232ffd700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02280b2900 0x7f02280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.407+0000 7f0241e54700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02280b2900 0x7f02280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.407+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f023400f020 con 0x7f023c109850 2024-05-17T09:35:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.408+0000 7f0241e54700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02280b2900 0x7f02280b4e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0238005950 tx=0x7f023800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.411+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0234029020 con 0x7f023c109850 2024-05-17T09:35:46.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 8.91 MB/sec warmup 103 sec latency 14970.329 ms 2024-05-17T09:35:46.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.677+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f023c02e5e0 con 0x7f023c109850 2024-05-17T09:35:46.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:46 smithi019 ceph-mon[31620]: pgmap v918: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 438 KiB/s wr, 70 op/s 2024-05-17T09:35:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.677+0000 7f0232ffd700 1 -- 172.21.15.19:0/335780589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11858 (secure 0 0 0) 0x7f02340ab810 con 0x7f023c109850 2024-05-17T09:35:46.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:35:46.677 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:34:12.645154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:35:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.680+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02280b2900 msgr2=0x7f02280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.680+0000 7f02440b8700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02280b2900 0x7f02280b4e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0238005950 tx=0x7f023800b410 comp rx=0 tx=0).stop 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.680+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 msgr2=0x7f023c077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.680+0000 7f02440b8700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c077640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0234009850 tx=0x7f0234007d90 comp rx=0 tx=0).stop 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 shutdown_connections 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02280b2900 0x7f02280b4e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f023c1045d0 0x7f023c076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023c109850 0x7f023c077640 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 --2- 172.21.15.19:0/335780589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c108f00 0x7f023c077100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 >> 172.21.15.19:0/335780589 conn(0x7f023c0ffec0 msgr2=0x7f023c108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 shutdown_connections 2024-05-17T09:35:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:46.681+0000 7f02440b8700 1 -- 172.21.15.19:0/335780589 wait complete. 2024-05-17T09:35:46.681 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:46 smithi069 ceph-mon[35081]: pgmap v918: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 438 KiB/s wr, 70 op/s 2024-05-17T09:35:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-05-17T09:35:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:46 smithi079 ceph-mon[35502]: pgmap v918: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 438 KiB/s wr, 70 op/s 2024-05-17T09:35:47.480 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 9d531f9d --release --await' 2024-05-17T09:35:47.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 9d531f9d --release --await 2024-05-17T09:35:47.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 8.82 MB/sec warmup 104 sec latency 15970.386 ms 2024-05-17T09:35:47.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:47 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/335780589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:47.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:47 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/335780589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:47.742 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:35:47.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:47 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/335780589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:48.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:48 smithi019 ceph-mon[31620]: pgmap v919: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 98 KiB/s wr, 25 op/s 2024-05-17T09:35:48.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 8.74 MB/sec warmup 105 sec latency 16970.429 ms 2024-05-17T09:35:48.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:48 smithi069 ceph-mon[35081]: pgmap v919: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 98 KiB/s wr, 25 op/s 2024-05-17T09:35:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:48 smithi079 ceph-mon[35502]: pgmap v919: 129 pgs: 129 active+clean; 322 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 98 KiB/s wr, 25 op/s 2024-05-17T09:35:49.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13763 8.66 MB/sec warmup 106 sec latency 17970.475 ms 2024-05-17T09:35:49.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 -- 172.21.15.19:0/1598866676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac108f10 msgr2=0x7f04ac109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 --2- 172.21.15.19:0/1598866676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac108f10 0x7f04ac109320 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f049c0099c0 tx=0x7f049c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 -- 172.21.15.19:0/1598866676 shutdown_connections 2024-05-17T09:35:49.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 --2- 172.21.15.19:0/1598866676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04ac1045e0 0x7f04ac1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 --2- 172.21.15.19:0/1598866676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ac109860 0x7f04ac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 --2- 172.21.15.19:0/1598866676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac108f10 0x7f04ac109320 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 -- 172.21.15.19:0/1598866676 >> 172.21.15.19:0/1598866676 conn(0x7f04ac0ffeb0 msgr2=0x7f04ac102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:49.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 -- 172.21.15.19:0/1598866676 shutdown_connections 2024-05-17T09:35:49.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.671+0000 7f04b35f3700 1 -- 172.21.15.19:0/1598866676 wait complete. 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 Processor -- start 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 -- start start 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 0x7f04ac19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04ac108f10 0x7f04ac19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ac109860 0x7f04ac19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac114320 con 0x7f04ac1045e0 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1141a0 con 0x7f04ac109860 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b35f3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1144a0 con 0x7f04ac108f10 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b138f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 0x7f04ac19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b138f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 0x7f04ac19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51298/0 (socket says 172.21.15.19:51298) 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b138f700 1 -- 172.21.15.19:0/3337669041 learned_addr learned my addr 172.21.15.19:0/3337669041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04ac108f10 0x7f04ac19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.672+0000 7f04b1b90700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ac109860 0x7f04ac19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b138f700 1 -- 172.21.15.19:0/3337669041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04ac108f10 msgr2=0x7f04ac19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b138f700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04ac108f10 0x7f04ac19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b138f700 1 -- 172.21.15.19:0/3337669041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ac109860 msgr2=0x7f04ac19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b138f700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ac109860 0x7f04ac19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b138f700 1 -- 172.21.15.19:0/3337669041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac19fc60 con 0x7f04ac1045e0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b138f700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 0x7f04ac19ad90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d950 tx=0x7f04a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04a27fc700 1 -- 172.21.15.19:0/3337669041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8016070 con 0x7f04ac1045e0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac19ff50 con 0x7f04ac1045e0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04a27fc700 1 -- 172.21.15.19:0/3337669041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a800fd20 con 0x7f04ac1045e0 2024-05-17T09:35:49.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04a27fc700 1 -- 172.21.15.19:0/3337669041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a800e3b0 con 0x7f04ac1045e0 2024-05-17T09:35:49.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.673+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac1a7870 con 0x7f04ac1045e0 2024-05-17T09:35:49.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.676+0000 7f04a27fc700 1 -- 172.21.15.19:0/3337669041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f04a8019070 con 0x7f04ac1045e0 2024-05-17T09:35:49.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.677+0000 7f04a27fc700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f04980b2880 0x7f04980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.677+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f04980b2880 0x7f04980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.677+0000 7f04a27fc700 1 -- 172.21.15.19:0/3337669041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f04a80e20e0 con 0x7f04ac1045e0 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.677+0000 7f04c2008b80 1 Processor -- start 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.677+0000 7f04c2008b80 1 -- start start 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f04980b2880 0x7f04980b4d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f049c009990 tx=0x7f049c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c496560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c496aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c490620 0x563f8c490a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04c2008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f8c3bed90 con 0x563f8c38f4d0 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04c2008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f8c3ae930 con 0x563f8c490620 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04c2008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f8c3b2970 con 0x563f8c311f40 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c496560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c496560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51300/0 (socket says 172.21.15.19:51300) 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 -- 172.21.15.19:0/1595288764 learned_addr learned my addr 172.21.15.19:0/1595288764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b1b90700 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c490620 0x563f8c490a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c496aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 -- 172.21.15.19:0/1595288764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 msgr2=0x563f8c496aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c496aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 -- 172.21.15.19:0/1595288764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c490620 msgr2=0x563f8c490a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c490620 0x563f8c490a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 -- 172.21.15.19:0/1595288764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8c2fe9d0 con 0x563f8c38f4d0 2024-05-17T09:35:49.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.678+0000 7f04b138f700 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c496560 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f04a80a5370 tx=0x7f04a80a8e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55a26e7cdfb6266 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:49.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.679+0000 7f04977fe700 1 -- 172.21.15.19:0/1595288764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8020070 con 0x563f8c38f4d0 2024-05-17T09:35:49.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.679+0000 7f04977fe700 1 -- 172.21.15.19:0/1595288764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a80a99f0 con 0x563f8c38f4d0 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.679+0000 7f04977fe700 1 -- 172.21.15.19:0/1595288764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a80b1cc0 con 0x563f8c38f4d0 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.679+0000 7f04c2008b80 1 -- 172.21.15.19:0/1595288764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 msgr2=0x563f8c496560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.679+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c496560 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f04a80a5370 tx=0x7f04a80a8e00 comp rx=0 tx=0).stop 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 -- 172.21.15.19:0/1595288764 shutdown_connections 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c490620 0x563f8c490a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c496560 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1595288764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c496aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 -- 172.21.15.19:0/1595288764 >> 172.21.15.19:0/1595288764 conn(0x563f8c350d80 msgr2=0x563f8c48b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 -- 172.21.15.19:0/1595288764 shutdown_connections 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 -- 172.21.15.19:0/1595288764 wait complete. 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 -- start start 2024-05-17T09:35:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.680+0000 7f04c2008b80 1 Processor -- start 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c49a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c38f4d0 0x563f8c49a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c49acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f8c2b50a0 con 0x563f8c490620 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f8c3508c0 con 0x563f8c311f40 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f8c491440 con 0x563f8c38f4d0 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04c2008b80 10 client.0 _ll_register_callbacks cb 0x563f8c349a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b1b90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c49acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b1b90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c49acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51306/0 (socket says 172.21.15.19:51306) 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b1b90700 1 -- 172.21.15.19:0/4161526570 learned_addr learned my addr 172.21.15.19:0/4161526570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c49a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c38f4d0 0x563f8c49a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 -- 172.21.15.19:0/4161526570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c38f4d0 msgr2=0x563f8c49a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c38f4d0 0x563f8c49a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 -- 172.21.15.19:0/4161526570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 msgr2=0x563f8c49acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c49acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 -- 172.21.15.19:0/4161526570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8c34d790 con 0x563f8c311f40 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b1b90700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c49acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f04b138f700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c49a250 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04a800ffa0 tx=0x7f04a80ad790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba84a3a9ca41144 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.681+0000 7f0494ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f0494ff9700 1 -- 172.21.15.19:0/4161526570 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a80c2020 con 0x563f8c311f40 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a80a99f0 con 0x563f8c311f40 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f0494ff9700 1 -- 172.21.15.19:0/4161526570 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f04a80b1cc0 con 0x563f8c311f40 2024-05-17T09:35:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f0494ff9700 1 -- 172.21.15.19:0/4161526570 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f04a8020070 con 0x563f8c311f40 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 10 client.27608 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563f8c4a7b60 con 0x563f8c311f40 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f0494ff9700 1 -- 172.21.15.19:0/4161526570 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a80a99f0 con 0x563f8c311f40 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f0494ff9700 1 -- 172.21.15.19:0/4161526570 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f04a80a8550 con 0x563f8c311f40 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 10 client.27608 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 10 client.27608 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 20 client.27608 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c4aa030 0x563f8c4ac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 4 client.27608 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.682+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8c3c0ce0 con 0x563f8c4aa030 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.683+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c4aa030 0x563f8c4ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.683+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c4aa030 0x563f8c4ac450 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1cebf2591cbbd1 server_cookie=3953293a20239883 in_seq=0 out_seq=0 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.683+0000 7f0494ff9700 10 client.27608 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f0494ff9700 1 -- 172.21.15.19:0/4161526570 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563f8c3c0ce0 con 0x563f8c4aa030 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f0494ff9700 10 client.27608 handle_command_reply: tid=0 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 client.27608 shutdown 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 msgr2=0x563f8c49a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c49a250 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04a800ffa0 tx=0x7f04a80ad790 comp rx=0 tx=0).stop 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 shutdown_connections 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c4aa030 0x563f8c4ac450 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c49acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c49a250 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 --2- 172.21.15.19:0/4161526570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c38f4d0 0x563f8c49a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 >> 172.21.15.19:0/4161526570 conn(0x563f8c350d80 msgr2=0x563f8c485280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f0494ff9700 0 client.27608 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 shutdown_connections 2024-05-17T09:35:49.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 1 -- 172.21.15.19:0/4161526570 wait complete. 2024-05-17T09:35:49.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.684+0000 7f04c2008b80 20 client.27608 trim_cache size 0 max 16384 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 Processor -- start 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 -- start start 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c4948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c38f4d0 0x563f8c49a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c4af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1b2c00 con 0x563f8c490620 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1b0430 con 0x563f8c38f4d0 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.705+0000 7f04c2008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1ae280 con 0x563f8c311f40 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c4af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c4af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51322/0 (socket says 172.21.15.19:51322) 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 -- 172.21.15.19:0/1413809480 learned_addr learned my addr 172.21.15.19:0/1413809480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b138f700 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c4948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c38f4d0 0x563f8c49a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 -- 172.21.15.19:0/1413809480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 msgr2=0x563f8c4948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c4948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 -- 172.21.15.19:0/1413809480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c38f4d0 msgr2=0x563f8c49a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c38f4d0 0x563f8c49a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 -- 172.21.15.19:0/1413809480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8c48cd40 con 0x563f8c490620 2024-05-17T09:35:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f04b1b90700 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c4af060 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f04a40076f0 tx=0x7f04a4007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acd762b02dcd8a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f0496ffd700 1 -- 172.21.15.19:0/1413809480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4020670 con 0x563f8c490620 2024-05-17T09:35:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f0496ffd700 1 -- 172.21.15.19:0/1413809480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a401b070 con 0x563f8c490620 2024-05-17T09:35:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.706+0000 7f0496ffd700 1 -- 172.21.15.19:0/1413809480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4017a40 con 0x563f8c490620 2024-05-17T09:35:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 -- 172.21.15.19:0/1413809480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 msgr2=0x563f8c4af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c4af060 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f04a40076f0 tx=0x7f04a4007720 comp rx=0 tx=0).stop 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 -- 172.21.15.19:0/1413809480 shutdown_connections 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c311f40 0x563f8c4948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c490620 0x563f8c4af060 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 --2- 172.21.15.19:0/1413809480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c38f4d0 0x563f8c49a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 -- 172.21.15.19:0/1413809480 >> 172.21.15.19:0/1413809480 conn(0x563f8c350d80 msgr2=0x563f8c4858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 -- 172.21.15.19:0/1413809480 shutdown_connections 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.707+0000 7f04c2008b80 1 -- 172.21.15.19:0/1413809480 wait complete. 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 -- start start 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 Processor -- start 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c4ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c4ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c490620 0x563f8c5138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1ae8f0 con 0x563f8c38f4d0 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1afd60 con 0x563f8c311f40 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1aefc0 con 0x563f8c490620 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04c2008b80 10 client.0 _ll_register_callbacks cb 0x563f8c349a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04b0b8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c4ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04b0b8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c4ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51338/0 (socket says 172.21.15.19:51338) 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.708+0000 7f04b0b8e700 1 -- 172.21.15.19:0/314781906 learned_addr learned my addr 172.21.15.19:0/314781906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b138f700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c4ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b1b90700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c490620 0x563f8c5138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b0b8e700 1 -- 172.21.15.19:0/314781906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c490620 msgr2=0x563f8c5138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c490620 0x563f8c5138a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b0b8e700 1 -- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 msgr2=0x563f8c4ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c4ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b0b8e700 1 -- 172.21.15.19:0/314781906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8c513de0 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b1b90700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c490620 0x563f8c5138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04b0b8e700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c4ada60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f049c0099c0 tx=0x7f049c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b1a2445f636643 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f0494ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f0494ff9700 1 -- 172.21.15.19:0/314781906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c003b50 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f0494ff9700 1 -- 172.21.15.19:0/314781906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f049c003cf0 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a401b070 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.709+0000 7f0494ff9700 1 -- 172.21.15.19:0/314781906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f049c045050 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 10 client.18057 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563f8c514fd0 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f0494ff9700 1 -- 172.21.15.19:0/314781906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a401b070 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f0494ff9700 1 -- 172.21.15.19:0/314781906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f049c03b030 con 0x563f8c38f4d0 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 10 client.18057 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 10 client.18057 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:35:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 20 client.18057 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c5156d0 0x563f8c517af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 4 client.18057 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9d531f9d", "release": true, "await": true}] 2024-05-17T09:35:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9d531f9d", "release": true, "await": true}) -- 0x563f8c3ba4c0 con 0x563f8c5156d0 2024-05-17T09:35:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.710+0000 7f04b138f700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c5156d0 0x563f8c517af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.711+0000 7f04b138f700 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c5156d0 0x563f8c517af0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9218005a85eed3 server_cookie=2c0b0ea8e3b36854 in_seq=0 out_seq=0 2024-05-17T09:35:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.711+0000 7f0494ff9700 10 client.18057 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f0494ff9700 1 -- 172.21.15.19:0/314781906 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563f8c3ba4c0 con 0x563f8c5156d0 2024-05-17T09:35:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f0494ff9700 10 client.18057 handle_command_reply: tid=0 2024-05-17T09:35:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 client.18057 shutdown 2024-05-17T09:35:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 msgr2=0x563f8c4ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c4ada60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f049c0099c0 tx=0x7f049c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 shutdown_connections 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f8c490620 0x563f8c5138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f8c5156d0 0x563f8c517af0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8c311f40 0x563f8c4ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 --2- 172.21.15.19:0/314781906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f8c38f4d0 0x563f8c4ada60 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.736+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 >> 172.21.15.19:0/314781906 conn(0x563f8c350d80 msgr2=0x563f8c4a52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.737+0000 7f0494ff9700 0 client.18057 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.737+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 shutdown_connections 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.737+0000 7f04c2008b80 1 -- 172.21.15.19:0/314781906 wait complete. 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.737+0000 7f04c2008b80 20 client.18057 trim_cache size 0 max 16384 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 35, 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 56, 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout: "9d531f9d": { 2024-05-17T09:35:49.737 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 56, 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 147.6, 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:35:49.738 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:35:49.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:49.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:49.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:49.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:49.739 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:49.739 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f04980b2880 msgr2=0x7f04980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f04980b2880 0x7f04980b4d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f049c009990 tx=0x7f049c005fb0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 msgr2=0x7f04ac19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 0x7f04ac19ad90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f04a800d950 tx=0x7f04a800dc60 comp rx=0 tx=0).stop 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 shutdown_connections 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ac109860 0x7f04ac19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ac1045e0 0x7f04ac19ad90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f04980b2880 0x7f04980b4d90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 --2- 172.21.15.19:0/3337669041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04ac108f10 0x7f04ac19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 >> 172.21.15.19:0/3337669041 conn(0x7f04ac0ffeb0 msgr2=0x7f04ac1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 shutdown_connections 2024-05-17T09:35:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:49.738+0000 7f04b35f3700 1 -- 172.21.15.19:0/3337669041 wait complete. 2024-05-17T09:35:50.476 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9d531f9d', total seconds elapsed: 33.8 2024-05-17T09:35:50.476 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.6 2024-05-17T09:35:50.477 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:35:50.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 13875 8.60 MB/sec warmup 107 sec latency 18120.921 ms 2024-05-17T09:35:50.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:50 smithi069 ceph-mon[35081]: pgmap v920: 129 pgs: 129 active+clean; 322 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-05-17T09:35:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:50 smithi019 ceph-mon[31620]: pgmap v920: 129 pgs: 129 active+clean; 322 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-05-17T09:35:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:50 smithi079 ceph-mon[35502]: pgmap v920: 129 pgs: 129 active+clean; 322 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-05-17T09:35:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:35:51.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14008 8.56 MB/sec warmup 108 sec latency 85.452 ms 2024-05-17T09:35:52.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:52 smithi019 ceph-mon[31620]: pgmap v921: 129 pgs: 129 active+clean; 322 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-05-17T09:35:52.565 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14159 8.55 MB/sec warmup 109 sec latency 78.345 ms 2024-05-17T09:35:52.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:52 smithi069 ceph-mon[35081]: pgmap v921: 129 pgs: 129 active+clean; 322 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-05-17T09:35:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:52 smithi079 ceph-mon[35502]: pgmap v921: 129 pgs: 129 active+clean; 322 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 -- 172.21.15.19:0/2606009139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94108f10 msgr2=0x7f1d94109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 --2- 172.21.15.19:0/2606009139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94108f10 0x7f1d94109320 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1d880099c0 tx=0x7f1d88009cd0 comp rx=0 tx=0).stop 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 -- 172.21.15.19:0/2606009139 shutdown_connections 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 --2- 172.21.15.19:0/2606009139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d94109860 0x7f1d941111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 --2- 172.21.15.19:0/2606009139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d941045e0 0x7f1d941049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 --2- 172.21.15.19:0/2606009139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94108f10 0x7f1d94109320 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:52.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 -- 172.21.15.19:0/2606009139 >> 172.21.15.19:0/2606009139 conn(0x7f1d94100030 msgr2=0x7f1d94102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 -- 172.21.15.19:0/2606009139 shutdown_connections 2024-05-17T09:35:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.829+0000 7f1d9a611700 1 -- 172.21.15.19:0/2606009139 wait complete. 2024-05-17T09:35:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.830+0000 7f1d9a611700 1 Processor -- start 2024-05-17T09:35:52.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.830+0000 7f1d9a611700 1 -- start start 2024-05-17T09:35:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d9a611700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 0x7f1d9419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d9a611700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d94108f10 0x7f1d9419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d9a611700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94109860 0x7f1d941a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:52.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d93fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 0x7f1d9419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d93fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 0x7f1d9419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52336/0 (socket says 172.21.15.19:52336) 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d937fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d94108f10 0x7f1d9419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d93fff700 1 -- 172.21.15.19:0/3973875903 learned_addr learned my addr 172.21.15.19:0/3973875903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d94114320 con 0x7f1d94109860 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d941141a0 con 0x7f1d94108f10 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1d941144a0 con 0x7f1d941045e0 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.831+0000 7f1d98bae700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94109860 0x7f1d941a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d93fff700 1 -- 172.21.15.19:0/3973875903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d94108f10 msgr2=0x7f1d9419d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d93fff700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d94108f10 0x7f1d9419d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d93fff700 1 -- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94109860 msgr2=0x7f1d941a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d93fff700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94109860 0x7f1d941a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d93fff700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d941a1e60 con 0x7f1d941045e0 2024-05-17T09:35:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d98bae700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94109860 0x7f1d941a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d937fe700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d94108f10 0x7f1d9419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d93fff700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 0x7f1d9419cf90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e9a0 tx=0x7f1d8400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400cbd0 con 0x7f1d941045e0 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d8400cd70 con 0x7f1d941045e0 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84018680 con 0x7f1d941045e0 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d941a2150 con 0x7f1d941045e0 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.832+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941a2550 con 0x7f1d941045e0 2024-05-17T09:35:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.834+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d940512b0 con 0x7f1d941045e0 2024-05-17T09:35:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.836+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d8404aab0 con 0x7f1d941045e0 2024-05-17T09:35:52.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.837+0000 7f1d917fa700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d7c0b2900 0x7f1d7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.837+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1d840e28e0 con 0x7f1d941045e0 2024-05-17T09:35:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.837+0000 7f1d937fe700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d7c0b2900 0x7f1d7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:52.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.838+0000 7f1d937fe700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d7c0b2900 0x7f1d7c0b4e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d8801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:52.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:52.841+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d840aa1f0 con 0x7f1d941045e0 2024-05-17T09:35:53.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.317+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d9419e2f0 con 0x7f1d941045e0 2024-05-17T09:35:53.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.318+0000 7f1d917fa700 1 -- 172.21.15.19:0/3973875903 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11858 (secure 0 0 0) 0x7f1d840b2270 con 0x7f1d941045e0 2024-05-17T09:35:53.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:35:53.318 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:34:12.645154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d7c0b2900 msgr2=0x7f1d7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d7c0b2900 0x7f1d7c0b4e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d8801a040 comp rx=0 tx=0).stop 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 msgr2=0x7f1d9419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 0x7f1d9419cf90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e9a0 tx=0x7f1d8400ecb0 comp rx=0 tx=0).stop 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 shutdown_connections 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1d7c0b2900 0x7f1d7c0b4e10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d941045e0 0x7f1d9419cf90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d94109860 0x7f1d941a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 --2- 172.21.15.19:0/3973875903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d94108f10 0x7f1d9419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 >> 172.21.15.19:0/3973875903 conn(0x7f1d94100030 msgr2=0x7f1d9410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 shutdown_connections 2024-05-17T09:35:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:53.322+0000 7f1d9a611700 1 -- 172.21.15.19:0/3973875903 wait complete. 2024-05-17T09:35:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-05-17T09:35:53.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14303 8.51 MB/sec warmup 110 sec latency 125.505 ms 2024-05-17T09:35:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:53 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3973875903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:53.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:53 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3973875903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:53.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:53 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3973875903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:35:54.184 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 147.6 --await-for 45.0' 2024-05-17T09:35:54.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 147.6 --await-for 45.0 2024-05-17T09:35:54.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14479 8.57 MB/sec warmup 111 sec latency 79.618 ms 2024-05-17T09:35:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:35:54.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:54 smithi069 ceph-mon[35081]: pgmap v922: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 318 KiB/s wr, 37 op/s 2024-05-17T09:35:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:54 smithi019 ceph-mon[31620]: pgmap v922: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 318 KiB/s wr, 37 op/s 2024-05-17T09:35:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:54 smithi079 ceph-mon[35502]: pgmap v922: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 318 KiB/s wr, 37 op/s 2024-05-17T09:35:55.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14618 8.53 MB/sec warmup 112 sec latency 82.507 ms 2024-05-17T09:35:56.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 -- 172.21.15.19:0/2911622812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 msgr2=0x7f23141081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 --2- 172.21.15.19:0/2911622812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f23141081f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2308009920 tx=0x7f2308009c30 comp rx=0 tx=0).stop 2024-05-17T09:35:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 -- 172.21.15.19:0/2911622812 shutdown_connections 2024-05-17T09:35:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 --2- 172.21.15.19:0/2911622812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 0x7f2314110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 --2- 172.21.15.19:0/2911622812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 0x7f23141038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 --2- 172.21.15.19:0/2911622812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f23141081f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 -- 172.21.15.19:0/2911622812 >> 172.21.15.19:0/2911622812 conn(0x7f23140fedb0 msgr2=0x7f23141011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 -- 172.21.15.19:0/2911622812 shutdown_connections 2024-05-17T09:35:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.481+0000 7f2319d50700 1 -- 172.21.15.19:0/2911622812 wait complete. 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.482+0000 7f2319d50700 1 Processor -- start 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.482+0000 7f2319d50700 1 -- start start 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2319d50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 0x7f2314199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2319d50700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f231419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2319d50700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 0x7f231419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2319d50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2314113240 con 0x7f23141034b0 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2319d50700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23141130c0 con 0x7f2314108730 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2319d50700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23141133c0 con 0x7f2314107de0 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 0x7f2314199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 0x7f2314199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45842/0 (socket says 172.21.15.19:45842) 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f23137fe700 1 -- 172.21.15.19:0/892527358 learned_addr learned my addr 172.21.15.19:0/892527358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f231419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2313fff700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 0x7f231419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 -- 172.21.15.19:0/892527358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 msgr2=0x7f231419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 0x7f231419e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 -- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 msgr2=0x7f2314199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 0x7f2314199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 -- 172.21.15.19:0/892527358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231419ebe0 con 0x7f2314107de0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2313fff700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 0x7f231419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.483+0000 7f2312ffd700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f231419a2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.484+0000 7f2310ff9700 1 -- 172.21.15.19:0/892527358 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230802c070 con 0x7f2314107de0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.484+0000 7f2310ff9700 1 -- 172.21.15.19:0/892527358 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f230800f8f0 con 0x7f2314107de0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.484+0000 7f2310ff9700 1 -- 172.21.15.19:0/892527358 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308005040 con 0x7f2314107de0 2024-05-17T09:35:56.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.484+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231419ee70 con 0x7f2314107de0 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.484+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231419f300 con 0x7f2314107de0 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.487+0000 7f2310ff9700 1 -- 172.21.15.19:0/892527358 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f230802f070 con 0x7f2314107de0 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.488+0000 7f2310ff9700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22fc0b2900 0x7f22fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.488+0000 7f23137fe700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22fc0b2900 0x7f22fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.489+0000 7f2310ff9700 1 -- 172.21.15.19:0/892527358 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f23080e7330 con 0x7f2314107de0 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.489+0000 7f23137fe700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22fc0b2900 0x7f22fc0b4e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2304005950 tx=0x7f2304009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.489+0000 7f2328765b80 1 Processor -- start 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 -- start start 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf7c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf7be6b0 0x562faf7beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562faf6ece20 con 0x562faf6bd560 2024-05-17T09:35:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562faf6dc9c0 con 0x562faf63ffd0 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2328765b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562faf6e0a00 con 0x562faf7be6b0 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf7c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2312ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2312ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:44266/0 (socket says 172.21.15.19:44266) 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2312ffd700 1 -- 172.21.15.19:0/3662523947 learned_addr learned my addr 172.21.15.19:0/3662523947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.490+0000 7f2313fff700 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf7be6b0 0x562faf7beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2312ffd700 1 -- 172.21.15.19:0/3662523947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf7be6b0 msgr2=0x562faf7beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2312ffd700 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf7be6b0 0x562faf7beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2312ffd700 1 -- 172.21.15.19:0/3662523947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 msgr2=0x562faf7c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2312ffd700 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf7c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2312ffd700 1 -- 172.21.15.19:0/3662523947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562faf62ca60 con 0x562faf63ffd0 2024-05-17T09:35:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2313fff700 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf7be6b0 0x562faf7beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f2312ffd700 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c4b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23080ae5c0 tx=0x7f230801de20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8a3b03e82a8538 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f22f9ffb700 1 -- 172.21.15.19:0/3662523947 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080aed30 con 0x562faf63ffd0 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f22f9ffb700 1 -- 172.21.15.19:0/3662523947 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23080ae630 con 0x562faf63ffd0 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.491+0000 7f22f9ffb700 1 -- 172.21.15.19:0/3662523947 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080ae890 con 0x562faf63ffd0 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.492+0000 7f2328765b80 1 -- 172.21.15.19:0/3662523947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 msgr2=0x562faf7c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.492+0000 7f2328765b80 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c4b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23080ae5c0 tx=0x7f230801de20 comp rx=0 tx=0).stop 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 -- 172.21.15.19:0/3662523947 shutdown_connections 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf7be6b0 0x562faf7beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf7c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 --2- 172.21.15.19:0/3662523947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c4b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 -- 172.21.15.19:0/3662523947 >> 172.21.15.19:0/3662523947 conn(0x562faf67ee10 msgr2=0x562faf7b95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 -- 172.21.15.19:0/3662523947 shutdown_connections 2024-05-17T09:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 -- 172.21.15.19:0/3662523947 wait complete. 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.493+0000 7f2328765b80 1 -- start start 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 Processor -- start 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 0x562faf7d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf7be6b0 0x562faf7d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562faf5e3130 con 0x562faf63ffd0 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562faf67e950 con 0x562faf7be6b0 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562faf6dc9c0 con 0x562faf6bd560 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2328765b80 10 client.0 _ll_register_callbacks cb 0x562faf677ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 0x562faf7d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 0x562faf7d62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45860/0 (socket says 172.21.15.19:45860) 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f23137fe700 1 -- 172.21.15.19:0/146104339 learned_addr learned my addr 172.21.15.19:0/146104339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2313fff700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf7be6b0 0x562faf7d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.494+0000 7f2312ffd700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f23137fe700 1 -- 172.21.15.19:0/146104339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 msgr2=0x562faf7d6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f23137fe700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7d6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f23137fe700 1 -- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf7be6b0 msgr2=0x562faf7d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f23137fe700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf7be6b0 0x562faf7d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f23137fe700 1 -- 172.21.15.19:0/146104339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562faf67b820 con 0x562faf63ffd0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f2313fff700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf7be6b0 0x562faf7d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f23137fe700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 0x562faf7d62f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f23040038e0 tx=0x7f23040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bac2476d692659 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f22f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.495+0000 7f22f2ffd700 1 -- 172.21.15.19:0/146104339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230401f980 con 0x562faf63ffd0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f22f2ffd700 1 -- 172.21.15.19:0/146104339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f230401fb20 con 0x562faf63ffd0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23080ae630 con 0x562faf63ffd0 2024-05-17T09:35:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f22f2ffd700 1 -- 172.21.15.19:0/146104339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f230401fd20 con 0x562faf63ffd0 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 10 client.18075 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562faf7ddf70 con 0x562faf63ffd0 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f22f2ffd700 1 -- 172.21.15.19:0/146104339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23080ae630 con 0x562faf63ffd0 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f22f2ffd700 1 -- 172.21.15.19:0/146104339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2304026070 con 0x562faf63ffd0 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 10 client.18075 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 10 client.18075 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 20 client.18075 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.496+0000 7f2328765b80 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7e0740 0x562faf7e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.497+0000 7f2328765b80 4 client.18075 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.497+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562faf6eed70 con 0x562faf7e0740 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.497+0000 7f2312ffd700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7e0740 0x562faf7e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.497+0000 7f22f2ffd700 10 client.18075 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.497+0000 7f2312ffd700 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7e0740 0x562faf7e2c10 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c846045f387d3e61 server_cookie=a007245d69aecb38 in_seq=0 out_seq=0 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.498+0000 7f22f2ffd700 1 -- 172.21.15.19:0/146104339 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562faf6eed70 con 0x562faf7e0740 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.498+0000 7f22f2ffd700 10 client.18075 handle_command_reply: tid=0 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.498+0000 7f2328765b80 1 client.18075 shutdown 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.498+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 msgr2=0x562faf7d62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.498+0000 7f2328765b80 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 0x562faf7d62f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f23040038e0 tx=0x7f23040047c0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 shutdown_connections 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7e0740 0x562faf7e2c10 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf7be6b0 0x562faf7d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf63ffd0 0x562faf7d62f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 --2- 172.21.15.19:0/146104339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 >> 172.21.15.19:0/146104339 conn(0x562faf67ee10 msgr2=0x562faf7b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f22f2ffd700 0 client.18075 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 shutdown_connections 2024-05-17T09:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 1 -- 172.21.15.19:0/146104339 wait complete. 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.499+0000 7f2328765b80 20 client.18075 trim_cache size 0 max 16384 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 Processor -- start 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 -- start start 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 0x562faf7c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23141b0970 con 0x562faf67ee10 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ae1a0 con 0x562faf63ffd0 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2328765b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23141abff0 con 0x562faf6bd560 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 0x562faf7c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f23137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 0x562faf7c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45876/0 (socket says 172.21.15.19:45876) 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f23137fe700 1 -- 172.21.15.19:0/3410492091 learned_addr learned my addr 172.21.15.19:0/3410492091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2312ffd700 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.534+0000 7f2313fff700 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f23137fe700 1 -- 172.21.15.19:0/3410492091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 msgr2=0x562faf7b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f23137fe700 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f23137fe700 1 -- 172.21.15.19:0/3410492091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 msgr2=0x562faf7c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f23137fe700 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f23137fe700 1 -- 172.21.15.19:0/3410492091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562faf7c6450 con 0x562faf67ee10 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f23137fe700 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 0x562faf7c5600 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2304016040 tx=0x7f2304009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23963e682b92bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f22f97fa700 1 -- 172.21.15.19:0/3410492091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230403b070 con 0x562faf67ee10 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f22f97fa700 1 -- 172.21.15.19:0/3410492091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f230401f240 con 0x562faf67ee10 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.535+0000 7f22f97fa700 1 -- 172.21.15.19:0/3410492091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304028440 con 0x562faf67ee10 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.536+0000 7f2328765b80 1 -- 172.21.15.19:0/3410492091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 msgr2=0x562faf7c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.536+0000 7f2328765b80 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 0x562faf7c5600 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2304016040 tx=0x7f2304009ad0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.536+0000 7f2328765b80 1 -- 172.21.15.19:0/3410492091 shutdown_connections 2024-05-17T09:35:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.536+0000 7f2328765b80 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf6bd560 0x562faf7b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf67ee10 0x562faf7c5600 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 --2- 172.21.15.19:0/3410492091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 -- 172.21.15.19:0/3410492091 >> 172.21.15.19:0/3410492091 conn(0x562faf7d0f80 msgr2=0x562faf7c2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 -- 172.21.15.19:0/3410492091 shutdown_connections 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 -- 172.21.15.19:0/3410492091 wait complete. 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 -- start start 2024-05-17T09:35:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.537+0000 7f2328765b80 1 Processor -- start 2024-05-17T09:35:56.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:56 smithi019 ceph-mon[31620]: pgmap v923: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 628 KiB/s wr, 67 op/s 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf67ee10 0x562faf7daf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf84a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ac660 con 0x562faf6bd560 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23141adad0 con 0x562faf63ffd0 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23141acd30 con 0x562faf67ee10 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f23137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f23137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:44292/0 (socket says 172.21.15.19:44292) 2024-05-17T09:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f23137fe700 1 -- 172.21.15.19:0/2140722546 learned_addr learned my addr 172.21.15.19:0/2140722546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2328765b80 10 client.0 _ll_register_callbacks cb 0x562faf677ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2312ffd700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf67ee10 0x562faf7daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.538+0000 7f2313fff700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf84a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f23137fe700 1 -- 172.21.15.19:0/2140722546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf67ee10 msgr2=0x562faf7daf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f23137fe700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf67ee10 0x562faf7daf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f23137fe700 1 -- 172.21.15.19:0/2140722546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 msgr2=0x562faf84a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f23137fe700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf84a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f23137fe700 1 -- 172.21.15.19:0/2140722546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562faf84a830 con 0x562faf63ffd0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f23137fe700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7da9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2304039040 tx=0x7f2304005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41baf6c82c54f7d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f22f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f22f2ffd700 1 -- 172.21.15.19:0/2140722546 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304046060 con 0x562faf63ffd0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f22f2ffd700 1 -- 172.21.15.19:0/2140722546 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f23040055f0 con 0x562faf63ffd0 2024-05-17T09:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230401f240 con 0x562faf63ffd0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f22f2ffd700 1 -- 172.21.15.19:0/2140722546 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f230403b070 con 0x562faf63ffd0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f2328765b80 10 client.27638 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.539+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562faf84b940 con 0x562faf63ffd0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f22f2ffd700 1 -- 172.21.15.19:0/2140722546 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2304004ba0 con 0x562faf63ffd0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f22f2ffd700 1 -- 172.21.15.19:0/2140722546 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2304026050 con 0x562faf63ffd0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2328765b80 10 client.27638 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2328765b80 10 client.27638 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2328765b80 20 client.27638 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2328765b80 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7be6b0 0x562faf84deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2328765b80 4 client.27638 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.6, "await_for": 45.0}] 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.6, "await_for": 45.0}) -- 0x7f230401f240 con 0x562faf7be6b0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.540+0000 7f2312ffd700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7be6b0 0x562faf84deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.541+0000 7f22f2ffd700 10 client.27638 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.541+0000 7f2312ffd700 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7be6b0 0x562faf84deb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9423d43b2a340c server_cookie=4cc0ffcef2b2342a in_seq=0 out_seq=0 2024-05-17T09:35:56.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14791 8.57 MB/sec warmup 113 sec latency 90.540 ms 2024-05-17T09:35:56.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:56 smithi069 ceph-mon[35081]: pgmap v923: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 628 KiB/s wr, 67 op/s 2024-05-17T09:35:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:56 smithi079 ceph-mon[35502]: pgmap v923: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 628 KiB/s wr, 67 op/s 2024-05-17T09:35:56.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f22f2ffd700 1 -- 172.21.15.19:0/2140722546 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f230401f240 con 0x562faf7be6b0 2024-05-17T09:35:56.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f22f2ffd700 10 client.27638 handle_command_reply: tid=0 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f2328765b80 1 client.27638 shutdown 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 msgr2=0x562faf7da9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f2328765b80 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7da9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2304039040 tx=0x7f2304005270 comp rx=0 tx=0).stop 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 shutdown_connections 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.843+0000 7f2328765b80 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562faf7be6b0 0x562faf84deb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562faf6bd560 0x562faf84a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562faf63ffd0 0x562faf7da9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f22f2ffd700 0 client.27638 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 1 --2- 172.21.15.19:0/2140722546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562faf67ee10 0x562faf7daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 >> 172.21.15.19:0/2140722546 conn(0x562faf7d0f80 msgr2=0x562faf7c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 shutdown_connections 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 1 -- 172.21.15.19:0/2140722546 wait complete. 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.844+0000 7f2328765b80 20 client.27638 trim_cache size 0 max 16384 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 35, 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 58, 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "b2624efb": { 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 58, 2024-05-17T09:35:56.845 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 147.6, 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:35:56.846 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22fc0b2900 msgr2=0x7f22fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22fc0b2900 0x7f22fc0b4e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2304005950 tx=0x7f2304009310 comp rx=0 tx=0).stop 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 msgr2=0x7f231419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f231419a2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308009400 comp rx=0 tx=0).stop 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 shutdown_connections 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2314108730 0x7f231419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23141034b0 0x7f2314199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f22fc0b2900 0x7f22fc0b4e10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 --2- 172.21.15.19:0/892527358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2314107de0 0x7f231419a2b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.845+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 >> 172.21.15.19:0/892527358 conn(0x7f23140fedb0 msgr2=0x7f23141011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.846+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 shutdown_connections 2024-05-17T09:35:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:35:56.846+0000 7f2319d50700 1 -- 172.21.15.19:0/892527358 wait complete. 2024-05-17T09:35:57.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 8.49 MB/sec warmup 114 sec latency 1032.393 ms 2024-05-17T09:35:57.799 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b2624efb', quiesce took 7.3 seconds. Will release after: 20.3 2024-05-17T09:35:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 -- 172.21.15.19:0/2314590923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 msgr2=0x7fc024103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 --2- 172.21.15.19:0/2314590923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc024103bd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-05-17T09:35:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 -- 172.21.15.19:0/2314590923 shutdown_connections 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 --2- 172.21.15.19:0/2314590923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024109a00 0x7fc02410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 --2- 172.21.15.19:0/2314590923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc024103bd0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 --2- 172.21.15.19:0/2314590923 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc024105100 0x7fc024103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 -- 172.21.15.19:0/2314590923 >> 172.21.15.19:0/2314590923 conn(0x7fc0240fecb0 msgr2=0x7fc024101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 -- 172.21.15.19:0/2314590923 shutdown_connections 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.272+0000 7fc02999d640 1 -- 172.21.15.19:0/2314590923 wait complete. 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 Processor -- start 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 -- start start 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc02419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024105100 0x7fc02419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc024109a00 0x7fc0241a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc02410f650 con 0x7fc0241037c0 2024-05-17T09:35:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc02410f4d0 con 0x7fc024105100 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.273+0000 7fc02999d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc02410f7d0 con 0x7fc024109a00 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc02419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc02419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45900/0 (socket says 172.21.15.19:45900) 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 -- 172.21.15.19:0/1742588070 learned_addr learned my addr 172.21.15.19:0/1742588070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc023fff640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024105100 0x7fc02419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02919c640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc024109a00 0x7fc0241a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 -- 172.21.15.19:0/1742588070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc024109a00 msgr2=0x7fc0241a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc024109a00 0x7fc0241a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 -- 172.21.15.19:0/1742588070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024105100 msgr2=0x7fc02419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024105100 0x7fc02419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 -- 172.21.15.19:0/1742588070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0241a1cd0 con 0x7fc0241037c0 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02899b640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc02419a190 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc01400e9f0 tx=0x7fc01400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc021ffb640 1 -- 172.21.15.19:0/1742588070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400ccc0 con 0x7fc0241037c0 2024-05-17T09:35:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc021ffb640 1 -- 172.21.15.19:0/1742588070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0140035f0 con 0x7fc0241037c0 2024-05-17T09:35:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc021ffb640 1 -- 172.21.15.19:0/1742588070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400ce60 con 0x7fc0241037c0 2024-05-17T09:35:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.274+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0241a1fc0 con 0x7fc0241037c0 2024-05-17T09:35:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.275+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a2420 con 0x7fc0241037c0 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.277+0000 7fc021ffb640 1 -- 172.21.15.19:0/1742588070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc014004000 con 0x7fc0241037c0 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 Processor -- start 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 -- start start 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3fac1b0 0x561bd434d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd434dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd43549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561bd42bfc30 con 0x561bd3f8b4d0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561bd42af7d0 con 0x561bd3fac1b0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc031b8e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561bd42b3810 con 0x561bd3f832f0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd43549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd43549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45916/0 (socket says 172.21.15.19:45916) 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc023fff640 1 -- 172.21.15.19:0/1062638970 learned_addr learned my addr 172.21.15.19:0/1062638970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc02899b640 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd434dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc02919c640 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3fac1b0 0x561bd434d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc023fff640 1 -- 172.21.15.19:0/1062638970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 msgr2=0x561bd434dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.278+0000 7fc023fff640 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd434dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc023fff640 1 -- 172.21.15.19:0/1062638970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3fac1b0 msgr2=0x561bd434d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc023fff640 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3fac1b0 0x561bd434d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc023fff640 1 -- 172.21.15.19:0/1062638970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd3fc4210 con 0x561bd3f8b4d0 2024-05-17T09:35:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc021ffb640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc00c0b2920 0x7fc00c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc02919c640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc00c0b2920 0x7fc00c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc023fff640 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd43549c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc018031460 tx=0x7fc018009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4832775cb314f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc021ffb640 1 -- 172.21.15.19:0/1742588070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0140e28b0 con 0x7fc0241037c0 2024-05-17T09:35:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc00affd640 1 -- 172.21.15.19:0/1062638970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035030 con 0x561bd3f8b4d0 2024-05-17T09:35:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc00affd640 1 -- 172.21.15.19:0/1062638970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc018040070 con 0x561bd3f8b4d0 2024-05-17T09:35:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.279+0000 7fc00affd640 1 -- 172.21.15.19:0/1062638970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018003850 con 0x561bd3f8b4d0 2024-05-17T09:35:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 -- 172.21.15.19:0/1062638970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 msgr2=0x561bd43549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc02919c640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc00c0b2920 0x7fc00c0b4e30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x561bd434e710 tx=0x7fc01c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd43549c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc018031460 tx=0x7fc018009640 comp rx=0 tx=0).stop 2024-05-17T09:35:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 -- 172.21.15.19:0/1062638970 shutdown_connections 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd43549c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd434dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1062638970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3fac1b0 0x561bd434d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 -- 172.21.15.19:0/1062638970 >> 172.21.15.19:0/1062638970 conn(0x561bd3fa0300 msgr2=0x561bd4348ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 -- 172.21.15.19:0/1062638970 shutdown_connections 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 -- 172.21.15.19:0/1062638970 wait complete. 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.280+0000 7fc031b8e740 1 -- start start 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 Processor -- start 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd4356df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd4357330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3fac1b0 0x561bd435e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561bd3f95790 con 0x561bd3f832f0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561bd426b890 con 0x561bd3f8b4d0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561bd43553b0 con 0x561bd3fac1b0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc031b8e740 10 client.0 _ll_register_callbacks cb 0x561bd426ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc02899b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd4356df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc02899b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd4356df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45932/0 (socket says 172.21.15.19:45932) 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc02899b640 1 -- 172.21.15.19:0/1605877604 learned_addr learned my addr 172.21.15.19:0/1605877604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc023fff640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd4357330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.281+0000 7fc02919c640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3fac1b0 0x561bd435e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc02899b640 1 -- 172.21.15.19:0/1605877604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3fac1b0 msgr2=0x561bd435e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc02899b640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3fac1b0 0x561bd435e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc02899b640 1 -- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 msgr2=0x561bd4357330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc02899b640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd4357330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc02899b640 1 -- 172.21.15.19:0/1605877604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd4090620 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc023fff640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd4357330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc02899b640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd4356df0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc0140a9ab0 tx=0x7fc0140a98f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a05ce6b7e4f09eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc003fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc003fff640 1 -- 172.21.15.19:0/1605877604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0140a9290 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc003fff640 1 -- 172.21.15.19:0/1605877604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0140037a0 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc018040070 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc003fff640 1 -- 172.21.15.19:0/1605877604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0140a9630 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc031b8e740 10 client.18093 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561bd4361290 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc003fff640 1 -- 172.21.15.19:0/1605877604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc018040070 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.282+0000 7fc003fff640 1 -- 172.21.15.19:0/1605877604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc0140042f0 con 0x561bd3f832f0 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc031b8e740 10 client.18093 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc031b8e740 10 client.18093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc031b8e740 20 client.18093 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3f6ad80 0x561bd4364a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc031b8e740 4 client.18093 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd42c1b80 con 0x561bd3f6ad80 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc023fff640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3f6ad80 0x561bd4364a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc003fff640 10 client.18093 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.283+0000 7fc023fff640 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3f6ad80 0x561bd4364a80 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ded51e23795460 server_cookie=f3df27922fc4f27d in_seq=0 out_seq=0 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc003fff640 1 -- 172.21.15.19:0/1605877604 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561bd42c1b80 con 0x561bd3f6ad80 2024-05-17T09:35:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc003fff640 10 client.18093 handle_command_reply: tid=0 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc031b8e740 1 client.18093 shutdown 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 msgr2=0x561bd4356df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd4356df0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc0140a9ab0 tx=0x7fc0140a98f0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 shutdown_connections 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3f6ad80 0x561bd4364a80 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3fac1b0 0x561bd435e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.284+0000 7fc003fff640 0 client.18093 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.285+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd4357330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.285+0000 7fc031b8e740 1 --2- 172.21.15.19:0/1605877604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd4356df0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.285+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 >> 172.21.15.19:0/1605877604 conn(0x561bd3fa0300 msgr2=0x561bd4347fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.285+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 shutdown_connections 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.285+0000 7fc031b8e740 1 -- 172.21.15.19:0/1605877604 wait complete. 2024-05-17T09:35:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.285+0000 7fc031b8e740 20 client.18093 trim_cache size 0 max 16384 2024-05-17T09:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 Processor -- start 2024-05-17T09:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 -- start start 2024-05-17T09:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f6ad80 0x561bd43573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd435e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd4347580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241af130 con 0x561bd3f8b4d0 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241ac9a0 con 0x561bd3f6ad80 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.300+0000 7fc031b8e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241aa7f0 con 0x561bd3f832f0 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd4347580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd4347580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45934/0 (socket says 172.21.15.19:45934) 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 -- 172.21.15.19:0/2237953160 learned_addr learned my addr 172.21.15.19:0/2237953160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02899b640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f6ad80 0x561bd43573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc023fff640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd435e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 -- 172.21.15.19:0/2237953160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 msgr2=0x561bd435e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd435e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 -- 172.21.15.19:0/2237953160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f6ad80 msgr2=0x561bd43573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f6ad80 0x561bd43573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 -- 172.21.15.19:0/2237953160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd43579f0 con 0x561bd3f8b4d0 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02899b640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f6ad80 0x561bd43573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc023fff640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd435e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc02919c640 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd4347580 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc01c003ed0 tx=0x7fc01c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6faefd1cd77166 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc00a7fc640 1 -- 172.21.15.19:0/2237953160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004600 con 0x561bd3f8b4d0 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc00a7fc640 1 -- 172.21.15.19:0/2237953160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc01c0047a0 con 0x561bd3f8b4d0 2024-05-17T09:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.301+0000 7fc00a7fc640 1 -- 172.21.15.19:0/2237953160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00fda0 con 0x561bd3f8b4d0 2024-05-17T09:35:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 -- 172.21.15.19:0/2237953160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 msgr2=0x561bd4347580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd4347580 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc01c003ed0 tx=0x7fc01c016000 comp rx=0 tx=0).stop 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 -- 172.21.15.19:0/2237953160 shutdown_connections 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f8b4d0 0x561bd4347580 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f832f0 0x561bd435e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2237953160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f6ad80 0x561bd43573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 -- 172.21.15.19:0/2237953160 >> 172.21.15.19:0/2237953160 conn(0x561bd3fa0300 msgr2=0x561bd4343ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 -- 172.21.15.19:0/2237953160 shutdown_connections 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 -- 172.21.15.19:0/2237953160 wait complete. 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.302+0000 7fc031b8e740 1 -- start start 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 Processor -- start 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f6ad80 0x561bd435b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd435bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd435cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241aae60 con 0x561bd3f832f0 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241ac2d0 con 0x561bd3f8b4d0 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241ab530 con 0x561bd3f6ad80 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc031b8e740 10 client.0 _ll_register_callbacks cb 0x7fc014003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd435bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd435bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45948/0 (socket says 172.21.15.19:45948) 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc023fff640 1 -- 172.21.15.19:0/2641309316 learned_addr learned my addr 172.21.15.19:0/2641309316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc02899b640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f6ad80 0x561bd435b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc02919c640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd435cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc023fff640 1 -- 172.21.15.19:0/2641309316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f6ad80 msgr2=0x561bd435b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.303+0000 7fc023fff640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f6ad80 0x561bd435b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc023fff640 1 -- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 msgr2=0x561bd435cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc023fff640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd435cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc023fff640 1 -- 172.21.15.19:0/2641309316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd435d0c0 con 0x561bd3f832f0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc02919c640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd435cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc023fff640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd435bbd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc0180057c0 tx=0x7fc01802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672ebe603eefd9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc003fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc003fff640 1 -- 172.21.15.19:0/2641309316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035040 con 0x561bd3f832f0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc003fff640 1 -- 172.21.15.19:0/2641309316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc018004040 con 0x561bd3f832f0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01c0047a0 con 0x561bd3f832f0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc003fff640 1 -- 172.21.15.19:0/2641309316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc01805d050 con 0x561bd3f832f0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc031b8e740 10 client.18099 fetch_fsmap learned FSMap version 35 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561bd435e3c0 con 0x561bd3f832f0 2024-05-17T09:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.304+0000 7fc003fff640 1 -- 172.21.15.19:0/2641309316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc01c0047a0 con 0x561bd3f832f0 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc003fff640 1 -- 172.21.15.19:0/2641309316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc0180035f0 con 0x561bd3f832f0 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc031b8e740 10 client.18099 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc031b8e740 10 client.18099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc031b8e740 20 client.18099 populate_metadata read hostname 'smithi019' 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3fac1b0 0x561bd43cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc031b8e740 4 client.18099 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bd42bb360 con 0x561bd3fac1b0 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.305+0000 7fc02899b640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3fac1b0 0x561bd43cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.306+0000 7fc003fff640 10 client.18099 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.306+0000 7fc02899b640 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3fac1b0 0x561bd43cf3e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28be642b3475b0e server_cookie=7c542977018da92c in_seq=0 out_seq=0 2024-05-17T09:35:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.350+0000 7fc003fff640 1 -- 172.21.15.19:0/2641309316 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561bd42bb360 con 0x561bd3fac1b0 2024-05-17T09:35:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.350+0000 7fc003fff640 10 client.18099 handle_command_reply: tid=0 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.350+0000 7fc031b8e740 1 client.18099 shutdown 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.350+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 msgr2=0x561bd435bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd435bbd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc0180057c0 tx=0x7fc01802ab10 comp rx=0 tx=0).stop 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 shutdown_connections 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561bd3fac1b0 0x561bd43cf3e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd3f8b4d0 0x561bd435cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561bd3f832f0 0x561bd435bbd0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 --2- 172.21.15.19:0/2641309316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561bd3f6ad80 0x561bd435b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc003fff640 0 client.18099 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 >> 172.21.15.19:0/2641309316 conn(0x561bd3fa0300 msgr2=0x561bd4343ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 shutdown_connections 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 1 -- 172.21.15.19:0/2641309316 wait complete. 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.351+0000 7fc031b8e740 20 client.18099 trim_cache size 0 max 16384 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:35:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:35:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc00c0b2920 msgr2=0x7fc00c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc00c0b2920 0x7fc00c0b4e30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x561bd434e710 tx=0x7fc01c009500 comp rx=0 tx=0).stop 2024-05-17T09:35:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 msgr2=0x7fc02419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:35:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc02419a190 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc01400e9f0 tx=0x7fc01400eeb0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 shutdown_connections 2024-05-17T09:35:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc024109a00 0x7fc0241a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024105100 0x7fc02419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc00c0b2920 0x7fc00c0b4e30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 --2- 172.21.15.19:0/1742588070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0241037c0 0x7fc02419a190 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:35:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 >> 172.21.15.19:0/1742588070 conn(0x7fc0240fecb0 msgr2=0x7fc0241059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:35:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 shutdown_connections 2024-05-17T09:35:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:35:58.352+0000 7fc02999d640 1 -- 172.21.15.19:0/1742588070 wait complete. 2024-05-17T09:35:58.566 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 8.42 MB/sec warmup 115 sec latency 2032.439 ms 2024-05-17T09:35:58.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:35:58 smithi069 ceph-mon[35081]: pgmap v924: 129 pgs: 129 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-05-17T09:35:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:35:58 smithi019 ceph-mon[31620]: pgmap v924: 129 pgs: 129 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-05-17T09:35:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:35:58 smithi079 ceph-mon[35502]: pgmap v924: 129 pgs: 129 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-05-17T09:35:59.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 8.35 MB/sec warmup 116 sec latency 3032.557 ms 2024-05-17T09:36:00.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 8.28 MB/sec warmup 117 sec latency 4032.604 ms 2024-05-17T09:36:00.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:00 smithi069 ceph-mon[35081]: pgmap v925: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-05-17T09:36:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:00 smithi019 ceph-mon[31620]: pgmap v925: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-05-17T09:36:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:00 smithi079 ceph-mon[35502]: pgmap v925: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-05-17T09:36:01.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 8.21 MB/sec warmup 118 sec latency 5032.668 ms 2024-05-17T09:36:02.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 8.14 MB/sec warmup 119 sec latency 6032.795 ms 2024-05-17T09:36:02.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:02 smithi069 ceph-mon[35081]: pgmap v926: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-05-17T09:36:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:02 smithi019 ceph-mon[31620]: pgmap v926: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-05-17T09:36:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:02 smithi079 ceph-mon[35502]: pgmap v926: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-05-17T09:36:04.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 1 sec latency 8032.941 ms 2024-05-17T09:36:04.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:04 smithi069 ceph-mon[35081]: pgmap v927: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-17T09:36:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:04 smithi019 ceph-mon[31620]: pgmap v927: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-17T09:36:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:04 smithi079 ceph-mon[35502]: pgmap v927: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-17T09:36:05.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 2 sec latency 9033.036 ms 2024-05-17T09:36:06.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 3 sec latency 10033.146 ms 2024-05-17T09:36:06.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:06 smithi069 ceph-mon[35081]: pgmap v928: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-05-17T09:36:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:06 smithi019 ceph-mon[31620]: pgmap v928: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-05-17T09:36:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:06 smithi079 ceph-mon[35502]: pgmap v928: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-05-17T09:36:07.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 4 sec latency 11033.282 ms 2024-05-17T09:36:08.567 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 5 sec latency 12033.380 ms 2024-05-17T09:36:08.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:08 smithi069 ceph-mon[35081]: pgmap v929: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-05-17T09:36:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:08 smithi019 ceph-mon[31620]: pgmap v929: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-05-17T09:36:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:08 smithi079 ceph-mon[35502]: pgmap v929: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-05-17T09:36:09.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 6 sec latency 13033.505 ms 2024-05-17T09:36:10.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 7 sec latency 14033.569 ms 2024-05-17T09:36:10.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:10 smithi069 ceph-mon[35081]: pgmap v930: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 615 KiB/s wr, 74 op/s 2024-05-17T09:36:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:10 smithi019 ceph-mon[31620]: pgmap v930: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 615 KiB/s wr, 74 op/s 2024-05-17T09:36:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:10 smithi079 ceph-mon[35502]: pgmap v930: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 615 KiB/s wr, 74 op/s 2024-05-17T09:36:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 -- 172.21.15.19:0/901423943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0108a60 msgr2=0x7f68e010ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 --2- 172.21.15.19:0/901423943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0108a60 0x7f68e010ae80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009900 tx=0x7f68d802a410 comp rx=0 tx=0).stop 2024-05-17T09:36:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 -- 172.21.15.19:0/901423943 shutdown_connections 2024-05-17T09:36:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 --2- 172.21.15.19:0/901423943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0108a60 0x7f68e010ae80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 --2- 172.21.15.19:0/901423943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 0x7f68e0104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 --2- 172.21.15.19:0/901423943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0102b20 0x7f68e0103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 -- 172.21.15.19:0/901423943 >> 172.21.15.19:0/901423943 conn(0x7f68e00fdd10 msgr2=0x7f68e01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 -- 172.21.15.19:0/901423943 shutdown_connections 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.466+0000 7f68e87ea640 1 -- 172.21.15.19:0/901423943 wait complete. 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 Processor -- start 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 -- start start 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 0x7f68e00759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 0x7f68e019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0108a60 0x7f68e01a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010e500 con 0x7f68e0102b20 2024-05-17T09:36:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010e380 con 0x7f68e0108a60 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e87ea640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010e680 con 0x7f68e0103d60 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 0x7f68e00759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 0x7f68e00759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42860/0 (socket says 172.21.15.19:42860) 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 -- 172.21.15.19:0/277512757 learned_addr learned my addr 172.21.15.19:0/277512757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6d87640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0108a60 0x7f68e01a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e5d85640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 0x7f68e019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 -- 172.21.15.19:0/277512757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 msgr2=0x7f68e019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 0x7f68e019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 -- 172.21.15.19:0/277512757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0108a60 msgr2=0x7f68e01a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0108a60 0x7f68e01a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6586640 1 -- 172.21.15.19:0/277512757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e01a1c90 con 0x7f68e0102b20 2024-05-17T09:36:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e5d85640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 0x7f68e019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.467+0000 7f68e6d87640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0108a60 0x7f68e01a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.468+0000 7f68e6586640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 0x7f68e00759c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f68d000ea50 tx=0x7f68d000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.468+0000 7f68df7fe640 1 -- 172.21.15.19:0/277512757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000cdc0 con 0x7f68e0102b20 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.468+0000 7f68df7fe640 1 -- 172.21.15.19:0/277512757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d0011900 con 0x7f68e0102b20 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.468+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e01a1f80 con 0x7f68e0102b20 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.468+0000 7f68df7fe640 1 -- 172.21.15.19:0/277512757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d001a970 con 0x7f68e0102b20 2024-05-17T09:36:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.468+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01a23e0 con 0x7f68e0102b20 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 Processor -- start 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 -- start start 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68df7fe640 1 -- 172.21.15.19:0/277512757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f68d0011470 con 0x7f68e0102b20 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818aa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ef8181cc30 con 0x55ef814e02f0 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef8180c7d0 con 0x55ef814e84d0 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68ef772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ef81810810 con 0x55ef815091b0 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42872/0 (socket says 172.21.15.19:42872) 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.471+0000 7f68e6586640 1 -- 172.21.15.19:0/3461063572 learned_addr learned my addr 172.21.15.19:0/3461063572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e5d85640 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6d87640 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6586640 1 -- 172.21.15.19:0/3461063572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 msgr2=0x55ef818aa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6586640 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818aa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6586640 1 -- 172.21.15.19:0/3461063572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 msgr2=0x55ef818b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6586640 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6586640 1 -- 172.21.15.19:0/3461063572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef81521210 con 0x55ef814e02f0 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68e6586640 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818aa920 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f68d00e6e40 tx=0x7f68d00eb690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e4cb31361f91cd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68dcff9640 1 -- 172.21.15.19:0/3461063572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00f2070 con 0x55ef814e02f0 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.472+0000 7f68df7fe640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f68c80b28a0 0x7f68c80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68dcff9640 1 -- 172.21.15.19:0/3461063572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d00a9940 con 0x55ef814e02f0 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68dcff9640 1 -- 172.21.15.19:0/3461063572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00a9780 con 0x55ef814e02f0 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68e6d87640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f68c80b28a0 0x7f68c80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68df7fe640 1 -- 172.21.15.19:0/277512757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f68d00e2bc0 con 0x7f68e0102b20 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68ef772740 1 -- 172.21.15.19:0/3461063572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 msgr2=0x55ef818aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68e6d87640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f68c80b28a0 0x7f68c80b4db0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55ef818ab570 tx=0x7f68d80058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.473+0000 7f68ef772740 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818aa920 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f68d00e6e40 tx=0x7f68d00eb690 comp rx=0 tx=0).stop 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- 172.21.15.19:0/3461063572 shutdown_connections 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818aa920 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 --2- 172.21.15.19:0/3461063572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- 172.21.15.19:0/3461063572 >> 172.21.15.19:0/3461063572 conn(0x55ef814fd300 msgr2=0x55ef818a5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- 172.21.15.19:0/3461063572 shutdown_connections 2024-05-17T09:36:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- 172.21.15.19:0/3461063572 wait complete. 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- start start 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 Processor -- start 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e02f0 0x55ef818b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 0x55ef818b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef818bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ef814f2790 con 0x55ef814e84d0 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef817c8890 con 0x55ef815091b0 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.474+0000 7f68ef772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ef818b2270 con 0x55ef814e02f0 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68ef772740 10 client.0 _ll_register_callbacks cb 0x55ef817cbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 0x55ef818b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 0x55ef818b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42886/0 (socket says 172.21.15.19:42886) 2024-05-17T09:36:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 -- 172.21.15.19:0/2987769781 learned_addr learned my addr 172.21.15.19:0/2987769781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e6586640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e02f0 0x55ef818b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e6d87640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef818bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 -- 172.21.15.19:0/2987769781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e02f0 msgr2=0x55ef818b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e02f0 0x55ef818b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 -- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 msgr2=0x55ef818bb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef818bb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 -- 172.21.15.19:0/2987769781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef815ed620 con 0x55ef814e84d0 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e6586640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e02f0 0x55ef818b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e6d87640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef818bb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68e5d85640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 0x55ef818b4190 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f68d4013710 tx=0x7f68d4013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5807772444217ef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:36:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.475+0000 7f68c5ffb640 1 -- 172.21.15.19:0/2987769781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4011dd0 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68c5ffb640 1 -- 172.21.15.19:0/2987769781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f68d401b3e0 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d00a9940 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68c5ffb640 1 -- 172.21.15.19:0/2987769781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f68d401b5e0 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 10 client.18111 fetch_fsmap learned FSMap version 35 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef818be0f0 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68c5ffb640 1 -- 172.21.15.19:0/2987769781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d00a9940 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68c5ffb640 1 -- 172.21.15.19:0/2987769781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f68d401a070 con 0x55ef814e84d0 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 10 client.18111 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 10 client.18111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 20 client.18111 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef814c7d80 0x55ef818c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 4 client.18111 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.476+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef8181eb80 con 0x55ef814c7d80 2024-05-17T09:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.477+0000 7f68e6586640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef814c7d80 0x55ef818c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.477+0000 7f68e6586640 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef814c7d80 0x55ef818c1a00 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa5bb3a82ba7f2d server_cookie=c300d14a4958b5e7 in_seq=0 out_seq=0 2024-05-17T09:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.477+0000 7f68c5ffb640 10 client.18111 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68c5ffb640 1 -- 172.21.15.19:0/2987769781 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ef8181eb80 con 0x55ef814c7d80 2024-05-17T09:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68c5ffb640 10 client.18111 handle_command_reply: tid=0 2024-05-17T09:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 client.18111 shutdown 2024-05-17T09:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 msgr2=0x55ef818b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 0x55ef818b4190 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f68d4013710 tx=0x7f68d4013b10 comp rx=0 tx=0).stop 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 shutdown_connections 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef814c7d80 0x55ef818c1a00 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef818bb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e84d0 0x55ef818b4190 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68c5ffb640 0 client.18111 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 --2- 172.21.15.19:0/2987769781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e02f0 0x55ef818b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 >> 172.21.15.19:0/2987769781 conn(0x55ef814fd300 msgr2=0x55ef818a8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 shutdown_connections 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 1 -- 172.21.15.19:0/2987769781 wait complete. 2024-05-17T09:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.478+0000 7f68ef772740 20 client.18111 trim_cache size 0 max 16384 2024-05-17T09:36:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 Processor -- start 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 -- start start 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01af350 con 0x55ef814e02f0 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01acbc0 con 0x55ef814e84d0 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68ef772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01aaa10 con 0x55ef815091b0 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42898/0 (socket says 172.21.15.19:42898) 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 -- 172.21.15.19:0/4139067963 learned_addr learned my addr 172.21.15.19:0/4139067963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6d87640 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e5d85640 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 -- 172.21.15.19:0/4139067963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 msgr2=0x55ef818bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 -- 172.21.15.19:0/4139067963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 msgr2=0x55ef818bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818bb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6586640 1 -- 172.21.15.19:0/4139067963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef818ad5b0 con 0x55ef814e02f0 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.494+0000 7f68e6d87640 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.495+0000 7f68e6586640 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818ae7f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f68d00a9dc0 tx=0x7f68d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d411bdfe6176710 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.495+0000 7f68c7fff640 1 -- 172.21.15.19:0/4139067963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00ba7d0 con 0x55ef814e02f0 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.495+0000 7f68c7fff640 1 -- 172.21.15.19:0/4139067963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d00b8ca0 con 0x55ef814e02f0 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.495+0000 7f68c7fff640 1 -- 172.21.15.19:0/4139067963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00baf10 con 0x55ef814e02f0 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.495+0000 7f68ef772740 1 -- 172.21.15.19:0/4139067963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 msgr2=0x55ef818ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.495+0000 7f68ef772740 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818ae7f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f68d00a9dc0 tx=0x7f68d000cf60 comp rx=0 tx=0).stop 2024-05-17T09:36:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- 172.21.15.19:0/4139067963 shutdown_connections 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef815091b0 0x55ef818bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef814e84d0 0x55ef818bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 --2- 172.21.15.19:0/4139067963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef818ae7f0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- 172.21.15.19:0/4139067963 >> 172.21.15.19:0/4139067963 conn(0x55ef814fd300 msgr2=0x55ef818a9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- 172.21.15.19:0/4139067963 shutdown_connections 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- 172.21.15.19:0/4139067963 wait complete. 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- start start 2024-05-17T09:36:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 Processor -- start 2024-05-17T09:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef819249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e84d0 0x55ef81924f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef8192be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01ab080 con 0x55ef814e02f0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01ac4f0 con 0x55ef815091b0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01ab750 con 0x55ef814e84d0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68ef772740 10 client.0 _ll_register_callbacks cb 0x7f68d4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef819249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68e6586640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef819249f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42910/0 (socket says 172.21.15.19:42910) 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.496+0000 7f68e6586640 1 -- 172.21.15.19:0/1460438594 learned_addr learned my addr 172.21.15.19:0/1460438594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6d87640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef8192be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6586640 1 -- 172.21.15.19:0/1460438594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e84d0 msgr2=0x55ef81924f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e5d85640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e84d0 0x55ef81924f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6586640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e84d0 0x55ef81924f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6586640 1 -- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 msgr2=0x55ef8192be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6586640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef8192be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6586640 1 -- 172.21.15.19:0/1460438594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8192c450 con 0x55ef814e02f0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6d87640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef8192be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68e6586640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef819249f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f68d00a9b30 tx=0x7f68d001a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d7c1d8108fe464 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68c5ffb640 1 -- 172.21.15.19:0/1460438594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00f2070 con 0x55ef814e02f0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68c5ffb640 1 -- 172.21.15.19:0/1460438594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f68d00e6960 con 0x55ef814e02f0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef81818360 con 0x55ef814e02f0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68c5ffb640 1 -- 172.21.15.19:0/1460438594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f68d00bbc10 con 0x55ef814e02f0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68ef772740 10 client.18117 fetch_fsmap learned FSMap version 35 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef8192d6e0 con 0x55ef814e02f0 2024-05-17T09:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.497+0000 7f68c5ffb640 1 -- 172.21.15.19:0/1460438594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ef81818360 con 0x55ef814e02f0 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68c5ffb640 1 -- 172.21.15.19:0/1460438594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f68d0022030 con 0x55ef814e02f0 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68ef772740 10 client.18117 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68ef772740 10 client.18117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68ef772740 20 client.18117 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68ef772740 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef8149d1c0 0x55ef8192fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68ef772740 4 client.18117 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef817925f0 con 0x55ef8149d1c0 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68e5d85640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef8149d1c0 0x55ef8192fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68c5ffb640 10 client.18117 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.498+0000 7f68e5d85640 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef8149d1c0 0x55ef8192fd10 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816b48874f225c51 server_cookie=2b23ef70dbeab937 in_seq=0 out_seq=0 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68c5ffb640 1 -- 172.21.15.19:0/1460438594 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef817925f0 con 0x55ef8149d1c0 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68c5ffb640 10 client.18117 handle_command_reply: tid=0 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 client.18117 shutdown 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 msgr2=0x55ef819249f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef819249f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f68d00a9b30 tx=0x7f68d001a400 comp rx=0 tx=0).stop 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 shutdown_connections 2024-05-17T09:36:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ef8149d1c0 0x55ef8192fd10 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef815091b0 0x55ef8192be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef814e84d0 0x55ef81924f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68c5ffb640 0 client.18117 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 --2- 172.21.15.19:0/1460438594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef814e02f0 0x55ef819249f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 >> 172.21.15.19:0/1460438594 conn(0x55ef814fd300 msgr2=0x55ef818a9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 shutdown_connections 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 1 -- 172.21.15.19:0/1460438594 wait complete. 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.521+0000 7f68ef772740 20 client.18117 trim_cache size 0 max 16384 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:36:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:36:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f68c80b28a0 msgr2=0x7f68c80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f68c80b28a0 0x7f68c80b4db0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55ef818ab570 tx=0x7f68d80058d0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 msgr2=0x7f68e00759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 0x7f68e00759c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f68d000ea50 tx=0x7f68d000ef10 comp rx=0 tx=0).stop 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 shutdown_connections 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e0108a60 0x7f68e01a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68e0103d60 0x7f68e019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f68c80b28a0 0x7f68c80b4db0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 --2- 172.21.15.19:0/277512757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e0102b20 0x7f68e00759c0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 >> 172.21.15.19:0/277512757 conn(0x7f68e00fdd10 msgr2=0x7f68e0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 shutdown_connections 2024-05-17T09:36:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:11.522+0000 7f68e87ea640 1 -- 172.21.15.19:0/277512757 wait complete. 2024-05-17T09:36:11.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 8 sec latency 15033.689 ms 2024-05-17T09:36:12.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 9 sec latency 16033.766 ms 2024-05-17T09:36:12.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:12 smithi069 ceph-mon[35081]: pgmap v931: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 228 KiB/s wr, 32 op/s 2024-05-17T09:36:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:12 smithi019 ceph-mon[31620]: pgmap v931: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 228 KiB/s wr, 32 op/s 2024-05-17T09:36:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:12 smithi079 ceph-mon[35502]: pgmap v931: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 228 KiB/s wr, 32 op/s 2024-05-17T09:36:13.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 10 sec latency 17033.872 ms 2024-05-17T09:36:14.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 11 sec latency 18033.972 ms 2024-05-17T09:36:14.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:14 smithi069 ceph-mon[35081]: pgmap v932: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 235 KiB/s wr, 37 op/s 2024-05-17T09:36:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:14 smithi019 ceph-mon[31620]: pgmap v932: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 235 KiB/s wr, 37 op/s 2024-05-17T09:36:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:14 smithi079 ceph-mon[35502]: pgmap v932: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 235 KiB/s wr, 37 op/s 2024-05-17T09:36:15.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 12 sec latency 19034.099 ms 2024-05-17T09:36:16.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 13 sec latency 20034.179 ms 2024-05-17T09:36:16.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:16 smithi069 ceph-mon[35081]: pgmap v933: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 KiB/s wr, 13 op/s 2024-05-17T09:36:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:16 smithi019 ceph-mon[31620]: pgmap v933: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 KiB/s wr, 13 op/s 2024-05-17T09:36:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:16 smithi079 ceph-mon[35502]: pgmap v933: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 KiB/s wr, 13 op/s 2024-05-17T09:36:17.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 14 sec latency 21034.291 ms 2024-05-17T09:36:18.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:36:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:36:18.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 15 sec latency 22034.325 ms 2024-05-17T09:36:18.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:18 smithi069 ceph-mon[35081]: pgmap v934: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-05-17T09:36:18.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:18 smithi019 ceph-mon[31620]: pgmap v934: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-05-17T09:36:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:18 smithi079 ceph-mon[35502]: pgmap v934: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-05-17T09:36:19.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 16 sec latency 23034.362 ms 2024-05-17T09:36:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.189+0000 7f94ac3bd700 1 -- 172.21.15.19:0/53500880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a4107de0 msgr2=0x7f94a41081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.189+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/53500880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a4107de0 0x7f94a41081f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f94940099c0 tx=0x7f9494009cd0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 -- 172.21.15.19:0/53500880 shutdown_connections 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/53500880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a4108730 0x7f94a4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/53500880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a41034b0 0x7f94a41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/53500880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a4107de0 0x7f94a41081f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 -- 172.21.15.19:0/53500880 >> 172.21.15.19:0/53500880 conn(0x7f94a40fedb0 msgr2=0x7f94a41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 -- 172.21.15.19:0/53500880 shutdown_connections 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 -- 172.21.15.19:0/53500880 wait complete. 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 Processor -- start 2024-05-17T09:36:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.190+0000 7f94ac3bd700 1 -- start start 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94ac3bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 0x7f94a419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94ac3bd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a4107de0 0x7f94a419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94ac3bd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a4108730 0x7f94a4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94ac3bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41131f0 con 0x7f94a41034b0 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94ac3bd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4113070 con 0x7f94a4107de0 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94ac3bd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4113370 con 0x7f94a4108730 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 0x7f94a419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 0x7f94a419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34526/0 (socket says 172.21.15.19:34526) 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 -- 172.21.15.19:0/2834018421 learned_addr learned my addr 172.21.15.19:0/2834018421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94a9958700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a4107de0 0x7f94a419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa95a700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a4108730 0x7f94a4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 -- 172.21.15.19:0/2834018421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a4108730 msgr2=0x7f94a4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a4108730 0x7f94a4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 -- 172.21.15.19:0/2834018421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a4107de0 msgr2=0x7f94a419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a4107de0 0x7f94a419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 -- 172.21.15.19:0/2834018421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a419a070 con 0x7f94a41034b0 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.191+0000 7f94aa159700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 0x7f94a419f810 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f94a0007e20 tx=0x7f94a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.192+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0016020 con 0x7f94a41034b0 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.192+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a419a360 con 0x7f94a41034b0 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.192+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94a000ad40 con 0x7f94a41034b0 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.192+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00115e0 con 0x7f94a41034b0 2024-05-17T09:36:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.192+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a406b600 con 0x7f94a41034b0 2024-05-17T09:36:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.193+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a4104960 con 0x7f94a41034b0 2024-05-17T09:36:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.195+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f94a0019020 con 0x7f94a41034b0 2024-05-17T09:36:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.196+0000 7f949b7fe700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94900b2890 0x7f94900b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.196+0000 7f94a9958700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94900b2890 0x7f94900b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.196+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f94a00e2800 con 0x7f94a41034b0 2024-05-17T09:36:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.197+0000 7f94a9958700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94900b2890 0x7f94900b4da0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9494009990 tx=0x7f9494005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.200+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94a0020070 con 0x7f94a41034b0 2024-05-17T09:36:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.468+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94a40512b0 con 0x7f94a41034b0 2024-05-17T09:36:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.469+0000 7f949b7fe700 1 -- 172.21.15.19:0/2834018421 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11858 (secure 0 0 0) 0x7f94a00aa110 con 0x7f94a41034b0 2024-05-17T09:36:20.469 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:36:20.469 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:34:12.645154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24487,24497,24470]},"id":1}]} 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94900b2890 msgr2=0x7f94900b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94900b2890 0x7f94900b4da0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9494009990 tx=0x7f9494005fb0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 msgr2=0x7f94a419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 0x7f94a419f810 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f94a0007e20 tx=0x7f94a000a340 comp rx=0 tx=0).stop 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 shutdown_connections 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f94900b2890 0x7f94900b4da0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a4108730 0x7f94a4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a41034b0 0x7f94a419f810 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 --2- 172.21.15.19:0/2834018421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a4107de0 0x7f94a419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 >> 172.21.15.19:0/2834018421 conn(0x7f94a40fedb0 msgr2=0x7f94a41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 shutdown_connections 2024-05-17T09:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:20.471+0000 7f94ac3bd700 1 -- 172.21.15.19:0/2834018421 wait complete. 2024-05-17T09:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-05-17T09:36:20.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 17 sec latency 24034.392 ms 2024-05-17T09:36:20.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:20 smithi069 ceph-mon[35081]: pgmap v935: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:36:20.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:20 smithi019 ceph-mon[31620]: pgmap v935: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:36:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:20 smithi079 ceph-mon[35502]: pgmap v935: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:36:21.138 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id b2624efb --release --await' 2024-05-17T09:36:21.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id b2624efb --release --await 2024-05-17T09:36:21.397 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:36:21.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 18 sec latency 25034.421 ms 2024-05-17T09:36:21.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:21 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2834018421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:36:21.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:21 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2834018421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:36:21.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:21 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2834018421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:36:22.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14800 0.00 MB/sec execute 19 sec latency 26034.514 ms 2024-05-17T09:36:22.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:22 smithi019 ceph-mon[31620]: pgmap v936: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-05-17T09:36:22.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:22 smithi069 ceph-mon[35081]: pgmap v936: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-05-17T09:36:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:22 smithi079 ceph-mon[35502]: pgmap v936: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-05-17T09:36:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.265+0000 7facba55b700 1 -- 172.21.15.19:0/3941249350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb4108740 msgr2=0x7facb41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.265+0000 7facba55b700 1 --2- 172.21.15.19:0/3941249350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb4108740 0x7facb41100a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7facac00b0d0 tx=0x7facac00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 -- 172.21.15.19:0/3941249350 shutdown_connections 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 --2- 172.21.15.19:0/3941249350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb41034c0 0x7facb41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 --2- 172.21.15.19:0/3941249350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb4108740 0x7facb41100a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 --2- 172.21.15.19:0/3941249350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 -- 172.21.15.19:0/3941249350 >> 172.21.15.19:0/3941249350 conn(0x7facb40fef10 msgr2=0x7facb4101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 -- 172.21.15.19:0/3941249350 shutdown_connections 2024-05-17T09:36:23.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 -- 172.21.15.19:0/3941249350 wait complete. 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.266+0000 7facba55b700 1 Processor -- start 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 -- start start 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb41034c0 0x7facb40ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb40ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 0x7facb40ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facb4113220 con 0x7facb41034c0 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facb41130a0 con 0x7facb4107df0 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facba55b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7facb41133a0 con 0x7facb4108740 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facabfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb40ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facb8af8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 0x7facb40ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facb8af8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 0x7facb40ffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33390/0 (socket says 172.21.15.19:33390) 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facb8af8700 1 -- 172.21.15.19:0/862562716 learned_addr learned my addr 172.21.15.19:0/862562716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.267+0000 7facabfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb40ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:39672/0 (socket says 172.21.15.19:39672) 2024-05-17T09:36:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facb3fff700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb41034c0 0x7facb40ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facabfff700 1 -- 172.21.15.19:0/862562716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 msgr2=0x7facb40ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facabfff700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 0x7facb40ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facabfff700 1 -- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb41034c0 msgr2=0x7facb40ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facabfff700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb41034c0 0x7facb40ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facabfff700 1 -- 172.21.15.19:0/862562716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb41a8fd0 con 0x7facb4107df0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facb8af8700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 0x7facb40ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facb3fff700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb41034c0 0x7facb40ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facabfff700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb40ff930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00c940 tx=0x7fac9c00cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facb1ffb700 1 -- 172.21.15.19:0/862562716 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c014070 con 0x7facb4107df0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb41a92c0 con 0x7facb4107df0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facb1ffb700 1 -- 172.21.15.19:0/862562716 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac9c00ce30 con 0x7facb4107df0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.268+0000 7facb1ffb700 1 -- 172.21.15.19:0/862562716 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003a00 con 0x7facb4107df0 2024-05-17T09:36:23.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.269+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a9850 con 0x7facb4107df0 2024-05-17T09:36:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.272+0000 7facb1ffb700 1 -- 172.21.15.19:0/862562716 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fac9c017070 con 0x7facb4107df0 2024-05-17T09:36:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.272+0000 7facc8f70b80 1 Processor -- start 2024-05-17T09:36:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.272+0000 7facb1ffb700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faca00b2880 0x7faca00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.272+0000 7facc8f70b80 1 -- start start 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb8af8700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faca00b2880 0x7faca00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb1ffb700 1 -- 172.21.15.19:0/862562716 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fac9c0e1ea0 con 0x7facb4107df0 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x555958201560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x555958201aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x5559581fba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facc8f70b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555958129d90 con 0x5559580fa4d0 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facc8f70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555958119930 con 0x5559581fb620 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facc8f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55595811d970 con 0x55595807cf40 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x555958201560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x555958201560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34552/0 (socket says 172.21.15.19:34552) 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 -- 172.21.15.19:0/3575080936 learned_addr learned my addr 172.21.15.19:0/3575080936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facabfff700 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x555958201aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 -- 172.21.15.19:0/3575080936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 msgr2=0x555958201aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x555958201aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 -- 172.21.15.19:0/3575080936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 msgr2=0x5559581fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x5559581fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb3fff700 1 -- 172.21.15.19:0/3575080936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559580699d0 con 0x5559580fa4d0 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facabfff700 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x555958201aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.273+0000 7facb8af8700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faca00b2880 0x7faca00b4d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7facac009e60 tx=0x7facac004a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.274+0000 7facb3fff700 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x555958201560 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7faca4008c70 tx=0x7faca4008f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadc5241a1ca07e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.274+0000 7facaa7fc700 1 -- 172.21.15.19:0/3575080936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4014070 con 0x5559580fa4d0 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.274+0000 7facaa7fc700 1 -- 172.21.15.19:0/3575080936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faca400eda0 con 0x5559580fa4d0 2024-05-17T09:36:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.274+0000 7facaa7fc700 1 -- 172.21.15.19:0/3575080936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca40056d0 con 0x5559580fa4d0 2024-05-17T09:36:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 -- 172.21.15.19:0/3575080936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 msgr2=0x555958201560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x555958201560 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7faca4008c70 tx=0x7faca4008f80 comp rx=0 tx=0).stop 2024-05-17T09:36:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 -- 172.21.15.19:0/3575080936 shutdown_connections 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x5559581fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x555958201560 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3575080936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x555958201aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 -- 172.21.15.19:0/3575080936 >> 172.21.15.19:0/3575080936 conn(0x5559580bbd80 msgr2=0x5559581f6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 -- 172.21.15.19:0/3575080936 shutdown_connections 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 -- 172.21.15.19:0/3575080936 wait complete. 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.275+0000 7facc8f70b80 1 -- start start 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 Processor -- start 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 0x5559582051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559580fa4d0 0x555958205700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559581fb620 0x555958205c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559580200a0 con 0x55595807cf40 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559580bb8c0 con 0x5559580fa4d0 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5559582113d0 con 0x5559581fb620 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facc8f70b80 10 client.0 _ll_register_callbacks cb 0x5559580b4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facb3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 0x5559582051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facb8af8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559581fb620 0x555958205c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facb3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 0x5559582051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34566/0 (socket says 172.21.15.19:34566) 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facb8af8700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559581fb620 0x555958205c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33420/0 (socket says 172.21.15.19:33420) 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facabfff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559580fa4d0 0x555958205700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.276+0000 7facb3fff700 1 -- 172.21.15.19:0/2021053680 learned_addr learned my addr 172.21.15.19:0/2021053680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facabfff700 1 -- 172.21.15.19:0/2021053680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559581fb620 msgr2=0x555958205c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facabfff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559581fb620 0x555958205c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facabfff700 1 -- 172.21.15.19:0/2021053680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 msgr2=0x5559582051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facabfff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 0x5559582051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facabfff700 1 -- 172.21.15.19:0/2021053680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559580b8790 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facb3fff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 0x5559582051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facabfff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559580fa4d0 0x555958205700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0a9040 tx=0x7fac9c0a8c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c32bc5d3d1002d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7fac937fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7fac937fe700 1 -- 172.21.15.19:0/2021053680 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0ad460 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7fac937fe700 1 -- 172.21.15.19:0/2021053680 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fac9c0a9310 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca400eda0 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7fac937fe700 1 -- 172.21.15.19:0/2021053680 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fac9c0b1f10 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facc8f70b80 10 client.27674 fetch_fsmap learned FSMap version 35 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.277+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555958212ca0 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7fac937fe700 1 -- 172.21.15.19:0/2021053680 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faca400eda0 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7fac937fe700 1 -- 172.21.15.19:0/2021053680 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fac9c068b20 con 0x5559580fa4d0 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facc8f70b80 10 client.27674 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:36:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facc8f70b80 10 client.27674 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facc8f70b80 20 client.27674 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facc8f70b80 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5559582152e0 0x555958217700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facc8f70b80 4 client.27674 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595812bce0 con 0x5559582152e0 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.278+0000 7facb3fff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5559582152e0 0x555958217700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.279+0000 7fac937fe700 10 client.27674 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.279+0000 7facb3fff700 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5559582152e0 0x555958217700 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa75538f4fd89b7 server_cookie=416d6ad7674e30db in_seq=0 out_seq=0 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7fac937fe700 1 -- 172.21.15.19:0/2021053680 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55595812bce0 con 0x5559582152e0 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7fac937fe700 10 client.27674 handle_command_reply: tid=0 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 client.27674 shutdown 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559580fa4d0 msgr2=0x555958205700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559580fa4d0 0x555958205700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0a9040 tx=0x7fac9c0a8c40 comp rx=0 tx=0).stop 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 shutdown_connections 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559581fb620 0x555958205c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5559582152e0 0x555958217700 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55595807cf40 0x5559582051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 --2- 172.21.15.19:0/2021053680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559580fa4d0 0x555958205700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7fac937fe700 0 client.27674 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 >> 172.21.15.19:0/2021053680 conn(0x5559580bbd80 msgr2=0x5559581f0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 shutdown_connections 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 1 -- 172.21.15.19:0/2021053680 wait complete. 2024-05-17T09:36:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.280+0000 7facc8f70b80 20 client.27674 trim_cache size 0 max 16384 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 Processor -- start 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 -- start start 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x5559581f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x55595821cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x55595821d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b4e20 con 0x5559580fa4d0 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b2650 con 0x5559581fb620 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facc8f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b04a0 con 0x55595807cf40 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facabfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x55595821cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facabfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x55595821cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34578/0 (socket says 172.21.15.19:34578) 2024-05-17T09:36:23.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.301+0000 7facabfff700 1 -- 172.21.15.19:0/3732754016 learned_addr learned my addr 172.21.15.19:0/3732754016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:23.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facb8af8700 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x55595821d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facb3fff700 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x5559581f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facabfff700 1 -- 172.21.15.19:0/3732754016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 msgr2=0x5559581f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facabfff700 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x5559581f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facabfff700 1 -- 172.21.15.19:0/3732754016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 msgr2=0x55595821d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facabfff700 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x55595821d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facabfff700 1 -- 172.21.15.19:0/3732754016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559581ee5e0 con 0x5559580fa4d0 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7facabfff700 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x55595821cf00 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0a8ff0 tx=0x7fac9c0a4fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdfdcf13cc8f4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7faca9ffb700 1 -- 172.21.15.19:0/3732754016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0a4d00 con 0x5559580fa4d0 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7faca9ffb700 1 -- 172.21.15.19:0/3732754016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac9c0c3070 con 0x5559580fa4d0 2024-05-17T09:36:23.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.302+0000 7faca9ffb700 1 -- 172.21.15.19:0/3732754016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0b1ce0 con 0x5559580fa4d0 2024-05-17T09:36:23.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 -- 172.21.15.19:0/3732754016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 msgr2=0x55595821cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x55595821cf00 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0a8ff0 tx=0x7fac9c0a4fb0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 -- 172.21.15.19:0/3732754016 shutdown_connections 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55595807cf40 0x5559581f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559581fb620 0x55595821d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3732754016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559580fa4d0 0x55595821cf00 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.303+0000 7facc8f70b80 1 -- 172.21.15.19:0/3732754016 >> 172.21.15.19:0/3732754016 conn(0x5559580bbd80 msgr2=0x5559581f0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.304+0000 7facc8f70b80 1 -- 172.21.15.19:0/3732754016 shutdown_connections 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.304+0000 7facc8f70b80 1 -- 172.21.15.19:0/3732754016 wait complete. 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.304+0000 7facc8f70b80 1 -- start start 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.304+0000 7facc8f70b80 1 Processor -- start 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55595807cf40 0x555958206710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559580fa4d0 0x555958206c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 0x555958207190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b0b10 con 0x5559581fb620 2024-05-17T09:36:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b1f80 con 0x55595807cf40 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b11e0 con 0x5559580fa4d0 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facc8f70b80 10 client.0 _ll_register_callbacks cb 0x5559580b4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 0x555958207190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 0x555958207190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34586/0 (socket says 172.21.15.19:34586) 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 -- 172.21.15.19:0/3825446222 learned_addr learned my addr 172.21.15.19:0/3825446222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facabfff700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559580fa4d0 0x555958206c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb3fff700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55595807cf40 0x555958206710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 -- 172.21.15.19:0/3825446222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559580fa4d0 msgr2=0x555958206c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559580fa4d0 0x555958206c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 -- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55595807cf40 msgr2=0x555958206710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55595807cf40 0x555958206710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 -- 172.21.15.19:0/3825446222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595827edd0 con 0x5559581fb620 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facabfff700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559580fa4d0 0x555958206c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:23.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb3fff700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55595807cf40 0x555958206710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7facb8af8700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 0x555958207190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7facac007ed0 tx=0x7facac00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bac03b845bc2a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.305+0000 7fac937fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7fac937fe700 1 -- 172.21.15.19:0/3825446222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac01caa0 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7fac937fe700 1 -- 172.21.15.19:0/3825446222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7facac01cc40 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9c0c3070 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7fac937fe700 1 -- 172.21.15.19:0/3825446222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7facac01ce40 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 10 client.18141 fetch_fsmap learned FSMap version 35 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555958280020 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7fac937fe700 1 -- 172.21.15.19:0/3825446222 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac9c0c3070 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7fac937fe700 1 -- 172.21.15.19:0/3825446222 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7facac011070 con 0x5559581fb620 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 10 client.18141 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 10 client.18141 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 20 client.18141 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.306+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555958280810 0x555958282c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.307+0000 7facc8f70b80 4 client.18141 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b2624efb", "release": true, "await": true}] 2024-05-17T09:36:23.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.307+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b2624efb", "release": true, "await": true}) -- 0x5559581254c0 con 0x555958280810 2024-05-17T09:36:23.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.307+0000 7facb3fff700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555958280810 0x555958282c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:23.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.307+0000 7facb3fff700 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555958280810 0x555958282c30 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb983b7c60b1195 server_cookie=2c1850dbb068627a in_seq=0 out_seq=0 2024-05-17T09:36:23.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.307+0000 7fac937fe700 10 client.18141 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7fac937fe700 1 -- 172.21.15.19:0/3825446222 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5559581254c0 con 0x555958280810 2024-05-17T09:36:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7fac937fe700 10 client.18141 handle_command_reply: tid=0 2024-05-17T09:36:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 client.18141 shutdown 2024-05-17T09:36:23.346 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:36:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 35, 2024-05-17T09:36:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:36:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 60, 2024-05-17T09:36:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:36:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "b2624efb": { 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 60, 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 147.6, 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:36:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:36:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:36:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:36:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:36:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:36:23.348 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 msgr2=0x555958207190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 0x555958207190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7facac007ed0 tx=0x7facac00b3e0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 shutdown_connections 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x555958280810 0x555958282c30 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559581fb620 0x555958207190 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55595807cf40 0x555958206710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7fac937fe700 0 client.18141 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 --2- 172.21.15.19:0/3825446222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5559580fa4d0 0x555958206c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 >> 172.21.15.19:0/3825446222 conn(0x5559580bbd80 msgr2=0x5559582036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.345+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 shutdown_connections 2024-05-17T09:36:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.346+0000 7facc8f70b80 1 -- 172.21.15.19:0/3825446222 wait complete. 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.346+0000 7facc8f70b80 20 client.18141 trim_cache size 0 max 16384 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faca00b2880 msgr2=0x7faca00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faca00b2880 0x7faca00b4d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7facac009e60 tx=0x7facac004a30 comp rx=0 tx=0).stop 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 msgr2=0x7facb40ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb40ff930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00c940 tx=0x7fac9c00cc50 comp rx=0 tx=0).stop 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 shutdown_connections 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7faca00b2880 0x7faca00b4d90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7facb4108740 0x7facb40ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facb41034c0 0x7facb40ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 --2- 172.21.15.19:0/862562716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facb4107df0 0x7facb40ff930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 >> 172.21.15.19:0/862562716 conn(0x7facb40fef10 msgr2=0x7facb406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.347+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 shutdown_connections 2024-05-17T09:36:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:36:23.348+0000 7facba55b700 1 -- 172.21.15.19:0/862562716 wait complete. 2024-05-17T09:36:23.568 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14832 0.02 MB/sec execute 20 sec latency 26787.140 ms 2024-05-17T09:36:24.093 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b2624efb', total seconds elapsed: 33.6 2024-05-17T09:36:24.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 14977 0.38 MB/sec execute 21 sec latency 83.903 ms 2024-05-17T09:36:24.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:24 smithi069 ceph-mon[35081]: pgmap v937: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:36:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 -- 172.21.15.19:0/1368841030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 msgr2=0x7fe058075300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/1368841030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe058075300 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe050009860 tx=0x7fe05002a340 comp rx=0 tx=0).stop 2024-05-17T09:36:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 -- 172.21.15.19:0/1368841030 shutdown_connections 2024-05-17T09:36:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/1368841030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe058075300 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/1368841030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058103d60 0x7fe058104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/1368841030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058102b20 0x7fe058103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.680+0000 7fe05ed0d640 1 -- 172.21.15.19:0/1368841030 >> 172.21.15.19:0/1368841030 conn(0x7fe0580fdd50 msgr2=0x7fe0581001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.681+0000 7fe05ed0d640 1 -- 172.21.15.19:0/1368841030 shutdown_connections 2024-05-17T09:36:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.681+0000 7fe05ed0d640 1 -- 172.21.15.19:0/1368841030 wait complete. 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 Processor -- start 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 -- start start 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe05819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058102b20 0x7fe05819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 0x7fe0581a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe058111810 con 0x7fe058074ef0 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe058111690 con 0x7fe058102b20 2024-05-17T09:36:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05ed0d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe058111990 con 0x7fe058103d60 2024-05-17T09:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05d2aa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 0x7fe0581a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05d2aa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 0x7fe0581a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55810/0 (socket says 172.21.15.19:55810) 2024-05-17T09:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05d2aa640 1 -- 172.21.15.19:0/4189478445 learned_addr learned my addr 172.21.15.19:0/4189478445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe05caa9640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe05819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.682+0000 7fe04ffff640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058102b20 0x7fe05819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05d2aa640 1 -- 172.21.15.19:0/4189478445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058102b20 msgr2=0x7fe05819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05d2aa640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058102b20 0x7fe05819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05d2aa640 1 -- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 msgr2=0x7fe05819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05d2aa640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe05819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05d2aa640 1 -- 172.21.15.19:0/4189478445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581a3e80 con 0x7fe058103d60 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe04ffff640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058102b20 0x7fe05819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05caa9640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe05819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.683+0000 7fe05d2aa640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 0x7fe0581a3810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe050030410 tx=0x7fe05002ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.684+0000 7fe0567fc640 1 -- 172.21.15.19:0/4189478445 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050042070 con 0x7fe058103d60 2024-05-17T09:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.684+0000 7fe0567fc640 1 -- 172.21.15.19:0/4189478445 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0500307f0 con 0x7fe058103d60 2024-05-17T09:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.684+0000 7fe0567fc640 1 -- 172.21.15.19:0/4189478445 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050037040 con 0x7fe058103d60 2024-05-17T09:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.684+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0581a4110 con 0x7fe058103d60 2024-05-17T09:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.685+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a44f0 con 0x7fe058103d60 2024-05-17T09:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.688+0000 7fe065bc4740 1 Processor -- start 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe0567fc640 1 -- 172.21.15.19:0/4189478445 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe050045070 con 0x7fe058103d60 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 -- start start 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d43a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d441820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc2d3acc30 con 0x55bc2d0991b0 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bc2d39c7d0 con 0x55bc2d0784d0 2024-05-17T09:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.689+0000 7fe065bc4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc2d3a0810 con 0x55bc2d0702f0 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe04ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d43a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55814/0 (socket says 172.21.15.19:55814) 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 -- 172.21.15.19:0/104088292 learned_addr learned my addr 172.21.15.19:0/104088292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe04ffff640 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d441820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 -- 172.21.15.19:0/104088292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 msgr2=0x55bc2d441820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d441820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 -- 172.21.15.19:0/104088292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 msgr2=0x55bc2d43a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d43a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 -- 172.21.15.19:0/104088292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc2d0b1210 con 0x55bc2d0702f0 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe05caa9640 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43a920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe048009ca0 tx=0x7fe0480117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa6f407252270b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.690+0000 7fe0567fc640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe03c0b2920 0x7fe03c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.691+0000 7fe04effd640 1 -- 172.21.15.19:0/104088292 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048005020 con 0x55bc2d0702f0 2024-05-17T09:36:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.691+0000 7fe04effd640 1 -- 172.21.15.19:0/104088292 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0480047a0 con 0x55bc2d0702f0 2024-05-17T09:36:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.691+0000 7fe04effd640 1 -- 172.21.15.19:0/104088292 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480126b0 con 0x55bc2d0702f0 2024-05-17T09:36:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.691+0000 7fe04ffff640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe03c0b2920 0x7fe03c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.691+0000 7fe0567fc640 1 -- 172.21.15.19:0/4189478445 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe050101e80 con 0x7fe058103d60 2024-05-17T09:36:24.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:24 smithi019 ceph-mon[31620]: pgmap v937: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:36:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 -- 172.21.15.19:0/104088292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 msgr2=0x55bc2d43a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43a920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe048009ca0 tx=0x7fe0480117b0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 -- 172.21.15.19:0/104088292 shutdown_connections 2024-05-17T09:36:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d441820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43a920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 --2- 172.21.15.19:0/104088292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d43a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 -- 172.21.15.19:0/104088292 >> 172.21.15.19:0/104088292 conn(0x55bc2d08d300 msgr2=0x55bc2d435d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe04ffff640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe03c0b2920 0x7fe03c0b4e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55bc2d43b570 tx=0x7fe0440099a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 -- 172.21.15.19:0/104088292 shutdown_connections 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.692+0000 7fe065bc4740 1 -- 172.21.15.19:0/104088292 wait complete. 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.693+0000 7fe065bc4740 1 -- start start 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.693+0000 7fe065bc4740 1 Processor -- start 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d443d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d444250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d443d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d443d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55954/0 (socket says 172.21.15.19:55954) 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe04ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d444250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe04ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d444250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55816/0 (socket says 172.21.15.19:55816) 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05d2aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05d2aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47822/0 (socket says 172.21.15.19:47822) 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc2d082790 con 0x55bc2d0991b0 2024-05-17T09:36:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 -- 172.21.15.19:0/1841118218 learned_addr learned my addr 172.21.15.19:0/1841118218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bc2d358890 con 0x55bc2d0702f0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc2d442270 con 0x55bc2d0784d0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe065bc4740 10 client.0 _ll_register_callbacks cb 0x55bc2d35bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 -- 172.21.15.19:0/1841118218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 msgr2=0x55bc2d444250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d444250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 -- 172.21.15.19:0/1841118218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 msgr2=0x55bc2d44b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.694+0000 7fe05caa9640 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc2d17d620 con 0x55bc2d0702f0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe05caa9640 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d443d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c00 tx=0x7fe048009fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89010cf3a3f0b0f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe03bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe03bfff640 1 -- 172.21.15.19:0/1841118218 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048028020 con 0x55bc2d0702f0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe03bfff640 1 -- 172.21.15.19:0/1841118218 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe048005200 con 0x55bc2d0702f0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0480047a0 con 0x55bc2d0702f0 2024-05-17T09:36:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe03bfff640 1 -- 172.21.15.19:0/1841118218 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe048033050 con 0x55bc2d0702f0 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe065bc4740 10 client.27689 fetch_fsmap learned FSMap version 35 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.695+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bc2d44e050 con 0x55bc2d0702f0 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe03bfff640 1 -- 172.21.15.19:0/1841118218 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0480047a0 con 0x55bc2d0702f0 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe03bfff640 1 -- 172.21.15.19:0/1841118218 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe048024410 con 0x55bc2d0702f0 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe065bc4740 10 client.27689 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe065bc4740 10 client.27689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe065bc4740 20 client.27689 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe065bc4740 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d057d80 0x55bc2d451500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe065bc4740 4 client.27689 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc2d3aeb80 con 0x55bc2d057d80 2024-05-17T09:36:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.696+0000 7fe04ffff640 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d057d80 0x55bc2d451500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.697+0000 7fe03bfff640 10 client.27689 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.697+0000 7fe04ffff640 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d057d80 0x55bc2d451500 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781879903d49d8b4 server_cookie=d5e1b3d652482b62 in_seq=0 out_seq=0 2024-05-17T09:36:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe03bfff640 1 -- 172.21.15.19:0/1841118218 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bc2d3aeb80 con 0x55bc2d057d80 2024-05-17T09:36:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe03bfff640 10 client.27689 handle_command_reply: tid=0 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 client.27689 shutdown 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 msgr2=0x55bc2d443d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d443d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c00 tx=0x7fe048009fc0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 shutdown_connections 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d057d80 0x55bc2d451500 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d444250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe03bfff640 0 client.27689 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 --2- 172.21.15.19:0/1841118218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d443d10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 >> 172.21.15.19:0/1841118218 conn(0x55bc2d08d300 msgr2=0x55bc2d438c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 shutdown_connections 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.698+0000 7fe065bc4740 1 -- 172.21.15.19:0/1841118218 wait complete. 2024-05-17T09:36:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.699+0000 7fe065bc4740 20 client.27689 trim_cache size 0 max 16384 2024-05-17T09:36:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.724+0000 7fe065bc4740 1 Processor -- start 2024-05-17T09:36:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.724+0000 7fe065bc4740 1 -- start start 2024-05-17T09:36:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d44b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d443f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe065bc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581b1490 con 0x55bc2d0991b0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe065bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581aed00 con 0x55bc2d0784d0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe065bc4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581acb50 con 0x55bc2d0702f0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe05d2aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d443f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe05d2aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d443f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47836/0 (socket says 172.21.15.19:47836) 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe05d2aa640 1 -- 172.21.15.19:0/3292111360 learned_addr learned my addr 172.21.15.19:0/3292111360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d44b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe05caa9640 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe04ffff640 1 -- 172.21.15.19:0/3292111360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 msgr2=0x55bc2d43e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe04ffff640 1 -- 172.21.15.19:0/3292111360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 msgr2=0x55bc2d443f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d443f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.725+0000 7fe04ffff640 1 -- 172.21.15.19:0/3292111360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc2d442150 con 0x55bc2d0784d0 2024-05-17T09:36:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.726+0000 7fe05d2aa640 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d443f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.726+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d44b0d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55bc2d451c00 tx=0x7fe044004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28af81a212120bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.726+0000 7fe04e7fc640 1 -- 172.21.15.19:0/3292111360 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044004690 con 0x55bc2d0784d0 2024-05-17T09:36:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.726+0000 7fe04e7fc640 1 -- 172.21.15.19:0/3292111360 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe044031070 con 0x55bc2d0784d0 2024-05-17T09:36:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.726+0000 7fe04e7fc640 1 -- 172.21.15.19:0/3292111360 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044022690 con 0x55bc2d0784d0 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 -- 172.21.15.19:0/3292111360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 msgr2=0x55bc2d44b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d44b0d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55bc2d451c00 tx=0x7fe044004a40 comp rx=0 tx=0).stop 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 -- 172.21.15.19:0/3292111360 shutdown_connections 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d443f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0784d0 0x55bc2d44b0d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3292111360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0702f0 0x55bc2d43e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.727+0000 7fe065bc4740 1 -- 172.21.15.19:0/3292111360 >> 172.21.15.19:0/3292111360 conn(0x55bc2d08d300 msgr2=0x55bc2d439560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.728+0000 7fe065bc4740 1 -- 172.21.15.19:0/3292111360 shutdown_connections 2024-05-17T09:36:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.728+0000 7fe065bc4740 1 -- 172.21.15.19:0/3292111360 wait complete. 2024-05-17T09:36:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.728+0000 7fe065bc4740 1 -- start start 2024-05-17T09:36:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.728+0000 7fe065bc4740 1 Processor -- start 2024-05-17T09:36:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d449a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d449f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581ad1c0 con 0x55bc2d0991b0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe05caa9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d449a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581ae630 con 0x55bc2d0702f0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581ad890 con 0x55bc2d0784d0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe05caa9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d449a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55972/0 (socket says 172.21.15.19:55972) 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe04ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d449f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe05caa9640 1 -- 172.21.15.19:0/3977800579 learned_addr learned my addr 172.21.15.19:0/3977800579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.729+0000 7fe065bc4740 10 client.0 _ll_register_callbacks cb 0x7fe048009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe05d2aa640 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe04ffff640 1 -- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 msgr2=0x55bc2d449a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d449a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe04ffff640 1 -- 172.21.15.19:0/3977800579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 msgr2=0x55bc2d44a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe04ffff640 1 -- 172.21.15.19:0/3977800579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc2d4b7c00 con 0x55bc2d0784d0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe04ffff640 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d449f50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55bc2d3589a0 tx=0x7fe044003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643ca775033af5e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe03bfff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe03bfff640 1 -- 172.21.15.19:0/3977800579 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044004cb0 con 0x55bc2d0784d0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe03bfff640 1 -- 172.21.15.19:0/3977800579 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe044004490 con 0x55bc2d0784d0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.730+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe044031070 con 0x55bc2d0784d0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe03bfff640 1 -- 172.21.15.19:0/3977800579 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe044047050 con 0x55bc2d0784d0 2024-05-17T09:36:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 10 client.27691 fetch_fsmap learned FSMap version 35 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bc2d4b8e80 con 0x55bc2d0784d0 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe03bfff640 1 -- 172.21.15.19:0/3977800579 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe044031070 con 0x55bc2d0784d0 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe03bfff640 1 -- 172.21.15.19:0/3977800579 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe044005040 con 0x55bc2d0784d0 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 10 client.27691 fetch_fsmap finished waiting for FSMap version 35 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 10 client.27691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 20 client.27691 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d02d1c0 0x55bc2d4bb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 4 client.27691 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc2d3a8360 con 0x55bc2d02d1c0 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.731+0000 7fe05caa9640 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d02d1c0 0x55bc2d4bb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.732+0000 7fe03bfff640 10 client.27691 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.732+0000 7fe05caa9640 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d02d1c0 0x55bc2d4bb930 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22dbce8e8e7a0e7 server_cookie=f4ad9f6f12c9b033 in_seq=0 out_seq=0 2024-05-17T09:36:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:24 smithi079 ceph-mon[35502]: pgmap v937: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-05-17T09:36:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.773+0000 7fe03bfff640 1 -- 172.21.15.19:0/3977800579 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bc2d3a8360 con 0x55bc2d02d1c0 2024-05-17T09:36:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.773+0000 7fe03bfff640 10 client.27691 handle_command_reply: tid=0 2024-05-17T09:36:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 client.27691 shutdown 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 msgr2=0x55bc2d449f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d449f50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55bc2d3589a0 tx=0x7fe044003ec0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 shutdown_connections 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55bc2d02d1c0 0x55bc2d4bb930 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc2d0991b0 0x55bc2d44a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe03bfff640 0 client.27691 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2d0784d0 0x55bc2d449f50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 --2- 172.21.15.19:0/3977800579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc2d0702f0 0x55bc2d449a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 >> 172.21.15.19:0/3977800579 conn(0x55bc2d08d300 msgr2=0x55bc2d439560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 shutdown_connections 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 1 -- 172.21.15.19:0/3977800579 wait complete. 2024-05-17T09:36:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.774+0000 7fe065bc4740 20 client.27691 trim_cache size 0 max 16384 2024-05-17T09:36:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:36:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:36:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:36:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe03c0b2920 msgr2=0x7fe03c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe03c0b2920 0x7fe03c0b4e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55bc2d43b570 tx=0x7fe0440099a0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 msgr2=0x7fe0581a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 0x7fe0581a3810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe050030410 tx=0x7fe05002ad30 comp rx=0 tx=0).stop 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 shutdown_connections 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe058103d60 0x7fe0581a3810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe058102b20 0x7fe05819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe03c0b2920 0x7fe03c0b4e30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 --2- 172.21.15.19:0/4189478445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe058074ef0 0x7fe05819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 >> 172.21.15.19:0/4189478445 conn(0x7fe0580fdd50 msgr2=0x7fe058104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 shutdown_connections 2024-05-17T09:36:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:24.775+0000 7fe05ed0d640 1 -- 172.21.15.19:0/4189478445 wait complete. 2024-05-17T09:36:25.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 15140 0.65 MB/sec execute 22 sec latency 83.697 ms 2024-05-17T09:36:26.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 15321 1.05 MB/sec execute 23 sec latency 66.956 ms 2024-05-17T09:36:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:26 smithi069 ceph-mon[35081]: pgmap v938: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-17T09:36:26.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:26 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:36:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:26 smithi019 ceph-mon[31620]: pgmap v938: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-17T09:36:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:26 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:36:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:26 smithi079 ceph-mon[35502]: pgmap v938: 129 pgs: 129 active+clean; 317 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-17T09:36:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:26 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:36:27.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 15497 1.29 MB/sec execute 24 sec latency 66.517 ms 2024-05-17T09:36:28.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 15648 1.48 MB/sec execute 25 sec latency 72.705 ms 2024-05-17T09:36:28.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:28 smithi069 ceph-mon[35081]: pgmap v939: 129 pgs: 129 active+clean; 321 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 452 KiB/s wr, 49 op/s 2024-05-17T09:36:28.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:28 smithi019 ceph-mon[31620]: pgmap v939: 129 pgs: 129 active+clean; 321 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 452 KiB/s wr, 49 op/s 2024-05-17T09:36:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:28 smithi079 ceph-mon[35502]: pgmap v939: 129 pgs: 129 active+clean; 321 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 452 KiB/s wr, 49 op/s 2024-05-17T09:36:29.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 15805 1.92 MB/sec execute 26 sec latency 61.189 ms 2024-05-17T09:36:30.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 15982 2.23 MB/sec execute 27 sec latency 66.486 ms 2024-05-17T09:36:30.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:30 smithi069 ceph-mon[35081]: pgmap v940: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 907 KiB/s wr, 84 op/s 2024-05-17T09:36:30.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:30 smithi019 ceph-mon[31620]: pgmap v940: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 907 KiB/s wr, 84 op/s 2024-05-17T09:36:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:30 smithi079 ceph-mon[35502]: pgmap v940: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 907 KiB/s wr, 84 op/s 2024-05-17T09:36:31.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 16204 2.88 MB/sec execute 28 sec latency 70.370 ms 2024-05-17T09:36:32.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 16403 3.14 MB/sec execute 29 sec latency 70.164 ms 2024-05-17T09:36:32.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:32 smithi069 ceph-mon[35081]: pgmap v941: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-05-17T09:36:32.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:32 smithi019 ceph-mon[31620]: pgmap v941: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-05-17T09:36:32.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:32 smithi079 ceph-mon[35502]: pgmap v941: 129 pgs: 129 active+clean; 324 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.0 MiB/s wr, 89 op/s 2024-05-17T09:36:33.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 16546 3.41 MB/sec execute 30 sec latency 62.017 ms 2024-05-17T09:36:34.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 16678 3.66 MB/sec execute 31 sec latency 89.119 ms 2024-05-17T09:36:34.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:34 smithi019 ceph-mon[31620]: pgmap v942: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.5 MiB/s wr, 152 op/s 2024-05-17T09:36:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:34 smithi079 ceph-mon[35502]: pgmap v942: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.5 MiB/s wr, 152 op/s 2024-05-17T09:36:34.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:34 smithi069 ceph-mon[35081]: pgmap v942: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.5 MiB/s wr, 152 op/s 2024-05-17T09:36:35.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 16836 3.80 MB/sec execute 32 sec latency 78.289 ms 2024-05-17T09:36:36.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 16982 3.95 MB/sec execute 33 sec latency 75.261 ms 2024-05-17T09:36:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:36 smithi019 ceph-mon[31620]: pgmap v943: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-05-17T09:36:36.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:36 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:36:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:36 smithi079 ceph-mon[35502]: pgmap v943: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-05-17T09:36:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:36 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:36:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:36 smithi069 ceph-mon[35081]: pgmap v943: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-05-17T09:36:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:36 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:36:37.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 17113 3.90 MB/sec execute 34 sec latency 73.348 ms 2024-05-17T09:36:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.928+0000 7f2f443b7640 1 -- 172.21.15.19:0/3312952199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10a660 msgr2=0x7f2f3c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.928+0000 7f2f443b7640 1 --2- 172.21.15.19:0/3312952199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10a660 0x7f2f3c10b360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005fd0 tx=0x7f2f3402c770 comp rx=0 tx=0).stop 2024-05-17T09:36:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.930+0000 7f2f443b7640 1 -- 172.21.15.19:0/3312952199 shutdown_connections 2024-05-17T09:36:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.930+0000 7f2f443b7640 1 --2- 172.21.15.19:0/3312952199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.930+0000 7f2f443b7640 1 --2- 172.21.15.19:0/3312952199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c10b8a0 0x7f2f3c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.930+0000 7f2f443b7640 1 --2- 172.21.15.19:0/3312952199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10a660 0x7f2f3c10b360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.930+0000 7f2f443b7640 1 -- 172.21.15.19:0/3312952199 >> 172.21.15.19:0/3312952199 conn(0x7f2f3c070a70 msgr2=0x7f2f3c072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.932+0000 7f2f443b7640 1 -- 172.21.15.19:0/3312952199 shutdown_connections 2024-05-17T09:36:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.932+0000 7f2f443b7640 1 -- 172.21.15.19:0/3312952199 wait complete. 2024-05-17T09:36:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.933+0000 7f2f443b7640 1 Processor -- start 2024-05-17T09:36:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.933+0000 7f2f443b7640 1 -- start start 2024-05-17T09:36:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f443b7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c10a660 0x7f2f3c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f443b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10b8a0 0x7f2f3c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f443b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f443b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c115c40 con 0x7f2f3c1105a0 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f443b7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c115ac0 con 0x7f2f3c10b8a0 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f443b7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c115dc0 con 0x7f2f3c10a660 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f42954640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f41952640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10b8a0 0x7f2f3c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f42954640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46828/0 (socket says 172.21.15.19:46828) 2024-05-17T09:36:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.934+0000 7f2f42954640 1 -- 172.21.15.19:0/2595248827 learned_addr learned my addr 172.21.15.19:0/2595248827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.935+0000 7f2f42954640 1 -- 172.21.15.19:0/2595248827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c10a660 msgr2=0x7f2f3c1a0810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T09:36:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.935+0000 7f2f42954640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c10a660 0x7f2f3c1a0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.935+0000 7f2f42954640 1 -- 172.21.15.19:0/2595248827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10b8a0 msgr2=0x7f2f3c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.935+0000 7f2f42954640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10b8a0 0x7f2f3c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.935+0000 7f2f42954640 1 -- 172.21.15.19:0/2595248827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c1a8350 con 0x7f2f3c1105a0 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.936+0000 7f2f42954640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1a7d60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800e970 tx=0x7f2f3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.937+0000 7f2f337fe640 1 -- 172.21.15.19:0/2595248827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800ccc0 con 0x7f2f3c1105a0 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.937+0000 7f2f337fe640 1 -- 172.21.15.19:0/2595248827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f3800ce60 con 0x7f2f3c1105a0 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.937+0000 7f2f337fe640 1 -- 172.21.15.19:0/2595248827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38010430 con 0x7f2f3c1105a0 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.937+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c1a8640 con 0x7f2f3c1105a0 2024-05-17T09:36:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.937+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c1a8b40 con 0x7f2f3c1105a0 2024-05-17T09:36:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.943+0000 7f2f337fe640 1 -- 172.21.15.19:0/2595248827 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2f380105d0 con 0x7f2f3c1105a0 2024-05-17T09:36:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.943+0000 7f2f4b26b740 1 Processor -- start 2024-05-17T09:36:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.944+0000 7f2f337fe640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f280b2ac0 0x7f2f280b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.945+0000 7f2f337fe640 1 -- 172.21.15.19:0/2595248827 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2f380e2f50 con 0x7f2f3c1105a0 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.945+0000 7f2f41952640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f280b2ac0 0x7f2f280b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.945+0000 7f2f4b26b740 1 -- start start 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266adc1b0 0x55e266e7d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e7d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 0x55e266e84820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e266defc30 con 0x55e266abb4d0 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e266ddf7d0 con 0x55e266adc1b0 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e266de3810 con 0x55e266ab32f0 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f42954640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 0x55e266e84820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f42954640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 0x55e266e84820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46836/0 (socket says 172.21.15.19:46836) 2024-05-17T09:36:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f42954640 1 -- 172.21.15.19:0/3426856786 learned_addr learned my addr 172.21.15.19:0/3426856786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.946+0000 7f2f41952640 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e7d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f42954640 1 -- 172.21.15.19:0/3426856786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 msgr2=0x55e266e7d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f42954640 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e7d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f42954640 1 -- 172.21.15.19:0/3426856786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266adc1b0 msgr2=0x55e266e7d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f41952640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f280b2ac0 0x7f2f280b4fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005e10 tx=0x7f2f2c0127c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f42954640 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266adc1b0 0x55e266e7d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f42954640 1 -- 172.21.15.19:0/3426856786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e266af4210 con 0x55e266abb4d0 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f42954640 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 0x55e266e84820 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2f380aa7a0 tx=0x7f2f380aa210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b000e98cd7c96b7c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f30bfe640 1 -- 172.21.15.19:0/3426856786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380aa5b0 con 0x55e266abb4d0 2024-05-17T09:36:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f30bfe640 1 -- 172.21.15.19:0/3426856786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f380ab1a0 con 0x55e266abb4d0 2024-05-17T09:36:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.947+0000 7f2f30bfe640 1 -- 172.21.15.19:0/3426856786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380a7090 con 0x55e266abb4d0 2024-05-17T09:36:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 -- 172.21.15.19:0/3426856786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 msgr2=0x55e266e84820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 0x55e266e84820 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2f380aa7a0 tx=0x7f2f380aa210 comp rx=0 tx=0).stop 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 -- 172.21.15.19:0/3426856786 shutdown_connections 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266abb4d0 0x55e266e84820 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e7d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/3426856786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266adc1b0 0x55e266e7d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.949+0000 7f2f4b26b740 1 -- 172.21.15.19:0/3426856786 >> 172.21.15.19:0/3426856786 conn(0x55e266ad0300 msgr2=0x55e266e78d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.951+0000 7f2f4b26b740 1 -- 172.21.15.19:0/3426856786 shutdown_connections 2024-05-17T09:36:37.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.951+0000 7f2f4b26b740 1 -- 172.21.15.19:0/3426856786 wait complete. 2024-05-17T09:36:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.952+0000 7f2f4b26b740 1 -- start start 2024-05-17T09:36:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.952+0000 7f2f4b26b740 1 Processor -- start 2024-05-17T09:36:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.953+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.953+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e87320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.953+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266adc1b0 0x55e266e8e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.953+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e266ac5790 con 0x55e266adc1b0 2024-05-17T09:36:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.953+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e266d9b890 con 0x55e266abb4d0 2024-05-17T09:36:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.953+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e266e85270 con 0x55e266ab32f0 2024-05-17T09:36:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.954+0000 7f2f4b26b740 10 client.0 _ll_register_callbacks cb 0x55e266d9ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.954+0000 7f2f42153640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.955+0000 7f2f42153640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e86de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:38530/0 (socket says 172.21.15.19:38530) 2024-05-17T09:36:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.955+0000 7f2f42153640 1 -- 172.21.15.19:0/1219096836 learned_addr learned my addr 172.21.15.19:0/1219096836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.955+0000 7f2f42954640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266adc1b0 0x55e266e8e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.955+0000 7f2f41952640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e87320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f42954640 1 -- 172.21.15.19:0/1219096836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 msgr2=0x55e266e86de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f42954640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e86de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f42954640 1 -- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 msgr2=0x55e266e87320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f42954640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e87320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f42954640 1 -- 172.21.15.19:0/1219096836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e266bc0620 con 0x55e266adc1b0 2024-05-17T09:36:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f42954640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266adc1b0 0x55e266e8e280 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2f380aa7a0 tx=0x7f2f380a9f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838b933af2b21288 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.956+0000 7f2f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:36:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.957+0000 7f2f25ffb640 1 -- 172.21.15.19:0/1219096836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380aa5b0 con 0x55e266adc1b0 2024-05-17T09:36:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.957+0000 7f2f25ffb640 1 -- 172.21.15.19:0/1219096836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2f380c4040 con 0x55e266adc1b0 2024-05-17T09:36:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.957+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f380ab1a0 con 0x55e266adc1b0 2024-05-17T09:36:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.957+0000 7f2f25ffb640 1 -- 172.21.15.19:0/1219096836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2f380aaaa0 con 0x55e266adc1b0 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 10 client.18171 fetch_fsmap learned FSMap version 36 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e266e90fa0 con 0x55e266adc1b0 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f25ffb640 1 -- 172.21.15.19:0/1219096836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f380aee50 con 0x55e266adc1b0 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f25ffb640 1 -- 172.21.15.19:0/1219096836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2f380a6410 con 0x55e266adc1b0 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 10 client.18171 fetch_fsmap finished waiting for FSMap version 36 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 10 client.18171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 20 client.18171 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266a9ad80 0x55e266e94300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 4 client.18171 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:36:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.958+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2f380ab1a0 con 0x55e266a9ad80 2024-05-17T09:36:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.960+0000 7f2f42153640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266a9ad80 0x55e266e94300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.961+0000 7f2f42153640 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266a9ad80 0x55e266e94300 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3692cb72e4f2fe server_cookie=b37741f5e73cb863 in_seq=0 out_seq=0 2024-05-17T09:36:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.962+0000 7f2f25ffb640 10 client.18171 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.963+0000 7f2f25ffb640 1 -- 172.21.15.19:0/1219096836 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f2f380ab1a0 con 0x55e266a9ad80 2024-05-17T09:36:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.963+0000 7f2f25ffb640 10 client.18171 handle_command_reply: tid=0 2024-05-17T09:36:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f4b26b740 1 client.18171 shutdown 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266adc1b0 msgr2=0x55e266e8e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266adc1b0 0x55e266e8e280 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2f380aa7a0 tx=0x7f2f380a9f30 comp rx=0 tx=0).stop 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 shutdown_connections 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266a9ad80 0x55e266e94300 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266adc1b0 0x55e266e8e280 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.964+0000 7f2f25ffb640 0 client.18171 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.965+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e87320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.965+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/1219096836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.965+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 >> 172.21.15.19:0/1219096836 conn(0x55e266ad0300 msgr2=0x55e266e77d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.965+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 shutdown_connections 2024-05-17T09:36:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.965+0000 7f2f4b26b740 1 -- 172.21.15.19:0/1219096836 wait complete. 2024-05-17T09:36:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.965+0000 7f2f4b26b740 20 client.18171 trim_cache size 0 max 16384 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 Processor -- start 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 -- start start 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e78340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266ab32f0 0x55e266e78880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266abb4d0 0x55e266e8e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b5a70 con 0x55e266a9ad80 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b3310 con 0x55e266ab32f0 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b1140 con 0x55e266abb4d0 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e78340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42954640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266abb4d0 0x55e266e8e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e78340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46846/0 (socket says 172.21.15.19:46846) 2024-05-17T09:36:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42954640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266abb4d0 0x55e266e8e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:38546/0 (socket says 172.21.15.19:38546) 2024-05-17T09:36:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42153640 1 -- 172.21.15.19:0/517687536 learned_addr learned my addr 172.21.15.19:0/517687536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f41952640 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266ab32f0 0x55e266e78880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42153640 1 -- 172.21.15.19:0/517687536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266abb4d0 msgr2=0x55e266e8e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42153640 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266abb4d0 0x55e266e8e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.989+0000 7f2f42153640 1 -- 172.21.15.19:0/517687536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266ab32f0 msgr2=0x55e266e78880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.990+0000 7f2f42153640 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266ab32f0 0x55e266e78880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.990+0000 7f2f42153640 1 -- 172.21.15.19:0/517687536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e266e78dc0 con 0x55e266a9ad80 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.990+0000 7f2f42153640 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e78340 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005fd0 tx=0x7f2f3402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99212c8e3d914a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.990+0000 7f2f27fff640 1 -- 172.21.15.19:0/517687536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34045070 con 0x55e266a9ad80 2024-05-17T09:36:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.990+0000 7f2f27fff640 1 -- 172.21.15.19:0/517687536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f34003e00 con 0x55e266a9ad80 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.990+0000 7f2f27fff640 1 -- 172.21.15.19:0/517687536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34038040 con 0x55e266a9ad80 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.991+0000 7f2f4b26b740 1 -- 172.21.15.19:0/517687536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 msgr2=0x55e266e78340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.991+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e78340 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005fd0 tx=0x7f2f3402cc70 comp rx=0 tx=0).stop 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 -- 172.21.15.19:0/517687536 shutdown_connections 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266abb4d0 0x55e266e8e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266ab32f0 0x55e266e78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/517687536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e78340 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 -- 172.21.15.19:0/517687536 >> 172.21.15.19:0/517687536 conn(0x55e266ad0300 msgr2=0x55e266e809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 -- 172.21.15.19:0/517687536 shutdown_connections 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 -- 172.21.15.19:0/517687536 wait complete. 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 -- start start 2024-05-17T09:36:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.992+0000 7f2f4b26b740 1 Processor -- start 2024-05-17T09:36:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e8c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e8d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b17d0 con 0x55e266a9ad80 2024-05-17T09:36:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b2c40 con 0x55e266abb4d0 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c1b1ea0 con 0x55e266ab32f0 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f42153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e8c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f42153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e8c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46862/0 (socket says 172.21.15.19:46862) 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f42153640 1 -- 172.21.15.19:0/2015824812 learned_addr learned my addr 172.21.15.19:0/2015824812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f41952640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f42954640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e8d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f4b26b740 10 client.0 _ll_register_callbacks cb 0x7f2f380b36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f41952640 1 -- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 msgr2=0x55e266e8d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f41952640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e8d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f41952640 1 -- 172.21.15.19:0/2015824812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 msgr2=0x55e266e8c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f41952640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e8c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.993+0000 7f2f41952640 1 -- 172.21.15.19:0/2015824812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e266e8deb0 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f42153640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e8c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f34003e00 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f41952640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e8c9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55e266e87e50 tx=0x7f2f2c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1ba344648af670 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f25ffb640 10 client.27703 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f25ffb640 1 -- 172.21.15.19:0/2015824812 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c02d060 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f25ffb640 1 -- 172.21.15.19:0/2015824812 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2f2c010430 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f25ffb640 1 -- 172.21.15.19:0/2015824812 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2f2c010670 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f4b26b740 10 client.27703 fetch_fsmap learned FSMap version 36 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.994+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55e266efc990 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f25ffb640 1 -- 172.21.15.19:0/2015824812 <== mon.1 v2:172.21.15.79:3300/0 4 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2f2c027030 con 0x55e266ab32f0 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f4b26b740 10 client.27703 fetch_fsmap finished waiting for FSMap version 36 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f4b26b740 10 client.27703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f4b26b740 20 client.27703 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266adc1b0 0x55e266eff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f4b26b740 4 client.27703 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:36:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e266df1b80 con 0x55e266adc1b0 2024-05-17T09:36:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.995+0000 7f2f42153640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266adc1b0 0x55e266eff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.996+0000 7f2f42153640 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266adc1b0 0x55e266eff210 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639126498e8e5c1f server_cookie=71bbdd3c02856440 in_seq=0 out_seq=0 2024-05-17T09:36:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:37.996+0000 7f2f25ffb640 10 client.27703 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f25ffb640 1 -- 172.21.15.19:0/2015824812 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e266df1b80 con 0x55e266adc1b0 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f25ffb640 10 client.27703 handle_command_reply: tid=0 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 client.27703 shutdown 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 msgr2=0x55e266e8c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e8c9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55e266e87e50 tx=0x7f2f2c004730 comp rx=0 tx=0).stop 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 shutdown_connections 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e266adc1b0 0x55e266eff210 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e266abb4d0 0x55e266e8d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e266ab32f0 0x55e266e8c9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f25ffb640 0 client.27703 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 --2- 172.21.15.19:0/2015824812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e266a9ad80 0x55e266e8c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 >> 172.21.15.19:0/2015824812 conn(0x55e266ad0300 msgr2=0x55e266e81360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 shutdown_connections 2024-05-17T09:36:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.087+0000 7f2f4b26b740 1 -- 172.21.15.19:0/2015824812 wait complete. 2024-05-17T09:36:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.088+0000 7f2f4b26b740 20 client.27703 trim_cache size 0 max 16384 2024-05-17T09:36:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:36:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:36:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.088+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f280b2ac0 msgr2=0x7f2f280b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.088+0000 7f2f443b7640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f280b2ac0 0x7f2f280b4fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c005e10 tx=0x7f2f2c0127c0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.088+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 msgr2=0x7f2f3c1a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.088+0000 7f2f443b7640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1a7d60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800e970 tx=0x7f2f3800ed70 comp rx=0 tx=0).stop 2024-05-17T09:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.088+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 shutdown_connections 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f280b2ac0 0x7f2f280b4fd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3c1105a0 0x7f2f3c1a7d60 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f3c10b8a0 0x7f2f3c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 --2- 172.21.15.19:0/2595248827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f3c10a660 0x7f2f3c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 >> 172.21.15.19:0/2595248827 conn(0x7f2f3c070a70 msgr2=0x7f2f3c10c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 shutdown_connections 2024-05-17T09:36:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:38.089+0000 7f2f443b7640 1 -- 172.21.15.19:0/2595248827 wait complete. 2024-05-17T09:36:38.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 17241 3.93 MB/sec execute 35 sec latency 120.000 ms 2024-05-17T09:36:38.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:38 smithi019 ceph-mon[31620]: pgmap v944: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 236 op/s 2024-05-17T09:36:38.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:36:38.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:36:38.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:36:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:38 smithi079 ceph-mon[35502]: pgmap v944: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 236 op/s 2024-05-17T09:36:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:36:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:36:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:36:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:38 smithi069 ceph-mon[35081]: pgmap v944: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 236 op/s 2024-05-17T09:36:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:36:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:36:38.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:36:39.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 17380 3.89 MB/sec execute 36 sec latency 74.500 ms 2024-05-17T09:36:40.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 17530 3.92 MB/sec execute 37 sec latency 70.417 ms 2024-05-17T09:36:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:40 smithi019 ceph-mon[31620]: pgmap v945: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2024-05-17T09:36:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:40 smithi079 ceph-mon[35502]: pgmap v945: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2024-05-17T09:36:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:40 smithi069 ceph-mon[35081]: pgmap v945: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2024-05-17T09:36:41.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 17688 3.99 MB/sec execute 38 sec latency 128.778 ms 2024-05-17T09:36:42.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 17854 4.21 MB/sec execute 39 sec latency 70.644 ms 2024-05-17T09:36:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:42 smithi079 ceph-mon[35502]: pgmap v946: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-05-17T09:36:42.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:42 smithi069 ceph-mon[35081]: pgmap v946: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-05-17T09:36:42.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:42 smithi019 ceph-mon[31620]: pgmap v946: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-05-17T09:36:43.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 18017 4.22 MB/sec execute 40 sec latency 64.070 ms 2024-05-17T09:36:43.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:43 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:36:43.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:43 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:36:43.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:43 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:36:44.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 18205 4.53 MB/sec execute 41 sec latency 68.689 ms 2024-05-17T09:36:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:44 smithi079 ceph-mon[35502]: pgmap v947: 129 pgs: 129 active+clean; 334 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 301 op/s 2024-05-17T09:36:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:44 smithi069 ceph-mon[35081]: pgmap v947: 129 pgs: 129 active+clean; 334 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 301 op/s 2024-05-17T09:36:44.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:44 smithi019 ceph-mon[31620]: pgmap v947: 129 pgs: 129 active+clean; 334 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 301 op/s 2024-05-17T09:36:45.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 18365 4.57 MB/sec execute 42 sec latency 66.066 ms 2024-05-17T09:36:46.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 18562 4.66 MB/sec execute 43 sec latency 74.903 ms 2024-05-17T09:36:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:46 smithi079 ceph-mon[35502]: pgmap v948: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-17T09:36:46.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:46 smithi069 ceph-mon[35081]: pgmap v948: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-17T09:36:46.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:46 smithi019 ceph-mon[31620]: pgmap v948: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-17T09:36:47.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 18714 4.63 MB/sec execute 44 sec latency 69.470 ms 2024-05-17T09:36:48.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 18849 4.61 MB/sec execute 45 sec latency 74.085 ms 2024-05-17T09:36:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:48 smithi079 ceph-mon[35502]: pgmap v949: 129 pgs: 129 active+clean; 334 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 294 op/s 2024-05-17T09:36:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:48 smithi069 ceph-mon[35081]: pgmap v949: 129 pgs: 129 active+clean; 334 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 294 op/s 2024-05-17T09:36:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:48 smithi019 ceph-mon[31620]: pgmap v949: 129 pgs: 129 active+clean; 334 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 294 op/s 2024-05-17T09:36:49.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19028 4.84 MB/sec execute 46 sec latency 76.539 ms 2024-05-17T09:36:50.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19168 4.90 MB/sec execute 47 sec latency 62.821 ms 2024-05-17T09:36:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:50 smithi079 ceph-mon[35502]: pgmap v950: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-05-17T09:36:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:50 smithi069 ceph-mon[35081]: pgmap v950: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-05-17T09:36:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:50 smithi019 ceph-mon[31620]: pgmap v950: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-05-17T09:36:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 -- 172.21.15.19:0/1254027084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901022a0 msgr2=0x7f9890102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 --2- 172.21.15.19:0/1254027084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901022a0 0x7f9890102fa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9880009860 tx=0x7f988002a340 comp rx=0 tx=0).stop 2024-05-17T09:36:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 -- 172.21.15.19:0/1254027084 shutdown_connections 2024-05-17T09:36:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 --2- 172.21.15.19:0/1254027084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98901081e0 0x7f989010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 --2- 172.21.15.19:0/1254027084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901034e0 0x7f98901038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 --2- 172.21.15.19:0/1254027084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901022a0 0x7f9890102fa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 -- 172.21.15.19:0/1254027084 >> 172.21.15.19:0/1254027084 conn(0x7f98900fd470 msgr2=0x7f98900ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 -- 172.21.15.19:0/1254027084 shutdown_connections 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.233+0000 7f9894ed1640 1 -- 172.21.15.19:0/1254027084 wait complete. 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.234+0000 7f9894ed1640 1 Processor -- start 2024-05-17T09:36:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.234+0000 7f9894ed1640 1 -- start start 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f9894ed1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 0x7f989019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f9894ed1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901034e0 0x7f989019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f9894ed1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98901081e0 0x7f98901a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f9894ed1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d3e0 con 0x7f98901022a0 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f9894ed1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d260 con 0x7f98901081e0 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f9894ed1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f989010d560 con 0x7f98901034e0 2024-05-17T09:36:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 0x7f989019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901034e0 0x7f989019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 0x7f989019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59994/0 (socket says 172.21.15.19:59994) 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988e59c640 1 -- 172.21.15.19:0/3345523427 learned_addr learned my addr 172.21.15.19:0/3345523427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988ed9d640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98901081e0 0x7f98901a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988dd9b640 1 -- 172.21.15.19:0/3345523427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98901081e0 msgr2=0x7f98901a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988dd9b640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98901081e0 0x7f98901a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988dd9b640 1 -- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 msgr2=0x7f989019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988dd9b640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 0x7f989019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988dd9b640 1 -- 172.21.15.19:0/3345523427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a1d00 con 0x7f98901034e0 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.235+0000 7f988e59c640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 0x7f989019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.236+0000 7f988dd9b640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901034e0 0x7f989019a700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9884005fd0 tx=0x7f988400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.236+0000 7f987f7fe640 1 -- 172.21.15.19:0/3345523427 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988400cd00 con 0x7f98901034e0 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.236+0000 7f987f7fe640 1 -- 172.21.15.19:0/3345523427 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98840035f0 con 0x7f98901034e0 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.236+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a1ff0 con 0x7f98901034e0 2024-05-17T09:36:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.236+0000 7f987f7fe640 1 -- 172.21.15.19:0/3345523427 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884010690 con 0x7f98901034e0 2024-05-17T09:36:51.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.236+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a24b0 con 0x7f98901034e0 2024-05-17T09:36:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.239+0000 7f987f7fe640 1 -- 172.21.15.19:0/3345523427 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9884003a80 con 0x7f98901034e0 2024-05-17T09:36:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.240+0000 7f987f7fe640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98780b29a0 0x7f98780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.240+0000 7f989bd81740 1 Processor -- start 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f988e59c640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98780b29a0 0x7f98780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f987f7fe640 1 -- 172.21.15.19:0/3345523427 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f98840e2750 con 0x7f98901034e0 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f989bd81740 1 -- start start 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f989bd81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa8f1b0 0x559c2fe303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f989bd81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 0x559c2fe30920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f989bd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.241+0000 7f989bd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c2fda2c30 con 0x559c2fa6e4d0 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f989bd81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559c2fd927d0 con 0x559c2fa662f0 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988e59c640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98780b29a0 0x7f98780b4eb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9880005e00 tx=0x7f988002bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 0x559c2fe30920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 0x559c2fe30920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37546/0 (socket says 172.21.15.19:37546) 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988dd9b640 1 -- 172.21.15.19:0/4018035924 learned_addr learned my addr 172.21.15.19:0/4018035924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988e59c640 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa8f1b0 0x559c2fe303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f989bd81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559c2fd96810 con 0x559c2fa8f1b0 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 -- 172.21.15.19:0/4018035924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa8f1b0 msgr2=0x559c2fe303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa8f1b0 0x559c2fe303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 -- 172.21.15.19:0/4018035924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 msgr2=0x559c2fe30920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 0x559c2fe30920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 -- 172.21.15.19:0/4018035924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2faa7210 con 0x559c2fa6e4d0 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988dd9b640 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 0x559c2fe30920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.242+0000 7f988ed9d640 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe37820 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9888005fd0 tx=0x7f988800a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2080c46fc9be7b5e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.243+0000 7f987cff9640 1 -- 172.21.15.19:0/4018035924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888003c30 con 0x559c2fa6e4d0 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.243+0000 7f987cff9640 1 -- 172.21.15.19:0/4018035924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9888003dd0 con 0x559c2fa6e4d0 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.243+0000 7f987cff9640 1 -- 172.21.15.19:0/4018035924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988800d9a0 con 0x559c2fa6e4d0 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 -- 172.21.15.19:0/4018035924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 msgr2=0x559c2fe37820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe37820 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9888005fd0 tx=0x7f988800a610 comp rx=0 tx=0).stop 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 -- 172.21.15.19:0/4018035924 shutdown_connections 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe37820 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa662f0 0x559c2fe30920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 --2- 172.21.15.19:0/4018035924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa8f1b0 0x559c2fe303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 -- 172.21.15.19:0/4018035924 >> 172.21.15.19:0/4018035924 conn(0x559c2fa83300 msgr2=0x559c2fe2bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 -- 172.21.15.19:0/4018035924 shutdown_connections 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.244+0000 7f989bd81740 1 -- 172.21.15.19:0/4018035924 wait complete. 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 -- start start 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 Processor -- start 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe39cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c2fa78790 con 0x559c2fa8f1b0 2024-05-17T09:36:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559c2fd4e890 con 0x559c2fa6e4d0 2024-05-17T09:36:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559c2fe38270 con 0x559c2fa662f0 2024-05-17T09:36:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f989bd81740 10 client.0 _ll_register_callbacks cb 0x559c2fd51d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.245+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37554/0 (socket says 172.21.15.19:37554) 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 -- 172.21.15.19:0/2559570972 learned_addr learned my addr 172.21.15.19:0/2559570972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988ed9d640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988e59c640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe39cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 -- 172.21.15.19:0/2559570972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 msgr2=0x559c2fe39cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe39cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 -- 172.21.15.19:0/2559570972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 msgr2=0x559c2fe41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe41160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 -- 172.21.15.19:0/2559570972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2fb73620 con 0x559c2fa6e4d0 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988ed9d640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f988dd9b640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3a200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f988400c6a0 tx=0x7f98840a50f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c6e48fb475e214 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f9875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.246+0000 7f9875ffb640 1 -- 172.21.15.19:0/2559570972 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840c3060 con 0x559c2fa6e4d0 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9888003dd0 con 0x559c2fa6e4d0 2024-05-17T09:36:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f9875ffb640 1 -- 172.21.15.19:0/2559570972 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f98840b2330 con 0x559c2fa6e4d0 2024-05-17T09:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f9875ffb640 1 -- 172.21.15.19:0/2559570972 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f98840a9470 con 0x559c2fa6e4d0 2024-05-17T09:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 10 client.27722 fetch_fsmap learned FSMap version 37 2024-05-17T09:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559c2fe441b0 con 0x559c2fa6e4d0 2024-05-17T09:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f9875ffb640 1 -- 172.21.15.19:0/2559570972 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9888003dd0 con 0x559c2fa6e4d0 2024-05-17T09:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f9875ffb640 1 -- 172.21.15.19:0/2559570972 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f98840a52d0 con 0x559c2fa6e4d0 2024-05-17T09:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 10 client.27722 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 10 client.27722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 20 client.27722 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa4dd80 0x559c2fe479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 4 client.27722 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.247+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c2fda4b80 con 0x559c2fa4dd80 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.248+0000 7f988e59c640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa4dd80 0x559c2fe479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.248+0000 7f988e59c640 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa4dd80 0x559c2fe479b0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97911da492202610 server_cookie=e822214c90ee99b in_seq=0 out_seq=0 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.248+0000 7f9875ffb640 10 client.27722 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f9875ffb640 1 -- 172.21.15.19:0/2559570972 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559c2fda4b80 con 0x559c2fa4dd80 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f9875ffb640 10 client.27722 handle_command_reply: tid=0 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 client.27722 shutdown 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 msgr2=0x559c2fe3a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3a200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f988400c6a0 tx=0x7f98840a50f0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 shutdown_connections 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa4dd80 0x559c2fe479b0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3a200 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 --2- 172.21.15.19:0/2559570972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe39cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 >> 172.21.15.19:0/2559570972 conn(0x559c2fa83300 msgr2=0x559c2fe2ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f9875ffb640 0 client.27722 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 shutdown_connections 2024-05-17T09:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 1 -- 172.21.15.19:0/2559570972 wait complete. 2024-05-17T09:36:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.249+0000 7f989bd81740 20 client.27722 trim_cache size 0 max 16384 2024-05-17T09:36:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 Processor -- start 2024-05-17T09:36:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 -- start start 2024-05-17T09:36:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe2aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe41080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa8f1b0 0x559c2fe415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98901ad110 con 0x559c2fa6e4d0 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98901aa9e0 con 0x559c2fa8f1b0 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f989bd81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a8830 con 0x559c2fa662f0 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe41080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60028/0 (socket says 172.21.15.19:60028) 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f988dd9b640 1 -- 172.21.15.19:0/1108568250 learned_addr learned my addr 172.21.15.19:0/1108568250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f988e59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe2aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.275+0000 7f988ed9d640 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa8f1b0 0x559c2fe415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988e59c640 1 -- 172.21.15.19:0/1108568250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa8f1b0 msgr2=0x559c2fe415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988e59c640 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa8f1b0 0x559c2fe415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988e59c640 1 -- 172.21.15.19:0/1108568250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 msgr2=0x559c2fe41080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988e59c640 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe41080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988e59c640 1 -- 172.21.15.19:0/1108568250 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2fa4dfb0 con 0x559c2fa662f0 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988dd9b640 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe41080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f988e59c640 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe2aa40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f988002a930 tx=0x7f9880041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fc01797199ad185 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f9877fff640 1 -- 172.21.15.19:0/1108568250 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880037040 con 0x559c2fa662f0 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f9877fff640 1 -- 172.21.15.19:0/1108568250 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98800054d0 con 0x559c2fa662f0 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.276+0000 7f9877fff640 1 -- 172.21.15.19:0/1108568250 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880032cd0 con 0x559c2fa662f0 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 -- 172.21.15.19:0/1108568250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 msgr2=0x559c2fe2aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe2aa40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f988002a930 tx=0x7f9880041000 comp rx=0 tx=0).stop 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 -- 172.21.15.19:0/1108568250 shutdown_connections 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa8f1b0 0x559c2fe415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa6e4d0 0x559c2fe41080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 --2- 172.21.15.19:0/1108568250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe2aa40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 -- 172.21.15.19:0/1108568250 >> 172.21.15.19:0/1108568250 conn(0x559c2fa83300 msgr2=0x559c2fe2a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 -- 172.21.15.19:0/1108568250 shutdown_connections 2024-05-17T09:36:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.277+0000 7f989bd81740 1 -- 172.21.15.19:0/1108568250 wait complete. 2024-05-17T09:36:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 -- start start 2024-05-17T09:36:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 Processor -- start 2024-05-17T09:36:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe3eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe3ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a8ea0 con 0x559c2fa8f1b0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98901aa310 con 0x559c2fa6e4d0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98901a9570 con 0x559c2fa662f0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.278+0000 7f989bd81740 10 client.0 _ll_register_callbacks cb 0x7f98840a9380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe3eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe3eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:55394/0 (socket says 172.21.15.19:55394) 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 -- 172.21.15.19:0/296266065 learned_addr learned my addr 172.21.15.19:0/296266065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe3ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 -- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 msgr2=0x559c2fe3f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 -- 172.21.15.19:0/296266065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 msgr2=0x559c2fe3ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:36:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe3ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 -- 172.21.15.19:0/296266065 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2fe404f0 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988dd9b640 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f988e59c640 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe3eb30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9880005cd0 tx=0x7f9880000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523c28160de8334f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f9875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f9875ffb640 1 -- 172.21.15.19:0/296266065 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880066070 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f9875ffb640 1 -- 172.21.15.19:0/296266065 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9880004040 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.279+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98800054d0 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f9875ffb640 1 -- 172.21.15.19:0/296266065 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9880037040 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 10 client.27721 fetch_fsmap learned FSMap version 37 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559c2fe417f0 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f9875ffb640 1 -- 172.21.15.19:0/296266065 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98800054d0 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f9875ffb640 1 -- 172.21.15.19:0/296266065 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f98800035f0 con 0x559c2fa662f0 2024-05-17T09:36:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 10 client.27721 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 10 client.27721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 20 client.27721 populate_metadata read hostname 'smithi019' 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa231c0 0x559c2feb2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 4 client.27721 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.280+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c2fd9e360 con 0x559c2fa231c0 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.281+0000 7f988dd9b640 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa231c0 0x559c2feb2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.281+0000 7f988dd9b640 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa231c0 0x559c2feb2700 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab424cceee7addb2 server_cookie=58ceea008d8f4756 in_seq=0 out_seq=0 2024-05-17T09:36:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.281+0000 7f9875ffb640 10 client.27721 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.376+0000 7f9875ffb640 1 -- 172.21.15.19:0/296266065 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559c2fd9e360 con 0x559c2fa231c0 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.376+0000 7f9875ffb640 10 client.27721 handle_command_reply: tid=0 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.376+0000 7f989bd81740 1 client.27721 shutdown 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 msgr2=0x559c2fe3eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe3eb30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9880005cd0 tx=0x7f9880000bc0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 shutdown_connections 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559c2fa231c0 0x559c2feb2700 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2fa8f1b0 0x559c2fe3ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c2fa6e4d0 0x559c2fe3f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f9875ffb640 0 client.27721 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 --2- 172.21.15.19:0/296266065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c2fa662f0 0x559c2fe3eb30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 >> 172.21.15.19:0/296266065 conn(0x559c2fa83300 msgr2=0x559c2fe2a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 shutdown_connections 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 1 -- 172.21.15.19:0/296266065 wait complete. 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.377+0000 7f989bd81740 20 client.27721 trim_cache size 0 max 16384 2024-05-17T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:36:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:36:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:36:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98780b29a0 msgr2=0x7f98780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98780b29a0 0x7f98780b4eb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9880005e00 tx=0x7f988002bf90 comp rx=0 tx=0).stop 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901034e0 msgr2=0x7f989019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901034e0 0x7f989019a700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9884005fd0 tx=0x7f988400ee50 comp rx=0 tx=0).stop 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 shutdown_connections 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98901081e0 0x7f98901a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98901034e0 0x7f989019a700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f98780b29a0 0x7f98780b4eb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 --2- 172.21.15.19:0/3345523427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98901022a0 0x7f989019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 >> 172.21.15.19:0/3345523427 conn(0x7f98900fd470 msgr2=0x7f98901061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 shutdown_connections 2024-05-17T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:36:51.378+0000 7f9894ed1640 1 -- 172.21.15.19:0/3345523427 wait complete. 2024-05-17T09:36:51.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19319 4.98 MB/sec execute 48 sec latency 130.209 ms 2024-05-17T09:36:52.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19557 5.31 MB/sec execute 49 sec latency 67.968 ms 2024-05-17T09:36:52.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:52 smithi019 ceph-mon[31620]: pgmap v951: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-05-17T09:36:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:52 smithi079 ceph-mon[35502]: pgmap v951: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-05-17T09:36:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:52 smithi069 ceph-mon[35081]: pgmap v951: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-05-17T09:36:53.569 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19753 5.37 MB/sec execute 50 sec latency 65.291 ms 2024-05-17T09:36:54.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19869 5.36 MB/sec execute 51 sec latency 78.625 ms 2024-05-17T09:36:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:54 smithi019 ceph-mon[31620]: pgmap v952: 129 pgs: 129 active+clean; 342 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-05-17T09:36:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:54 smithi079 ceph-mon[35502]: pgmap v952: 129 pgs: 129 active+clean; 342 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-05-17T09:36:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:54 smithi069 ceph-mon[35081]: pgmap v952: 129 pgs: 129 active+clean; 342 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-05-17T09:36:55.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 19984 5.36 MB/sec execute 52 sec latency 86.791 ms 2024-05-17T09:36:56.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 20166 5.61 MB/sec execute 53 sec latency 70.608 ms 2024-05-17T09:36:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:56 smithi079 ceph-mon[35502]: pgmap v953: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-05-17T09:36:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:56 smithi069 ceph-mon[35081]: pgmap v953: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-05-17T09:36:56.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:56 smithi019 ceph-mon[31620]: pgmap v953: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-05-17T09:36:57.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 20324 5.69 MB/sec execute 54 sec latency 68.708 ms 2024-05-17T09:36:58.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 20479 5.80 MB/sec execute 55 sec latency 70.815 ms 2024-05-17T09:36:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:36:58 smithi079 ceph-mon[35502]: pgmap v954: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 280 op/s 2024-05-17T09:36:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:36:58 smithi069 ceph-mon[35081]: pgmap v954: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 280 op/s 2024-05-17T09:36:58.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:36:58 smithi019 ceph-mon[31620]: pgmap v954: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 280 op/s 2024-05-17T09:36:59.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 20609 5.73 MB/sec execute 56 sec latency 68.636 ms 2024-05-17T09:37:00.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 20752 5.66 MB/sec execute 57 sec latency 69.836 ms 2024-05-17T09:37:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:00 smithi079 ceph-mon[35502]: pgmap v955: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-05-17T09:37:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:00 smithi069 ceph-mon[35081]: pgmap v955: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-05-17T09:37:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:00 smithi019 ceph-mon[31620]: pgmap v955: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-05-17T09:37:01.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 20910 5.64 MB/sec execute 58 sec latency 85.433 ms 2024-05-17T09:37:02.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 21050 5.60 MB/sec execute 59 sec latency 104.904 ms 2024-05-17T09:37:02.731 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:02 smithi069 ceph-mon[35081]: pgmap v956: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T09:37:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:02 smithi079 ceph-mon[35502]: pgmap v956: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T09:37:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:02 smithi019 ceph-mon[31620]: pgmap v956: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 240 op/s 2024-05-17T09:37:03.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 21198 5.62 MB/sec execute 60 sec latency 85.580 ms 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 -- 172.21.15.19:0/457579275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 msgr2=0x7f2f8010bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 --2- 172.21.15.19:0/457579275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 0x7f2f8010bde0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009be0 tx=0x7f2f7802a6c0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 -- 172.21.15.19:0/457579275 shutdown_connections 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 --2- 172.21.15.19:0/457579275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 0x7f2f8010bde0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 --2- 172.21.15.19:0/457579275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 0x7f2f80109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 --2- 172.21.15.19:0/457579275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f801067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 -- 172.21.15.19:0/457579275 >> 172.21.15.19:0/457579275 conn(0x7f2f800fdd30 msgr2=0x7f2f801001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 -- 172.21.15.19:0/457579275 shutdown_connections 2024-05-17T09:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.520+0000 7f2f84a92640 1 -- 172.21.15.19:0/457579275 wait complete. 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 Processor -- start 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 -- start start 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f80197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 0x7f2f80198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 0x7f2f8019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f80197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010d3f0 con 0x7f2f801043b0 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010d270 con 0x7f2f80106da0 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f80197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47676/0 (socket says 172.21.15.19:47676) 2024-05-17T09:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f84a92640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010d570 con 0x7f2f801098d0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f7e59c640 1 -- 172.21.15.19:0/2158079493 learned_addr learned my addr 172.21.15.19:0/2158079493 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f77fff640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 0x7f2f80198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.521+0000 7f2f7ed9d640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 0x7f2f8019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f77fff640 1 -- 172.21.15.19:0/2158079493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 msgr2=0x7f2f8019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f77fff640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 0x7f2f8019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f77fff640 1 -- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 msgr2=0x7f2f80197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f77fff640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f80197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f77fff640 1 -- 172.21.15.19:0/2158079493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8019fa80 con 0x7f2f80106da0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f80197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f77fff640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 0x7f2f80198480 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2f68013ef0 tx=0x7f2f68007590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f777fe640 1 -- 172.21.15.19:0/2158079493 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68005020 con 0x7f2f80106da0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8019fd70 con 0x7f2f80106da0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f777fe640 1 -- 172.21.15.19:0/2158079493 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f68005650 con 0x7f2f80106da0 2024-05-17T09:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f777fe640 1 -- 172.21.15.19:0/2158079493 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6801d650 con 0x7f2f80106da0 2024-05-17T09:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.522+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f801a0230 con 0x7f2f80106da0 2024-05-17T09:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 Processor -- start 2024-05-17T09:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 -- start start 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2007eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f777fe640 1 -- 172.21.15.19:0/2158079493 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2f68024020 con 0x7f2f80106da0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a20075dc30 con 0x55a20044a1b0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a20074d7d0 con 0x55a2004212f0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a200751810 con 0x55a2004294d0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007eb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34702/0 (socket says 172.21.15.19:34702) 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f7ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2007eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.526+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3799262310 learned_addr learned my addr 172.21.15.19:0/3799262310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f77fff640 1 -- 172.21.15.19:0/3799262310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 msgr2=0x55a2007eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f77fff640 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007eb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f77fff640 1 -- 172.21.15.19:0/3799262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 msgr2=0x55a2007eb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f77fff640 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2007eb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f77fff640 1 -- 172.21.15.19:0/3799262310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a200462210 con 0x55a2004294d0 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f77fff640 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f2820 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f680e78f0 tx=0x7f2f680f2630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42922e690c02703c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.527+0000 7f2f74ff9640 1 -- 172.21.15.19:0/3799262310 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680fd070 con 0x55a2004294d0 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.528+0000 7f2f74ff9640 1 -- 172.21.15.19:0/3799262310 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f680f2e30 con 0x55a2004294d0 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.528+0000 7f2f74ff9640 1 -- 172.21.15.19:0/3799262310 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680f8660 con 0x55a2004294d0 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.528+0000 7f2f777fe640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f6c0b2920 0x7f2f6c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.528+0000 7f2f7ed9d640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f6c0b2920 0x7f2f6c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.528+0000 7f2f777fe640 1 -- 172.21.15.19:0/2158079493 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2f680ea260 con 0x7f2f80106da0 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3799262310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 msgr2=0x55a2007f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f7ed9d640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f6c0b2920 0x7f2f6c0b4e30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2f78030800 tx=0x7f2f78030860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f2820 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f680e78f0 tx=0x7f2f680f2630 comp rx=0 tx=0).stop 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3799262310 shutdown_connections 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f2820 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3799262310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2007eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3799262310 >> 172.21.15.19:0/3799262310 conn(0x55a20043e300 msgr2=0x55a2007e6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3799262310 shutdown_connections 2024-05-17T09:37:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.529+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3799262310 wait complete. 2024-05-17T09:37:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 -- start start 2024-05-17T09:37:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 Processor -- start 2024-05-17T09:37:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004294d0 0x55a2007f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a20044a1b0 0x55a2007fc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a200433790 con 0x55a2004294d0 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a200709890 con 0x55a2004212f0 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a2007f3270 con 0x55a20044a1b0 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f8ba1d740 10 client.0 _ll_register_callbacks cb 0x55a20070cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004294d0 0x55a2007f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34716/0 (socket says 172.21.15.19:34716) 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f7ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a20044a1b0 0x55a2007fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.530+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3257553114 learned_addr learned my addr 172.21.15.19:0/3257553114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3257553114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a20044a1b0 msgr2=0x55a2007fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a20044a1b0 0x55a2007fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3257553114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004294d0 msgr2=0x55a2007f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004294d0 0x55a2007f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3257553114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a20052e620 con 0x55a2004212f0 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7ed9d640 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a20044a1b0 0x55a2007fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f70004620 tx=0x7f2f70009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a704759d8f7c5089 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f65ffb640 1 -- 172.21.15.19:0/3257553114 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7001e030 con 0x55a2004212f0 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f65ffb640 1 -- 172.21.15.19:0/3257553114 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2f7000d840 con 0x55a2004212f0 2024-05-17T09:37:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f680f2e30 con 0x55a2004212f0 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f65ffb640 1 -- 172.21.15.19:0/3257553114 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f70028050 con 0x55a2004212f0 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f8ba1d740 10 client.27740 fetch_fsmap learned FSMap version 37 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.531+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a2007ff160 con 0x55a2004212f0 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f65ffb640 1 -- 172.21.15.19:0/3257553114 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f680f2e30 con 0x55a2004212f0 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f65ffb640 1 -- 172.21.15.19:0/3257553114 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2f7001ab80 con 0x55a2004212f0 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f8ba1d740 10 client.27740 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f8ba1d740 10 client.27740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f8ba1d740 20 client.27740 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a200408d80 0x55a200802a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f8ba1d740 4 client.27740 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a20075fb80 con 0x55a200408d80 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.532+0000 7f2f77fff640 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a200408d80 0x55a200802a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.533+0000 7f2f65ffb640 10 client.27740 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.533+0000 7f2f77fff640 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a200408d80 0x55a200802a20 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d594270d0cd25c53 server_cookie=4782295dc6cc1d79 in_seq=0 out_seq=0 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f65ffb640 1 -- 172.21.15.19:0/3257553114 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a20075fb80 con 0x55a200408d80 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f65ffb640 10 client.27740 handle_command_reply: tid=0 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f8ba1d740 1 client.27740 shutdown 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 msgr2=0x55a2007f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f70004620 tx=0x7f2f70009b70 comp rx=0 tx=0).stop 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 shutdown_connections 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a200408d80 0x55a200802a20 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.534+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a20044a1b0 0x55a2007fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004294d0 0x55a2007f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3257553114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 >> 172.21.15.19:0/3257553114 conn(0x55a20043e300 msgr2=0x55a2007e5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f65ffb640 0 client.27740 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 shutdown_connections 2024-05-17T09:37:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3257553114 wait complete. 2024-05-17T09:37:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.535+0000 7f2f8ba1d740 20 client.27740 trim_cache size 0 max 16384 2024-05-17T09:37:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 Processor -- start 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 -- start start 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 0x55a2007e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a20044a1b0 0x55a2007ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801ad140 con 0x55a2004212f0 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801aaa10 con 0x55a20044a1b0 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f8ba1d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801a8860 con 0x55a2004294d0 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 0x55a2007e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 0x55a2007e5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47710/0 (socket says 172.21.15.19:47710) 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3779986030 learned_addr learned my addr 172.21.15.19:0/3779986030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.559+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7ed9d640 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a20044a1b0 0x55a2007ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3779986030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 msgr2=0x55a2007fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3779986030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a20044a1b0 msgr2=0x55a2007ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a20044a1b0 0x55a2007ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7e59c640 1 -- 172.21.15.19:0/3779986030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a200408fb0 con 0x55a2004212f0 2024-05-17T09:37:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 0x55a2007e5a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2f70011010 tx=0x7f2f70009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18025894a402345a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f67fff640 1 -- 172.21.15.19:0/3779986030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70032020 con 0x55a2004212f0 2024-05-17T09:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f67fff640 1 -- 172.21.15.19:0/3779986030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f7000dcd0 con 0x55a2004212f0 2024-05-17T09:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.560+0000 7f2f67fff640 1 -- 172.21.15.19:0/3779986030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7001e040 con 0x55a2004212f0 2024-05-17T09:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.561+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3779986030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 msgr2=0x55a2007e5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.561+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 0x55a2007e5a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2f70011010 tx=0x7f2f70009670 comp rx=0 tx=0).stop 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3779986030 shutdown_connections 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a20044a1b0 0x55a2007ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/3779986030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2004212f0 0x55a2007e5a40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3779986030 >> 172.21.15.19:0/3779986030 conn(0x55a20043e300 msgr2=0x55a2007e5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3779986030 shutdown_connections 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/3779986030 wait complete. 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 -- start start 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.562+0000 7f2f8ba1d740 1 Processor -- start 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f8ba1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2008690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f7ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2008690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:36832/0 (socket says 172.21.15.19:36832) 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f77fff640 1 -- 172.21.15.19:0/1769652313 learned_addr learned my addr 172.21.15.19:0/1769652313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801a8ed0 con 0x55a20044a1b0 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801aa340 con 0x55a2004212f0 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801a95a0 con 0x55a2004294d0 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f77fff640 1 -- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 msgr2=0x55a2007f9080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f77fff640 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f9080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.563+0000 7f2f77fff640 1 -- 172.21.15.19:0/1769652313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 msgr2=0x55a2008690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f77fff640 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2008690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f8ba1d740 10 client.0 _ll_register_callbacks cb 0x7f2f7000f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f77fff640 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2008695e0 con 0x55a2004294d0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f7ed9d640 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2008690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f7000dcd0 con 0x55a2004294d0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f77fff640 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f95c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009fd0 tx=0x7f2f68013e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc1dd80f46338332 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f65ffb640 10 client.27739 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f65ffb640 1 -- 172.21.15.19:0/1769652313 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680f8660 con 0x55a2004294d0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.564+0000 7f2f65ffb640 1 -- 172.21.15.19:0/1769652313 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2f680ada60 con 0x55a2004294d0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f65ffb640 1 -- 172.21.15.19:0/1769652313 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f680ad330 con 0x55a2004294d0 2024-05-17T09:37:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 10 client.27739 fetch_fsmap learned FSMap version 37 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55a20086a670 con 0x55a2004294d0 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f65ffb640 1 -- 172.21.15.19:0/1769652313 <== mon.1 v2:172.21.15.79:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2f680e79d0 con 0x55a2004294d0 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 10 client.27739 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 10 client.27739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 20 client.27739 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a2003de1c0 0x55a20086cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 4 client.27739 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.565+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a200759360 con 0x55a2003de1c0 2024-05-17T09:37:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.566+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a2003de1c0 0x55a20086cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.567+0000 7f2f7e59c640 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a2003de1c0 0x55a20086cba0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22843eef266f24ca server_cookie=aa6df79d8981e809 in_seq=0 out_seq=0 2024-05-17T09:37:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.567+0000 7f2f65ffb640 10 client.27739 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 21374 5.79 MB/sec execute 61 sec latency 72.804 ms 2024-05-17T09:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f65ffb640 1 -- 172.21.15.19:0/1769652313 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a200759360 con 0x55a2003de1c0 2024-05-17T09:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f65ffb640 10 client.27739 handle_command_reply: tid=0 2024-05-17T09:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f8ba1d740 1 client.27739 shutdown 2024-05-17T09:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 msgr2=0x55a2007f95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f95c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009fd0 tx=0x7f2f68013e00 comp rx=0 tx=0).stop 2024-05-17T09:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 shutdown_connections 2024-05-17T09:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.652+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55a2003de1c0 0x55a20086cba0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f65ffb640 0 client.27739 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a20044a1b0 0x55a2008690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a2004294d0 0x55a2007f95c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 1 --2- 172.21.15.19:0/1769652313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2004212f0 0x55a2007f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 >> 172.21.15.19:0/1769652313 conn(0x55a20043e300 msgr2=0x55a2007f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 shutdown_connections 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 1 -- 172.21.15.19:0/1769652313 wait complete. 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.653+0000 7f2f8ba1d740 20 client.27739 trim_cache size 0 max 16384 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.654+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f6c0b2920 msgr2=0x7f2f6c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.654+0000 7f2f84a92640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f6c0b2920 0x7f2f6c0b4e30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2f78030800 tx=0x7f2f78030860 comp rx=0 tx=0).stop 2024-05-17T09:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.654+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 msgr2=0x7f2f80198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.654+0000 7f2f84a92640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 0x7f2f80198480 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2f68013ef0 tx=0x7f2f68007590 comp rx=0 tx=0).stop 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 shutdown_connections 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f801098d0 0x7f2f8019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f80106da0 0x7f2f80198480 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2f6c0b2920 0x7f2f6c0b4e30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 --2- 172.21.15.19:0/2158079493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f801043b0 0x7f2f80197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 >> 172.21.15.19:0/2158079493 conn(0x7f2f800fdd30 msgr2=0x7f2f80105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 shutdown_connections 2024-05-17T09:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:04.655+0000 7f2f84a92640 1 -- 172.21.15.19:0/2158079493 wait complete. 2024-05-17T09:37:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:04 smithi079 ceph-mon[35502]: pgmap v957: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 291 op/s 2024-05-17T09:37:04.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:04 smithi019 ceph-mon[31620]: pgmap v957: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 291 op/s 2024-05-17T09:37:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:04 smithi069 ceph-mon[35081]: pgmap v957: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 291 op/s 2024-05-17T09:37:05.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 21541 5.89 MB/sec execute 62 sec latency 110.897 ms 2024-05-17T09:37:06.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 21716 5.95 MB/sec execute 63 sec latency 102.267 ms 2024-05-17T09:37:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:06 smithi079 ceph-mon[35502]: pgmap v958: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-05-17T09:37:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:06 smithi069 ceph-mon[35081]: pgmap v958: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-05-17T09:37:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:06 smithi019 ceph-mon[31620]: pgmap v958: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-05-17T09:37:07.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 21858 5.90 MB/sec execute 64 sec latency 135.439 ms 2024-05-17T09:37:08.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 22029 5.95 MB/sec execute 65 sec latency 67.624 ms 2024-05-17T09:37:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:08 smithi079 ceph-mon[35502]: pgmap v959: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 261 op/s 2024-05-17T09:37:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:08 smithi069 ceph-mon[35081]: pgmap v959: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 261 op/s 2024-05-17T09:37:08.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:08 smithi019 ceph-mon[31620]: pgmap v959: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 261 op/s 2024-05-17T09:37:09.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 22208 5.98 MB/sec execute 66 sec latency 75.155 ms 2024-05-17T09:37:10.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 22361 6.00 MB/sec execute 67 sec latency 74.650 ms 2024-05-17T09:37:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:10 smithi079 ceph-mon[35502]: pgmap v960: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-05-17T09:37:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:10 smithi069 ceph-mon[35081]: pgmap v960: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-05-17T09:37:10.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:10 smithi019 ceph-mon[31620]: pgmap v960: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 299 op/s 2024-05-17T09:37:11.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 22484 5.99 MB/sec execute 68 sec latency 206.259 ms 2024-05-17T09:37:12.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 22666 6.11 MB/sec execute 69 sec latency 74.070 ms 2024-05-17T09:37:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:12 smithi079 ceph-mon[35502]: pgmap v961: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-05-17T09:37:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:12 smithi069 ceph-mon[35081]: pgmap v961: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-05-17T09:37:12.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:12 smithi019 ceph-mon[31620]: pgmap v961: 129 pgs: 129 active+clean; 344 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.3 MiB/s wr, 244 op/s 2024-05-17T09:37:13.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 22881 6.23 MB/sec execute 70 sec latency 68.602 ms 2024-05-17T09:37:14.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23070 6.29 MB/sec execute 71 sec latency 70.666 ms 2024-05-17T09:37:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:14 smithi079 ceph-mon[35502]: pgmap v962: 129 pgs: 129 active+clean; 350 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 284 op/s 2024-05-17T09:37:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:14 smithi069 ceph-mon[35081]: pgmap v962: 129 pgs: 129 active+clean; 350 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 284 op/s 2024-05-17T09:37:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:14 smithi019 ceph-mon[31620]: pgmap v962: 129 pgs: 129 active+clean; 350 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 284 op/s 2024-05-17T09:37:15.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23205 6.28 MB/sec execute 72 sec latency 67.264 ms 2024-05-17T09:37:16.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23333 6.33 MB/sec execute 73 sec latency 88.194 ms 2024-05-17T09:37:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:16 smithi079 ceph-mon[35502]: pgmap v963: 129 pgs: 129 active+clean; 355 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-05-17T09:37:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:16 smithi069 ceph-mon[35081]: pgmap v963: 129 pgs: 129 active+clean; 355 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-05-17T09:37:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:16 smithi019 ceph-mon[31620]: pgmap v963: 129 pgs: 129 active+clean; 355 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-05-17T09:37:17.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23519 6.48 MB/sec execute 74 sec latency 120.929 ms 2024-05-17T09:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 -- 172.21.15.19:0/3176463939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924104e80 msgr2=0x7fb924105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3176463939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924104e80 0x7fb924105290 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb91802a410 comp rx=0 tx=0).stop 2024-05-17T09:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 -- 172.21.15.19:0/3176463939 shutdown_connections 2024-05-17T09:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3176463939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924109b80 0x7fb92410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3176463939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924104e80 0x7fb924105290 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3176463939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb924103c40 0x7fb924104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 -- 172.21.15.19:0/3176463939 >> 172.21.15.19:0/3176463939 conn(0x7fb9240fee50 msgr2=0x7fb9241012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 -- 172.21.15.19:0/3176463939 shutdown_connections 2024-05-17T09:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.802+0000 7fb929d7d640 1 -- 172.21.15.19:0/3176463939 wait complete. 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.803+0000 7fb929d7d640 1 Processor -- start 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.803+0000 7fb929d7d640 1 -- start start 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.803+0000 7fb929d7d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 0x7fb92419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.803+0000 7fb929d7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb924104e80 0x7fb92419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.803+0000 7fb929d7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924109b80 0x7fb9241a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb929d7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410f5f0 con 0x7fb924109b80 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb929d7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410f470 con 0x7fb924104e80 2024-05-17T09:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb929d7d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410f770 con 0x7fb924103c40 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 0x7fb92419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 0x7fb92419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47118/0 (socket says 172.21.15.19:47118) 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 -- 172.21.15.19:0/3870159401 learned_addr learned my addr 172.21.15.19:0/3870159401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb923fff640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924109b80 0x7fb9241a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb924104e80 0x7fb92419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 -- 172.21.15.19:0/3870159401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb924104e80 msgr2=0x7fb92419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb924104e80 0x7fb92419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 -- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924109b80 msgr2=0x7fb9241a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924109b80 0x7fb9241a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 -- 172.21.15.19:0/3870159401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9241a3fa0 con 0x7fb924103c40 2024-05-17T09:37:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.804+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 0x7fb92419c460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb91400e9f0 tx=0x7fb91400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.805+0000 7fb920ff9640 1 -- 172.21.15.19:0/3870159401 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400ccc0 con 0x7fb924103c40 2024-05-17T09:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.805+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241a4290 con 0x7fb924103c40 2024-05-17T09:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.805+0000 7fb920ff9640 1 -- 172.21.15.19:0/3870159401 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9140044b0 con 0x7fb924103c40 2024-05-17T09:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.805+0000 7fb920ff9640 1 -- 172.21.15.19:0/3870159401 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914010430 con 0x7fb924103c40 2024-05-17T09:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.805+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a4830 con 0x7fb924103c40 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.808+0000 7fb930c33740 1 Processor -- start 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.808+0000 7fb930c33740 1 -- start start 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb930c33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 0x55b85b2db3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb920ff9640 1 -- 172.21.15.19:0/3870159401 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb914003f90 con 0x7fb924103c40 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb930c33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af112f0 0x55b85b2db920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb930c33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af194d0 0x55b85b2e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb930c33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b85b24dc30 con 0x55b85af3a1b0 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb930c33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b85b23d7d0 con 0x55b85af112f0 2024-05-17T09:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb930c33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b85b241810 con 0x55b85af194d0 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af112f0 0x55b85b2db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb923fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 0x55b85b2db3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af194d0 0x55b85b2e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb923fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 0x55b85b2db3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35668/0 (socket says 172.21.15.19:35668) 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb923fff640 1 -- 172.21.15.19:0/3763134286 learned_addr learned my addr 172.21.15.19:0/3763134286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb922ffd640 1 -- 172.21.15.19:0/3763134286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af112f0 msgr2=0x55b85b2db920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af112f0 0x55b85b2db920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb922ffd640 1 -- 172.21.15.19:0/3763134286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 msgr2=0x55b85b2db3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 0x55b85b2db3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.809+0000 7fb922ffd640 1 -- 172.21.15.19:0/3763134286 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b85af52210 con 0x55b85af194d0 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb923fff640 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 0x55b85b2db3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af112f0 0x55b85b2db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af194d0 0x55b85b2e2820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb918005ab0 tx=0x7fb918009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=600fa3d7455e0bed server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb909ffb640 1 -- 172.21.15.19:0/3763134286 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918035030 con 0x55b85af194d0 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb909ffb640 1 -- 172.21.15.19:0/3763134286 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb918040070 con 0x55b85af194d0 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb920ff9640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb90c0b2920 0x7fb90c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb909ffb640 1 -- 172.21.15.19:0/3763134286 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918003850 con 0x55b85af194d0 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.810+0000 7fb923fff640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb90c0b2920 0x7fb90c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb920ff9640 1 -- 172.21.15.19:0/3870159401 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb9140e2820 con 0x7fb924103c40 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 -- 172.21.15.19:0/3763134286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af194d0 msgr2=0x55b85b2e2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb923fff640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb90c0b2920 0x7fb90c0b4e30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55b85b2dc570 tx=0x7fb91c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af194d0 0x55b85b2e2820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb918005ab0 tx=0x7fb918009620 comp rx=0 tx=0).stop 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 -- 172.21.15.19:0/3763134286 shutdown_connections 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af194d0 0x55b85b2e2820 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af112f0 0x55b85b2db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 --2- 172.21.15.19:0/3763134286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af3a1b0 0x55b85b2db3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 -- 172.21.15.19:0/3763134286 >> 172.21.15.19:0/3763134286 conn(0x55b85af2e300 msgr2=0x55b85b2d6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 -- 172.21.15.19:0/3763134286 shutdown_connections 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.811+0000 7fb930c33740 1 -- 172.21.15.19:0/3763134286 wait complete. 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 -- start start 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 Processor -- start 2024-05-17T09:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b85af23790 con 0x55b85af112f0 2024-05-17T09:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b85b1f9890 con 0x55b85af194d0 2024-05-17T09:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b85b2e3210 con 0x55b85af3a1b0 2024-05-17T09:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb930c33740 10 client.0 _ll_register_callbacks cb 0x55b85b1fcd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb923fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33206/0 (socket says 172.21.15.19:33206) 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb922ffd640 1 -- 172.21.15.19:0/745110386 learned_addr learned my addr 172.21.15.19:0/745110386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb923fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47122/0 (socket says 172.21.15.19:47122) 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.812+0000 7fb9237fe640 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb922ffd640 1 -- 172.21.15.19:0/745110386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 msgr2=0x55b85b2ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb922ffd640 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb922ffd640 1 -- 172.21.15.19:0/745110386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 msgr2=0x55b85b2e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb922ffd640 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb922ffd640 1 -- 172.21.15.19:0/745110386 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b85b01e620 con 0x55b85af194d0 2024-05-17T09:37:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb922ffd640 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e5190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb918000c00 tx=0x7fb918004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e0c9af462f7146 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb902ffd640 1 -- 172.21.15.19:0/745110386 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918035070 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918040070 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.813+0000 7fb902ffd640 1 -- 172.21.15.19:0/745110386 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb91803c440 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb902ffd640 1 -- 172.21.15.19:0/745110386 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb918056050 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 10 client.27755 fetch_fsmap learned FSMap version 37 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b85b2ef020 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb902ffd640 1 -- 172.21.15.19:0/745110386 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb918040070 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb902ffd640 1 -- 172.21.15.19:0/745110386 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb918044680 con 0x55b85af194d0 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 10 client.27755 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 10 client.27755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 20 client.27755 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aef8d80 0x55b85b2f2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 4 client.27755 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:37:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b85b24fb80 con 0x55b85aef8d80 2024-05-17T09:37:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.814+0000 7fb9237fe640 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aef8d80 0x55b85b2f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.815+0000 7fb902ffd640 10 client.27755 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.815+0000 7fb9237fe640 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aef8d80 0x55b85b2f2380 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a55925aa6b4153 server_cookie=2ed0f2faf4bd60e0 in_seq=0 out_seq=0 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb902ffd640 1 -- 172.21.15.19:0/745110386 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b85b24fb80 con 0x55b85aef8d80 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb902ffd640 10 client.27755 handle_command_reply: tid=0 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 client.27755 shutdown 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 msgr2=0x55b85b2e5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e5190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb918000c00 tx=0x7fb918004590 comp rx=0 tx=0).stop 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 shutdown_connections 2024-05-17T09:37:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aef8d80 0x55b85b2f2380 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e5190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 --2- 172.21.15.19:0/745110386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 >> 172.21.15.19:0/745110386 conn(0x55b85af2e300 msgr2=0x55b85b2d5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb902ffd640 0 client.27755 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 shutdown_connections 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 1 -- 172.21.15.19:0/745110386 wait complete. 2024-05-17T09:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.818+0000 7fb930c33740 20 client.27755 trim_cache size 0 max 16384 2024-05-17T09:37:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.848+0000 7fb930c33740 1 Processor -- start 2024-05-17T09:37:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.848+0000 7fb930c33740 1 -- start start 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb930c33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb930c33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2ec010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb930c33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb930c33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241b15c0 con 0x55b85af112f0 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb930c33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241aee90 con 0x55b85af194d0 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb930c33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241acce0 con 0x55b85af3a1b0 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2ec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2d5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35686/0 (socket says 172.21.15.19:35686) 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2ec010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33212/0 (socket says 172.21.15.19:33212) 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb9237fe640 1 -- 172.21.15.19:0/3807885654 learned_addr learned my addr 172.21.15.19:0/3807885654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb923fff640 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 -- 172.21.15.19:0/3807885654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 msgr2=0x55b85b2ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 -- 172.21.15.19:0/3807885654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 msgr2=0x55b85b2d5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2d5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb922ffd640 1 -- 172.21.15.19:0/3807885654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b85b2e3320 con 0x55b85af194d0 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.849+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.850+0000 7fb922ffd640 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2ec010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb918009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d7ae1a4673ad5e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.850+0000 7fb9097fa640 1 -- 172.21.15.19:0/3807885654 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918035040 con 0x55b85af194d0 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.850+0000 7fb9097fa640 1 -- 172.21.15.19:0/3807885654 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb918040070 con 0x55b85af194d0 2024-05-17T09:37:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.850+0000 7fb9097fa640 1 -- 172.21.15.19:0/3807885654 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918052b70 con 0x55b85af194d0 2024-05-17T09:37:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.851+0000 7fb930c33740 1 -- 172.21.15.19:0/3807885654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 msgr2=0x55b85b2ec010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.851+0000 7fb930c33740 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2ec010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb918009620 comp rx=0 tx=0).stop 2024-05-17T09:37:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 -- 172.21.15.19:0/3807885654 shutdown_connections 2024-05-17T09:37:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b2ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2ec010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 --2- 172.21.15.19:0/3807885654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 -- 172.21.15.19:0/3807885654 >> 172.21.15.19:0/3807885654 conn(0x55b85af2e300 msgr2=0x55b85b2d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 -- 172.21.15.19:0/3807885654 shutdown_connections 2024-05-17T09:37:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 -- 172.21.15.19:0/3807885654 wait complete. 2024-05-17T09:37:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.852+0000 7fb930c33740 1 -- start start 2024-05-17T09:37:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 Processor -- start 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b3589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ad350 con 0x55b85af112f0 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ae7c0 con 0x55b85af194d0 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ada20 con 0x55b85af3a1b0 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb930c33740 10 client.0 _ll_register_callbacks cb 0x7fb91802ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.853+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35694/0 (socket says 172.21.15.19:35694) 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb9237fe640 1 -- 172.21.15.19:0/3017753445 learned_addr learned my addr 172.21.15.19:0/3017753445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b3589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 -- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 msgr2=0x55b85b2e9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 -- 172.21.15.19:0/3017753445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 msgr2=0x55b85b2e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 -- 172.21.15.19:0/3017753445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b85b2ecaf0 con 0x55b85af3a1b0 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb923fff640 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b3589e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb91c005fd0 tx=0x7fb91c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4198c95225307196 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.854+0000 7fb902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb902ffd640 1 -- 172.21.15.19:0/3017753445 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c02e060 con 0x55b85af3a1b0 2024-05-17T09:37:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918040070 con 0x55b85af3a1b0 2024-05-17T09:37:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb902ffd640 1 -- 172.21.15.19:0/3017753445 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb91c004590 con 0x55b85af3a1b0 2024-05-17T09:37:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb902ffd640 1 -- 172.21.15.19:0/3017753445 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb91c004790 con 0x55b85af3a1b0 2024-05-17T09:37:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb930c33740 10 client.27757 fetch_fsmap learned FSMap version 37 2024-05-17T09:37:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b85b359fa0 con 0x55b85af3a1b0 2024-05-17T09:37:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.855+0000 7fb902ffd640 1 -- 172.21.15.19:0/3017753445 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb918040070 con 0x55b85af3a1b0 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb902ffd640 1 -- 172.21.15.19:0/3017753445 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb91c01d420 con 0x55b85af3a1b0 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb930c33740 10 client.27757 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb930c33740 10 client.27757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb930c33740 20 client.27757 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb930c33740 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aece1c0 0x55b85b35c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb930c33740 4 client.27757 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b85b249360 con 0x55b85aece1c0 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.856+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aece1c0 0x55b85b35c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.857+0000 7fb902ffd640 10 client.27757 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.857+0000 7fb9237fe640 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aece1c0 0x55b85b35c9d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3c5cea2301fa99 server_cookie=67e32bc9c714778d in_seq=0 out_seq=0 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.934+0000 7fb902ffd640 1 -- 172.21.15.19:0/3017753445 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b85b249360 con 0x55b85aece1c0 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.934+0000 7fb902ffd640 10 client.27757 handle_command_reply: tid=0 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.934+0000 7fb930c33740 1 client.27757 shutdown 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.934+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 msgr2=0x55b85b3589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.934+0000 7fb930c33740 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b3589e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb91c005fd0 tx=0x7fb91c004a40 comp rx=0 tx=0).stop 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 shutdown_connections 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b85aece1c0 0x55b85b35c9d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b85af3a1b0 0x55b85b3589e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b85af194d0 0x55b85b2e9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb902ffd640 0 client.27757 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 --2- 172.21.15.19:0/3017753445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b85af112f0 0x55b85b2e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 >> 172.21.15.19:0/3017753445 conn(0x55b85af2e300 msgr2=0x55b85b2d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 shutdown_connections 2024-05-17T09:37:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 1 -- 172.21.15.19:0/3017753445 wait complete. 2024-05-17T09:37:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb930c33740 20 client.27757 trim_cache size 0 max 16384 2024-05-17T09:37:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:37:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:37:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.935+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb90c0b2920 msgr2=0x7fb90c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb90c0b2920 0x7fb90c0b4e30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55b85b2dc570 tx=0x7fb91c009500 comp rx=0 tx=0).stop 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 msgr2=0x7fb92419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 0x7fb92419c460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb91400e9f0 tx=0x7fb91400eeb0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 shutdown_connections 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb90c0b2920 0x7fb90c0b4e30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb924109b80 0x7fb9241a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb924104e80 0x7fb92419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 --2- 172.21.15.19:0/3870159401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb924103c40 0x7fb92419c460 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 >> 172.21.15.19:0/3870159401 conn(0x7fb9240fee50 msgr2=0x7fb9240ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 shutdown_connections 2024-05-17T09:37:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:17.936+0000 7fb929d7d640 1 -- 172.21.15.19:0/3870159401 wait complete. 2024-05-17T09:37:18.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23690 6.56 MB/sec execute 75 sec latency 111.608 ms 2024-05-17T09:37:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:18 smithi079 ceph-mon[35502]: pgmap v964: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 268 op/s 2024-05-17T09:37:18.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:18 smithi069 ceph-mon[35081]: pgmap v964: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 268 op/s 2024-05-17T09:37:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:18 smithi019 ceph-mon[31620]: pgmap v964: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 268 op/s 2024-05-17T09:37:19.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23838 6.52 MB/sec execute 76 sec latency 57.546 ms 2024-05-17T09:37:20.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 23972 6.47 MB/sec execute 77 sec latency 74.984 ms 2024-05-17T09:37:20.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:20 smithi079 ceph-mon[35502]: pgmap v965: 129 pgs: 129 active+clean; 361 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 307 op/s 2024-05-17T09:37:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:20 smithi069 ceph-mon[35081]: pgmap v965: 129 pgs: 129 active+clean; 361 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 307 op/s 2024-05-17T09:37:20.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:20 smithi019 ceph-mon[31620]: pgmap v965: 129 pgs: 129 active+clean; 361 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 307 op/s 2024-05-17T09:37:21.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 24103 6.44 MB/sec execute 78 sec latency 101.316 ms 2024-05-17T09:37:22.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 24254 6.38 MB/sec execute 79 sec latency 88.930 ms 2024-05-17T09:37:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:22 smithi079 ceph-mon[35502]: pgmap v966: 129 pgs: 129 active+clean; 360 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-05-17T09:37:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:22 smithi069 ceph-mon[35081]: pgmap v966: 129 pgs: 129 active+clean; 360 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-05-17T09:37:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:22 smithi019 ceph-mon[31620]: pgmap v966: 129 pgs: 129 active+clean; 360 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-05-17T09:37:23.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 24427 6.40 MB/sec execute 80 sec latency 91.052 ms 2024-05-17T09:37:24.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 24570 6.42 MB/sec execute 81 sec latency 92.350 ms 2024-05-17T09:37:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:24 smithi079 ceph-mon[35502]: pgmap v967: 129 pgs: 129 active+clean; 359 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-17T09:37:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:24 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:37:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:24 smithi069 ceph-mon[35081]: pgmap v967: 129 pgs: 129 active+clean; 359 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-17T09:37:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:24 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:37:24.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:24 smithi019 ceph-mon[31620]: pgmap v967: 129 pgs: 129 active+clean; 359 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-17T09:37:24.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:24 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:37:25.570 INFO:tasks.workunit.client.0.smithi019.stdout: 10 24708 6.43 MB/sec execute 82 sec latency 127.850 ms 2024-05-17T09:37:26.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 24881 6.50 MB/sec execute 83 sec latency 71.576 ms 2024-05-17T09:37:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:26 smithi079 ceph-mon[35502]: pgmap v968: 129 pgs: 129 active+clean; 362 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2024-05-17T09:37:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:26 smithi069 ceph-mon[35081]: pgmap v968: 129 pgs: 129 active+clean; 362 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2024-05-17T09:37:26.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:26 smithi019 ceph-mon[31620]: pgmap v968: 129 pgs: 129 active+clean; 362 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2024-05-17T09:37:27.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 25095 6.63 MB/sec execute 84 sec latency 64.121 ms 2024-05-17T09:37:28.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 25265 6.59 MB/sec execute 85 sec latency 69.620 ms 2024-05-17T09:37:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:28 smithi079 ceph-mon[35502]: pgmap v969: 129 pgs: 129 active+clean; 356 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-05-17T09:37:28.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:28 smithi069 ceph-mon[35081]: pgmap v969: 129 pgs: 129 active+clean; 356 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-05-17T09:37:28.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:28 smithi019 ceph-mon[31620]: pgmap v969: 129 pgs: 129 active+clean; 356 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-05-17T09:37:29.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 25410 6.57 MB/sec execute 86 sec latency 69.298 ms 2024-05-17T09:37:29.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:29 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:37:29.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:29 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:37:30.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:29 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:37:30.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 25566 6.58 MB/sec execute 87 sec latency 77.641 ms 2024-05-17T09:37:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:30 smithi069 ceph-mon[35081]: pgmap v970: 129 pgs: 129 active+clean; 359 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-05-17T09:37:30.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:30 smithi019 ceph-mon[31620]: pgmap v970: 129 pgs: 129 active+clean; 359 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-05-17T09:37:31.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:30 smithi079 ceph-mon[35502]: pgmap v970: 129 pgs: 129 active+clean; 359 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-05-17T09:37:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/3840857656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4104e80 msgr2=0x7f6dc4105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/3840857656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4104e80 0x7f6dc4105290 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009860 tx=0x7f6db402a340 comp rx=0 tx=0).stop 2024-05-17T09:37:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/3840857656 shutdown_connections 2024-05-17T09:37:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/3840857656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc4109b80 0x7f6dc410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/3840857656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4104e80 0x7f6dc4105290 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/3840857656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc4103c40 0x7f6dc4104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/3840857656 >> 172.21.15.19:0/3840857656 conn(0x7f6dc40fee50 msgr2=0x7f6dc41012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/3840857656 shutdown_connections 2024-05-17T09:37:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.082+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/3840857656 wait complete. 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.083+0000 7f6dcc2ea640 1 Processor -- start 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.083+0000 7f6dcc2ea640 1 -- start start 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dcc2ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 0x7f6dc4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dcc2ea640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc4104e80 0x7f6dc41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dcc2ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc4109b80 0x7f6dc41a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dcc2ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410f5f0 con 0x7f6dc4103c40 2024-05-17T09:37:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dcc2ea640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410f470 con 0x7f6dc4109b80 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dcc2ea640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410f770 con 0x7f6dc4104e80 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dca086640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 0x7f6dc4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dca086640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 0x7f6dc4198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51236/0 (socket says 172.21.15.19:51236) 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dca086640 1 -- 172.21.15.19:0/2862725117 learned_addr learned my addr 172.21.15.19:0/2862725117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dc9885640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc4104e80 0x7f6dc41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.084+0000 7f6dca887640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc4109b80 0x7f6dc41a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dca086640 1 -- 172.21.15.19:0/2862725117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc4104e80 msgr2=0x7f6dc41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dca086640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc4104e80 0x7f6dc41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dca086640 1 -- 172.21.15.19:0/2862725117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc4109b80 msgr2=0x7f6dc41a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dca086640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc4109b80 0x7f6dc41a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dca086640 1 -- 172.21.15.19:0/2862725117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc41a8370 con 0x7f6dc4103c40 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dca086640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 0x7f6dc4198070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000e9c0 tx=0x7f6dc000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dbb7fe640 1 -- 172.21.15.19:0/2862725117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000ccc0 con 0x7f6dc4103c40 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dbb7fe640 1 -- 172.21.15.19:0/2862725117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc00035f0 con 0x7f6dc4103c40 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dbb7fe640 1 -- 172.21.15.19:0/2862725117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000ce60 con 0x7f6dc4103c40 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.085+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41a8660 con 0x7f6dc4103c40 2024-05-17T09:37:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.086+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41a8b30 con 0x7f6dc4103c40 2024-05-17T09:37:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.090+0000 7f6dbb7fe640 1 -- 172.21.15.19:0/2862725117 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6dc0004000 con 0x7f6dc4103c40 2024-05-17T09:37:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.090+0000 7f6dd3195740 1 Processor -- start 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 -- start start 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cf6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08cfd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a08c68c30 con 0x557a0892c2f0 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a08c587d0 con 0x557a089344d0 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dd3195740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557a08c5c810 con 0x557a089551b0 2024-05-17T09:37:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dca887640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dca887640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:53250/0 (socket says 172.21.15.19:53250) 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dca887640 1 -- 172.21.15.19:0/3321580132 learned_addr learned my addr 172.21.15.19:0/3321580132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08cfd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.091+0000 7f6dca086640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cf6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca887640 1 -- 172.21.15.19:0/3321580132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 msgr2=0x557a08cfd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca887640 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08cfd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca887640 1 -- 172.21.15.19:0/3321580132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 msgr2=0x557a08cf6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca887640 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cf6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca887640 1 -- 172.21.15.19:0/3321580132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0896d210 con 0x557a089551b0 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca086640 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cf6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dbb7fe640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6db00b2920 0x7f6db00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca887640 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf63a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc011440 tx=0x7f6dbc011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b0f1c1cedc35191 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dca086640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6db00b2920 0x7f6db00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6dbb7fe640 1 -- 172.21.15.19:0/2862725117 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6dc00e2700 con 0x7f6dc4103c40 2024-05-17T09:37:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6db8bfe640 1 -- 172.21.15.19:0/3321580132 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc004d20 con 0x557a089551b0 2024-05-17T09:37:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6db8bfe640 1 -- 172.21.15.19:0/3321580132 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dbc004050 con 0x557a089551b0 2024-05-17T09:37:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.092+0000 7f6db8bfe640 1 -- 172.21.15.19:0/3321580132 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00f740 con 0x557a089551b0 2024-05-17T09:37:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.093+0000 7f6dca086640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6db00b2920 0x7f6db00b4e30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x557a08cf7780 tx=0x7f6dc0003a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.093+0000 7f6dd3195740 1 -- 172.21.15.19:0/3321580132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 msgr2=0x557a08cf63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.093+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf63a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc011440 tx=0x7f6dbc011900 comp rx=0 tx=0).stop 2024-05-17T09:37:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 -- 172.21.15.19:0/3321580132 shutdown_connections 2024-05-17T09:37:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08cfd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cf6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3321580132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf63a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 -- 172.21.15.19:0/3321580132 >> 172.21.15.19:0/3321580132 conn(0x557a08949300 msgr2=0x557a08cf1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 -- 172.21.15.19:0/3321580132 shutdown_connections 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.094+0000 7f6dd3195740 1 -- 172.21.15.19:0/3321580132 wait complete. 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dd3195740 1 -- start start 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dd3195740 1 Processor -- start 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a0892c2f0 0x557a08cffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d00280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dc9885640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d00280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dca887640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a0892c2f0 0x557a08cffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dc9885640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d00280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51266/0 (socket says 172.21.15.19:51266) 2024-05-17T09:37:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dca887640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a0892c2f0 0x557a08cffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:53266/0 (socket says 172.21.15.19:53266) 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089551b0 0x557a08d071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.095+0000 7f6dc9885640 1 -- 172.21.15.19:0/3322680634 learned_addr learned my addr 172.21.15.19:0/3322680634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a0893e790 con 0x557a089344d0 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a08c14890 con 0x557a089551b0 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557a08cfe270 con 0x557a0892c2f0 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dc9885640 1 -- 172.21.15.19:0/3322680634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a0892c2f0 msgr2=0x557a08cffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a0892c2f0 0x557a08cffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dc9885640 1 -- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089551b0 msgr2=0x557a08d071e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089551b0 0x557a08d071e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dc9885640 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a08a39620 con 0x557a089344d0 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.096+0000 7f6dd3195740 10 client.0 _ll_register_callbacks cb 0x557a08c17d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.097+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc004050 con 0x557a089344d0 2024-05-17T09:37:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.097+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d00280 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6db402a930 tx=0x7f6db40308d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21007a22c8a39a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.097+0000 7f6dadffb640 10 client.18243 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:37:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.097+0000 7f6dadffb640 1 -- 172.21.15.19:0/3322680634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4037040 con 0x557a089344d0 2024-05-17T09:37:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.097+0000 7f6dadffb640 1 -- 172.21.15.19:0/3322680634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6db4030df0 con 0x557a089344d0 2024-05-17T09:37:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.097+0000 7f6dadffb640 1 -- 172.21.15.19:0/3322680634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6db4004040 con 0x557a089344d0 2024-05-17T09:37:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 10 client.18243 fetch_fsmap learned FSMap version 39 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=39}) -- 0x557a08d09f90 con 0x557a089344d0 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dadffb640 1 -- 172.21.15.19:0/3322680634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6db40041e0 con 0x557a089344d0 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 10 client.18243 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 10 client.18243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 20 client.18243 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a08913d80 0x557a08d0d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 4 client.18243 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.098+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a08c6ab80 con 0x557a08913d80 2024-05-17T09:37:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.099+0000 7f6dca887640 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a08913d80 0x557a08d0d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.100+0000 7f6dadffb640 10 client.18243 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.100+0000 7f6dca887640 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a08913d80 0x557a08d0d790 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33748787810a3bbf server_cookie=ce50a3d0e726b188 in_seq=0 out_seq=0 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dadffb640 1 -- 172.21.15.19:0/3322680634 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557a08c6ab80 con 0x557a08913d80 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dadffb640 10 client.18243 handle_command_reply: tid=0 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 client.18243 shutdown 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 msgr2=0x557a08d00280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d00280 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6db402a930 tx=0x7f6db40308d0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 shutdown_connections 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a08913d80 0x557a08d0d790 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089551b0 0x557a08d071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d00280 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3322680634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a0892c2f0 0x557a08cffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 >> 172.21.15.19:0/3322680634 conn(0x557a08949300 msgr2=0x557a08cf0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.101+0000 7f6dadffb640 0 client.18243 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.102+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 shutdown_connections 2024-05-17T09:37:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.102+0000 7f6dd3195740 1 -- 172.21.15.19:0/3322680634 wait complete. 2024-05-17T09:37:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.102+0000 7f6dd3195740 20 client.18243 trim_cache size 0 max 16384 2024-05-17T09:37:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 Processor -- start 2024-05-17T09:37:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 -- start start 2024-05-17T09:37:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cfa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08d07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b5a10 con 0x557a0892c2f0 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b3270 con 0x557a089344d0 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.125+0000 7f6dd3195740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b10c0 con 0x557a089551b0 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cfa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cfa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51272/0 (socket says 172.21.15.19:51272) 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 -- 172.21.15.19:0/243869262 learned_addr learned my addr 172.21.15.19:0/243869262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dc9885640 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08d07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 -- 172.21.15.19:0/243869262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 msgr2=0x557a08cf0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 -- 172.21.15.19:0/243869262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 msgr2=0x557a08d07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08d07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 -- 172.21.15.19:0/243869262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a08cf75a0 con 0x557a0892c2f0 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dc9885640 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08d07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6dca887640 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cfa7d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006010 tx=0x7f6dbc005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16b23767c219d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6daffff640 1 -- 172.21.15.19:0/243869262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc031020 con 0x557a0892c2f0 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6daffff640 1 -- 172.21.15.19:0/243869262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dbc0045a0 con 0x557a0892c2f0 2024-05-17T09:37:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.126+0000 7f6daffff640 1 -- 172.21.15.19:0/243869262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc01d040 con 0x557a0892c2f0 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 -- 172.21.15.19:0/243869262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 msgr2=0x557a08cfa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cfa7d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006010 tx=0x7f6dbc005c80 comp rx=0 tx=0).stop 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 -- 172.21.15.19:0/243869262 shutdown_connections 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08cf0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a089344d0 0x557a08d07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 --2- 172.21.15.19:0/243869262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a0892c2f0 0x557a08cfa7d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 -- 172.21.15.19:0/243869262 >> 172.21.15.19:0/243869262 conn(0x557a08949300 msgr2=0x557a08cf5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 -- 172.21.15.19:0/243869262 shutdown_connections 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 -- 172.21.15.19:0/243869262 wait complete. 2024-05-17T09:37:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.127+0000 7f6dd3195740 1 -- start start 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 Processor -- start 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a0892c2f0 0x557a08d042b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08d05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b1730 con 0x557a089344d0 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b2ba0 con 0x557a0892c2f0 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b1e00 con 0x557a089551b0 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dd3195740 10 client.0 _ll_register_callbacks cb 0x7f6db4030560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51284/0 (socket says 172.21.15.19:51284) 2024-05-17T09:37:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 -- 172.21.15.19:0/3192292588 learned_addr learned my addr 172.21.15.19:0/3192292588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dca887640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08d05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dca887640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a0892c2f0 0x557a08d042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 -- 172.21.15.19:0/3192292588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 msgr2=0x557a08d05720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08d05720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 -- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a0892c2f0 msgr2=0x557a08d042b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a0892c2f0 0x557a08d042b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.128+0000 7f6dc9885640 1 -- 172.21.15.19:0/3192292588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a08d05ce0 con 0x557a089344d0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dc9885640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d047f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6db402a930 tx=0x7f6db4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ed4e34b99ec90d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dadffb640 1 -- 172.21.15.19:0/3192292588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4037040 con 0x557a089344d0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dadffb640 1 -- 172.21.15.19:0/3192292588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6db4003a00 con 0x557a089344d0 2024-05-17T09:37:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc0045a0 con 0x557a089344d0 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dadffb640 1 -- 172.21.15.19:0/3192292588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6db405f050 con 0x557a089344d0 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 10 client.18249 fetch_fsmap learned FSMap version 39 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a08d06fe0 con 0x557a089344d0 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dadffb640 1 -- 172.21.15.19:0/3192292588 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dbc0045a0 con 0x557a089344d0 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dadffb640 1 -- 172.21.15.19:0/3192292588 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6db4003d20 con 0x557a089344d0 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 10 client.18249 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 10 client.18249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 20 client.18249 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a088e91c0 0x557a08d78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 4 client.18249 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.129+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a08c64360 con 0x557a088e91c0 2024-05-17T09:37:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.130+0000 7f6dca887640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a088e91c0 0x557a08d78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.130+0000 7f6dca887640 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a088e91c0 0x557a08d78260 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da624de66e4ae8a0 server_cookie=ca4d31257a79c557 in_seq=0 out_seq=0 2024-05-17T09:37:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.130+0000 7f6dadffb640 10 client.18249 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.296+0000 7f6dadffb640 1 -- 172.21.15.19:0/3192292588 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f6dbc02a030 con 0x557a088e91c0 2024-05-17T09:37:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.296+0000 7f6dadffb640 10 client.18249 handle_command_reply: tid=0 2024-05-17T09:37:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.296+0000 7f6dd3195740 1 client.18249 shutdown 2024-05-17T09:37:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.296+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 msgr2=0x557a08d047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.296+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d047f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6db402a930 tx=0x7f6db4005f90 comp rx=0 tx=0).stop 2024-05-17T09:37:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 shutdown_connections 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557a088e91c0 0x557a08d78260 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a089551b0 0x557a08d05720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a089344d0 0x557a08d047f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dadffb640 0 client.18249 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 --2- 172.21.15.19:0/3192292588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a0892c2f0 0x557a08d042b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 >> 172.21.15.19:0/3192292588 conn(0x557a08949300 msgr2=0x557a08cf52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 shutdown_connections 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 1 -- 172.21.15.19:0/3192292588 wait complete. 2024-05-17T09:37:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.297+0000 7f6dd3195740 20 client.18249 trim_cache size 0 max 16384 2024-05-17T09:37:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:37:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:37:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6db00b2920 msgr2=0x7f6db00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6db00b2920 0x7f6db00b4e30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x557a08cf7780 tx=0x7f6dc0003a10 comp rx=0 tx=0).stop 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 msgr2=0x7f6dc4198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 0x7f6dc4198070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000e9c0 tx=0x7f6dc000edc0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 shutdown_connections 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc4109b80 0x7f6dc41a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6dc4104e80 0x7f6dc41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6db00b2920 0x7f6db00b4e30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 --2- 172.21.15.19:0/2862725117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc4103c40 0x7f6dc4198070 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 >> 172.21.15.19:0/2862725117 conn(0x7f6dc40fee50 msgr2=0x7f6dc40ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 shutdown_connections 2024-05-17T09:37:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:31.298+0000 7f6dcc2ea640 1 -- 172.21.15.19:0/2862725117 wait complete. 2024-05-17T09:37:31.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 25697 6.57 MB/sec execute 88 sec latency 225.745 ms 2024-05-17T09:37:32.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 25893 6.69 MB/sec execute 89 sec latency 70.971 ms 2024-05-17T09:37:32.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:32 smithi079 ceph-mon[35502]: pgmap v971: 129 pgs: 129 active+clean; 360 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-05-17T09:37:32.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:32 smithi069 ceph-mon[35081]: pgmap v971: 129 pgs: 129 active+clean; 360 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-05-17T09:37:32.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:32 smithi019 ceph-mon[31620]: pgmap v971: 129 pgs: 129 active+clean; 360 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-05-17T09:37:33.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26044 6.69 MB/sec execute 90 sec latency 68.083 ms 2024-05-17T09:37:34.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26220 6.82 MB/sec execute 91 sec latency 64.974 ms 2024-05-17T09:37:34.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:34 smithi019 ceph-mon[31620]: pgmap v972: 129 pgs: 129 active+clean; 363 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-05-17T09:37:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:34 smithi069 ceph-mon[35081]: pgmap v972: 129 pgs: 129 active+clean; 363 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-05-17T09:37:35.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:34 smithi079 ceph-mon[35502]: pgmap v972: 129 pgs: 129 active+clean; 363 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-05-17T09:37:35.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26413 6.86 MB/sec execute 92 sec latency 122.151 ms 2024-05-17T09:37:36.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26585 6.90 MB/sec execute 93 sec latency 75.400 ms 2024-05-17T09:37:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:36 smithi069 ceph-mon[35081]: pgmap v973: 129 pgs: 129 active+clean; 360 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-05-17T09:37:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:36 smithi019 ceph-mon[31620]: pgmap v973: 129 pgs: 129 active+clean; 360 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-05-17T09:37:37.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:36 smithi079 ceph-mon[35502]: pgmap v973: 129 pgs: 129 active+clean; 360 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-05-17T09:37:37.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26690 6.87 MB/sec execute 94 sec latency 78.691 ms 2024-05-17T09:37:38.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26804 6.87 MB/sec execute 95 sec latency 187.817 ms 2024-05-17T09:37:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:38 smithi069 ceph-mon[35081]: pgmap v974: 129 pgs: 129 active+clean; 367 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-17T09:37:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:38 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:37:38.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:38 smithi019 ceph-mon[31620]: pgmap v974: 129 pgs: 129 active+clean; 367 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-17T09:37:38.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:38 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:37:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:38 smithi079 ceph-mon[35502]: pgmap v974: 129 pgs: 129 active+clean; 367 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-05-17T09:37:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:38 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:37:39.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 26945 6.91 MB/sec execute 96 sec latency 149.659 ms 2024-05-17T09:37:40.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27071 6.90 MB/sec execute 97 sec latency 147.896 ms 2024-05-17T09:37:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:40 smithi069 ceph-mon[35081]: pgmap v975: 129 pgs: 129 active+clean; 371 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 272 op/s 2024-05-17T09:37:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:40 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:40.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:40 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:40.935 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:40 smithi019 ceph-mon[31620]: pgmap v975: 129 pgs: 129 active+clean; 371 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 272 op/s 2024-05-17T09:37:40.935 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:40 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:40.935 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:40 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:40 smithi079 ceph-mon[35502]: pgmap v975: 129 pgs: 129 active+clean; 371 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 272 op/s 2024-05-17T09:37:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:40 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:40 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:41.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27188 6.87 MB/sec execute 98 sec latency 153.707 ms 2024-05-17T09:37:42.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27306 6.84 MB/sec execute 99 sec latency 144.268 ms 2024-05-17T09:37:42.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:42 smithi069 ceph-mon[35081]: pgmap v976: 129 pgs: 129 active+clean; 371 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-05-17T09:37:42.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:42.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:42 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:42 smithi019 ceph-mon[31620]: pgmap v976: 129 pgs: 129 active+clean; 371 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-05-17T09:37:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:42 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:42 smithi079 ceph-mon[35502]: pgmap v976: 129 pgs: 129 active+clean; 371 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-05-17T09:37:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:42 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:43.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27412 6.79 MB/sec execute 100 sec latency 148.410 ms 2024-05-17T09:37:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 -- 172.21.15.19:0/4022830866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 msgr2=0x7f9aa410ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/4022830866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa410ae80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009ca0 tx=0x7f9a9c02a8c0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 -- 172.21.15.19:0/4022830866 shutdown_connections 2024-05-17T09:37:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/4022830866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa410ae80 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/4022830866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9aa4103d60 0x7f9aa4104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/4022830866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4102b20 0x7f9aa4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 -- 172.21.15.19:0/4022830866 >> 172.21.15.19:0/4022830866 conn(0x7f9aa40fdd10 msgr2=0x7f9aa41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 -- 172.21.15.19:0/4022830866 shutdown_connections 2024-05-17T09:37:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.432+0000 7f9aaacaf640 1 -- 172.21.15.19:0/4022830866 wait complete. 2024-05-17T09:37:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 Processor -- start 2024-05-17T09:37:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 -- start start 2024-05-17T09:37:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9aa4102b20 0x7f9aa419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 0x7f9aa419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa41a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa4111830 con 0x7f9aa4108a60 2024-05-17T09:37:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41116b0 con 0x7f9aa4102b20 2024-05-17T09:37:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aaacaf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41119b0 con 0x7f9aa4103d60 2024-05-17T09:37:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aa924c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa41a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aa3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 0x7f9aa419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aa3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 0x7f9aa419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:53310/0 (socket says 172.21.15.19:53310) 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aa3fff640 1 -- 172.21.15.19:0/2900858040 learned_addr learned my addr 172.21.15.19:0/2900858040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aa924c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa41a3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50118/0 (socket says 172.21.15.19:50118) 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.433+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9aa4102b20 0x7f9aa419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa3fff640 1 -- 172.21.15.19:0/2900858040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9aa4102b20 msgr2=0x7f9aa419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9aa4102b20 0x7f9aa419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa3fff640 1 -- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 msgr2=0x7f9aa41a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa41a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa3fff640 1 -- 172.21.15.19:0/2900858040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa41a3ec0 con 0x7f9aa4103d60 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 0x7f9aa419c8c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000d950 tx=0x7f9a9000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa1ffb640 1 -- 172.21.15.19:0/2900858040 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90017070 con 0x7f9aa4103d60 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa1ffb640 1 -- 172.21.15.19:0/2900858040 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a90009c30 con 0x7f9aa4103d60 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa41a41b0 con 0x7f9aa4103d60 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.434+0000 7f9aa1ffb640 1 -- 172.21.15.19:0/2900858040 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000f620 con 0x7f9aa4103d60 2024-05-17T09:37:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.435+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa41a4700 con 0x7f9aa4103d60 2024-05-17T09:37:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 Processor -- start 2024-05-17T09:37:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 -- start start 2024-05-17T09:37:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9aa1ffb640 1 -- 172.21.15.19:0/2900858040 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9a90010460 con 0x7f9aa4103d60 2024-05-17T09:37:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x5650858383a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085838920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x56508583f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5650857aac30 con 0x5650854971b0 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56508579a7d0 con 0x56508546e2f0 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56508579e810 con 0x5650854764d0 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9aa924c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x5650858383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9aa924c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x5650858383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50130/0 (socket says 172.21.15.19:50130) 2024-05-17T09:37:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9aa924c640 1 -- 172.21.15.19:0/794410631 learned_addr learned my addr 172.21.15.19:0/794410631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.438+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085838920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x56508583f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa924c640 1 -- 172.21.15.19:0/794410631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 msgr2=0x56508583f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa924c640 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x56508583f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa924c640 1 -- 172.21.15.19:0/794410631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 msgr2=0x565085838920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa924c640 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085838920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa924c640 1 -- 172.21.15.19:0/794410631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650854af210 con 0x5650854971b0 2024-05-17T09:37:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085838920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa924c640 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x5650858383a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c035000 tx=0x7f9a9c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a870504244448 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9a8effd640 1 -- 172.21.15.19:0/794410631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c031a80 con 0x5650854971b0 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.439+0000 7f9aa1ffb640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a940b28a0 0x7f9a940b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.440+0000 7f9a8effd640 1 -- 172.21.15.19:0/794410631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a9c036030 con 0x5650854971b0 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.440+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a940b28a0 0x7f9a940b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.440+0000 7f9aa1ffb640 1 -- 172.21.15.19:0/2900858040 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9a900e2190 con 0x7f9aa4103d60 2024-05-17T09:37:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.440+0000 7f9a8effd640 1 -- 172.21.15.19:0/794410631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c004440 con 0x5650854971b0 2024-05-17T09:37:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 -- 172.21.15.19:0/794410631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 msgr2=0x5650858383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x5650858383a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c035000 tx=0x7f9a9c0098b0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a940b28a0 0x7f9a940b4db0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x565085839780 tx=0x7f9a98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 -- 172.21.15.19:0/794410631 shutdown_connections 2024-05-17T09:37:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x56508583f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085838920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/794410631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x5650858383a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 -- 172.21.15.19:0/794410631 >> 172.21.15.19:0/794410631 conn(0x56508548b300 msgr2=0x565085833d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 -- 172.21.15.19:0/794410631 shutdown_connections 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 -- 172.21.15.19:0/794410631 wait complete. 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 -- start start 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.441+0000 7f9ab1b65740 1 Processor -- start 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085841ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x565085842010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x565085848f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565085480790 con 0x5650854971b0 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565085756890 con 0x56508546e2f0 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565085840090 con 0x5650854764d0 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9ab1b65740 10 client.0 _ll_register_callbacks cb 0x565085759d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085841ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x565085842010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085841ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54380/0 (socket says 172.21.15.19:54380) 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/458359226 learned_addr learned my addr 172.21.15.19:0/458359226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa924c640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x565085848f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/458359226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 msgr2=0x565085842010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x565085842010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/458359226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 msgr2=0x565085848f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x565085848f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/458359226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56508557b620 con 0x56508546e2f0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.442+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x565085842010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085841ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009fd0 tx=0x7f9a98004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0e0824e03a9d48 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9a83fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9a83fff640 1 -- 172.21.15.19:0/458359226 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9802e060 con 0x56508546e2f0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9a83fff640 1 -- 172.21.15.19:0/458359226 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9a98018030 con 0x56508546e2f0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a9c036030 con 0x56508546e2f0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9a83fff640 1 -- 172.21.15.19:0/458359226 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9a98035050 con 0x56508546e2f0 2024-05-17T09:37:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9ab1b65740 10 client.27788 fetch_fsmap learned FSMap version 39 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.443+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56508584bf70 con 0x56508546e2f0 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9a83fff640 1 -- 172.21.15.19:0/458359226 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a9c036030 con 0x56508546e2f0 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9a83fff640 1 -- 172.21.15.19:0/458359226 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9a9801c3a0 con 0x56508546e2f0 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9ab1b65740 10 client.27788 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9ab1b65740 10 client.27788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9ab1b65740 20 client.27788 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565085455d80 0x56508584f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9ab1b65740 4 client.27788 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650857acb80 con 0x565085455d80 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.444+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565085455d80 0x56508584f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.445+0000 7f9a83fff640 10 client.27788 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.445+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565085455d80 0x56508584f650 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8957a3c30c2477ca server_cookie=6b4eb7204c7ad7c in_seq=0 out_seq=0 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9a83fff640 1 -- 172.21.15.19:0/458359226 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5650857acb80 con 0x565085455d80 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9a83fff640 10 client.27788 handle_command_reply: tid=0 2024-05-17T09:37:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 client.27788 shutdown 2024-05-17T09:37:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 msgr2=0x565085841ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085841ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009fd0 tx=0x7f9a98004730 comp rx=0 tx=0).stop 2024-05-17T09:37:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 shutdown_connections 2024-05-17T09:37:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565085455d80 0x56508584f650 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854971b0 0x565085848f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854764d0 0x565085842010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/458359226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x565085841ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 >> 172.21.15.19:0/458359226 conn(0x56508548b300 msgr2=0x565085832d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9a83fff640 0 client.27788 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 shutdown_connections 2024-05-17T09:37:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 1 -- 172.21.15.19:0/458359226 wait complete. 2024-05-17T09:37:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.446+0000 7f9ab1b65740 20 client.27788 trim_cache size 0 max 16384 2024-05-17T09:37:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.469+0000 7f9ab1b65740 1 Processor -- start 2024-05-17T09:37:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.469+0000 7f9ab1b65740 1 -- start start 2024-05-17T09:37:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x56508583b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 0x565085848f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085849440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41b1510 con 0x5650854764d0 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41aede0 con 0x56508546e2f0 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41acc30 con 0x5650854971b0 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 0x565085848f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 0x565085848f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50154/0 (socket says 172.21.15.19:50154) 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 -- 172.21.15.19:0/662963036 learned_addr learned my addr 172.21.15.19:0/662963036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa924c640 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085849440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x56508583b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 -- 172.21.15.19:0/662963036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 msgr2=0x565085849440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085849440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 -- 172.21.15.19:0/662963036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 msgr2=0x56508583b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x56508583b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.470+0000 7f9aa3fff640 1 -- 172.21.15.19:0/662963036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650858395a0 con 0x5650854764d0 2024-05-17T09:37:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.471+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 0x565085848f00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9a900df8e0 tx=0x7f9a90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a882c25b9faeab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.471+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/662963036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90020040 con 0x5650854764d0 2024-05-17T09:37:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.471+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/662963036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a900a94b0 con 0x5650854764d0 2024-05-17T09:37:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.471+0000 7f9a8e7fc640 1 -- 172.21.15.19:0/662963036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a900a5740 con 0x5650854764d0 2024-05-17T09:37:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 -- 172.21.15.19:0/662963036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 msgr2=0x565085848f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 0x565085848f00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9a900df8e0 tx=0x7f9a90000bc0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 -- 172.21.15.19:0/662963036 shutdown_connections 2024-05-17T09:37:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085849440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650854764d0 0x565085848f00 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/662963036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56508546e2f0 0x56508583b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 -- 172.21.15.19:0/662963036 >> 172.21.15.19:0/662963036 conn(0x56508548b300 msgr2=0x565085832d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 -- 172.21.15.19:0/662963036 shutdown_connections 2024-05-17T09:37:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 -- 172.21.15.19:0/662963036 wait complete. 2024-05-17T09:37:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.472+0000 7f9ab1b65740 1 -- start start 2024-05-17T09:37:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 Processor -- start 2024-05-17T09:37:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 0x5650858469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650854764d0 0x565085846f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085847e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41ad2a0 con 0x56508546e2f0 2024-05-17T09:37:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41ae710 con 0x5650854764d0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41ad970 con 0x5650854971b0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9ab1b65740 10 client.0 _ll_register_callbacks cb 0x7f9a9801e7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9aa8a4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 0x5650858469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9aa8a4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 0x5650858469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50166/0 (socket says 172.21.15.19:50166) 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9aa924c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085847e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/1903841934 learned_addr learned my addr 172.21.15.19:0/1903841934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.473+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650854764d0 0x565085846f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/1903841934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 msgr2=0x565085847e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085847e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650854764d0 msgr2=0x565085846f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650854764d0 0x565085846f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa8a4b640 1 -- 172.21.15.19:0/1903841934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565085848400 con 0x56508546e2f0 2024-05-17T09:37:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650854764d0 0x565085846f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9aa8a4b640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 0x5650858469d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9a98019040 tx=0x7f9a9801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4e9e2309a963a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9a83fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9a83fff640 1 -- 172.21.15.19:0/1903841934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9803f020 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9a83fff640 1 -- 172.21.15.19:0/1903841934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9a9801cbf0 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.474+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a900a94b0 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9a83fff640 1 -- 172.21.15.19:0/1903841934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9a9804a050 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 10 client.18267 fetch_fsmap learned FSMap version 39 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565085849700 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9a83fff640 1 -- 172.21.15.19:0/1903841934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a900a94b0 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9a83fff640 1 -- 172.21.15.19:0/1903841934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9a9803b9f0 con 0x56508546e2f0 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 10 client.18267 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 10 client.18267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 20 client.18267 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56508542b1c0 0x5650858ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 4 client.18267 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:37:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650857a6360 con 0x56508542b1c0 2024-05-17T09:37:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.475+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56508542b1c0 0x5650858ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.476+0000 7f9a83fff640 10 client.18267 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.476+0000 7f9aa3fff640 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56508542b1c0 0x5650858ba4a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf864f91b7b5997 server_cookie=177525afba2a276 in_seq=0 out_seq=0 2024-05-17T09:37:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9a83fff640 1 -- 172.21.15.19:0/1903841934 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5650857a6360 con 0x56508542b1c0 2024-05-17T09:37:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9a83fff640 10 client.18267 handle_command_reply: tid=0 2024-05-17T09:37:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 client.18267 shutdown 2024-05-17T09:37:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 msgr2=0x5650858469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 0x5650858469d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9a98019040 tx=0x7f9a9801c630 comp rx=0 tx=0).stop 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 shutdown_connections 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56508542b1c0 0x5650858ba4a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5650854971b0 0x565085847e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9a83fff640 0 client.18267 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650854764d0 0x565085846f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 --2- 172.21.15.19:0/1903841934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56508546e2f0 0x5650858469d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 >> 172.21.15.19:0/1903841934 conn(0x56508548b300 msgr2=0x565085832330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 shutdown_connections 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.557+0000 7f9ab1b65740 1 -- 172.21.15.19:0/1903841934 wait complete. 2024-05-17T09:37:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9ab1b65740 20 client.18267 trim_cache size 0 max 16384 2024-05-17T09:37:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:37:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:37:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a940b28a0 msgr2=0x7f9a940b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a940b28a0 0x7f9a940b4db0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x565085839780 tx=0x7f9a98005f90 comp rx=0 tx=0).stop 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 msgr2=0x7f9aa419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 0x7f9aa419c8c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000d950 tx=0x7f9a9000de10 comp rx=0 tx=0).stop 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 shutdown_connections 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9a940b28a0 0x7f9a940b4db0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa4108a60 0x7f9aa41a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9aa4103d60 0x7f9aa419c8c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 --2- 172.21.15.19:0/2900858040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9aa4102b20 0x7f9aa419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.558+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 >> 172.21.15.19:0/2900858040 conn(0x7f9aa40fdd10 msgr2=0x7f9aa4104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.559+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 shutdown_connections 2024-05-17T09:37:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:44.559+0000 7f9aaacaf640 1 -- 172.21.15.19:0/2900858040 wait complete. 2024-05-17T09:37:44.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27524 6.78 MB/sec execute 101 sec latency 132.892 ms 2024-05-17T09:37:44.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:44 smithi069 ceph-mon[35081]: pgmap v977: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2024-05-17T09:37:44.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:44 smithi019 ceph-mon[31620]: pgmap v977: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2024-05-17T09:37:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:44 smithi079 ceph-mon[35502]: pgmap v977: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 253 op/s 2024-05-17T09:37:45.571 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27671 6.75 MB/sec execute 102 sec latency 100.309 ms 2024-05-17T09:37:46.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27841 6.81 MB/sec execute 103 sec latency 112.084 ms 2024-05-17T09:37:46.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:46 smithi069 ceph-mon[35081]: pgmap v978: 129 pgs: 129 active+clean; 374 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 250 op/s 2024-05-17T09:37:46.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:37:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:37:46.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:46 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:46 smithi019 ceph-mon[31620]: pgmap v978: 129 pgs: 129 active+clean; 374 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 250 op/s 2024-05-17T09:37:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:37:46.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:37:46.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:46 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:46 smithi079 ceph-mon[35502]: pgmap v978: 129 pgs: 129 active+clean; 374 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 250 op/s 2024-05-17T09:37:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:37:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:37:46.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:46 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:37:47.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 27997 6.82 MB/sec execute 104 sec latency 106.783 ms 2024-05-17T09:37:48.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 28153 6.81 MB/sec execute 105 sec latency 70.312 ms 2024-05-17T09:37:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:48 smithi019 ceph-mon[31620]: pgmap v979: 129 pgs: 129 active+clean; 372 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-05-17T09:37:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:48 smithi079 ceph-mon[35502]: pgmap v979: 129 pgs: 129 active+clean; 372 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-05-17T09:37:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:48 smithi069 ceph-mon[35081]: pgmap v979: 129 pgs: 129 active+clean; 372 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-05-17T09:37:49.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 28336 6.89 MB/sec execute 106 sec latency 80.102 ms 2024-05-17T09:37:50.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 28494 6.89 MB/sec execute 107 sec latency 76.236 ms 2024-05-17T09:37:50.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:50 smithi019 ceph-mon[31620]: pgmap v980: 129 pgs: 129 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-05-17T09:37:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:50 smithi079 ceph-mon[35502]: pgmap v980: 129 pgs: 129 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-05-17T09:37:50.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:50 smithi069 ceph-mon[35081]: pgmap v980: 129 pgs: 129 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-05-17T09:37:51.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 28664 6.87 MB/sec execute 108 sec latency 106.487 ms 2024-05-17T09:37:52.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 28838 6.88 MB/sec execute 109 sec latency 65.347 ms 2024-05-17T09:37:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:52 smithi019 ceph-mon[31620]: pgmap v981: 129 pgs: 129 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-05-17T09:37:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:52 smithi079 ceph-mon[35502]: pgmap v981: 129 pgs: 129 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-05-17T09:37:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:52 smithi069 ceph-mon[35081]: pgmap v981: 129 pgs: 129 active+clean; 365 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-05-17T09:37:53.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 28983 6.85 MB/sec execute 110 sec latency 64.381 ms 2024-05-17T09:37:54.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 29151 6.89 MB/sec execute 111 sec latency 77.851 ms 2024-05-17T09:37:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:54 smithi019 ceph-mon[31620]: pgmap v982: 129 pgs: 129 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-05-17T09:37:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:54 smithi079 ceph-mon[35502]: pgmap v982: 129 pgs: 129 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-05-17T09:37:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:54 smithi069 ceph-mon[35081]: pgmap v982: 129 pgs: 129 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-05-17T09:37:55.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 29320 6.92 MB/sec execute 112 sec latency 59.704 ms 2024-05-17T09:37:56.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 29492 6.95 MB/sec execute 113 sec latency 123.511 ms 2024-05-17T09:37:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:56 smithi019 ceph-mon[31620]: pgmap v983: 129 pgs: 129 active+clean; 366 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-05-17T09:37:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:56 smithi079 ceph-mon[35502]: pgmap v983: 129 pgs: 129 active+clean; 366 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-05-17T09:37:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:56 smithi069 ceph-mon[35081]: pgmap v983: 129 pgs: 129 active+clean; 366 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-05-17T09:37:57.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 29693 7.07 MB/sec execute 114 sec latency 183.651 ms 2024-05-17T09:37:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/335608443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48104ea0 msgr2=0x7f9b481052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/335608443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48104ea0 0x7f9b481052b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009860 tx=0x7f9b3c02a340 comp rx=0 tx=0).stop 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/335608443 shutdown_connections 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/335608443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48109ba0 0x7f9b4810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/335608443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48104ea0 0x7f9b481052b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/335608443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b48103c60 0x7f9b48104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/335608443 >> 172.21.15.19:0/335608443 conn(0x7f9b480fee50 msgr2=0x7f9b48101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/335608443 shutdown_connections 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.703+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/335608443 wait complete. 2024-05-17T09:37:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.704+0000 7f9b4d3b2640 1 Processor -- start 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.704+0000 7f9b4d3b2640 1 -- start start 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b4d3b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48103c60 0x7f9b4819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b4d3b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b48104ea0 0x7f9b4819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b4d3b2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48109ba0 0x7f9b481a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b4d3b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810f610 con 0x7f9b48104ea0 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b4d3b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810f490 con 0x7f9b48103c60 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b4d3b2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810f790 con 0x7f9b48109ba0 2024-05-17T09:37:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48103c60 0x7f9b4819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b48104ea0 0x7f9b4819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48103c60 0x7f9b4819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47476/0 (socket says 172.21.15.19:47476) 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48109ba0 0x7f9b481a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b46ffd640 1 -- 172.21.15.19:0/1941197546 learned_addr learned my addr 172.21.15.19:0/1941197546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b477fe640 1 -- 172.21.15.19:0/1941197546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48103c60 msgr2=0x7f9b4819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b477fe640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48103c60 0x7f9b4819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b477fe640 1 -- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b48104ea0 msgr2=0x7f9b4819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b477fe640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b48104ea0 0x7f9b4819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.705+0000 7f9b477fe640 1 -- 172.21.15.19:0/1941197546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b481a1cc0 con 0x7f9b48109ba0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.706+0000 7f9b477fe640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48109ba0 0x7f9b481a1650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009fd0 tx=0x7f9b4000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.706+0000 7f9b2ffff640 1 -- 172.21.15.19:0/1941197546 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000cd60 con 0x7f9b48109ba0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.706+0000 7f9b2ffff640 1 -- 172.21.15.19:0/1941197546 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b40005020 con 0x7f9b48109ba0 2024-05-17T09:37:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.706+0000 7f9b2ffff640 1 -- 172.21.15.19:0/1941197546 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40010690 con 0x7f9b48109ba0 2024-05-17T09:37:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.706+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b48074f50 con 0x7f9b48109ba0 2024-05-17T09:37:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.706+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b48075470 con 0x7f9b48109ba0 2024-05-17T09:37:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 Processor -- start 2024-05-17T09:37:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b2ffff640 1 -- 172.21.15.19:0/1941197546 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9b40004a40 con 0x7f9b48109ba0 2024-05-17T09:37:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 -- start start 2024-05-17T09:37:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc5ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 0x55cebc5ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc606820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cebc571c30 con 0x55cebc2352f0 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cebc5617d0 con 0x55cebc23d4d0 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.710+0000 7f9b54269740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cebc565810 con 0x55cebc25e1b0 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc5ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 0x55cebc5ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc5ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52652/0 (socket says 172.21.15.19:52652) 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b467fc640 1 -- 172.21.15.19:0/2759400149 learned_addr learned my addr 172.21.15.19:0/2759400149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 0x55cebc5ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44046/0 (socket says 172.21.15.19:44046) 2024-05-17T09:37:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc606820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b2ffff640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b300b6db0 0x7f9b300b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2759400149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 msgr2=0x55cebc5ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc5ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2759400149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 msgr2=0x55cebc606820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc606820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2759400149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebc276210 con 0x55cebc2352f0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b477fe640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b300b6db0 0x7f9b300b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.711+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc5ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.712+0000 7f9b2ffff640 1 -- 172.21.15.19:0/1941197546 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9b400e2c50 con 0x7f9b48109ba0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.712+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 0x55cebc5ff920 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009ca0 tx=0x7f9b38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae01b850add33872 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.712+0000 7f9b2d7fa640 1 -- 172.21.15.19:0/2759400149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38009e50 con 0x55cebc2352f0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.712+0000 7f9b2d7fa640 1 -- 172.21.15.19:0/2759400149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b380054b0 con 0x55cebc2352f0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.712+0000 7f9b477fe640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b300b6db0 0x7f9b300b92c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9b4001f000 tx=0x7f9b4001c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.712+0000 7f9b2d7fa640 1 -- 172.21.15.19:0/2759400149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38019410 con 0x55cebc2352f0 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 -- 172.21.15.19:0/2759400149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 msgr2=0x55cebc5ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 0x55cebc5ff920 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009ca0 tx=0x7f9b38011810 comp rx=0 tx=0).stop 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 -- 172.21.15.19:0/2759400149 shutdown_connections 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc606820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc2352f0 0x55cebc5ff920 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 --2- 172.21.15.19:0/2759400149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc5ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 -- 172.21.15.19:0/2759400149 >> 172.21.15.19:0/2759400149 conn(0x55cebc252300 msgr2=0x55cebc5fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 -- 172.21.15.19:0/2759400149 shutdown_connections 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.713+0000 7f9b54269740 1 -- 172.21.15.19:0/2759400149 wait complete. 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 -- start start 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 Processor -- start 2024-05-17T09:37:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc2352f0 0x55cebc608d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc609260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc6101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cebc247790 con 0x55cebc25e1b0 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cebc51d890 con 0x55cebc23d4d0 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cebc607270 con 0x55cebc2352f0 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b54269740 10 client.0 _ll_register_callbacks cb 0x55cebc520d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc609260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc609260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47486/0 (socket says 172.21.15.19:47486) 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.714+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2826277171 learned_addr learned my addr 172.21.15.19:0/2826277171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc6101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2826277171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc2352f0 msgr2=0x55cebc608d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc2352f0 0x55cebc608d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2826277171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 msgr2=0x55cebc6101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc6101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2826277171 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebc342620 con 0x55cebc23d4d0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc6101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc609260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9b38004a40 tx=0x7f9b38005310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de64a8cf4b7e6f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b2aff5640 1 -- 172.21.15.19:0/2826277171 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38028070 con 0x55cebc23d4d0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b380054b0 con 0x55cebc23d4d0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.715+0000 7f9b2aff5640 1 -- 172.21.15.19:0/2826277171 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9b38024440 con 0x55cebc23d4d0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b2aff5640 1 -- 172.21.15.19:0/2826277171 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b38034050 con 0x55cebc23d4d0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 10 client.27806 fetch_fsmap learned FSMap version 39 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cebc613140 con 0x55cebc23d4d0 2024-05-17T09:37:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b2aff5640 1 -- 172.21.15.19:0/2826277171 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b380054b0 con 0x55cebc23d4d0 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b2aff5640 1 -- 172.21.15.19:0/2826277171 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9b38022410 con 0x55cebc23d4d0 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 10 client.27806 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 10 client.27806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 20 client.27806 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc21cd80 0x55cebc6166e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 4 client.27806 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cebc573b80 con 0x55cebc21cd80 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.716+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc21cd80 0x55cebc6166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.717+0000 7f9b2aff5640 10 client.27806 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.717+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc21cd80 0x55cebc6166e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1e51b74495fb0b server_cookie=1cc2b07e70981f38 in_seq=0 out_seq=0 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.718+0000 7f9b2aff5640 1 -- 172.21.15.19:0/2826277171 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cebc573b80 con 0x55cebc21cd80 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.718+0000 7f9b2aff5640 10 client.27806 handle_command_reply: tid=0 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 client.27806 shutdown 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 msgr2=0x55cebc609260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc609260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9b38004a40 tx=0x7f9b38005310 comp rx=0 tx=0).stop 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 shutdown_connections 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc21cd80 0x55cebc6166e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc6101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc23d4d0 0x55cebc609260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b2aff5640 0 client.27806 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 --2- 172.21.15.19:0/2826277171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc2352f0 0x55cebc608d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 >> 172.21.15.19:0/2826277171 conn(0x55cebc252300 msgr2=0x55cebc5fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 shutdown_connections 2024-05-17T09:37:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 1 -- 172.21.15.19:0/2826277171 wait complete. 2024-05-17T09:37:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.719+0000 7f9b54269740 20 client.27806 trim_cache size 0 max 16384 2024-05-17T09:37:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 Processor -- start 2024-05-17T09:37:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 -- start start 2024-05-17T09:37:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc61bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 0x55cebc61c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc6100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481acfe0 con 0x55cebc23d4d0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481aa8b0 con 0x55cebc2352f0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b54269740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481a8700 con 0x55cebc25e1b0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 0x55cebc61c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 0x55cebc61c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44076/0 (socket says 172.21.15.19:44076) 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc61bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b46ffd640 1 -- 172.21.15.19:0/2349912934 learned_addr learned my addr 172.21.15.19:0/2349912934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.743+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc6100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b467fc640 1 -- 172.21.15.19:0/2349912934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 msgr2=0x55cebc6100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc6100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b467fc640 1 -- 172.21.15.19:0/2349912934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 msgr2=0x55cebc61c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 0x55cebc61c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b467fc640 1 -- 172.21.15.19:0/2349912934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebc606680 con 0x55cebc2352f0 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 0x55cebc61c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b467fc640 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc61bea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55cebc600570 tx=0x7f9b3c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8ef5cd1f1da103 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b2cff9640 1 -- 172.21.15.19:0/2349912934 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c037040 con 0x55cebc2352f0 2024-05-17T09:37:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b2cff9640 1 -- 172.21.15.19:0/2349912934 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b3c030e40 con 0x55cebc2352f0 2024-05-17T09:37:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.744+0000 7f9b2cff9640 1 -- 172.21.15.19:0/2349912934 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c004920 con 0x55cebc2352f0 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 -- 172.21.15.19:0/2349912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 msgr2=0x55cebc61bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc61bea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55cebc600570 tx=0x7f9b3c02a340 comp rx=0 tx=0).stop 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 -- 172.21.15.19:0/2349912934 shutdown_connections 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc25e1b0 0x55cebc6100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc23d4d0 0x55cebc61c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 --2- 172.21.15.19:0/2349912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc61bea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.745+0000 7f9b54269740 1 -- 172.21.15.19:0/2349912934 >> 172.21.15.19:0/2349912934 conn(0x55cebc252300 msgr2=0x55cebc6028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.746+0000 7f9b54269740 1 -- 172.21.15.19:0/2349912934 shutdown_connections 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.746+0000 7f9b54269740 1 -- 172.21.15.19:0/2349912934 wait complete. 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.746+0000 7f9b54269740 1 -- start start 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.746+0000 7f9b54269740 1 Processor -- start 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc60fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc23d4d0 0x55cebc610280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc67d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481a8d70 con 0x55cebc25e1b0 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481aa1e0 con 0x55cebc2352f0 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481a9440 con 0x55cebc23d4d0 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b54269740 10 client.0 _ll_register_callbacks cb 0x7f9b38009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc67d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc67d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44090/0 (socket says 172.21.15.19:44090) 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b467fc640 1 -- 172.21.15.19:0/625489013 learned_addr learned my addr 172.21.15.19:0/625489013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc23d4d0 0x55cebc610280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b467fc640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc60fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b46ffd640 1 -- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 msgr2=0x55cebc60fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc60fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b46ffd640 1 -- 172.21.15.19:0/625489013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 msgr2=0x55cebc67d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc67d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b46ffd640 1 -- 172.21.15.19:0/625489013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebc67d600 con 0x55cebc23d4d0 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.747+0000 7f9b467fc640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc67d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b46ffd640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc23d4d0 0x55cebc610280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55cebc606f00 tx=0x7f9b38022a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1ed9bc4059e7a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b2aff5640 1 -- 172.21.15.19:0/625489013 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38031070 con 0x55cebc23d4d0 2024-05-17T09:37:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b2aff5640 1 -- 172.21.15.19:0/625489013 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9b38005380 con 0x55cebc23d4d0 2024-05-17T09:37:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b3c030e40 con 0x55cebc23d4d0 2024-05-17T09:37:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b2aff5640 1 -- 172.21.15.19:0/625489013 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b38028040 con 0x55cebc23d4d0 2024-05-17T09:37:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b54269740 10 client.27805 fetch_fsmap learned FSMap version 39 2024-05-17T09:37:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.748+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cebc67e890 con 0x55cebc23d4d0 2024-05-17T09:37:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b2aff5640 1 -- 172.21.15.19:0/625489013 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b3c030e40 con 0x55cebc23d4d0 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b2aff5640 1 -- 172.21.15.19:0/625489013 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9b380035f0 con 0x55cebc23d4d0 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b54269740 10 client.27805 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b54269740 10 client.27805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b54269740 20 client.27805 populate_metadata read hostname 'smithi019' 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b54269740 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc1f21c0 0x55cebc681210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b54269740 4 client.27805 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.749+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cebc56d360 con 0x55cebc1f21c0 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.750+0000 7f9b467fc640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc1f21c0 0x55cebc681210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.750+0000 7f9b2aff5640 10 client.27805 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.750+0000 7f9b467fc640 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc1f21c0 0x55cebc681210 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a7a03f5d78a106 server_cookie=d021392ea9e0b8eb in_seq=0 out_seq=0 2024-05-17T09:37:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b2aff5640 1 -- 172.21.15.19:0/625489013 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cebc56d360 con 0x55cebc1f21c0 2024-05-17T09:37:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b2aff5640 10 client.27805 handle_command_reply: tid=0 2024-05-17T09:37:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 client.27805 shutdown 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc23d4d0 msgr2=0x55cebc610280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc23d4d0 0x55cebc610280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55cebc606f00 tx=0x7f9b38022a70 comp rx=0 tx=0).stop 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 shutdown_connections 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cebc1f21c0 0x55cebc681210 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cebc25e1b0 0x55cebc67d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cebc23d4d0 0x55cebc610280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b2aff5640 0 client.27805 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 --2- 172.21.15.19:0/625489013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cebc2352f0 0x55cebc60fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 >> 172.21.15.19:0/625489013 conn(0x55cebc252300 msgr2=0x55cebc603210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 shutdown_connections 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 1 -- 172.21.15.19:0/625489013 wait complete. 2024-05-17T09:37:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.851+0000 7f9b54269740 20 client.27805 trim_cache size 0 max 16384 2024-05-17T09:37:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:37:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:37:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b300b6db0 msgr2=0x7f9b300b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b300b6db0 0x7f9b300b92c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9b4001f000 tx=0x7f9b4001c000 comp rx=0 tx=0).stop 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48109ba0 msgr2=0x7f9b481a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48109ba0 0x7f9b481a1650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009fd0 tx=0x7f9b4000ee50 comp rx=0 tx=0).stop 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 shutdown_connections 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b48109ba0 0x7f9b481a1650 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b300b6db0 0x7f9b300b92c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b48104ea0 0x7f9b4819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 --2- 172.21.15.19:0/1941197546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b48103c60 0x7f9b4819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 >> 172.21.15.19:0/1941197546 conn(0x7f9b480fee50 msgr2=0x7f9b48105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:37:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 shutdown_connections 2024-05-17T09:37:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:37:57.852+0000 7f9b4d3b2640 1 -- 172.21.15.19:0/1941197546 wait complete. 2024-05-17T09:37:58.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 29908 7.11 MB/sec execute 115 sec latency 120.063 ms 2024-05-17T09:37:58.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:37:58 smithi019 ceph-mon[31620]: pgmap v984: 129 pgs: 129 active+clean; 375 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-05-17T09:37:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:37:58 smithi079 ceph-mon[35502]: pgmap v984: 129 pgs: 129 active+clean; 375 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-05-17T09:37:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:37:58 smithi069 ceph-mon[35081]: pgmap v984: 129 pgs: 129 active+clean; 375 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-05-17T09:37:59.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30018 7.10 MB/sec execute 116 sec latency 77.420 ms 2024-05-17T09:38:00.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30123 7.10 MB/sec execute 117 sec latency 101.175 ms 2024-05-17T09:38:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:00 smithi079 ceph-mon[35502]: pgmap v985: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 4.2 MiB/s wr, 260 op/s 2024-05-17T09:38:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:00 smithi069 ceph-mon[35081]: pgmap v985: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 4.2 MiB/s wr, 260 op/s 2024-05-17T09:38:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:00 smithi019 ceph-mon[31620]: pgmap v985: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 4.2 MiB/s wr, 260 op/s 2024-05-17T09:38:01.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30303 7.18 MB/sec execute 118 sec latency 64.694 ms 2024-05-17T09:38:02.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30474 7.20 MB/sec execute 119 sec latency 76.845 ms 2024-05-17T09:38:02.731 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:02 smithi069 ceph-mon[35081]: pgmap v986: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-05-17T09:38:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:02 smithi079 ceph-mon[35502]: pgmap v986: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-05-17T09:38:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:02 smithi019 ceph-mon[31620]: pgmap v986: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-05-17T09:38:03.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30613 7.17 MB/sec execute 120 sec latency 71.286 ms 2024-05-17T09:38:04.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30746 7.15 MB/sec execute 121 sec latency 65.749 ms 2024-05-17T09:38:04.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:04 smithi079 ceph-mon[35502]: pgmap v987: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 284 op/s 2024-05-17T09:38:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:04 smithi069 ceph-mon[35081]: pgmap v987: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 284 op/s 2024-05-17T09:38:04.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:04 smithi019 ceph-mon[31620]: pgmap v987: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 284 op/s 2024-05-17T09:38:05.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 30876 7.12 MB/sec execute 122 sec latency 69.200 ms 2024-05-17T09:38:06.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31032 7.10 MB/sec execute 123 sec latency 67.320 ms 2024-05-17T09:38:06.608 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:38:06.608 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-05-17T09:38:06.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:38:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:06 smithi079 ceph-mon[35502]: pgmap v988: 129 pgs: 129 active+clean; 384 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-17T09:38:06.799 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:06 smithi069 ceph-mon[35081]: pgmap v988: 129 pgs: 129 active+clean; 384 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-17T09:38:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:06 smithi019 ceph-mon[31620]: pgmap v988: 129 pgs: 129 active+clean; 384 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-17T09:38:07.010 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:07.572 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31172 7.10 MB/sec execute 124 sec latency 130.006 ms 2024-05-17T09:38:08.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31296 7.07 MB/sec execute 125 sec latency 199.730 ms 2024-05-17T09:38:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:08 smithi019 ceph-mon[31620]: pgmap v989: 129 pgs: 129 active+clean; 384 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2024-05-17T09:38:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:08 smithi079 ceph-mon[35502]: pgmap v989: 129 pgs: 129 active+clean; 384 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2024-05-17T09:38:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:08 smithi069 ceph-mon[35081]: pgmap v989: 129 pgs: 129 active+clean; 384 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2024-05-17T09:38:09.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31459 7.08 MB/sec execute 126 sec latency 83.872 ms 2024-05-17T09:38:10.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31587 7.06 MB/sec execute 127 sec latency 148.186 ms 2024-05-17T09:38:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:10 smithi079 ceph-mon[35502]: pgmap v990: 129 pgs: 129 active+clean; 382 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 280 op/s 2024-05-17T09:38:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:10 smithi069 ceph-mon[35081]: pgmap v990: 129 pgs: 129 active+clean; 382 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 280 op/s 2024-05-17T09:38:10.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:10 smithi019 ceph-mon[31620]: pgmap v990: 129 pgs: 129 active+clean; 382 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 280 op/s 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 -- 172.21.15.19:0/2355311326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080103d80 msgr2=0x7fc080104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 --2- 172.21.15.19:0/2355311326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080103d80 0x7fc080104190 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc070009860 tx=0x7fc07002a340 comp rx=0 tx=0).stop 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 -- 172.21.15.19:0/2355311326 shutdown_connections 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 --2- 172.21.15.19:0/2355311326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080108a80 0x7fc08010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 --2- 172.21.15.19:0/2355311326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080103d80 0x7fc080104190 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 --2- 172.21.15.19:0/2355311326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc080102b40 0x7fc080103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 -- 172.21.15.19:0/2355311326 >> 172.21.15.19:0/2355311326 conn(0x7fc0800fdd30 msgr2=0x7fc0801001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 -- 172.21.15.19:0/2355311326 shutdown_connections 2024-05-17T09:38:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.995+0000 7fc086f3f640 1 -- 172.21.15.19:0/2355311326 wait complete. 2024-05-17T09:38:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 Processor -- start 2024-05-17T09:38:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 -- start start 2024-05-17T09:38:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 0x7fc08019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc080103d80 0x7fc08019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080108a80 0x7fc0801a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010f620 con 0x7fc080108a80 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010f4a0 con 0x7fc080103d80 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc086f3f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010f7a0 con 0x7fc080102b40 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc084cdb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 0x7fc08019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080108a80 0x7fc0801a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc080103d80 0x7fc08019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc084cdb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 0x7fc08019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37708/0 (socket says 172.21.15.19:37708) 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.996+0000 7fc084cdb640 1 -- 172.21.15.19:0/1708941127 learned_addr learned my addr 172.21.15.19:0/1708941127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc084cdb640 1 -- 172.21.15.19:0/1708941127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc080103d80 msgr2=0x7fc08019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc084cdb640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc080103d80 0x7fc08019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc084cdb640 1 -- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080108a80 msgr2=0x7fc0801a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc084cdb640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080108a80 0x7fc0801a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc084cdb640 1 -- 172.21.15.19:0/1708941127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0801a3f30 con 0x7fc080102b40 2024-05-17T09:38:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc084cdb640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 0x7fc08019c3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e980 tx=0x7fc06800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc07dffb640 1 -- 172.21.15.19:0/1708941127 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800cd80 con 0x7fc080102b40 2024-05-17T09:38:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc07dffb640 1 -- 172.21.15.19:0/1708941127 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0680108c0 con 0x7fc080102b40 2024-05-17T09:38:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0801a4220 con 0x7fc080102b40 2024-05-17T09:38:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc07dffb640 1 -- 172.21.15.19:0/1708941127 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068018880 con 0x7fc080102b40 2024-05-17T09:38:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:10.997+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801a4790 con 0x7fc080102b40 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc07dffb640 1 -- 172.21.15.19:0/1708941127 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc06801f020 con 0x7fc080102b40 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 Processor -- start 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 -- start start 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a1e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607396542f0 0x560739a1e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a25820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560739990c30 con 0x5607396542f0 2024-05-17T09:38:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5607399807d0 con 0x56073965c4d0 2024-05-17T09:38:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560739984810 con 0x56073967d1b0 2024-05-17T09:38:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a1e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc084cdb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607396542f0 0x560739a1e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a1e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37722/0 (socket says 172.21.15.19:37722) 2024-05-17T09:38:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc0854dc640 1 -- 172.21.15.19:0/449601547 learned_addr learned my addr 172.21.15.19:0/449601547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.001+0000 7fc07ffff640 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a25820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc0854dc640 1 -- 172.21.15.19:0/449601547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 msgr2=0x560739a25820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc0854dc640 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a25820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc0854dc640 1 -- 172.21.15.19:0/449601547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607396542f0 msgr2=0x560739a1e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc0854dc640 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607396542f0 0x560739a1e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc0854dc640 1 -- 172.21.15.19:0/449601547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560739695210 con 0x56073967d1b0 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc084cdb640 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607396542f0 0x560739a1e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc0854dc640 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a1e3a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc07800ae90 tx=0x7fc07800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe51ce87609d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc07dffb640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc06c0b2920 0x7fc06c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc076ffd640 1 -- 172.21.15.19:0/449601547 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078003b70 con 0x56073967d1b0 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.002+0000 7fc076ffd640 1 -- 172.21.15.19:0/449601547 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc078003d10 con 0x56073967d1b0 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.003+0000 7fc076ffd640 1 -- 172.21.15.19:0/449601547 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078010ae0 con 0x56073967d1b0 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.003+0000 7fc084cdb640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc06c0b2920 0x7fc06c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc07dffb640 1 -- 172.21.15.19:0/1708941127 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0680e2630 con 0x7fc080102b40 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 -- 172.21.15.19:0/449601547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 msgr2=0x560739a1e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a1e3a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc07800ae90 tx=0x7fc07800f790 comp rx=0 tx=0).stop 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc084cdb640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc06c0b2920 0x7fc06c0b4e30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560739a1f780 tx=0x7fc0680c6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 -- 172.21.15.19:0/449601547 shutdown_connections 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a25820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607396542f0 0x560739a1e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/449601547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a1e3a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 -- 172.21.15.19:0/449601547 >> 172.21.15.19:0/449601547 conn(0x560739671300 msgr2=0x560739a19d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 -- 172.21.15.19:0/449601547 shutdown_connections 2024-05-17T09:38:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.004+0000 7fc08ddf5740 1 -- 172.21.15.19:0/449601547 wait complete. 2024-05-17T09:38:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.005+0000 7fc08ddf5740 1 -- start start 2024-05-17T09:38:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.005+0000 7fc08ddf5740 1 Processor -- start 2024-05-17T09:38:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 0x560739a27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a2f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560739666790 con 0x56073965c4d0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56073993c890 con 0x5607396542f0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560739a26270 con 0x56073967d1b0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc08ddf5740 10 client.0 _ll_register_callbacks cb 0x56073993fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a2f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 0x560739a27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 0x560739a27cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45622/0 (socket says 172.21.15.19:45622) 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 -- 172.21.15.19:0/2200558832 learned_addr learned my addr 172.21.15.19:0/2200558832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 -- 172.21.15.19:0/2200558832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 msgr2=0x560739a2f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a2f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.006+0000 7fc0854dc640 1 -- 172.21.15.19:0/2200558832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 msgr2=0x560739a281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc0854dc640 1 -- 172.21.15.19:0/2200558832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560739761620 con 0x5607396542f0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 0x560739a27cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc078010890 tx=0x7fc078000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86db459fb005147a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc05ffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc05ffff640 1 -- 172.21.15.19:0/2200558832 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078010b00 con 0x5607396542f0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc078003d10 con 0x5607396542f0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.007+0000 7fc05ffff640 1 -- 172.21.15.19:0/2200558832 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc078004a40 con 0x5607396542f0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc05ffff640 1 -- 172.21.15.19:0/2200558832 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc078031050 con 0x5607396542f0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 10 client.27824 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560739a32080 con 0x5607396542f0 2024-05-17T09:38:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc05ffff640 1 -- 172.21.15.19:0/2200558832 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc078003d10 con 0x5607396542f0 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc05ffff640 1 -- 172.21.15.19:0/2200558832 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc078020020 con 0x5607396542f0 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 10 client.27824 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 10 client.27824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 20 client.27824 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56073963bd80 0x560739a35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 4 client.27824 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560739992b80 con 0x56073963bd80 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.008+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56073963bd80 0x560739a35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.009+0000 7fc05ffff640 10 client.27824 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.009+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56073963bd80 0x560739a35ba0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bda5ebc2bc09e server_cookie=8fac2a182150d1da in_seq=0 out_seq=0 2024-05-17T09:38:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc05ffff640 1 -- 172.21.15.19:0/2200558832 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560739992b80 con 0x56073963bd80 2024-05-17T09:38:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc05ffff640 10 client.27824 handle_command_reply: tid=0 2024-05-17T09:38:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 client.27824 shutdown 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 msgr2=0x560739a27cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 0x560739a27cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc078010890 tx=0x7fc078000f40 comp rx=0 tx=0).stop 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 shutdown_connections 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56073963bd80 0x560739a35ba0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56073967d1b0 0x560739a2f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.010+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2200558832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607396542f0 0x560739a27cb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.011+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 >> 172.21.15.19:0/2200558832 conn(0x560739671300 msgr2=0x560739a18d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.011+0000 7fc05ffff640 0 client.27824 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.011+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 shutdown_connections 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.011+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2200558832 wait complete. 2024-05-17T09:38:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.011+0000 7fc08ddf5740 20 client.27824 trim_cache size 0 max 16384 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 Processor -- start 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 -- start start 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a27290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a2f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073967d1b0 0x560739a2f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801b1610 con 0x56073967d1b0 2024-05-17T09:38:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801aeee0 con 0x56073965c4d0 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.037+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801acd30 con 0x5607396542f0 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073967d1b0 0x560739a2f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a27290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a2f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a27290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37750/0 (socket says 172.21.15.19:37750) 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 -- 172.21.15.19:0/1468903479 learned_addr learned my addr 172.21.15.19:0/1468903479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 -- 172.21.15.19:0/1468903479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 msgr2=0x560739a2f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a2f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 -- 172.21.15.19:0/1468903479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073967d1b0 msgr2=0x560739a2f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073967d1b0 0x560739a2f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 -- 172.21.15.19:0/1468903479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560739a25630 con 0x5607396542f0 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc07ffff640 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a2f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.038+0000 7fc0854dc640 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a27290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0780108c0 tx=0x7fc07800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c480af7e27038e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.039+0000 7fc0767fc640 1 -- 172.21.15.19:0/1468903479 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800a4d0 con 0x5607396542f0 2024-05-17T09:38:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.039+0000 7fc0767fc640 1 -- 172.21.15.19:0/1468903479 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc078010b00 con 0x5607396542f0 2024-05-17T09:38:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.039+0000 7fc0767fc640 1 -- 172.21.15.19:0/1468903479 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0780184d0 con 0x5607396542f0 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 -- 172.21.15.19:0/1468903479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 msgr2=0x560739a27290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a27290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0780108c0 tx=0x7fc07800f790 comp rx=0 tx=0).stop 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 -- 172.21.15.19:0/1468903479 shutdown_connections 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073967d1b0 0x560739a2f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073965c4d0 0x560739a2f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/1468903479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a27290 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 -- 172.21.15.19:0/1468903479 >> 172.21.15.19:0/1468903479 conn(0x560739671300 msgr2=0x560739a18d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 -- 172.21.15.19:0/1468903479 shutdown_connections 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.040+0000 7fc08ddf5740 1 -- 172.21.15.19:0/1468903479 wait complete. 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 -- start start 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 Processor -- start 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a2bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073967d1b0 0x560739a2c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801ad3a0 con 0x56073965c4d0 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801ae810 con 0x56073967d1b0 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801ada70 con 0x5607396542f0 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc08ddf5740 10 client.0 _ll_register_callbacks cb 0x7fc078010ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc0854dc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a2f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37758/0 (socket says 172.21.15.19:37758) 2024-05-17T09:38:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc0854dc640 1 -- 172.21.15.19:0/2698064658 learned_addr learned my addr 172.21.15.19:0/2698064658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.041+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a2bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073967d1b0 0x560739a2c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 -- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073967d1b0 msgr2=0x560739a2c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073967d1b0 0x560739a2c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 -- 172.21.15.19:0/2698064658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 msgr2=0x560739a2bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a2bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 -- 172.21.15.19:0/2698064658 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560739a9c260 con 0x5607396542f0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc0854dc640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a2f400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0780106a0 tx=0x7fc078000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ebbc71019256ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc05ffff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc05ffff640 1 -- 172.21.15.19:0/2698064658 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07803e040 con 0x5607396542f0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc05ffff640 1 -- 172.21.15.19:0/2698064658 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0780184b0 con 0x5607396542f0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.042+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56073998c360 con 0x5607396542f0 2024-05-17T09:38:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc05ffff640 1 -- 172.21.15.19:0/2698064658 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc078018930 con 0x5607396542f0 2024-05-17T09:38:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 10 client.27820 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560739a9d350 con 0x5607396542f0 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc05ffff640 1 -- 172.21.15.19:0/2698064658 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56073998c360 con 0x5607396542f0 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc05ffff640 1 -- 172.21.15.19:0/2698064658 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc078020030 con 0x5607396542f0 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 10 client.27820 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 10 client.27820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 20 client.27820 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5607396111c0 0x560739a9fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 4 client.27820 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607399065f0 con 0x5607396111c0 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.043+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5607396111c0 0x560739a9fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.044+0000 7fc05ffff640 10 client.27820 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.044+0000 7fc07ffff640 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5607396111c0 0x560739a9fbd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b5dbf7be24495f server_cookie=b71f50b9f560537 in_seq=0 out_seq=0 2024-05-17T09:38:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.132+0000 7fc05ffff640 1 -- 172.21.15.19:0/2698064658 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607399065f0 con 0x5607396111c0 2024-05-17T09:38:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.132+0000 7fc05ffff640 10 client.27820 handle_command_reply: tid=0 2024-05-17T09:38:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.132+0000 7fc08ddf5740 1 client.27820 shutdown 2024-05-17T09:38:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.133+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 msgr2=0x560739a2f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.133+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a2f400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0780106a0 tx=0x7fc078000f40 comp rx=0 tx=0).stop 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 shutdown_connections 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5607396111c0 0x560739a9fbd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56073967d1b0 0x560739a2c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc05ffff640 0 client.27820 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56073965c4d0 0x560739a2bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 --2- 172.21.15.19:0/2698064658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607396542f0 0x560739a2f400 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 >> 172.21.15.19:0/2698064658 conn(0x560739671300 msgr2=0x560739a18d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 shutdown_connections 2024-05-17T09:38:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 1 -- 172.21.15.19:0/2698064658 wait complete. 2024-05-17T09:38:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.134+0000 7fc08ddf5740 20 client.27820 trim_cache size 0 max 16384 2024-05-17T09:38:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:38:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:38:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc06c0b2920 msgr2=0x7fc06c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc06c0b2920 0x7fc06c0b4e30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560739a1f780 tx=0x7fc0680c6040 comp rx=0 tx=0).stop 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 msgr2=0x7fc08019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 0x7fc08019c3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e980 tx=0x7fc06800ee40 comp rx=0 tx=0).stop 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 shutdown_connections 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc06c0b2920 0x7fc06c0b4e30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc080108a80 0x7fc0801a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc080103d80 0x7fc08019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 --2- 172.21.15.19:0/1708941127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc080102b40 0x7fc08019c3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 >> 172.21.15.19:0/1708941127 conn(0x7fc0800fdd30 msgr2=0x7fc0800fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 shutdown_connections 2024-05-17T09:38:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:11.135+0000 7fc086f3f640 1 -- 172.21.15.19:0/1708941127 wait complete. 2024-05-17T09:38:11.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31739 7.13 MB/sec execute 128 sec latency 136.778 ms 2024-05-17T09:38:12.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31877 7.13 MB/sec execute 129 sec latency 155.929 ms 2024-05-17T09:38:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:12 smithi079 ceph-mon[35502]: pgmap v991: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.2 MiB/s wr, 230 op/s 2024-05-17T09:38:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:12 smithi069 ceph-mon[35081]: pgmap v991: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.2 MiB/s wr, 230 op/s 2024-05-17T09:38:12.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:12 smithi019 ceph-mon[31620]: pgmap v991: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.2 MiB/s wr, 230 op/s 2024-05-17T09:38:13.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 31991 7.10 MB/sec execute 130 sec latency 207.878 ms 2024-05-17T09:38:14.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 32157 7.11 MB/sec execute 131 sec latency 109.630 ms 2024-05-17T09:38:14.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:14 smithi079 ceph-mon[35502]: pgmap v992: 129 pgs: 129 active+clean; 377 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-05-17T09:38:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:14 smithi069 ceph-mon[35081]: pgmap v992: 129 pgs: 129 active+clean; 377 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-05-17T09:38:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:14 smithi019 ceph-mon[31620]: pgmap v992: 129 pgs: 129 active+clean; 377 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-05-17T09:38:15.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 32291 7.09 MB/sec execute 132 sec latency 161.128 ms 2024-05-17T09:38:16.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 32449 7.10 MB/sec execute 133 sec latency 106.192 ms 2024-05-17T09:38:16.594 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:16 smithi019 ceph-mon[31620]: pgmap v993: 129 pgs: 129 active+clean; 373 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-05-17T09:38:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:16 smithi079 ceph-mon[35502]: pgmap v993: 129 pgs: 129 active+clean; 373 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-05-17T09:38:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 -- 172.21.15.19:0/2477280721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241089e0 msgr2=0x7fe824108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 --2- 172.21.15.19:0/2477280721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241089e0 0x7fe824108df0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe814009990 tx=0x7fe814009ca0 comp rx=0 tx=0).stop 2024-05-17T09:38:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 -- 172.21.15.19:0/2477280721 shutdown_connections 2024-05-17T09:38:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 --2- 172.21.15.19:0/2477280721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 0x7fe82410f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 --2- 172.21.15.19:0/2477280721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241089e0 0x7fe824108df0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 --2- 172.21.15.19:0/2477280721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241029b0 0x7fe824102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 -- 172.21.15.19:0/2477280721 >> 172.21.15.19:0/2477280721 conn(0x7fe8240fe6e0 msgr2=0x7fe824100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 -- 172.21.15.19:0/2477280721 shutdown_connections 2024-05-17T09:38:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.898+0000 7fe82a361700 1 -- 172.21.15.19:0/2477280721 wait complete. 2024-05-17T09:38:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.900+0000 7fe82a361700 1 Processor -- start 2024-05-17T09:38:16.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.900+0000 7fe82a361700 1 -- start start 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe82a361700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 0x7fe82419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe82a361700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 0x7fe82419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe82a361700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241089e0 0x7fe8241a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe823fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 0x7fe82419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe82a361700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe824114370 con 0x7fe8241029b0 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe823fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 0x7fe82419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44658/0 (socket says 172.21.15.19:44658) 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe823fff700 1 -- 172.21.15.19:0/197425154 learned_addr learned my addr 172.21.15.19:0/197425154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe8288fe700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241089e0 0x7fe8241a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:16.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.901+0000 7fe8237fe700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 0x7fe82419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241141f0 con 0x7fe8241089e0 2024-05-17T09:38:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241144f0 con 0x7fe824103300 2024-05-17T09:38:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe8237fe700 1 -- 172.21.15.19:0/197425154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241089e0 msgr2=0x7fe8241a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe8237fe700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241089e0 0x7fe8241a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe8237fe700 1 -- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 msgr2=0x7fe82419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe8237fe700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 0x7fe82419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe8237fe700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8241a1cb0 con 0x7fe824103300 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe823fff700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 0x7fe82419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.902+0000 7fe8288fe700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241089e0 0x7fe8241a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.903+0000 7fe8237fe700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 0x7fe82419d380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe81800e9c0 tx=0x7fe81800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.903+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81800c9c0 con 0x7fe824103300 2024-05-17T09:38:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.903+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe818010430 con 0x7fe824103300 2024-05-17T09:38:16.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.903+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818018680 con 0x7fe824103300 2024-05-17T09:38:16.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.903+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8241a1fa0 con 0x7fe824103300 2024-05-17T09:38:16.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.904+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8241a98c0 con 0x7fe824103300 2024-05-17T09:38:16.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.905+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8240512b0 con 0x7fe824103300 2024-05-17T09:38:16.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.907+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe8180105e0 con 0x7fe824103300 2024-05-17T09:38:16.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.908+0000 7fe8217fa700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe80c0b2900 0x7fe80c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:16.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.909+0000 7fe823fff700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe80c0b2900 0x7fe80c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:16.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.909+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe8180e2a00 con 0x7fe824103300 2024-05-17T09:38:16.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.909+0000 7fe823fff700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe80c0b2900 0x7fe80c0b4e10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe814006010 tx=0x7fe81400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:16.912+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8180aa310 con 0x7fe824103300 2024-05-17T09:38:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:16 smithi069 ceph-mon[35081]: pgmap v993: 129 pgs: 129 active+clean; 373 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-05-17T09:38:17.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.358+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe82419e140 con 0x7fe824103300 2024-05-17T09:38:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.359+0000 7fe8217fa700 1 -- 172.21.15.19:0/197425154 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fe8180a9c10 con 0x7fe824103300 2024-05-17T09:38:17.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:38:17.359 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:38:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.362+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe80c0b2900 msgr2=0x7fe80c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.362+0000 7fe82a361700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe80c0b2900 0x7fe80c0b4e10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe814006010 tx=0x7fe81400b610 comp rx=0 tx=0).stop 2024-05-17T09:38:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.362+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 msgr2=0x7fe82419d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.362+0000 7fe82a361700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 0x7fe82419d380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe81800e9c0 tx=0x7fe81800ecd0 comp rx=0 tx=0).stop 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 shutdown_connections 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8241089e0 0x7fe8241a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8241029b0 0x7fe82419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe80c0b2900 0x7fe80c0b4e10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 --2- 172.21.15.19:0/197425154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe824103300 0x7fe82419d380 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 >> 172.21.15.19:0/197425154 conn(0x7fe8240fe6e0 msgr2=0x7fe82410dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 shutdown_connections 2024-05-17T09:38:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:17.363+0000 7fe82a361700 1 -- 172.21.15.19:0/197425154 wait complete. 2024-05-17T09:38:17.364 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:38:17.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 32593 7.11 MB/sec execute 134 sec latency 78.971 ms 2024-05-17T09:38:17.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:17 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/197425154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:17.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:17 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/197425154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:17.828 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:17 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/197425154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:18.065 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-05-17T09:38:18.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-05-17T09:38:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:18.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 32704 7.09 MB/sec execute 135 sec latency 73.441 ms 2024-05-17T09:38:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:18 smithi079 ceph-mon[35502]: pgmap v994: 129 pgs: 129 active+clean; 378 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-05-17T09:38:18.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:18 smithi069 ceph-mon[35081]: pgmap v994: 129 pgs: 129 active+clean; 378 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-05-17T09:38:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:18 smithi019 ceph-mon[31620]: pgmap v994: 129 pgs: 129 active+clean; 378 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-05-17T09:38:19.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 32839 7.09 MB/sec execute 136 sec latency 126.042 ms 2024-05-17T09:38:20.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:20 smithi019 ceph-mon[31620]: pgmap v995: 129 pgs: 129 active+clean; 381 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.1 MiB/s wr, 243 op/s 2024-05-17T09:38:20.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33055 7.21 MB/sec execute 137 sec latency 88.995 ms 2024-05-17T09:38:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 -- 172.21.15.19:0/3554169754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108f10 msgr2=0x7f3a4c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/3554169754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108f10 0x7f3a4c109320 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099c0 tx=0x7f3a3c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 -- 172.21.15.19:0/3554169754 shutdown_connections 2024-05-17T09:38:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/3554169754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a4c109860 0x7f3a4c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/3554169754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c1045e0 0x7f3a4c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/3554169754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108f10 0x7f3a4c109320 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 -- 172.21.15.19:0/3554169754 >> 172.21.15.19:0/3554169754 conn(0x7f3a4c0fffd0 msgr2=0x7f3a4c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 -- 172.21.15.19:0/3554169754 shutdown_connections 2024-05-17T09:38:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.682+0000 7f3a53b5a700 1 -- 172.21.15.19:0/3554169754 wait complete. 2024-05-17T09:38:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.683+0000 7f3a53b5a700 1 Processor -- start 2024-05-17T09:38:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.684+0000 7f3a53b5a700 1 -- start start 2024-05-17T09:38:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.684+0000 7f3a53b5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1045e0 0x7f3a4c10e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.684+0000 7f3a53b5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 0x7f3a4c10e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a53b5a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a4c109860 0x7f3a4c10eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a53b5a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c0773d0 con 0x7f3a4c1045e0 2024-05-17T09:38:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a53b5a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c077250 con 0x7f3a4c108f10 2024-05-17T09:38:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a53b5a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c077550 con 0x7f3a4c109860 2024-05-17T09:38:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 0x7f3a4c10e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 0x7f3a4c10e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:49684/0 (socket says 172.21.15.19:49684) 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 -- 172.21.15.19:0/1659859167 learned_addr learned my addr 172.21.15.19:0/1659859167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a520f7700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a4c109860 0x7f3a4c10eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 -- 172.21.15.19:0/1659859167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a4c109860 msgr2=0x7f3a4c10eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a4c109860 0x7f3a4c10eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 -- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1045e0 msgr2=0x7f3a4c10e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1045e0 0x7f3a4c10e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 -- 172.21.15.19:0/1659859167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c1ad2c0 con 0x7f3a4c108f10 2024-05-17T09:38:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.685+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 0x7f3a4c10e9a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0045f0 tx=0x7f3a3c004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.686+0000 7f3a42ffd700 1 -- 172.21.15.19:0/1659859167 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c026070 con 0x7f3a4c108f10 2024-05-17T09:38:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.686+0000 7f3a42ffd700 1 -- 172.21.15.19:0/1659859167 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a3c00f460 con 0x7f3a4c108f10 2024-05-17T09:38:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.686+0000 7f3a42ffd700 1 -- 172.21.15.19:0/1659859167 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c009e60 con 0x7f3a4c108f10 2024-05-17T09:38:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.686+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c1ad550 con 0x7f3a4c108f10 2024-05-17T09:38:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.686+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1ad950 con 0x7f3a4c108f10 2024-05-17T09:38:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.690+0000 7f3a42ffd700 1 -- 172.21.15.19:0/1659859167 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3a3c0039d0 con 0x7f3a4c108f10 2024-05-17T09:38:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.692+0000 7f3a42ffd700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3a380b2900 0x7f3a380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.692+0000 7f3a42ffd700 1 -- 172.21.15.19:0/1659859167 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3a3c0e3120 con 0x7f3a4c108f10 2024-05-17T09:38:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.692+0000 7f3a518f6700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3a380b2900 0x7f3a380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.693+0000 7f3a518f6700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3a380b2900 0x7f3a380b4e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005d90 tx=0x7f3a48005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.693+0000 7f3a6256fb80 1 Processor -- start 2024-05-17T09:38:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.693+0000 7f3a6256fb80 1 -- start start 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d6805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d680b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d67aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b06d5a8e20 con 0x55b06d579560 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d6805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b06d5989c0 con 0x55b06d4fbfd0 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b06d59ca00 con 0x55b06d67a6b0 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d6805f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44684/0 (socket says 172.21.15.19:44684) 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a518f6700 1 -- 172.21.15.19:0/3263897721 learned_addr learned my addr 172.21.15.19:0/3263897721 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a510f5700 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d680b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a520f7700 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d67aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.694+0000 7f3a510f5700 1 -- 172.21.15.19:0/3263897721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 msgr2=0x55b06d67aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a510f5700 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d67aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a510f5700 1 -- 172.21.15.19:0/3263897721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 msgr2=0x55b06d6805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a510f5700 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d6805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a510f5700 1 -- 172.21.15.19:0/3263897721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06d4e8a60 con 0x55b06d4fbfd0 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a520f7700 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d67aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a510f5700 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d680b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0e0870 tx=0x7f3a3c0aa2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8383beb26eadd03 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a37fff700 1 -- 172.21.15.19:0/3263897721 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0a9f30 con 0x55b06d4fbfd0 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a37fff700 1 -- 172.21.15.19:0/3263897721 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a3c0aab20 con 0x55b06d4fbfd0 2024-05-17T09:38:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.695+0000 7f3a37fff700 1 -- 172.21.15.19:0/3263897721 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0aa680 con 0x55b06d4fbfd0 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 -- 172.21.15.19:0/3263897721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 msgr2=0x55b06d680b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d680b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0e0870 tx=0x7f3a3c0aa2c0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 -- 172.21.15.19:0/3263897721 shutdown_connections 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d67aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d6805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/3263897721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d680b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 -- 172.21.15.19:0/3263897721 >> 172.21.15.19:0/3263897721 conn(0x55b06d53ae10 msgr2=0x55b06d6755b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 -- 172.21.15.19:0/3263897721 shutdown_connections 2024-05-17T09:38:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.697+0000 7f3a6256fb80 1 -- 172.21.15.19:0/3263897721 wait complete. 2024-05-17T09:38:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.698+0000 7f3a6256fb80 1 -- start start 2024-05-17T09:38:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.698+0000 7f3a6256fb80 1 Processor -- start 2024-05-17T09:38:20.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d689d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d68a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d684100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b06d49f130 con 0x55b06d579560 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b06d53a950 con 0x55b06d4fbfd0 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b06d67b2f0 con 0x55b06d67a6b0 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d689d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a6256fb80 10 client.0 _ll_register_callbacks cb 0x55b06d533ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a510f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d68a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a510f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d68a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44694/0 (socket says 172.21.15.19:44694) 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a510f5700 1 -- 172.21.15.19:0/1238301401 learned_addr learned my addr 172.21.15.19:0/1238301401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.699+0000 7f3a520f7700 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d684100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a510f5700 1 -- 172.21.15.19:0/1238301401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 msgr2=0x55b06d684100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d684100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a510f5700 1 -- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 msgr2=0x55b06d689d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d689d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a510f5700 1 -- 172.21.15.19:0/1238301401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06d537820 con 0x55b06d579560 2024-05-17T09:38:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d68a240 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0e0be0 tx=0x7f3a3c092070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4718bf3a3af8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a357fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a357fa700 1 -- 172.21.15.19:0/1238301401 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0ae7d0 con 0x55b06d579560 2024-05-17T09:38:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a357fa700 1 -- 172.21.15.19:0/1238301401 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3a3c00f610 con 0x55b06d579560 2024-05-17T09:38:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.700+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3c0aab20 con 0x55b06d579560 2024-05-17T09:38:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a357fa700 1 -- 172.21.15.19:0/1238301401 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a3c0b31f0 con 0x55b06d579560 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a6256fb80 10 client.18315 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b06d691920 con 0x55b06d579560 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a357fa700 1 -- 172.21.15.19:0/1238301401 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a3c0cb1a0 con 0x55b06d579560 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a357fa700 1 -- 172.21.15.19:0/1238301401 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3a3c084c10 con 0x55b06d579560 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a6256fb80 10 client.18315 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a6256fb80 10 client.18315 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.701+0000 7f3a6256fb80 20 client.18315 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.702+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d693df0 0x55b06d696210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.702+0000 7f3a6256fb80 4 client.18315 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.703+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3a3c0aab20 con 0x55b06d693df0 2024-05-17T09:38:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.703+0000 7f3a518f6700 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d693df0 0x55b06d696210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.703+0000 7f3a518f6700 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d693df0 0x55b06d696210 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb0b9aace188d90 server_cookie=f876ab4e8bfa8130 in_seq=0 out_seq=0 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.703+0000 7f3a357fa700 10 client.18315 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a357fa700 1 -- 172.21.15.19:0/1238301401 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f3a3c0aab20 con 0x55b06d693df0 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a357fa700 10 client.18315 handle_command_reply: tid=0 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a6256fb80 1 client.18315 shutdown 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 msgr2=0x55b06d68a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d68a240 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0e0be0 tx=0x7f3a3c092070 comp rx=0 tx=0).stop 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 shutdown_connections 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.704+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d684100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d693df0 0x55b06d696210 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d4fbfd0 0x55b06d689d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1238301401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d68a240 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a357fa700 0 client.18315 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 >> 172.21.15.19:0/1238301401 conn(0x55b06d53ae10 msgr2=0x55b06d66f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 shutdown_connections 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1238301401 wait complete. 2024-05-17T09:38:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.705+0000 7f3a6256fb80 20 client.18315 trim_cache size 0 max 16384 2024-05-17T09:38:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 Processor -- start 2024-05-17T09:38:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 -- start start 2024-05-17T09:38:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 0x55b06d67e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d681600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d67a6b0 0x55b06d681b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b8e40 con 0x55b06d579560 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b66d0 con 0x55b06d67a6b0 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a6256fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b4520 con 0x55b06d4fbfd0 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 0x55b06d67e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 0x55b06d67e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:37618/0 (socket says 172.21.15.19:37618) 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a518f6700 1 -- 172.21.15.19:0/1740213539 learned_addr learned my addr 172.21.15.19:0/1740213539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a520f7700 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d67a6b0 0x55b06d681b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.734+0000 7f3a510f5700 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d681600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a520f7700 1 -- 172.21.15.19:0/1740213539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 msgr2=0x55b06d67e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a520f7700 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 0x55b06d67e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a520f7700 1 -- 172.21.15.19:0/1740213539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 msgr2=0x55b06d681600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a520f7700 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d681600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a520f7700 1 -- 172.21.15.19:0/1740213539 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06d6764b0 con 0x55b06d67a6b0 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a518f6700 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 0x55b06d67e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a520f7700 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d67a6b0 0x55b06d681b40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009540 tx=0x7f3a4400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a20eb11e5cf8c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a377fe700 1 -- 172.21.15.19:0/1740213539 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44004db0 con 0x55b06d67a6b0 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a377fe700 1 -- 172.21.15.19:0/1740213539 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a4401b070 con 0x55b06d67a6b0 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.735+0000 7f3a377fe700 1 -- 172.21.15.19:0/1740213539 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44005020 con 0x55b06d67a6b0 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.736+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1740213539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d67a6b0 msgr2=0x55b06d681b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.736+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d67a6b0 0x55b06d681b40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009540 tx=0x7f3a4400b040 comp rx=0 tx=0).stop 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.736+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1740213539 shutdown_connections 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d4fbfd0 0x55b06d67e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d67a6b0 0x55b06d681b40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/1740213539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d579560 0x55b06d681600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1740213539 >> 172.21.15.19:0/1740213539 conn(0x55b06d53ae10 msgr2=0x55b06d67df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:20.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1740213539 shutdown_connections 2024-05-17T09:38:20.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 -- 172.21.15.19:0/1740213539 wait complete. 2024-05-17T09:38:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 -- start start 2024-05-17T09:38:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.737+0000 7f3a6256fb80 1 Processor -- start 2024-05-17T09:38:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 0x55b06d690190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d579560 0x55b06d6906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d68e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 0x55b06d690190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a518f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 0x55b06d690190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44714/0 (socket says 172.21.15.19:44714) 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a518f6700 1 -- 172.21.15.19:0/2042071277 learned_addr learned my addr 172.21.15.19:0/2042071277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b4b90 con 0x55b06d4fbfd0 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b6000 con 0x55b06d579560 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b5260 con 0x55b06d67a6b0 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a6256fb80 10 client.0 _ll_register_callbacks cb 0x55b06d533ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a510f5700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d579560 0x55b06d6906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.738+0000 7f3a520f7700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d68e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a518f6700 1 -- 172.21.15.19:0/2042071277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 msgr2=0x55b06d68e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a518f6700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d68e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a518f6700 1 -- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d579560 msgr2=0x55b06d6906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a518f6700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d579560 0x55b06d6906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a518f6700 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06d66ef70 con 0x55b06d4fbfd0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a510f5700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d579560 0x55b06d6906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a518f6700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 0x55b06d690190 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800edb0 tx=0x7f3a48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fa9fefe24dcd84 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a357fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a357fa700 1 -- 172.21.15.19:0/2042071277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4802c070 con 0x55b06d4fbfd0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a357fa700 1 -- 172.21.15.19:0/2042071277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3a48010ad0 con 0x55b06d4fbfd0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.739+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a4401b070 con 0x55b06d4fbfd0 2024-05-17T09:38:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a357fa700 1 -- 172.21.15.19:0/2042071277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a4803a050 con 0x55b06d4fbfd0 2024-05-17T09:38:20.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 10 client.18321 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:20.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b06d6ff050 con 0x55b06d4fbfd0 2024-05-17T09:38:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a357fa700 1 -- 172.21.15.19:0/2042071277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a4401b070 con 0x55b06d4fbfd0 2024-05-17T09:38:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a357fa700 1 -- 172.21.15.19:0/2042071277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3a48031480 con 0x55b06d4fbfd0 2024-05-17T09:38:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 10 client.18321 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 10 client.18321 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:38:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 20 client.18321 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d6ff800 0x55b06d701c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.740+0000 7f3a6256fb80 4 client.18321 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-05-17T09:38:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.741+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x55b06d5aad70 con 0x55b06d6ff800 2024-05-17T09:38:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.741+0000 7f3a510f5700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d6ff800 0x55b06d701c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.741+0000 7f3a357fa700 10 client.18321 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:20.741+0000 7f3a510f5700 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d6ff800 0x55b06d701c20 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6929ea93d928fd40 server_cookie=8c7d33ddadbdb22d in_seq=0 out_seq=0 2024-05-17T09:38:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:20 smithi079 ceph-mon[35502]: pgmap v995: 129 pgs: 129 active+clean; 381 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.1 MiB/s wr, 243 op/s 2024-05-17T09:38:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:20 smithi069 ceph-mon[35081]: pgmap v995: 129 pgs: 129 active+clean; 381 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.1 MiB/s wr, 243 op/s 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a357fa700 1 -- 172.21.15.19:0/2042071277 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b06d5aad70 con 0x55b06d6ff800 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a357fa700 10 client.18321 handle_command_reply: tid=0 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 client.18321 shutdown 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 msgr2=0x55b06d690190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 0x55b06d690190 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800edb0 tx=0x7f3a48009260 comp rx=0 tx=0).stop 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 shutdown_connections 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b06d67a6b0 0x55b06d68e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b06d6ff800 0x55b06d701c20 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b06d4fbfd0 0x55b06d690190 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 --2- 172.21.15.19:0/2042071277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b06d579560 0x55b06d6906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 >> 172.21.15.19:0/2042071277 conn(0x55b06d53ae10 msgr2=0x55b06d66faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.201+0000 7f3a357fa700 0 client.18321 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 shutdown_connections 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a6256fb80 1 -- 172.21.15.19:0/2042071277 wait complete. 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a6256fb80 20 client.18321 trim_cache size 0 max 16384 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:38:21.203 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 62, 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "2d55d7de": { 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 62, 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.2, 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:38:21.204 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:21.205 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3a380b2900 msgr2=0x7f3a380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3a380b2900 0x7f3a380b4e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005d90 tx=0x7f3a48005ce0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 msgr2=0x7f3a4c10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 0x7f3a4c10e9a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0045f0 tx=0x7f3a3c004620 comp rx=0 tx=0).stop 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 shutdown_connections 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3a380b2900 0x7f3a380b4e10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a4c109860 0x7f3a4c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1045e0 0x7f3a4c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 --2- 172.21.15.19:0/1659859167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a4c108f10 0x7f3a4c10e9a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 >> 172.21.15.19:0/1659859167 conn(0x7f3a4c0fffd0 msgr2=0x7f3a4c106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.202+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 shutdown_connections 2024-05-17T09:38:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:21.203+0000 7f3a53b5a700 1 -- 172.21.15.19:0/1659859167 wait complete. 2024-05-17T09:38:21.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 7.18 MB/sec execute 138 sec latency 831.731 ms 2024-05-17T09:38:21.903 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2d55d7de', quiesce took 15.3 seconds. Will release after: 6.899999999999999 2024-05-17T09:38:22.573 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 7.13 MB/sec execute 139 sec latency 1831.838 ms 2024-05-17T09:38:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:22 smithi079 ceph-mon[35502]: pgmap v996: 129 pgs: 129 active+clean; 382 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-05-17T09:38:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:22 smithi069 ceph-mon[35081]: pgmap v996: 129 pgs: 129 active+clean; 382 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-05-17T09:38:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:22 smithi019 ceph-mon[31620]: pgmap v996: 129 pgs: 129 active+clean; 382 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-05-17T09:38:23.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 7.08 MB/sec execute 140 sec latency 2831.874 ms 2024-05-17T09:38:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 -- 172.21.15.19:0/2260950161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 msgr2=0x7fe2dc103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/2260950161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc103840 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009900 tx=0x7fe2cc02a410 comp rx=0 tx=0).stop 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 -- 172.21.15.19:0/2260950161 shutdown_connections 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/2260950161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2dc108a80 0x7fe2dc10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/2260950161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2dc103d80 0x7fe2dc104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/2260950161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc103840 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.241+0000 7fe2e22e0640 1 -- 172.21.15.19:0/2260950161 >> 172.21.15.19:0/2260950161 conn(0x7fe2dc0fdd30 msgr2=0x7fe2dc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 -- 172.21.15.19:0/2260950161 shutdown_connections 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 -- 172.21.15.19:0/2260950161 wait complete. 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 Processor -- start 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 -- start start 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2dc103d80 0x7fe2dc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2dc108a80 0x7fe2dc1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc10e4f0 con 0x7fe2dc102b40 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc10e370 con 0x7fe2dc103d80 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.242+0000 7fe2e22e0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc10e670 con 0x7fe2dc108a80 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48522/0 (socket says 172.21.15.19:48522) 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 -- 172.21.15.19:0/1245712301 learned_addr learned my addr 172.21.15.19:0/1245712301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2e087d640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2dc108a80 0x7fe2dc1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2dc103d80 0x7fe2dc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 -- 172.21.15.19:0/1245712301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2dc108a80 msgr2=0x7fe2dc1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2dc108a80 0x7fe2dc1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 -- 172.21.15.19:0/1245712301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2dc103d80 msgr2=0x7fe2dc199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2dc103d80 0x7fe2dc199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 -- 172.21.15.19:0/1245712301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a0be0 con 0x7fe2dc102b40 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc199150 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc005760 tx=0x7fe2cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2d9ffb640 1 -- 172.21.15.19:0/1245712301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc03f030 con 0x7fe2dc102b40 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2d9ffb640 1 -- 172.21.15.19:0/1245712301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2cc003a80 con 0x7fe2dc102b40 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2d9ffb640 1 -- 172.21.15.19:0/1245712301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc035070 con 0x7fe2dc102b40 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a0e70 con 0x7fe2dc102b40 2024-05-17T09:38:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.243+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a1270 con 0x7fe2dc102b40 2024-05-17T09:38:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.246+0000 7fe2e9198740 1 Processor -- start 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 -- start start 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a3031b0 0x55664a6a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a6a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55664a616c30 con 0x55664a2e24d0 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55664a6067d0 con 0x55664a3031b0 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e9198740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55664a60a810 con 0x55664a2da2f0 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d9ffb640 1 -- 172.21.15.19:0/1245712301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe2cc042030 con 0x7fe2dc102b40 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48528/0 (socket says 172.21.15.19:48528) 2024-05-17T09:38:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 -- 172.21.15.19:0/1846590658 learned_addr learned my addr 172.21.15.19:0/1846590658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a6a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2e087d640 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a3031b0 0x55664a6a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 -- 172.21.15.19:0/1846590658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 msgr2=0x55664a6a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a6a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 -- 172.21.15.19:0/1846590658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a3031b0 msgr2=0x55664a6a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.247+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a3031b0 0x55664a6a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d3fff640 1 -- 172.21.15.19:0/1846590658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664a31b210 con 0x55664a2e24d0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2e087d640 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a3031b0 0x55664a6a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6ada30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4004650 tx=0x7fe2c400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703a1a852c5f74ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d27fc640 1 -- 172.21.15.19:0/1846590658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4003bd0 con 0x55664a2e24d0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d27fc640 1 -- 172.21.15.19:0/1846590658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2c4015070 con 0x55664a2e24d0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d27fc640 1 -- 172.21.15.19:0/1846590658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4004c10 con 0x55664a2e24d0 2024-05-17T09:38:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d9ffb640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2c80b28a0 0x7fe2c80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2e087d640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2c80b28a0 0x7fe2c80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.248+0000 7fe2d9ffb640 1 -- 172.21.15.19:0/1245712301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe2cc102c00 con 0x7fe2dc102b40 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 -- 172.21.15.19:0/1846590658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 msgr2=0x55664a6ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6ada30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4004650 tx=0x7fe2c400bba0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e087d640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2c80b28a0 0x7fe2c80b4db0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55664a6a7780 tx=0x7fe2d4007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 -- 172.21.15.19:0/1846590658 shutdown_connections 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6ada30 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a6a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 --2- 172.21.15.19:0/1846590658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a3031b0 0x55664a6a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 -- 172.21.15.19:0/1846590658 >> 172.21.15.19:0/1846590658 conn(0x55664a2f7300 msgr2=0x55664a69fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 -- 172.21.15.19:0/1846590658 shutdown_connections 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 -- 172.21.15.19:0/1846590658 wait complete. 2024-05-17T09:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.249+0000 7fe2e9198740 1 -- start start 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 Processor -- start 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 0x55664a6aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2e24d0 0x55664a6b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a3031b0 0x55664a6b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55664a2ec790 con 0x55664a2da2f0 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55664a5c2890 con 0x55664a2e24d0 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55664a6ae480 con 0x55664a3031b0 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e9198740 10 client.0 _ll_register_callbacks cb 0x55664a5c5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 0x55664a6aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 0x55664a6aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48532/0 (socket says 172.21.15.19:48532) 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 -- 172.21.15.19:0/3937652756 learned_addr learned my addr 172.21.15.19:0/3937652756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2e087d640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a3031b0 0x55664a6b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2e24d0 0x55664a6b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 -- 172.21.15.19:0/3937652756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a3031b0 msgr2=0x55664a6b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a3031b0 0x55664a6b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 -- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2e24d0 msgr2=0x55664a6b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2e24d0 0x55664a6b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.250+0000 7fe2dbfff640 1 -- 172.21.15.19:0/3937652756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664a3e7620 con 0x55664a2da2f0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 0x55664a6aff20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc032fd0 tx=0x7fe2cc0c5de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb38415287d17c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3937652756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc049040 con 0x55664a2da2f0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c4015070 con 0x55664a2da2f0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3937652756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe2cc0c61b0 con 0x55664a2da2f0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3937652756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe2cc0c9690 con 0x55664a2da2f0 2024-05-17T09:38:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 10 client.18333 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55664a6ba2f0 con 0x55664a2da2f0 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3937652756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2c4015070 con 0x55664a2da2f0 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3937652756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe2cc0ca180 con 0x55664a2da2f0 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 10 client.18333 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 10 client.18333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 20 client.18333 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.251+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a2c1d80 0x55664a6bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.252+0000 7fe2e9198740 4 client.18333 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.252+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55664a618b80 con 0x55664a2c1d80 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.252+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a2c1d80 0x55664a6bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.252+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a2c1d80 0x55664a6bdbc0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bca55f7b27f4ab server_cookie=94faa55453106c49 in_seq=0 out_seq=0 2024-05-17T09:38:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.252+0000 7fe2bb7fe640 10 client.18333 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3937652756 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55664a618b80 con 0x55664a2c1d80 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2bb7fe640 10 client.18333 handle_command_reply: tid=0 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 client.18333 shutdown 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 msgr2=0x55664a6aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 0x55664a6aff20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc032fd0 tx=0x7fe2cc0c5de0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 shutdown_connections 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a2c1d80 0x55664a6bdbc0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a3031b0 0x55664a6b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2e24d0 0x55664a6b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2bb7fe640 0 client.18333 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3937652756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2da2f0 0x55664a6aff20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 >> 172.21.15.19:0/3937652756 conn(0x55664a2f7300 msgr2=0x55664a6a2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:24.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 shutdown_connections 2024-05-17T09:38:24.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 1 -- 172.21.15.19:0/3937652756 wait complete. 2024-05-17T09:38:24.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.253+0000 7fe2e9198740 20 client.18333 trim_cache size 0 max 16384 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 Processor -- start 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 -- start start 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2f7300 0x55664a6aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2da2f0 0x55664a6b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6c2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1acfa0 con 0x55664a2e24d0 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1aa870 con 0x55664a2da2f0 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e9198740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1a86c0 con 0x55664a2f7300 2024-05-17T09:38:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6c2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48538/0 (socket says 172.21.15.19:48538) 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 -- 172.21.15.19:0/643934574 learned_addr learned my addr 172.21.15.19:0/643934574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2f7300 0x55664a6aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 -- 172.21.15.19:0/643934574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2f7300 msgr2=0x55664a6aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2f7300 0x55664a6aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 -- 172.21.15.19:0/643934574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2da2f0 msgr2=0x55664a6b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2da2f0 0x55664a6b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2e087d640 1 -- 172.21.15.19:0/643934574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664a6a6160 con 0x55664a2e24d0 2024-05-17T09:38:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.269+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2f7300 0x55664a6aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.270+0000 7fe2e087d640 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6c2370 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4003be0 tx=0x7fe2d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ae93f2642f484c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.270+0000 7fe2d1ffb640 1 -- 172.21.15.19:0/643934574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40044c0 con 0x55664a2e24d0 2024-05-17T09:38:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.270+0000 7fe2d1ffb640 1 -- 172.21.15.19:0/643934574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2d4004660 con 0x55664a2e24d0 2024-05-17T09:38:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.270+0000 7fe2d1ffb640 1 -- 172.21.15.19:0/643934574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d401d670 con 0x55664a2e24d0 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 -- 172.21.15.19:0/643934574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 msgr2=0x55664a6c2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6c2370 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4003be0 tx=0x7fe2d4014000 comp rx=0 tx=0).stop 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 -- 172.21.15.19:0/643934574 shutdown_connections 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a6c2370 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2da2f0 0x55664a6b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 --2- 172.21.15.19:0/643934574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2f7300 0x55664a6aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 -- 172.21.15.19:0/643934574 >> 172.21.15.19:0/643934574 conn(0x55664a6b0630 msgr2=0x55664a6a3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 -- 172.21.15.19:0/643934574 shutdown_connections 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 -- 172.21.15.19:0/643934574 wait complete. 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 -- start start 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.271+0000 7fe2e9198740 1 Processor -- start 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a720220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a722770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2f7300 0x55664a724cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1a8d30 con 0x55664a2e24d0 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1aa1a0 con 0x55664a2f7300 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1a9400 con 0x55664a2da2f0 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e9198740 10 client.0 _ll_register_callbacks cb 0x7fe2cc032e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a722770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a722770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48546/0 (socket says 172.21.15.19:48546) 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 -- 172.21.15.19:0/3147911105 learned_addr learned my addr 172.21.15.19:0/3147911105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2e087d640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2f7300 0x55664a724cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a720220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 -- 172.21.15.19:0/3147911105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 msgr2=0x55664a720220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a720220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 -- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2f7300 msgr2=0x55664a724cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2f7300 0x55664a724cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 -- 172.21.15.19:0/3147911105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664a725200 con 0x55664a2e24d0 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a720220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2d3fff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a722770 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4004650 tx=0x7fe2c4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ee9381f4758933 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.272+0000 7fe2bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3147911105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4004060 con 0x55664a2e24d0 2024-05-17T09:38:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3147911105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe2c4011440 con 0x55664a2e24d0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d4004660 con 0x55664a2e24d0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3147911105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe2c4036050 con 0x55664a2e24d0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 10 client.18339 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55664a7263c0 con 0x55664a2e24d0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3147911105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2d4004660 con 0x55664a2e24d0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3147911105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe2c40195e0 con 0x55664a2e24d0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 10 client.18339 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 10 client.18339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 20 client.18339 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a3031b0 0x55664a728ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 4 client.18339 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.273+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55664a612360 con 0x55664a3031b0 2024-05-17T09:38:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.274+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a3031b0 0x55664a728ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.274+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a3031b0 0x55664a728ca0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a547a9757270d331 server_cookie=52070b1dd0baeb69 in_seq=0 out_seq=0 2024-05-17T09:38:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.274+0000 7fe2bb7fe640 10 client.18339 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.338+0000 7fe2bb7fe640 1 -- 172.21.15.19:0/3147911105 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55664a612360 con 0x55664a3031b0 2024-05-17T09:38:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.338+0000 7fe2bb7fe640 10 client.18339 handle_command_reply: tid=0 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.338+0000 7fe2e9198740 1 client.18339 shutdown 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 msgr2=0x55664a722770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a722770 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4004650 tx=0x7fe2c4003ce0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 shutdown_connections 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55664a3031b0 0x55664a728ca0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55664a2f7300 0x55664a724cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55664a2e24d0 0x55664a722770 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2bb7fe640 0 client.18339 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 --2- 172.21.15.19:0/3147911105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55664a2da2f0 0x55664a720220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 >> 172.21.15.19:0/3147911105 conn(0x55664a6b0630 msgr2=0x55664a6a32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 shutdown_connections 2024-05-17T09:38:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 1 -- 172.21.15.19:0/3147911105 wait complete. 2024-05-17T09:38:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e9198740 20 client.18339 trim_cache size 0 max 16384 2024-05-17T09:38:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:38:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:38:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:38:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2c80b28a0 msgr2=0x7fe2c80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2c80b28a0 0x7fe2c80b4db0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55664a6a7780 tx=0x7fe2d4007520 comp rx=0 tx=0).stop 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 msgr2=0x7fe2dc199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.339+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc199150 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc005760 tx=0x7fe2cc004270 comp rx=0 tx=0).stop 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 shutdown_connections 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2dc108a80 0x7fe2dc1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2dc103d80 0x7fe2dc199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe2c80b28a0 0x7fe2c80b4db0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 --2- 172.21.15.19:0/1245712301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2dc102b40 0x7fe2dc199150 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 >> 172.21.15.19:0/1245712301 conn(0x7fe2dc0fdd30 msgr2=0x7fe2dc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 shutdown_connections 2024-05-17T09:38:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:24.340+0000 7fe2e22e0640 1 -- 172.21.15.19:0/1245712301 wait complete. 2024-05-17T09:38:24.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 7.03 MB/sec execute 141 sec latency 3831.943 ms 2024-05-17T09:38:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:24 smithi079 ceph-mon[35502]: pgmap v997: 129 pgs: 129 active+clean; 388 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-05-17T09:38:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:24 smithi069 ceph-mon[35081]: pgmap v997: 129 pgs: 129 active+clean; 388 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-05-17T09:38:24.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:24 smithi019 ceph-mon[31620]: pgmap v997: 129 pgs: 129 active+clean; 388 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-05-17T09:38:25.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.98 MB/sec execute 142 sec latency 4832.088 ms 2024-05-17T09:38:26.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.93 MB/sec execute 143 sec latency 5832.171 ms 2024-05-17T09:38:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:26 smithi079 ceph-mon[35502]: pgmap v998: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 3.3 MiB/s wr, 216 op/s 2024-05-17T09:38:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:26 smithi069 ceph-mon[35081]: pgmap v998: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 3.3 MiB/s wr, 216 op/s 2024-05-17T09:38:26.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:26 smithi019 ceph-mon[31620]: pgmap v998: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 3.3 MiB/s wr, 216 op/s 2024-05-17T09:38:27.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.88 MB/sec execute 144 sec latency 6832.282 ms 2024-05-17T09:38:28.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.83 MB/sec execute 145 sec latency 7832.350 ms 2024-05-17T09:38:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:28 smithi079 ceph-mon[35502]: pgmap v999: 129 pgs: 129 active+clean; 389 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-05-17T09:38:28.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:38:28.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:28 smithi019 ceph-mon[31620]: pgmap v999: 129 pgs: 129 active+clean; 389 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-05-17T09:38:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:28 smithi069 ceph-mon[35081]: pgmap v999: 129 pgs: 129 active+clean; 389 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 2.9 MiB/s wr, 192 op/s 2024-05-17T09:38:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:29.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.79 MB/sec execute 146 sec latency 8832.462 ms 2024-05-17T09:38:30.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.74 MB/sec execute 147 sec latency 9832.602 ms 2024-05-17T09:38:30.633 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:30 smithi019 ceph-mon[31620]: pgmap v1000: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-05-17T09:38:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:30 smithi079 ceph-mon[35502]: pgmap v1000: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-05-17T09:38:30.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 -- 172.21.15.19:0/441506020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 msgr2=0x7f2938109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:30.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 --2- 172.21.15.19:0/441506020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f2938109330 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f292c0099c0 tx=0x7f292c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:38:30.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 -- 172.21.15.19:0/441506020 shutdown_connections 2024-05-17T09:38:30.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 --2- 172.21.15.19:0/441506020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2938104640 0x7f2938104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:30.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 --2- 172.21.15.19:0/441506020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938109870 0x7f29381111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:30.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 --2- 172.21.15.19:0/441506020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f2938109330 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:30.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 -- 172.21.15.19:0/441506020 >> 172.21.15.19:0/441506020 conn(0x7f2938079ef0 msgr2=0x7f293807a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:30.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 -- 172.21.15.19:0/441506020 shutdown_connections 2024-05-17T09:38:30.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.896+0000 7f293cc30700 1 -- 172.21.15.19:0/441506020 wait complete. 2024-05-17T09:38:30.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 Processor -- start 2024-05-17T09:38:30.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 -- start start 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938104640 0x7f293819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f293819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2938109870 0x7f293819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2938114330 con 0x7f2938108f20 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29381141b0 con 0x7f2938104640 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.897+0000 7f293cc30700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f29381144b0 con 0x7f2938109870 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f293819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f293819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48574/0 (socket says 172.21.15.19:48574) 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 -- 172.21.15.19:0/1439410186 learned_addr learned my addr 172.21.15.19:0/1439410186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2936d9d700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2938109870 0x7f293819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:30.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f293659c700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938104640 0x7f293819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 -- 172.21.15.19:0/1439410186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2938109870 msgr2=0x7f293819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2938109870 0x7f293819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 -- 172.21.15.19:0/1439410186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938104640 msgr2=0x7f293819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938104640 0x7f293819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 -- 172.21.15.19:0/1439410186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293819fd10 con 0x7f2938108f20 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f293659c700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938104640 0x7f293819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f2935d9b700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f293819b3e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f292c00b580 tx=0x7f292c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c00f8a0 con 0x7f2938108f20 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f292c004830 con 0x7f2938108f20 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293819ffa0 con 0x7f2938108f20 2024-05-17T09:38:30.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c005350 con 0x7f2938108f20 2024-05-17T09:38:30.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.898+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29381a0420 con 0x7f2938108f20 2024-05-17T09:38:30.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.900+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29380512b0 con 0x7f2938108f20 2024-05-17T09:38:30.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.901+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f292c029060 con 0x7f2938108f20 2024-05-17T09:38:30.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.902+0000 7f29277fe700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29200b2880 0x7f29200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:30.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.902+0000 7f293659c700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29200b2880 0x7f29200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:30.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.902+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f292c0e3180 con 0x7f2938108f20 2024-05-17T09:38:30.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.903+0000 7f293659c700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29200b2880 0x7f29200b4d90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2928005950 tx=0x7f29280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:30.907+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f292c013070 con 0x7f2938108f20 2024-05-17T09:38:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:30 smithi069 ceph-mon[35081]: pgmap v1000: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-05-17T09:38:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.166+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f293802e5e0 con 0x7f2938108f20 2024-05-17T09:38:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.167+0000 7f29277fe700 1 -- 172.21.15.19:0/1439410186 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f292c0aab20 con 0x7f2938108f20 2024-05-17T09:38:31.167 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:38:31.167 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:38:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29200b2880 msgr2=0x7f29200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29200b2880 0x7f29200b4d90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2928005950 tx=0x7f29280058e0 comp rx=0 tx=0).stop 2024-05-17T09:38:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 msgr2=0x7f293819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f293819b3e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f292c00b580 tx=0x7f292c004790 comp rx=0 tx=0).stop 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 shutdown_connections 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f29200b2880 0x7f29200b4d90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2938109870 0x7f293819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2938104640 0x7f293819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 --2- 172.21.15.19:0/1439410186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2938108f20 0x7f293819b3e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.169+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 >> 172.21.15.19:0/1439410186 conn(0x7f2938079ef0 msgr2=0x7f2938102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.170+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 shutdown_connections 2024-05-17T09:38:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:31.170+0000 7f293cc30700 1 -- 172.21.15.19:0/1439410186 wait complete. 2024-05-17T09:38:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:38:31.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.69 MB/sec execute 148 sec latency 10832.648 ms 2024-05-17T09:38:31.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:31 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1439410186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:31.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:31 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1439410186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:31.893 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 2d55d7de --release --await' 2024-05-17T09:38:31.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 2d55d7de --release --await 2024-05-17T09:38:31.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:31 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1439410186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:32.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.65 MB/sec execute 149 sec latency 11832.693 ms 2024-05-17T09:38:32.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:32 smithi019 ceph-mon[31620]: pgmap v1001: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-17T09:38:32.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:32 smithi079 ceph-mon[35502]: pgmap v1001: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-17T09:38:32.831 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:32 smithi069 ceph-mon[35081]: pgmap v1001: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-05-17T09:38:33.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33092 6.60 MB/sec execute 150 sec latency 12832.820 ms 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 -- 172.21.15.19:0/475972027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910108750 msgr2=0x7fb9101100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 --2- 172.21.15.19:0/475972027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910108750 0x7fb9101100b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b0d0 tx=0x7fb90800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 -- 172.21.15.19:0/475972027 shutdown_connections 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 --2- 172.21.15.19:0/475972027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910108750 0x7fb9101100b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 --2- 172.21.15.19:0/475972027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9101038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 --2- 172.21.15.19:0/475972027 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb910107e00 0x7fb910108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 -- 172.21.15.19:0/475972027 >> 172.21.15.19:0/475972027 conn(0x7fb9100fef40 msgr2=0x7fb910101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 -- 172.21.15.19:0/475972027 shutdown_connections 2024-05-17T09:38:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.994+0000 7fb916c78700 1 -- 172.21.15.19:0/475972027 wait complete. 2024-05-17T09:38:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 Processor -- start 2024-05-17T09:38:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 -- start start 2024-05-17T09:38:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9100ff420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910107e00 0x7fb9100ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb910108750 0x7fb9100ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb910113230 con 0x7fb910107e00 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101130b0 con 0x7fb9101034d0 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb916c78700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101133b0 con 0x7fb910108750 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb914a14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9100ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb914a14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9100ff420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:44760/0 (socket says 172.21.15.19:44760) 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb914a14700 1 -- 172.21.15.19:0/132183500 learned_addr learned my addr 172.21.15.19:0/132183500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb915215700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb910108750 0x7fb9100ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.995+0000 7fb90ffff700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910107e00 0x7fb9100ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb914a14700 1 -- 172.21.15.19:0/132183500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb910108750 msgr2=0x7fb9100ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb914a14700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb910108750 0x7fb9100ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb914a14700 1 -- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910107e00 msgr2=0x7fb9100ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb914a14700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910107e00 0x7fb9100ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb914a14700 1 -- 172.21.15.19:0/132183500 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9101a9070 con 0x7fb9101034d0 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb914a14700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9100ff420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800d950 tx=0x7fb8f800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb90dffb700 1 -- 172.21.15.19:0/132183500 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8016070 con 0x7fb9101034d0 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9101a9360 con 0x7fb9101034d0 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb90dffb700 1 -- 172.21.15.19:0/132183500 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f800fd20 con 0x7fb9101034d0 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.996+0000 7fb90dffb700 1 -- 172.21.15.19:0/132183500 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800e3b0 con 0x7fb9101034d0 2024-05-17T09:38:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.997+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a98f0 con 0x7fb9101034d0 2024-05-17T09:38:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:33.999+0000 7fb90dffb700 1 -- 172.21.15.19:0/132183500 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb8f8019070 con 0x7fb9101034d0 2024-05-17T09:38:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.000+0000 7fb90dffb700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8fc0b2880 0x7fb8fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.000+0000 7fb92568db80 1 Processor -- start 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.000+0000 7fb92568db80 1 -- start start 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.000+0000 7fb90ffff700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8fc0b2880 0x7fb8fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.000+0000 7fb90dffb700 1 -- 172.21.15.19:0/132183500 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb8f80e2080 con 0x7fb9101034d0 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb92568db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc2fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb92568db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc176f40 0x5578fc2fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb92568db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc2f5620 0x5578fc2f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb92568db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578fc223d90 con 0x5578fc1f44d0 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb92568db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5578fc213930 con 0x5578fc2f5620 2024-05-17T09:38:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb92568db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5578fc217970 con 0x5578fc176f40 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc2fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb90ffff700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8fc0b2880 0x7fb8fc0b4d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb910109980 tx=0x7fb90000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc2fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60228/0 (socket says 172.21.15.19:60228) 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 -- 172.21.15.19:0/3310208928 learned_addr learned my addr 172.21.15.19:0/3310208928 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb915215700 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc2f5620 0x5578fc2f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb90ffff700 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc176f40 0x5578fc2fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 -- 172.21.15.19:0/3310208928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc176f40 msgr2=0x5578fc2fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc176f40 0x5578fc2fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 -- 172.21.15.19:0/3310208928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc2f5620 msgr2=0x5578fc2f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc2f5620 0x5578fc2f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 -- 172.21.15.19:0/3310208928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578fc1639d0 con 0x5578fc1f44d0 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.001+0000 7fb914a14700 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc2fb560 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b2f0 tx=0x7fb8f80a8e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807e493f43feb88e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.002+0000 7fb906ffd700 1 -- 172.21.15.19:0/3310208928 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8020070 con 0x5578fc1f44d0 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.002+0000 7fb906ffd700 1 -- 172.21.15.19:0/3310208928 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f80ad6a0 con 0x5578fc1f44d0 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.002+0000 7fb906ffd700 1 -- 172.21.15.19:0/3310208928 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80b1f20 con 0x5578fc1f44d0 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.002+0000 7fb92568db80 1 -- 172.21.15.19:0/3310208928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 msgr2=0x5578fc2fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.002+0000 7fb92568db80 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc2fb560 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b2f0 tx=0x7fb8f80a8e80 comp rx=0 tx=0).stop 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 -- 172.21.15.19:0/3310208928 shutdown_connections 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc2f5620 0x5578fc2f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc2fb560 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 --2- 172.21.15.19:0/3310208928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc176f40 0x5578fc2fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 -- 172.21.15.19:0/3310208928 >> 172.21.15.19:0/3310208928 conn(0x5578fc1b5d80 msgr2=0x5578fc2f0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:34.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 -- 172.21.15.19:0/3310208928 shutdown_connections 2024-05-17T09:38:34.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 -- 172.21.15.19:0/3310208928 wait complete. 2024-05-17T09:38:34.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 -- start start 2024-05-17T09:38:34.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.003+0000 7fb92568db80 1 Processor -- start 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2ff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578fc11a0a0 con 0x5578fc2f5620 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5578fc1b58c0 con 0x5578fc176f40 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5578fc2f63e0 con 0x5578fc1f44d0 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb92568db80 10 client.0 _ll_register_callbacks cb 0x5578fc1aea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60234/0 (socket says 172.21.15.19:60234) 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 -- 172.21.15.19:0/135516746 learned_addr learned my addr 172.21.15.19:0/135516746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb90ffff700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb914a14700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 -- 172.21.15.19:0/135516746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 msgr2=0x5578fc2ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 -- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 msgr2=0x5578fc2ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb915215700 1 -- 172.21.15.19:0/135516746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578fc1b2790 con 0x5578fc2f5620 2024-05-17T09:38:34.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.004+0000 7fb90ffff700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb915215700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2ffcd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb908009540 tx=0x7fb908009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641f2dd8e57d499 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb8effff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb8effff700 1 -- 172.21.15.19:0/135516746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908003c10 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb8effff700 1 -- 172.21.15.19:0/135516746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb908003db0 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f80ad6a0 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb8effff700 1 -- 172.21.15.19:0/135516746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb90802e050 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 10 client.18357 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578fc30caf0 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb8effff700 1 -- 172.21.15.19:0/135516746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f80ad6a0 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb8effff700 1 -- 172.21.15.19:0/135516746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb908011070 con 0x5578fc2f5620 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 10 client.18357 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 10 client.18357 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:38:34.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 20 client.18357 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.005+0000 7fb92568db80 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc30f0d0 0x5578fc3114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.006+0000 7fb92568db80 4 client.18357 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.006+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578fc225ce0 con 0x5578fc30f0d0 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.006+0000 7fb914a14700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc30f0d0 0x5578fc3114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.006+0000 7fb8effff700 10 client.18357 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.006+0000 7fb914a14700 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc30f0d0 0x5578fc3114f0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f55213b3e9d7fe7 server_cookie=f54ceb4744a566ec in_seq=0 out_seq=0 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb8effff700 1 -- 172.21.15.19:0/135516746 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5578fc225ce0 con 0x5578fc30f0d0 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb8effff700 10 client.18357 handle_command_reply: tid=0 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 client.18357 shutdown 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 msgr2=0x5578fc2ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2ffcd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb908009540 tx=0x7fb908009570 comp rx=0 tx=0).stop 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 shutdown_connections 2024-05-17T09:38:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc30f0d0 0x5578fc3114f0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2ffcd0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 --2- 172.21.15.19:0/135516746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 >> 172.21.15.19:0/135516746 conn(0x5578fc1b5d80 msgr2=0x5578fc2ea280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb8effff700 0 client.18357 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 shutdown_connections 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.007+0000 7fb92568db80 1 -- 172.21.15.19:0/135516746 wait complete. 2024-05-17T09:38:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.008+0000 7fb92568db80 20 client.18357 trim_cache size 0 max 16384 2024-05-17T09:38:34.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 Processor -- start 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 -- start start 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b4d70 con 0x5578fc2f5620 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b2600 con 0x5578fc176f40 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb92568db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b0450 con 0x5578fc1f44d0 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb915215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb915215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60242/0 (socket says 172.21.15.19:60242) 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.028+0000 7fb915215700 1 -- 172.21.15.19:0/27792386 learned_addr learned my addr 172.21.15.19:0/27792386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb90ffff700 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb914a14700 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb915215700 1 -- 172.21.15.19:0/27792386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 msgr2=0x5578fc2faa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb915215700 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2faa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb915215700 1 -- 172.21.15.19:0/27792386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 msgr2=0x5578fc2f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb915215700 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb915215700 1 -- 172.21.15.19:0/27792386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578fc2fd7c0 con 0x5578fc2f5620 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb90ffff700 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb915215700 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2f5040 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb9080057c0 tx=0x7fb908003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fce47e7c06de9ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb9067fc700 1 -- 172.21.15.19:0/27792386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908005c60 con 0x5578fc2f5620 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb9067fc700 1 -- 172.21.15.19:0/27792386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb908005e00 con 0x5578fc2f5620 2024-05-17T09:38:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.029+0000 7fb9067fc700 1 -- 172.21.15.19:0/27792386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908007c10 con 0x5578fc2f5620 2024-05-17T09:38:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 -- 172.21.15.19:0/27792386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 msgr2=0x5578fc2f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2f5040 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb9080057c0 tx=0x7fb908003e20 comp rx=0 tx=0).stop 2024-05-17T09:38:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 -- 172.21.15.19:0/27792386 shutdown_connections 2024-05-17T09:38:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc2f5620 0x5578fc2f5040 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc2f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 --2- 172.21.15.19:0/27792386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc1f44d0 0x5578fc2faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 -- 172.21.15.19:0/27792386 >> 172.21.15.19:0/27792386 conn(0x5578fc1b5d80 msgr2=0x5578fc300900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 -- 172.21.15.19:0/27792386 shutdown_connections 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.030+0000 7fb92568db80 1 -- 172.21.15.19:0/27792386 wait complete. 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 -- start start 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 Processor -- start 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc307120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc307660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc2f5620 0x5578fc301ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b0ac0 con 0x5578fc1f44d0 2024-05-17T09:38:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b1f30 con 0x5578fc176f40 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b1190 con 0x5578fc2f5620 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb92568db80 10 client.0 _ll_register_callbacks cb 0x5578fc1aea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc307660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc307660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60258/0 (socket says 172.21.15.19:60258) 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb90ffff700 1 -- 172.21.15.19:0/2147638530 learned_addr learned my addr 172.21.15.19:0/2147638530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.031+0000 7fb914a14700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc307120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb915215700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc2f5620 0x5578fc301ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb90ffff700 1 -- 172.21.15.19:0/2147638530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc2f5620 msgr2=0x5578fc301ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb90ffff700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc2f5620 0x5578fc301ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb90ffff700 1 -- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 msgr2=0x5578fc307120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb90ffff700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc307120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb90ffff700 1 -- 172.21.15.19:0/2147638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578fc302650 con 0x5578fc1f44d0 2024-05-17T09:38:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb914a14700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc307120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb90ffff700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc307660 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb900005950 tx=0x7fb900012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95927af1316c07c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb8effff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb8effff700 1 -- 172.21.15.19:0/2147638530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900016070 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb908005e00 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb8effff700 1 -- 172.21.15.19:0/2147638530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb900004830 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb8effff700 1 -- 172.21.15.19:0/2147638530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb900034050 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb92568db80 10 client.18363 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.032+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578fc37a5c0 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb8effff700 1 -- 172.21.15.19:0/2147638530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb908005e00 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb8effff700 1 -- 172.21.15.19:0/2147638530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb90001c730 con 0x5578fc1f44d0 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb92568db80 10 client.18363 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb92568db80 10 client.18363 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb92568db80 20 client.18363 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb92568db80 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc37aca0 0x5578fc37d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb92568db80 4 client.18363 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2d55d7de", "release": true, "await": true}] 2024-05-17T09:38:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2d55d7de", "release": true, "await": true}) -- 0x5578fc21f4c0 con 0x5578fc37aca0 2024-05-17T09:38:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.033+0000 7fb914a14700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc37aca0 0x5578fc37d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.034+0000 7fb8effff700 10 client.18363 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.034+0000 7fb914a14700 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc37aca0 0x5578fc37d0c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f725406200ba91f9 server_cookie=4db0ac385d34081f in_seq=0 out_seq=0 2024-05-17T09:38:34.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb8effff700 1 -- 172.21.15.19:0/2147638530 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5578fc21f4c0 con 0x5578fc37aca0 2024-05-17T09:38:34.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb8effff700 10 client.18363 handle_command_reply: tid=0 2024-05-17T09:38:34.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 client.18363 shutdown 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 msgr2=0x5578fc307660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc307660 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb900005950 tx=0x7fb900012070 comp rx=0 tx=0).stop 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 shutdown_connections 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5578fc2f5620 0x5578fc301ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5578fc37aca0 0x5578fc37d0c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5578fc176f40 0x5578fc307120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb8effff700 0 client.18363 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 --2- 172.21.15.19:0/2147638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578fc1f44d0 0x5578fc307660 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.058+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 >> 172.21.15.19:0/2147638530 conn(0x5578fc1b5d80 msgr2=0x5578fc2f4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.059+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 shutdown_connections 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.059+0000 7fb92568db80 1 -- 172.21.15.19:0/2147638530 wait complete. 2024-05-17T09:38:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.059+0000 7fb92568db80 20 client.18363 trim_cache size 0 max 16384 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 64, 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "2d55d7de": { 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 64, 2024-05-17T09:38:34.060 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.2, 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:34.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:34.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:34.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:34.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:34.062 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:38:34.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8fc0b2880 msgr2=0x7fb8fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:34.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8fc0b2880 0x7fb8fc0b4d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb910109980 tx=0x7fb90000a300 comp rx=0 tx=0).stop 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 msgr2=0x7fb9100ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9100ff420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800d950 tx=0x7fb8f800dc60 comp rx=0 tx=0).stop 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 shutdown_connections 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb8fc0b2880 0x7fb8fc0b4d90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb910108750 0x7fb9100ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9101034d0 0x7fb9100ff420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 --2- 172.21.15.19:0/132183500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910107e00 0x7fb9100ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 >> 172.21.15.19:0/132183500 conn(0x7fb9100fef40 msgr2=0x7fb910075d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 shutdown_connections 2024-05-17T09:38:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:34.060+0000 7fb916c78700 1 -- 172.21.15.19:0/132183500 wait complete. 2024-05-17T09:38:34.574 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33171 6.59 MB/sec execute 151 sec latency 13338.702 ms 2024-05-17T09:38:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:34 smithi079 ceph-mon[35502]: pgmap v1002: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-17T09:38:34.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:34 smithi019 ceph-mon[31620]: pgmap v1002: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-17T09:38:34.921 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2d55d7de', total seconds elapsed: 28.3 2024-05-17T09:38:34.921 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-05-17T09:38:34.921 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:38:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:34 smithi069 ceph-mon[35081]: pgmap v1002: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-05-17T09:38:35.297 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:35.575 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33377 6.62 MB/sec execute 152 sec latency 67.328 ms 2024-05-17T09:38:36.575 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33467 6.59 MB/sec execute 153 sec latency 62.784 ms 2024-05-17T09:38:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:36 smithi079 ceph-mon[35502]: pgmap v1003: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 364 KiB/s wr, 39 op/s 2024-05-17T09:38:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:36 smithi069 ceph-mon[35081]: pgmap v1003: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 364 KiB/s wr, 39 op/s 2024-05-17T09:38:36.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:36 smithi019 ceph-mon[31620]: pgmap v1003: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 364 KiB/s wr, 39 op/s 2024-05-17T09:38:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 -- 172.21.15.19:0/1009235678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824102b40 msgr2=0x7f8824103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 --2- 172.21.15.19:0/1009235678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824102b40 0x7f8824103840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8814009900 tx=0x7f881402a410 comp rx=0 tx=0).stop 2024-05-17T09:38:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 -- 172.21.15.19:0/1009235678 shutdown_connections 2024-05-17T09:38:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 --2- 172.21.15.19:0/1009235678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8824108a80 0x7f882410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 --2- 172.21.15.19:0/1009235678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824103d80 0x7f8824104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 --2- 172.21.15.19:0/1009235678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824102b40 0x7f8824103840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 -- 172.21.15.19:0/1009235678 >> 172.21.15.19:0/1009235678 conn(0x7f88240fdd30 msgr2=0x7f88241001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 -- 172.21.15.19:0/1009235678 shutdown_connections 2024-05-17T09:38:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.485+0000 7f8829b38640 1 -- 172.21.15.19:0/1009235678 wait complete. 2024-05-17T09:38:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.486+0000 7f8829b38640 1 Processor -- start 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.486+0000 7f8829b38640 1 -- start start 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8829b38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824103d80 0x7f882419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8829b38640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824108a80 0x7f882419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8829b38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 0x7f88241a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8829b38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f882410e4d0 con 0x7f882419f7a0 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8829b38640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f882410e350 con 0x7f8824103d80 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8829b38640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f882410e650 con 0x7f8824108a80 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f88237fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824103d80 0x7f882419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8823fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 0x7f88241a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8823fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 0x7f88241a1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60280/0 (socket says 172.21.15.19:60280) 2024-05-17T09:38:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8822ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824108a80 0x7f882419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8823fff640 1 -- 172.21.15.19:0/4014192719 learned_addr learned my addr 172.21.15.19:0/4014192719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f88237fe640 1 -- 172.21.15.19:0/4014192719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824108a80 msgr2=0x7f882419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f88237fe640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824108a80 0x7f882419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f88237fe640 1 -- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 msgr2=0x7f88241a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f88237fe640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 0x7f88241a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f88237fe640 1 -- 172.21.15.19:0/4014192719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88241a22f0 con 0x7f8824103d80 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.487+0000 7f8823fff640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 0x7f88241a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f8822ffd640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824108a80 0x7f882419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f88237fe640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824103d80 0x7f882419a2e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88241021e0 tx=0x7f8814004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f8820ff9640 1 -- 172.21.15.19:0/4014192719 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881403f070 con 0x7f8824103d80 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f8820ff9640 1 -- 172.21.15.19:0/4014192719 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8814003c80 con 0x7f8824103d80 2024-05-17T09:38:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88240fe140 con 0x7f8824103d80 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f8820ff9640 1 -- 172.21.15.19:0/4014192719 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814035070 con 0x7f8824103d80 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.488+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88240fe5b0 con 0x7f8824103d80 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 Processor -- start 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 -- start start 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa0264d0 0x561daa3ef820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561daa35ac30 con 0x561daa01e2f0 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561daa34a7d0 con 0x561daa0264d0 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.492+0000 7f88309ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561daa34e810 con 0x561daa0471b0 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8823fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8822ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa0264d0 0x561daa3ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8823fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46856/0 (socket says 172.21.15.19:46856) 2024-05-17T09:38:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8823fff640 1 -- 172.21.15.19:0/1612834841 learned_addr learned my addr 172.21.15.19:0/1612834841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8820ff9640 1 -- 172.21.15.19:0/4014192719 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8814043400 con 0x7f8824103d80 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8822ffd640 1 -- 172.21.15.19:0/1612834841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 msgr2=0x561daa3e83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8822ffd640 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3e83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8822ffd640 1 -- 172.21.15.19:0/1612834841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 msgr2=0x561daa3e8920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8822ffd640 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3e8920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.493+0000 7f8822ffd640 1 -- 172.21.15.19:0/1612834841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daa05f210 con 0x561daa0264d0 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.495+0000 7f8822ffd640 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa0264d0 0x561daa3ef820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8818009630 tx=0x7f8818005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f215c7591f78ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.495+0000 7f8820ff9640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f880c0b28a0 0x7f880c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.495+0000 7f8820ff9640 1 -- 172.21.15.19:0/4014192719 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8814102c20 con 0x7f8824103d80 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.496+0000 7f8823fff640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f880c0b28a0 0x7f880c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.497+0000 7f8809ffb640 1 -- 172.21.15.19:0/1612834841 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004230 con 0x561daa0264d0 2024-05-17T09:38:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.497+0000 7f8809ffb640 1 -- 172.21.15.19:0/1612834841 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88180043d0 con 0x561daa0264d0 2024-05-17T09:38:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 -- 172.21.15.19:0/1612834841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa0264d0 msgr2=0x561daa3ef820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa0264d0 0x561daa3ef820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8818009630 tx=0x7f8818005f90 comp rx=0 tx=0).stop 2024-05-17T09:38:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f8823fff640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f880c0b28a0 0x7f880c0b4db0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f881c009fd0 tx=0x7f881c009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 -- 172.21.15.19:0/1612834841 shutdown_connections 2024-05-17T09:38:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa0264d0 0x561daa3ef820 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3e8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 --2- 172.21.15.19:0/1612834841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3e83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.498+0000 7f88309ee740 1 -- 172.21.15.19:0/1612834841 >> 172.21.15.19:0/1612834841 conn(0x561daa03b300 msgr2=0x561daa3e3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.499+0000 7f88309ee740 1 -- 172.21.15.19:0/1612834841 shutdown_connections 2024-05-17T09:38:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.499+0000 7f88309ee740 1 -- 172.21.15.19:0/1612834841 wait complete. 2024-05-17T09:38:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.499+0000 7f88309ee740 1 -- start start 2024-05-17T09:38:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 Processor -- start 2024-05-17T09:38:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa0264d0 0x561daa3f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561daa030790 con 0x561daa0264d0 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561daa306890 con 0x561daa01e2f0 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561daa3f02c0 con 0x561daa0471b0 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88309ee740 10 client.0 _ll_register_callbacks cb 0x561daa309d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f8823fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88237fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88237fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:44836/0 (socket says 172.21.15.19:44836) 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f88237fe640 1 -- 172.21.15.19:0/2989174695 learned_addr learned my addr 172.21.15.19:0/2989174695 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.500+0000 7f8822ffd640 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa0264d0 0x561daa3f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88237fe640 1 -- 172.21.15.19:0/2989174695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 msgr2=0x561daa3f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88237fe640 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88237fe640 1 -- 172.21.15.19:0/2989174695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa0264d0 msgr2=0x561daa3f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88237fe640 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa0264d0 0x561daa3f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88237fe640 1 -- 172.21.15.19:0/2989174695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daa12b620 con 0x561daa01e2f0 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88237fe640 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f1cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8814005ab0 tx=0x7f88140c97e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735df34d6caabb7c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f8802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f8802ffd640 1 -- 172.21.15.19:0/2989174695 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140ca590 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f8802ffd640 1 -- 172.21.15.19:0/2989174695 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f88140c6120 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88180043d0 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f8802ffd640 1 -- 172.21.15.19:0/2989174695 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88140c9e90 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.501+0000 7f88309ee740 10 client.27902 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.502+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561daa3fbeb0 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.502+0000 7f8802ffd640 1 -- 172.21.15.19:0/2989174695 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88140ce240 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.502+0000 7f8802ffd640 1 -- 172.21.15.19:0/2989174695 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f88140d28f0 con 0x561daa01e2f0 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.502+0000 7f88309ee740 10 client.27902 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.503+0000 7f88309ee740 10 client.27902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.503+0000 7f88309ee740 20 client.27902 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.503+0000 7f88309ee740 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa005d80 0x561daa3ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.503+0000 7f88309ee740 4 client.27902 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.503+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88180043d0 con 0x561daa005d80 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.503+0000 7f8822ffd640 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa005d80 0x561daa3ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.504+0000 7f8802ffd640 10 client.27902 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.504+0000 7f8822ffd640 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa005d80 0x561daa3ff610 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8da29bb9274786 server_cookie=56fec83f861fc0b0 in_seq=0 out_seq=0 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f8802ffd640 1 -- 172.21.15.19:0/2989174695 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f88180043d0 con 0x561daa005d80 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f8802ffd640 10 client.27902 handle_command_reply: tid=0 2024-05-17T09:38:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f88309ee740 1 client.27902 shutdown 2024-05-17T09:38:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 msgr2=0x561daa3f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f88309ee740 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f1cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8814005ab0 tx=0x7f88140c97e0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 shutdown_connections 2024-05-17T09:38:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.506+0000 7f88309ee740 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa005d80 0x561daa3ff610 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f8802ffd640 0 client.27902 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0471b0 0x561daa3f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa0264d0 0x561daa3f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 1 --2- 172.21.15.19:0/2989174695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f1cf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 >> 172.21.15.19:0/2989174695 conn(0x561daa03b300 msgr2=0x561daa3e2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 shutdown_connections 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 1 -- 172.21.15.19:0/2989174695 wait complete. 2024-05-17T09:38:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.507+0000 7f88309ee740 20 client.27902 trim_cache size 0 max 16384 2024-05-17T09:38:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 Processor -- start 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 -- start start 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa005d80 0x561daa3e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88241ad620 con 0x561daa01e2f0 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88241aaef0 con 0x561daa005d80 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88309ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f88241a8d40 con 0x561daa0264d0 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f8822ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f8822ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60298/0 (socket says 172.21.15.19:60298) 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f8822ffd640 1 -- 172.21.15.19:0/4163481597 learned_addr learned my addr 172.21.15.19:0/4163481597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f8823fff640 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.537+0000 7f88237fe640 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa005d80 0x561daa3e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f8822ffd640 1 -- 172.21.15.19:0/4163481597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 msgr2=0x561daa3f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f8822ffd640 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f8822ffd640 1 -- 172.21.15.19:0/4163481597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa005d80 msgr2=0x561daa3e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f8822ffd640 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa005d80 0x561daa3e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f8822ffd640 1 -- 172.21.15.19:0/4163481597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daa3f0440 con 0x561daa01e2f0 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f88237fe640 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa005d80 0x561daa3e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f8822ffd640 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3f9120 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8818009250 tx=0x7f88180035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb34d6b9a72b1394 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f88097fa640 1 -- 172.21.15.19:0/4163481597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88180050f0 con 0x561daa01e2f0 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f88097fa640 1 -- 172.21.15.19:0/4163481597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f881800b680 con 0x561daa01e2f0 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.538+0000 7f88097fa640 1 -- 172.21.15.19:0/4163481597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881802f650 con 0x561daa01e2f0 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.539+0000 7f88309ee740 1 -- 172.21.15.19:0/4163481597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 msgr2=0x561daa3f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.539+0000 7f88309ee740 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3f9120 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8818009250 tx=0x7f88180035f0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.539+0000 7f88309ee740 1 -- 172.21.15.19:0/4163481597 shutdown_connections 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa01e2f0 0x561daa3f9120 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 --2- 172.21.15.19:0/4163481597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa005d80 0x561daa3e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 -- 172.21.15.19:0/4163481597 >> 172.21.15.19:0/4163481597 conn(0x561daa03b300 msgr2=0x561daa3e2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 -- 172.21.15.19:0/4163481597 shutdown_connections 2024-05-17T09:38:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 -- 172.21.15.19:0/4163481597 wait complete. 2024-05-17T09:38:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.540+0000 7f88309ee740 1 -- start start 2024-05-17T09:38:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88309ee740 1 Processor -- start 2024-05-17T09:38:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88309ee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 0x561daa3f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88309ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88309ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 0x561daa3f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 0x561daa3f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60310/0 (socket says 172.21.15.19:60310) 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88237fe640 1 -- 172.21.15.19:0/1045851643 learned_addr learned my addr 172.21.15.19:0/1045851643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8822ffd640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.541+0000 7f88309ee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88241a93b0 con 0x561daa005d80 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88241aa820 con 0x561daa01e2f0 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f88241a9a80 con 0x561daa0264d0 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8823fff640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f88309ee740 10 client.0 _ll_register_callbacks cb 0x7f88140c5d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8823fff640 1 -- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 msgr2=0x561daa3f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8823fff640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8823fff640 1 -- 172.21.15.19:0/1045851643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 msgr2=0x561daa3f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8823fff640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 0x561daa3f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8823fff640 1 -- 172.21.15.19:0/1045851643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daa3f88f0 con 0x561daa0264d0 2024-05-17T09:38:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f88237fe640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 0x561daa3f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.542+0000 7f8822ffd640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.543+0000 7f8823fff640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f8330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f881c005c40 tx=0x7f881c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e439ebeb1c7b2f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.543+0000 7f8802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.544+0000 7f8802ffd640 1 -- 172.21.15.19:0/1045851643 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c004b20 con 0x561daa0264d0 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.544+0000 7f8802ffd640 1 -- 172.21.15.19:0/1045851643 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f881c004cc0 con 0x561daa0264d0 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.544+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881800b680 con 0x561daa0264d0 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.544+0000 7f8802ffd640 1 -- 172.21.15.19:0/1045851643 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f881c00f750 con 0x561daa0264d0 2024-05-17T09:38:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.544+0000 7f88309ee740 10 client.27898 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.544+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561daa467d50 con 0x561daa0264d0 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.545+0000 7f8802ffd640 1 -- 172.21.15.19:0/1045851643 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f881c00fd80 con 0x561daa0264d0 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.545+0000 7f8802ffd640 1 -- 172.21.15.19:0/1045851643 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f881c005670 con 0x561daa0264d0 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.545+0000 7f88309ee740 10 client.27898 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.545+0000 7f88309ee740 10 client.27898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.545+0000 7f88309ee740 20 client.27898 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.546+0000 7f88309ee740 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa0471b0 0x561daa46a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.546+0000 7f88309ee740 4 client.27898 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:38:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.546+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f881800b680 con 0x561daa0471b0 2024-05-17T09:38:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.546+0000 7f88237fe640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa0471b0 0x561daa46a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.547+0000 7f88237fe640 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa0471b0 0x561daa46a720 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9490ac5a7a068598 server_cookie=e5122d6c04832253 in_seq=0 out_seq=0 2024-05-17T09:38:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.547+0000 7f8802ffd640 10 client.27898 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:37.579 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33556 6.56 MB/sec execute 154 sec latency 95.963 ms 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f8802ffd640 1 -- 172.21.15.19:0/1045851643 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f881800b680 con 0x561daa0471b0 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f8802ffd640 10 client.27898 handle_command_reply: tid=0 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 client.27898 shutdown 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 msgr2=0x561daa3f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f8330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f881c005c40 tx=0x7f881c004a40 comp rx=0 tx=0).stop 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 shutdown_connections 2024-05-17T09:38:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561daa0471b0 0x561daa46a720 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561daa0264d0 0x561daa3f8330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561daa01e2f0 0x561daa3f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 --2- 172.21.15.19:0/1045851643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561daa005d80 0x561daa3f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f8802ffd640 0 client.27898 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 >> 172.21.15.19:0/1045851643 conn(0x561daa03b300 msgr2=0x561daa3e2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 shutdown_connections 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.635+0000 7f88309ee740 1 -- 172.21.15.19:0/1045851643 wait complete. 2024-05-17T09:38:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f88309ee740 20 client.27898 trim_cache size 0 max 16384 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f880c0b28a0 msgr2=0x7f880c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f880c0b28a0 0x7f880c0b4db0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f881c009fd0 tx=0x7f881c009760 comp rx=0 tx=0).stop 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824103d80 msgr2=0x7f882419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824103d80 0x7f882419a2e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88241021e0 tx=0x7f8814004270 comp rx=0 tx=0).stop 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 shutdown_connections 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f880c0b28a0 0x7f880c0b4db0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882419f7a0 0x7f88241a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8824108a80 0x7f882419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 --2- 172.21.15.19:0/4014192719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8824103d80 0x7f882419a2e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.636+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 >> 172.21.15.19:0/4014192719 conn(0x7f88240fdd30 msgr2=0x7f882410a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.637+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 shutdown_connections 2024-05-17T09:38:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:37.637+0000 7f8829b38640 1 -- 172.21.15.19:0/4014192719 wait complete. 2024-05-17T09:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.641+0000 7f41d159a700 1 -- 172.21.15.19:0/310688994 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c400bab0 con 0x7f41cc097730 2024-05-17T09:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 -- 172.21.15.19:0/310688994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc097730 msgr2=0x7f41cc097b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 --2- 172.21.15.19:0/310688994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc097730 0x7f41cc097b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009990 tx=0x7f41c4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 -- 172.21.15.19:0/310688994 shutdown_connections 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 --2- 172.21.15.19:0/310688994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc097730 0x7f41cc097b40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 --2- 172.21.15.19:0/310688994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc0a0ba0 0x7f41cc0a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 --2- 172.21.15.19:0/310688994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc098080 0x7f41cc0a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 -- 172.21.15.19:0/310688994 >> 172.21.15.19:0/310688994 conn(0x7f41cc00b060 msgr2=0x7f41cc00b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 -- 172.21.15.19:0/310688994 shutdown_connections 2024-05-17T09:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.642+0000 7f41d359e700 1 -- 172.21.15.19:0/310688994 wait complete. 2024-05-17T09:38:37.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 Processor -- start 2024-05-17T09:38:37.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 -- start start 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 0x7f41cc0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc098080 0x7f41cc0ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0a0ba0 0x7f41cc09f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0a44b0 con 0x7f41cc098080 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0a4330 con 0x7f41cc097730 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d359e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0a4630 con 0x7f41cc0a0ba0 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 0x7f41cc0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0a0ba0 0x7f41cc09f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 0x7f41cc0aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:44880/0 (socket says 172.21.15.19:44880) 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d259c700 1 -- 172.21.15.19:0/1317208166 learned_addr learned my addr 172.21.15.19:0/1317208166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.644+0000 7f41d1d9b700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc098080 0x7f41cc0ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d259c700 1 -- 172.21.15.19:0/1317208166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0a0ba0 msgr2=0x7f41cc09f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d259c700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0a0ba0 0x7f41cc09f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d259c700 1 -- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc098080 msgr2=0x7f41cc0ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d259c700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc098080 0x7f41cc0ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d259c700 1 -- 172.21.15.19:0/1317208166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc0a01d0 con 0x7f41cc097730 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d259c700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 0x7f41cc0aae00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f41c40098e0 tx=0x7f41c40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4027070 con 0x7f41cc097730 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc0a0460 con 0x7f41cc097730 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41c4022470 con 0x7f41cc097730 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.645+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc0aeb10 con 0x7f41cc097730 2024-05-17T09:38:37.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.646+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4017650 con 0x7f41cc097730 2024-05-17T09:38:37.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.647+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc004910 con 0x7f41cc097730 2024-05-17T09:38:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.649+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f41c402a070 con 0x7f41cc097730 2024-05-17T09:38:37.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.650+0000 7f41c37fe700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41bc0b2900 0x7f41bc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.650+0000 7f41d1d9b700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41bc0b2900 0x7f41bc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:37.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.650+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f41c40e37f0 con 0x7f41cc097730 2024-05-17T09:38:37.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.651+0000 7f41d1d9b700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41bc0b2900 0x7f41bc0b4e10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0a53d0 tx=0x7f41c8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:37.654+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41c405fdf0 con 0x7f41cc097730 2024-05-17T09:38:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:38:38.102 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:38:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.099+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41cc0aed70 con 0x7f41cc097730 2024-05-17T09:38:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.101+0000 7f41c37fe700 1 -- 172.21.15.19:0/1317208166 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f41c40a6020 con 0x7f41cc097730 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41bc0b2900 msgr2=0x7f41bc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41bc0b2900 0x7f41bc0b4e10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0a53d0 tx=0x7f41c8009190 comp rx=0 tx=0).stop 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 msgr2=0x7f41cc0aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 0x7f41cc0aae00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f41c40098e0 tx=0x7f41c40057c0 comp rx=0 tx=0).stop 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 shutdown_connections 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f41bc0b2900 0x7f41bc0b4e10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0a0ba0 0x7f41cc09f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc097730 0x7f41cc0aae00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 --2- 172.21.15.19:0/1317208166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41cc098080 0x7f41cc0ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 >> 172.21.15.19:0/1317208166 conn(0x7f41cc00b060 msgr2=0x7f41cc094110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 shutdown_connections 2024-05-17T09:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:38.111+0000 7f41d359e700 1 -- 172.21.15.19:0/1317208166 wait complete. 2024-05-17T09:38:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:38:38.579 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33721 6.62 MB/sec execute 155 sec latency 110.134 ms 2024-05-17T09:38:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:38 smithi079 ceph-mon[35502]: pgmap v1004: 129 pgs: 129 active+clean; 391 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 594 KiB/s wr, 47 op/s 2024-05-17T09:38:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:38 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1317208166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:38.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:38 smithi019 ceph-mon[31620]: pgmap v1004: 129 pgs: 129 active+clean; 391 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 594 KiB/s wr, 47 op/s 2024-05-17T09:38:38.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:38 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1317208166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:38.927 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-05-17T09:38:38.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-05-17T09:38:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:38 smithi069 ceph-mon[35081]: pgmap v1004: 129 pgs: 129 active+clean; 391 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 594 KiB/s wr, 47 op/s 2024-05-17T09:38:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:38 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1317208166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:38:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:39.579 INFO:tasks.workunit.client.0.smithi019.stdout: 10 33864 6.62 MB/sec execute 156 sec latency 97.920 ms 2024-05-17T09:38:40.583 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34000 6.63 MB/sec execute 157 sec latency 91.229 ms 2024-05-17T09:38:40.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:40 smithi079 ceph-mon[35502]: pgmap v1005: 129 pgs: 129 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.0 MiB/s wr, 80 op/s 2024-05-17T09:38:40.817 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:40 smithi019 ceph-mon[31620]: pgmap v1005: 129 pgs: 129 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.0 MiB/s wr, 80 op/s 2024-05-17T09:38:40.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:40 smithi069 ceph-mon[35081]: pgmap v1005: 129 pgs: 129 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.0 MiB/s wr, 80 op/s 2024-05-17T09:38:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 -- 172.21.15.19:0/1120022428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 msgr2=0x7f4c84108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 --2- 172.21.15.19:0/1120022428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c84108210 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009920 tx=0x7f4c74009c30 comp rx=0 tx=0).stop 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 -- 172.21.15.19:0/1120022428 shutdown_connections 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 --2- 172.21.15.19:0/1120022428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c84108750 0x7f4c841100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 --2- 172.21.15.19:0/1120022428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c841034d0 0x7f4c841038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 --2- 172.21.15.19:0/1120022428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c84108210 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 -- 172.21.15.19:0/1120022428 >> 172.21.15.19:0/1120022428 conn(0x7f4c840fef20 msgr2=0x7f4c84101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 -- 172.21.15.19:0/1120022428 shutdown_connections 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.338+0000 7f4c89c06700 1 -- 172.21.15.19:0/1120022428 wait complete. 2024-05-17T09:38:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.339+0000 7f4c89c06700 1 Processor -- start 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.339+0000 7f4c89c06700 1 -- start start 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c89c06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c841034d0 0x7f4c84100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c89c06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c840ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c840ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c840ff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60350/0 (socket says 172.21.15.19:60350) 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c82ffd700 1 -- 172.21.15.19:0/2187319864 learned_addr learned my addr 172.21.15.19:0/2187319864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c837fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c841034d0 0x7f4c84100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c84108750 0x7f4c840ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84113260 con 0x7f4c84107e00 2024-05-17T09:38:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841130e0 con 0x7f4c84108750 2024-05-17T09:38:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841133e0 con 0x7f4c841034d0 2024-05-17T09:38:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.340+0000 7f4c83fff700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c84108750 0x7f4c840ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c82ffd700 1 -- 172.21.15.19:0/2187319864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c841034d0 msgr2=0x7f4c84100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c841034d0 0x7f4c84100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c82ffd700 1 -- 172.21.15.19:0/2187319864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c84108750 msgr2=0x7f4c840ff9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c84108750 0x7f4c840ff9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c82ffd700 1 -- 172.21.15.19:0/2187319864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c84100210 con 0x7f4c84107e00 2024-05-17T09:38:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c837fe700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c841034d0 0x7f4c84100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:38:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.341+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c840ff400 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b7c0 tx=0x7f4c7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.342+0000 7f4c80ff9700 1 -- 172.21.15.19:0/2187319864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74025070 con 0x7f4c84107e00 2024-05-17T09:38:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.342+0000 7f4c80ff9700 1 -- 172.21.15.19:0/2187319864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c740054c0 con 0x7f4c84107e00 2024-05-17T09:38:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.342+0000 7f4c80ff9700 1 -- 172.21.15.19:0/2187319864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74004cd0 con 0x7f4c84107e00 2024-05-17T09:38:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.343+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c841ad690 con 0x7f4c84107e00 2024-05-17T09:38:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.343+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c841ada90 con 0x7f4c84107e00 2024-05-17T09:38:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.348+0000 7f4c80ff9700 1 -- 172.21.15.19:0/2187319864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4c74028070 con 0x7f4c84107e00 2024-05-17T09:38:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.348+0000 7f4c9861bb80 1 Processor -- start 2024-05-17T09:38:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 -- start start 2024-05-17T09:38:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x5622102f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622102f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562210219f00 con 0x56221016cfd0 2024-05-17T09:38:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562210209aa0 con 0x5622102585c0 2024-05-17T09:38:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56221020dae0 con 0x5622102ec0b0 2024-05-17T09:38:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c83fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x5622102f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46920/0 (socket says 172.21.15.19:46920) 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c82ffd700 1 -- 172.21.15.19:0/351819590 learned_addr learned my addr 172.21.15.19:0/351819590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.349+0000 7f4c80ff9700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4c6c0b2a50 0x7f4c6c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c837fe700 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622102f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c82ffd700 1 -- 172.21.15.19:0/351819590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 msgr2=0x5622102f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x5622102f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c82ffd700 1 -- 172.21.15.19:0/351819590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 msgr2=0x5622102f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622102f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c82ffd700 1 -- 172.21.15.19:0/351819590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622101b3140 con 0x5622102ec0b0 2024-05-17T09:38:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c83fff700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4c6c0b2a50 0x7f4c6c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ec580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c740a6320 tx=0x7f4c740a9cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d748b3156eab3208 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c80ff9700 1 -- 172.21.15.19:0/2187319864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4c740e3000 con 0x7f4c84107e00 2024-05-17T09:38:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.350+0000 7f4c69ffb700 1 -- 172.21.15.19:0/351819590 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400cc10 con 0x5622102ec0b0 2024-05-17T09:38:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.351+0000 7f4c69ffb700 1 -- 172.21.15.19:0/351819590 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c7400bd90 con 0x5622102ec0b0 2024-05-17T09:38:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.351+0000 7f4c69ffb700 1 -- 172.21.15.19:0/351819590 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74027920 con 0x5622102ec0b0 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 -- 172.21.15.19:0/351819590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 msgr2=0x5622102ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ec580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c740a6320 tx=0x7f4c740a9cf0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c83fff700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4c6c0b2a50 0x7f4c6c0b4f60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5622102f19e0 tx=0x7f4c7c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 -- 172.21.15.19:0/351819590 shutdown_connections 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ec580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x5622102f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/351819590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622102f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 -- 172.21.15.19:0/351819590 >> 172.21.15.19:0/351819590 conn(0x5622101cd350 msgr2=0x5622102e6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 -- 172.21.15.19:0/351819590 shutdown_connections 2024-05-17T09:38:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.352+0000 7f4c9861bb80 1 -- 172.21.15.19:0/351819590 wait complete. 2024-05-17T09:38:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.353+0000 7f4c9861bb80 1 -- start start 2024-05-17T09:38:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.353+0000 7f4c9861bb80 1 Processor -- start 2024-05-17T09:38:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56221016cfd0 0x5622102fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 0x5622102fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102ec0b0 0x5622102f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562210110130 con 0x5622102ec0b0 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562210171f30 con 0x56221016cfd0 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5622102ecdd0 con 0x5622102585c0 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 0x5622102fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 0x5622102fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46922/0 (socket says 172.21.15.19:46922) 2024-05-17T09:38:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c9861bb80 10 client.0 _ll_register_callbacks cb 0x5622101a4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 -- 172.21.15.19:0/263802391 learned_addr learned my addr 172.21.15.19:0/263802391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c837fe700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56221016cfd0 0x5622102fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c83fff700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102ec0b0 0x5622102f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 -- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56221016cfd0 msgr2=0x5622102fb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56221016cfd0 0x5622102fb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 -- 172.21.15.19:0/263802391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102ec0b0 msgr2=0x5622102f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102ec0b0 0x5622102f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.354+0000 7f4c82ffd700 1 -- 172.21.15.19:0/263802391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562210159a60 con 0x5622102585c0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 0x5622102fbc00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c740a9fa0 tx=0x7f4c74005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6112ec3fbe5e74ac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c62ffd700 1 -- 172.21.15.19:0/263802391 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74027b90 con 0x5622102585c0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c62ffd700 1 -- 172.21.15.19:0/263802391 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4c7408e240 con 0x5622102585c0 2024-05-17T09:38:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7400bd90 con 0x5622102585c0 2024-05-17T09:38:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c62ffd700 1 -- 172.21.15.19:0/263802391 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4c74027db0 con 0x5622102585c0 2024-05-17T09:38:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c9861bb80 10 client.27916 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5622101c38f0 con 0x5622102585c0 2024-05-17T09:38:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.355+0000 7f4c62ffd700 1 -- 172.21.15.19:0/263802391 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c740a9db0 con 0x5622102585c0 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c62ffd700 1 -- 172.21.15.19:0/263802391 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4c740aa500 con 0x5622102585c0 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c9861bb80 10 client.27916 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c9861bb80 10 client.27916 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c9861bb80 20 client.27916 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562210304840 0x562210306c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c9861bb80 4 client.27916 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c7400bd90 con 0x562210304840 2024-05-17T09:38:41.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c837fe700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562210304840 0x562210306c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.356+0000 7f4c62ffd700 10 client.27916 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:41.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.357+0000 7f4c837fe700 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562210304840 0x562210306c60 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c354ff0ef52d3d8 server_cookie=e47b579f29816d76 in_seq=0 out_seq=0 2024-05-17T09:38:41.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c62ffd700 1 -- 172.21.15.19:0/263802391 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f4c7400bd90 con 0x562210304840 2024-05-17T09:38:41.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c62ffd700 10 client.27916 handle_command_reply: tid=0 2024-05-17T09:38:41.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 client.27916 shutdown 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 msgr2=0x5622102fbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 0x5622102fbc00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c740a9fa0 tx=0x7f4c74005dc0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 shutdown_connections 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x562210304840 0x562210306c60 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102ec0b0 0x5622102f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56221016cfd0 0x5622102fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/263802391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102585c0 0x5622102fbc00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c62ffd700 0 client.27916 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 >> 172.21.15.19:0/263802391 conn(0x5622101cd350 msgr2=0x5622102eb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.359+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 shutdown_connections 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.360+0000 7f4c9861bb80 1 -- 172.21.15.19:0/263802391 wait complete. 2024-05-17T09:38:41.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.360+0000 7f4c9861bb80 20 client.27916 trim_cache size 0 max 16384 2024-05-17T09:38:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 Processor -- start 2024-05-17T09:38:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 -- start start 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 0x5622102f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 0x56221030c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102ec0b0 0x56221030c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8406fa40 con 0x5622102585c0 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c837fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 0x5622102f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 0x56221030c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8406d310 con 0x5622102ec0b0 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841a54f0 con 0x56221016cfd0 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c837fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 0x5622102f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46938/0 (socket says 172.21.15.19:46938) 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c837fe700 1 -- 172.21.15.19:0/1818762391 learned_addr learned my addr 172.21.15.19:0/1818762391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 0x56221030c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60378/0 (socket says 172.21.15.19:60378) 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.392+0000 7f4c83fff700 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102ec0b0 0x56221030c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c837fe700 1 -- 172.21.15.19:0/1818762391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102ec0b0 msgr2=0x56221030c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c837fe700 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102ec0b0 0x56221030c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c837fe700 1 -- 172.21.15.19:0/1818762391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 msgr2=0x56221030c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c837fe700 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 0x56221030c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c837fe700 1 -- 172.21.15.19:0/1818762391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622102e7d90 con 0x56221016cfd0 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 0x56221030c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c837fe700 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 0x5622102f1f20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c78005fd0 tx=0x7f4c78009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81acae6159285b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c697fa700 1 -- 172.21.15.19:0/1818762391 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78004250 con 0x56221016cfd0 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c697fa700 1 -- 172.21.15.19:0/1818762391 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c78012e50 con 0x56221016cfd0 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.393+0000 7f4c697fa700 1 -- 172.21.15.19:0/1818762391 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c780053a0 con 0x56221016cfd0 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.394+0000 7f4c9861bb80 1 -- 172.21.15.19:0/1818762391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 msgr2=0x5622102f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.394+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 0x5622102f1f20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c78005fd0 tx=0x7f4c78009310 comp rx=0 tx=0).stop 2024-05-17T09:38:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.394+0000 7f4c9861bb80 1 -- 172.21.15.19:0/1818762391 shutdown_connections 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.394+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56221016cfd0 0x5622102f1f20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.394+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102ec0b0 0x56221030c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.395+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/1818762391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622102585c0 0x56221030c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.395+0000 7f4c9861bb80 1 -- 172.21.15.19:0/1818762391 >> 172.21.15.19:0/1818762391 conn(0x5622101cd350 msgr2=0x5622102f43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.395+0000 7f4c9861bb80 1 -- 172.21.15.19:0/1818762391 shutdown_connections 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.395+0000 7f4c9861bb80 1 -- 172.21.15.19:0/1818762391 wait complete. 2024-05-17T09:38:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.395+0000 7f4c9861bb80 1 -- start start 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.395+0000 7f4c9861bb80 1 Processor -- start 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622103016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x562210301c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841a5b60 con 0x56221016cfd0 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841a6fd0 con 0x5622102585c0 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841a6230 con 0x5622102ec0b0 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c9861bb80 10 client.0 _ll_register_callbacks cb 0x5622101a4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622103016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622103016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60394/0 (socket says 172.21.15.19:60394) 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c837fe700 1 -- 172.21.15.19:0/4110567324 learned_addr learned my addr 172.21.15.19:0/4110567324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x562210301c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c83fff700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c82ffd700 1 -- 172.21.15.19:0/4110567324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 msgr2=0x5622102ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c82ffd700 1 -- 172.21.15.19:0/4110567324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 msgr2=0x5622103016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622103016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.396+0000 7f4c82ffd700 1 -- 172.21.15.19:0/4110567324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562210300310 con 0x5622102585c0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.397+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x562210301c30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c740aa230 tx=0x7f4c740b2e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac61e0cfea5a376 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.397+0000 7f4c62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.397+0000 7f4c62ffd700 1 -- 172.21.15.19:0/4110567324 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400c2d0 con 0x5622102585c0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.397+0000 7f4c62ffd700 1 -- 172.21.15.19:0/4110567324 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4c7400cde0 con 0x5622102585c0 2024-05-17T09:38:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.397+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c78012e50 con 0x5622102585c0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.397+0000 7f4c62ffd700 1 -- 172.21.15.19:0/4110567324 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4c7400c4c0 con 0x5622102585c0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 10 client.27932 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562210300310 con 0x5622102585c0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c62ffd700 1 -- 172.21.15.19:0/4110567324 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c78012e50 con 0x5622102585c0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c62ffd700 1 -- 172.21.15.19:0/4110567324 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4c740e0bb0 con 0x5622102585c0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 10 client.27932 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 10 client.27932 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 20 client.27932 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622103712c0 0x5622103736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 4 client.27932 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x5622102b8060 con 0x5622103712c0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.398+0000 7f4c837fe700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622103712c0 0x5622103736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.399+0000 7f4c62ffd700 10 client.27932 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.399+0000 7f4c837fe700 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622103712c0 0x5622103736e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5e6ff6a6993828 server_cookie=e47fde59fbc21089 in_seq=0 out_seq=0 2024-05-17T09:38:41.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34103 6.62 MB/sec execute 158 sec latency 200.882 ms 2024-05-17T09:38:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.839+0000 7f4c62ffd700 1 -- 172.21.15.19:0/4110567324 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5622102b8060 con 0x5622103712c0 2024-05-17T09:38:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.839+0000 7f4c62ffd700 10 client.27932 handle_command_reply: tid=0 2024-05-17T09:38:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.839+0000 7f4c9861bb80 1 client.27932 shutdown 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.839+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 msgr2=0x562210301c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.839+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x562210301c30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c740aa230 tx=0x7f4c740b2e90 comp rx=0 tx=0).stop 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 shutdown_connections 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622102ec0b0 0x5622102ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622103712c0 0x5622103736e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56221016cfd0 0x5622103016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 --2- 172.21.15.19:0/4110567324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622102585c0 0x562210301c30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 >> 172.21.15.19:0/4110567324 conn(0x5622101cd350 msgr2=0x5622102f4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c62ffd700 0 client.27932 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 shutdown_connections 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 1 -- 172.21.15.19:0/4110567324 wait complete. 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.840+0000 7f4c9861bb80 20 client.27932 trim_cache size 0 max 16384 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 66, 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout: "334c53bf": { 2024-05-17T09:38:41.841 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 66, 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.2, 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:38:41.842 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:38:41.843 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:38:41.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4c6c0b2a50 msgr2=0x7f4c6c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4c6c0b2a50 0x7f4c6c0b4f60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5622102f19e0 tx=0x7f4c7c004b40 comp rx=0 tx=0).stop 2024-05-17T09:38:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 msgr2=0x7f4c840ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c840ff400 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b7c0 tx=0x7f4c7400b8a0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 shutdown_connections 2024-05-17T09:38:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f4c6c0b2a50 0x7f4c6c0b4f60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c841034d0 0x7f4c84100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c84108750 0x7f4c840ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 --2- 172.21.15.19:0/2187319864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c84107e00 0x7f4c840ff400 secure :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b7c0 tx=0x7f4c7400b8a0 comp rx=0 tx=0).stop 2024-05-17T09:38:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.842+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 >> 172.21.15.19:0/2187319864 conn(0x7f4c840fef20 msgr2=0x7f4c84107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.843+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 shutdown_connections 2024-05-17T09:38:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:41.843+0000 7f4c89c06700 1 -- 172.21.15.19:0/2187319864 wait complete. 2024-05-17T09:38:42.579 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.57 MB/sec execute 159 sec latency 1114.394 ms 2024-05-17T09:38:42.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:42 smithi019 ceph-mon[31620]: pgmap v1006: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-05-17T09:38:42.667 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '334c53bf', quiesce took 7.7 seconds. Will release after: 14.5 2024-05-17T09:38:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:42 smithi079 ceph-mon[35502]: pgmap v1006: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-05-17T09:38:42.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:42 smithi069 ceph-mon[35081]: pgmap v1006: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-05-17T09:38:43.579 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.53 MB/sec execute 160 sec latency 2114.475 ms 2024-05-17T09:38:44.579 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.49 MB/sec execute 161 sec latency 3114.568 ms 2024-05-17T09:38:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:44 smithi079 ceph-mon[35502]: pgmap v1007: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-05-17T09:38:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:44 smithi069 ceph-mon[35081]: pgmap v1007: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-05-17T09:38:44.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:44 smithi019 ceph-mon[31620]: pgmap v1007: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-05-17T09:38:45.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.45 MB/sec execute 162 sec latency 4114.668 ms 2024-05-17T09:38:46.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.41 MB/sec execute 163 sec latency 5114.757 ms 2024-05-17T09:38:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:46 smithi079 ceph-mon[35502]: pgmap v1008: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-05-17T09:38:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:46 smithi069 ceph-mon[35081]: pgmap v1008: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-05-17T09:38:46.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:46 smithi019 ceph-mon[31620]: pgmap v1008: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-05-17T09:38:47.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.37 MB/sec execute 164 sec latency 6114.806 ms 2024-05-17T09:38:47.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:38:47.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:47.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:47 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:47.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:38:47.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:47.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:47 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:47.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:38:47.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:47.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:47 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.33 MB/sec execute 165 sec latency 7114.837 ms 2024-05-17T09:38:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: pgmap v1009: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-05-17T09:38:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:48 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: pgmap v1009: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-05-17T09:38:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:48.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:48 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: pgmap v1009: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-05-17T09:38:49.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:48 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:49.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.30 MB/sec execute 166 sec latency 8114.929 ms 2024-05-17T09:38:50.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:38:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:38:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.272 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:49 smithi079 ceph-mon[35502]: pgmap v1010: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-05-17T09:38:50.429 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.429 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:38:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:38:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.430 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:49 smithi069 ceph-mon[35081]: pgmap v1010: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-05-17T09:38:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:38:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:38:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:38:50.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:49 smithi019 ceph-mon[31620]: pgmap v1010: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-05-17T09:38:50.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.26 MB/sec execute 167 sec latency 9115.024 ms 2024-05-17T09:38:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 -- 172.21.15.19:0/1616911630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648104390 msgr2=0x7f86481067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1616911630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648104390 0x7f86481067b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8644009900 tx=0x7f864402a410 comp rx=0 tx=0).stop 2024-05-17T09:38:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 -- 172.21.15.19:0/1616911630 shutdown_connections 2024-05-17T09:38:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1616911630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 0x7f864810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1616911630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8648106d80 0x7f8648109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1616911630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648104390 0x7f86481067b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 -- 172.21.15.19:0/1616911630 >> 172.21.15.19:0/1616911630 conn(0x7f86480fdd30 msgr2=0x7f86481001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 -- 172.21.15.19:0/1616911630 shutdown_connections 2024-05-17T09:38:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 -- 172.21.15.19:0/1616911630 wait complete. 2024-05-17T09:38:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 Processor -- start 2024-05-17T09:38:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.759+0000 7f864fcb6640 1 -- start start 2024-05-17T09:38:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864fcb6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8648104390 0x7f864819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864fcb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 0x7f864819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864fcb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 0x7f86481a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864fcb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f620 con 0x7f8648106d80 2024-05-17T09:38:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864fcb6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f4a0 con 0x7f86481098f0 2024-05-17T09:38:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864fcb6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f7a0 con 0x7f8648104390 2024-05-17T09:38:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 0x7f864819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 0x7f864819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39512/0 (socket says 172.21.15.19:39512) 2024-05-17T09:38:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 -- 172.21.15.19:0/1365446794 learned_addr learned my addr 172.21.15.19:0/1365446794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864e253640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 0x7f86481a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864da52640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8648104390 0x7f864819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 -- 172.21.15.19:0/1365446794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8648104390 msgr2=0x7f864819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8648104390 0x7f864819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 -- 172.21.15.19:0/1365446794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 msgr2=0x7f86481a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 0x7f86481a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 -- 172.21.15.19:0/1365446794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481a1d00 con 0x7f8648106d80 2024-05-17T09:38:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864e253640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 0x7f86481a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.760+0000 7f864d251640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 0x7f864819a700 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f863800e9f0 tx=0x7f863800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.761+0000 7f863effd640 1 -- 172.21.15.19:0/1365446794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863800cd80 con 0x7f8648106d80 2024-05-17T09:38:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.761+0000 7f863effd640 1 -- 172.21.15.19:0/1365446794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86380054b0 con 0x7f8648106d80 2024-05-17T09:38:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.761+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481a1ff0 con 0x7f8648106d80 2024-05-17T09:38:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.761+0000 7f863effd640 1 -- 172.21.15.19:0/1365446794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638010430 con 0x7f8648106d80 2024-05-17T09:38:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.761+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a2460 con 0x7f8648106d80 2024-05-17T09:38:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 Processor -- start 2024-05-17T09:38:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 -- start start 2024-05-17T09:38:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f863effd640 1 -- 172.21.15.19:0/1365446794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f86380046c0 con 0x7f8648106d80 2024-05-17T09:38:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4a72f0 0x55b8ae871920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4af4d0 0x55b8ae878820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b8ae7e3c30 con 0x55b8ae4af4d0 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b8ae7d37d0 con 0x55b8ae4a72f0 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f8656b6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b8ae7d7810 con 0x55b8ae4d01b0 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f864d251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4af4d0 0x55b8ae878820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f864e253640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f864d251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4af4d0 0x55b8ae878820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39518/0 (socket says 172.21.15.19:39518) 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f864d251640 1 -- 172.21.15.19:0/2209391128 learned_addr learned my addr 172.21.15.19:0/2209391128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.764+0000 7f864da52640 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4a72f0 0x55b8ae871920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864e253640 1 -- 172.21.15.19:0/2209391128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4a72f0 msgr2=0x55b8ae871920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864e253640 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4a72f0 0x55b8ae871920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864e253640 1 -- 172.21.15.19:0/2209391128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4af4d0 msgr2=0x55b8ae878820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864e253640 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4af4d0 0x55b8ae878820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864e253640 1 -- 172.21.15.19:0/2209391128 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ae4e8210 con 0x55b8ae4d01b0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864e253640 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8713a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8640009f80 tx=0x7f864000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160e6d0dc1fa0853 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f8633fff640 1 -- 172.21.15.19:0/2209391128 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640015070 con 0x55b8ae4d01b0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f863effd640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86340b2920 0x7f86340b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f8633fff640 1 -- 172.21.15.19:0/2209391128 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8640011b90 con 0x55b8ae4d01b0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f8633fff640 1 -- 172.21.15.19:0/2209391128 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864001ab10 con 0x55b8ae4d01b0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f864da52640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86340b2920 0x7f86340b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.765+0000 7f863effd640 1 -- 172.21.15.19:0/1365446794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86380e2a10 con 0x7f8648106d80 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 -- 172.21.15.19:0/2209391128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 msgr2=0x55b8ae8713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f864da52640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86340b2920 0x7f86340b4e30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8644006010 tx=0x7f8644031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8713a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8640009f80 tx=0x7f864000f7f0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 -- 172.21.15.19:0/2209391128 shutdown_connections 2024-05-17T09:38:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4af4d0 0x55b8ae878820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4a72f0 0x55b8ae871920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2209391128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8713a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 -- 172.21.15.19:0/2209391128 >> 172.21.15.19:0/2209391128 conn(0x55b8ae4c4300 msgr2=0x55b8ae86cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 -- 172.21.15.19:0/2209391128 shutdown_connections 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.766+0000 7f8656b6b740 1 -- 172.21.15.19:0/2209391128 wait complete. 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 -- start start 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 Processor -- start 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae87ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae87b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b8ae4b9790 con 0x55b8ae4a72f0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b8ae78f890 con 0x55b8ae4af4d0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b8ae879270 con 0x55b8ae4d01b0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f8656b6b740 10 client.0 _ll_register_callbacks cb 0x55b8ae792d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f864da52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae87ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f864da52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae87ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39526/0 (socket says 172.21.15.19:39526) 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.767+0000 7f864da52640 1 -- 172.21.15.19:0/1703401355 learned_addr learned my addr 172.21.15.19:0/1703401355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864d251640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae87b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864e253640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864da52640 1 -- 172.21.15.19:0/1703401355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 msgr2=0x55b8ae8821a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864da52640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8821a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864da52640 1 -- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 msgr2=0x55b8ae87b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864da52640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae87b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864da52640 1 -- 172.21.15.19:0/1703401355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ae5b4620 con 0x55b8ae4a72f0 2024-05-17T09:38:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864d251640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae87b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f864da52640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae87ad70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8644005f70 tx=0x7f8644009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e96120dc29ed37c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f86317fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f86317fa640 1 -- 172.21.15.19:0/1703401355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864404e070 con 0x55b8ae4a72f0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f86317fa640 1 -- 172.21.15.19:0/1703401355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f864404a450 con 0x55b8ae4a72f0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8640011b90 con 0x55b8ae4a72f0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f86317fa640 1 -- 172.21.15.19:0/1703401355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f864405a050 con 0x55b8ae4a72f0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f8656b6b740 10 client.18411 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.768+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8ae885140 con 0x55b8ae4a72f0 2024-05-17T09:38:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f86317fa640 1 -- 172.21.15.19:0/1703401355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8640011b90 con 0x55b8ae4a72f0 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f86317fa640 1 -- 172.21.15.19:0/1703401355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f864403d780 con 0x55b8ae4a72f0 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f8656b6b740 10 client.18411 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f8656b6b740 10 client.18411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f8656b6b740 20 client.18411 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f8656b6b740 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae48ed80 0x55b8ae888880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f8656b6b740 4 client.18411 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ae7e5b80 con 0x55b8ae48ed80 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.769+0000 7f864d251640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae48ed80 0x55b8ae888880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.770+0000 7f864d251640 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae48ed80 0x55b8ae888880 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28464281988d3fd server_cookie=4cb7f318ce936585 in_seq=0 out_seq=0 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.770+0000 7f86317fa640 10 client.18411 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.770+0000 7f86317fa640 1 -- 172.21.15.19:0/1703401355 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8ae7e5b80 con 0x55b8ae48ed80 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f86317fa640 10 client.18411 handle_command_reply: tid=0 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 client.18411 shutdown 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 msgr2=0x55b8ae87ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae87ad70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8644005f70 tx=0x7f8644009260 comp rx=0 tx=0).stop 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 shutdown_connections 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae48ed80 0x55b8ae888880 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4d01b0 0x55b8ae8821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f86317fa640 0 client.18411 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae87b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 --2- 172.21.15.19:0/1703401355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae87ad70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 >> 172.21.15.19:0/1703401355 conn(0x55b8ae4c4300 msgr2=0x55b8ae86fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 shutdown_connections 2024-05-17T09:38:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 1 -- 172.21.15.19:0/1703401355 wait complete. 2024-05-17T09:38:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.771+0000 7f8656b6b740 20 client.18411 trim_cache size 0 max 16384 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.786+0000 7f8656b6b740 1 Processor -- start 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.786+0000 7f8656b6b740 1 -- start start 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 0x55b8ae8758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4a72f0 0x55b8ae882130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae882670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f8656b6b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86481af2b0 con 0x55b8ae48ed80 2024-05-17T09:38:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f8656b6b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86481acb80 con 0x55b8ae4af4d0 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f8656b6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86481aa9d0 con 0x55b8ae4a72f0 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 0x55b8ae8758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 0x55b8ae8758f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39540/0 (socket says 172.21.15.19:39540) 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 -- 172.21.15.19:0/2969956826 learned_addr learned my addr 172.21.15.19:0/2969956826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864e253640 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae882670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864d251640 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4a72f0 0x55b8ae882130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 -- 172.21.15.19:0/2969956826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4a72f0 msgr2=0x55b8ae882130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4a72f0 0x55b8ae882130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 -- 172.21.15.19:0/2969956826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 msgr2=0x55b8ae882670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae882670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 -- 172.21.15.19:0/2969956826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ae8786c0 con 0x55b8ae48ed80 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864d251640 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4a72f0 0x55b8ae882130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:38:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.787+0000 7f864da52640 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 0x55b8ae8758f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f864403db50 tx=0x7f864403ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d28c6ec8b969023 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f86337fe640 1 -- 172.21.15.19:0/2969956826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644062070 con 0x55b8ae48ed80 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f86337fe640 1 -- 172.21.15.19:0/2969956826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f864403d710 con 0x55b8ae48ed80 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f86337fe640 1 -- 172.21.15.19:0/2969956826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864404e070 con 0x55b8ae48ed80 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 -- 172.21.15.19:0/2969956826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 msgr2=0x55b8ae8758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 0x55b8ae8758f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f864403db50 tx=0x7f864403ddc0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 -- 172.21.15.19:0/2969956826 shutdown_connections 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae4af4d0 0x55b8ae882670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4a72f0 0x55b8ae882130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2969956826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae48ed80 0x55b8ae8758f0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.788+0000 7f8656b6b740 1 -- 172.21.15.19:0/2969956826 >> 172.21.15.19:0/2969956826 conn(0x55b8ae4c4300 msgr2=0x55b8ae8701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 -- 172.21.15.19:0/2969956826 shutdown_connections 2024-05-17T09:38:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 -- 172.21.15.19:0/2969956826 wait complete. 2024-05-17T09:38:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 -- start start 2024-05-17T09:38:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 Processor -- start 2024-05-17T09:38:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae48ed80 0x55b8ae881a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae881f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4af4d0 0x55b8ae8eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ab040 con 0x55b8ae4a72f0 2024-05-17T09:38:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ac4b0 con 0x55b8ae48ed80 2024-05-17T09:38:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ab710 con 0x55b8ae4af4d0 2024-05-17T09:38:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f8656b6b740 10 client.0 _ll_register_callbacks cb 0x7f86440315d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:38:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.789+0000 7f864d251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae881f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae881f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39554/0 (socket says 172.21.15.19:39554) 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 -- 172.21.15.19:0/2465852987 learned_addr learned my addr 172.21.15.19:0/2465852987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864da52640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae48ed80 0x55b8ae881a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864e253640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4af4d0 0x55b8ae8eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 -- 172.21.15.19:0/2465852987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4af4d0 msgr2=0x55b8ae8eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4af4d0 0x55b8ae8eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 -- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae48ed80 msgr2=0x55b8ae881a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:38:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae48ed80 0x55b8ae881a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 -- 172.21.15.19:0/2465852987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ae8ef500 con 0x55b8ae4a72f0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864e253640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4af4d0 0x55b8ae8eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f864d251640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae881f60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f86380e01b0 tx=0x7f86380e8550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e2d1141890652e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f86317fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f86317fa640 1 -- 172.21.15.19:0/2465852987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86380a5410 con 0x55b8ae4a72f0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f86317fa640 1 -- 172.21.15.19:0/2465852987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86380a6270 con 0x55b8ae4a72f0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864403d710 con 0x55b8ae4a72f0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f86317fa640 1 -- 172.21.15.19:0/2465852987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f86380b2420 con 0x55b8ae4a72f0 2024-05-17T09:38:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f8656b6b740 10 client.18417 fetch_fsmap learned FSMap version 39 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.790+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8ae8f0750 con 0x55b8ae4a72f0 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f86317fa640 1 -- 172.21.15.19:0/2465852987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f864403d710 con 0x55b8ae4a72f0 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f86317fa640 1 -- 172.21.15.19:0/2465852987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8638016330 con 0x55b8ae4a72f0 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f8656b6b740 10 client.18417 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f8656b6b740 10 client.18417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f8656b6b740 20 client.18417 populate_metadata read hostname 'smithi019' 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae4d01b0 0x55b8ae8f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f8656b6b740 4 client.18417 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:38:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8ae7df360 con 0x55b8ae4d01b0 2024-05-17T09:38:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.791+0000 7f864da52640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae4d01b0 0x55b8ae8f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.792+0000 7f86317fa640 10 client.18417 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.792+0000 7f864da52640 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae4d01b0 0x55b8ae8f3120 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f847fd6f9b83218a server_cookie=418842cc2b92fcc9 in_seq=0 out_seq=0 2024-05-17T09:38:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f86317fa640 1 -- 172.21.15.19:0/2465852987 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b8ae7df360 con 0x55b8ae4d01b0 2024-05-17T09:38:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f86317fa640 10 client.18417 handle_command_reply: tid=0 2024-05-17T09:38:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 client.18417 shutdown 2024-05-17T09:38:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 msgr2=0x55b8ae881f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae881f60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f86380e01b0 tx=0x7f86380e8550 comp rx=0 tx=0).stop 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 shutdown_connections 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b8ae4d01b0 0x55b8ae8f3120 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8ae4af4d0 0x55b8ae8eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f86317fa640 0 client.18417 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b8ae4a72f0 0x55b8ae881f60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 --2- 172.21.15.19:0/2465852987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8ae48ed80 0x55b8ae881a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 >> 172.21.15.19:0/2465852987 conn(0x55b8ae4c4300 msgr2=0x55b8ae87b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 shutdown_connections 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 1 -- 172.21.15.19:0/2465852987 wait complete. 2024-05-17T09:38:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.848+0000 7f8656b6b740 20 client.18417 trim_cache size 0 max 16384 2024-05-17T09:38:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:38:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:38:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:38:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86340b2920 msgr2=0x7f86340b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86340b2920 0x7f86340b4e30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8644006010 tx=0x7f8644031400 comp rx=0 tx=0).stop 2024-05-17T09:38:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 msgr2=0x7f864819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 0x7f864819a700 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f863800e9f0 tx=0x7f863800edf0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 shutdown_connections 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86481098f0 0x7f86481a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86340b2920 0x7f86340b4e30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8648106d80 0x7f864819a700 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 --2- 172.21.15.19:0/1365446794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8648104390 0x7f864819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 >> 172.21.15.19:0/1365446794 conn(0x7f86480fdd30 msgr2=0x7f86481051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 shutdown_connections 2024-05-17T09:38:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:38:50.849+0000 7f864fcb6640 1 -- 172.21.15.19:0/1365446794 wait complete. 2024-05-17T09:38:51.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.22 MB/sec execute 168 sec latency 10115.143 ms 2024-05-17T09:38:52.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.18 MB/sec execute 169 sec latency 11115.206 ms 2024-05-17T09:38:52.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:52 smithi069 ceph-mon[35081]: pgmap v1011: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-05-17T09:38:52.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:52 smithi019 ceph-mon[31620]: pgmap v1011: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-05-17T09:38:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:52 smithi079 ceph-mon[35502]: pgmap v1011: 129 pgs: 129 active+clean; 394 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-05-17T09:38:53.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.15 MB/sec execute 170 sec latency 12115.243 ms 2024-05-17T09:38:54.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.11 MB/sec execute 171 sec latency 13115.281 ms 2024-05-17T09:38:54.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:54 smithi069 ceph-mon[35081]: pgmap v1012: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-05-17T09:38:54.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:54 smithi019 ceph-mon[31620]: pgmap v1012: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-05-17T09:38:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:54 smithi079 ceph-mon[35502]: pgmap v1012: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-05-17T09:38:55.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.08 MB/sec execute 172 sec latency 14115.321 ms 2024-05-17T09:38:56.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.04 MB/sec execute 173 sec latency 15115.369 ms 2024-05-17T09:38:56.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:56 smithi069 ceph-mon[35081]: pgmap v1013: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 387 KiB/s wr, 49 op/s 2024-05-17T09:38:56.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:56 smithi019 ceph-mon[31620]: pgmap v1013: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 387 KiB/s wr, 49 op/s 2024-05-17T09:38:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:56 smithi079 ceph-mon[35502]: pgmap v1013: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 387 KiB/s wr, 49 op/s 2024-05-17T09:38:57.168 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:38:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:38:57.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 6.01 MB/sec execute 174 sec latency 16115.434 ms 2024-05-17T09:38:58.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 5.97 MB/sec execute 175 sec latency 17115.486 ms 2024-05-17T09:38:58.633 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:38:58 smithi019 ceph-mon[31620]: pgmap v1014: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-05-17T09:38:58.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:38:58 smithi069 ceph-mon[35081]: pgmap v1014: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-05-17T09:38:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:38:58 smithi079 ceph-mon[35502]: pgmap v1014: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-05-17T09:38:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2761599337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc341034b0 msgr2=0x7fbc341038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2761599337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc341034b0 0x7fbc341038c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbc300099c0 tx=0x7fbc30009cd0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2761599337 shutdown_connections 2024-05-17T09:38:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2761599337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc34108730 0x7fbc34110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2761599337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc341034b0 0x7fbc341038c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2761599337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc34107de0 0x7fbc341081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2761599337 >> 172.21.15.19:0/2761599337 conn(0x7fbc340fedb0 msgr2=0x7fbc341011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2761599337 shutdown_connections 2024-05-17T09:38:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.483+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2761599337 wait complete. 2024-05-17T09:38:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 Processor -- start 2024-05-17T09:38:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 -- start start 2024-05-17T09:38:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc341034b0 0x7fbc3419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc34107de0 0x7fbc3419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 0x7fbc3419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34113230 con 0x7fbc34108730 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341130b0 con 0x7fbc341034b0 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc3b67d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341133b0 con 0x7fbc34107de0 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.484+0000 7fbc39c1a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 0x7fbc3419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 0x7fbc3419c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40916/0 (socket says 172.21.15.19:40916) 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 -- 172.21.15.19:0/2783706820 learned_addr learned my addr 172.21.15.19:0/2783706820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39419700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc341034b0 0x7fbc3419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc38c18700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc34107de0 0x7fbc3419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 -- 172.21.15.19:0/2783706820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc34107de0 msgr2=0x7fbc3419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc34107de0 0x7fbc3419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 -- 172.21.15.19:0/2783706820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc341034b0 msgr2=0x7fbc3419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc341034b0 0x7fbc3419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 -- 172.21.15.19:0/2783706820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc341a6ec0 con 0x7fbc34108730 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc39c1a700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 0x7fbc3419c610 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00ef30 tx=0x7fbc2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0092e0 con 0x7fbc34108730 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc2c004bb0 con 0x7fbc34108730 2024-05-17T09:38:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c004020 con 0x7fbc34108730 2024-05-17T09:38:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc341a7150 con 0x7fbc34108730 2024-05-17T09:38:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.485+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a7640 con 0x7fbc34108730 2024-05-17T09:38:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.486+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc340512b0 con 0x7fbc34108730 2024-05-17T09:38:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.488+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbc2c01b030 con 0x7fbc34108730 2024-05-17T09:38:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.489+0000 7fbc2a7fc700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc200b2880 0x7fbc200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:38:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.489+0000 7fbc39419700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc200b2880 0x7fbc200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:38:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.489+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbc2c0e2a90 con 0x7fbc34108730 2024-05-17T09:38:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.489+0000 7fbc39419700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc200b2880 0x7fbc200b4d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbc30000c00 tx=0x7fbc3001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:38:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.492+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc2c022020 con 0x7fbc34108730 2024-05-17T09:38:59.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 5.94 MB/sec execute 176 sec latency 18115.520 ms 2024-05-17T09:38:59.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.754+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc3402e5e0 con 0x7fbc34108730 2024-05-17T09:38:59.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.755+0000 7fbc2a7fc700 1 -- 172.21.15.19:0/2783706820 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fbc2c010040 con 0x7fbc34108730 2024-05-17T09:38:59.755 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:38:59.755 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:38:59.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc200b2880 msgr2=0x7fbc200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc200b2880 0x7fbc200b4d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbc30000c00 tx=0x7fbc3001a040 comp rx=0 tx=0).stop 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 msgr2=0x7fbc3419c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 0x7fbc3419c610 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00ef30 tx=0x7fbc2c00c3a0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 shutdown_connections 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc34108730 0x7fbc3419c610 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc341034b0 0x7fbc3419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbc200b2880 0x7fbc200b4d90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 --2- 172.21.15.19:0/2783706820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc34107de0 0x7fbc3419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 >> 172.21.15.19:0/2783706820 conn(0x7fbc340fedb0 msgr2=0x7fbc341011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 shutdown_connections 2024-05-17T09:38:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:38:59.757+0000 7fbc3b67d700 1 -- 172.21.15.19:0/2783706820 wait complete. 2024-05-17T09:38:59.758 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:39:00.412 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 334c53bf --release --await' 2024-05-17T09:39:00.412 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 334c53bf --release --await 2024-05-17T09:39:00.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 5.91 MB/sec execute 177 sec latency 19115.551 ms 2024-05-17T09:39:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:39:00.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:00 smithi069 ceph-mon[35081]: pgmap v1015: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-17T09:39:00.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:00 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2783706820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:39:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:00 smithi019 ceph-mon[31620]: pgmap v1015: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-17T09:39:00.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:00 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2783706820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:39:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:00 smithi079 ceph-mon[35502]: pgmap v1015: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-17T09:39:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:00 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2783706820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:39:01.580 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34106 5.87 MB/sec execute 178 sec latency 20115.587 ms 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.441+0000 7f18a232a700 1 -- 172.21.15.19:0/1622147331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c108740 msgr2=0x7f189c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.441+0000 7f18a232a700 1 --2- 172.21.15.19:0/1622147331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c108740 0x7f189c1100a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f189400b0d0 tx=0x7f189400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 -- 172.21.15.19:0/1622147331 shutdown_connections 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 --2- 172.21.15.19:0/1622147331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c108740 0x7f189c1100a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 --2- 172.21.15.19:0/1622147331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f189c1034c0 0x7f189c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 --2- 172.21.15.19:0/1622147331 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f189c107df0 0x7f189c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 -- 172.21.15.19:0/1622147331 >> 172.21.15.19:0/1622147331 conn(0x7f189c0fef10 msgr2=0x7f189c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 -- 172.21.15.19:0/1622147331 shutdown_connections 2024-05-17T09:39:02.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 -- 172.21.15.19:0/1622147331 wait complete. 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 Processor -- start 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.442+0000 7f18a232a700 1 -- start start 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a232a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f189c1034c0 0x7f189c0ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a232a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 0x7f189c0ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a232a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f189c108740 0x7f189c0ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a232a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f189c113220 con 0x7f189c107df0 2024-05-17T09:39:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a232a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1130a0 con 0x7f189c1034c0 2024-05-17T09:39:02.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a232a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1133a0 con 0x7f189c108740 2024-05-17T09:39:02.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:02 smithi019 ceph-mon[31620]: pgmap v1016: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-05-17T09:39:02.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 0x7f189c0ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 0x7f189c0ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40934/0 (socket says 172.21.15.19:40934) 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 -- 172.21.15.19:0/2527282386 learned_addr learned my addr 172.21.15.19:0/2527282386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f189bfff700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f189c1034c0 0x7f189c0ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f18a08c7700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f189c108740 0x7f189c0ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 -- 172.21.15.19:0/2527282386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f189c108740 msgr2=0x7f189c0ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f189c108740 0x7f189c0ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 -- 172.21.15.19:0/2527282386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f189c1034c0 msgr2=0x7f189c0ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f189c1034c0 0x7f189c0ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.443+0000 7f1893fff700 1 -- 172.21.15.19:0/2527282386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1a90a0 con 0x7f189c107df0 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.444+0000 7f1893fff700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 0x7f189c0ff930 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f188400d980 tx=0x7f188400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.444+0000 7f1899ffb700 1 -- 172.21.15.19:0/2527282386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884017070 con 0x7f189c107df0 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.444+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a9390 con 0x7f189c107df0 2024-05-17T09:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.444+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a9920 con 0x7f189c107df0 2024-05-17T09:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.444+0000 7f1899ffb700 1 -- 172.21.15.19:0/2527282386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f188400fd20 con 0x7f189c107df0 2024-05-17T09:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.444+0000 7f1899ffb700 1 -- 172.21.15.19:0/2527282386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884010df0 con 0x7f189c107df0 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.447+0000 7f1899ffb700 1 -- 172.21.15.19:0/2527282386 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f188400e3b0 con 0x7f189c107df0 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 Processor -- start 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f1899ffb700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18880b2880 0x7f18880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 -- start start 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18a08c7700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18880b2880 0x7f18880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f1899ffb700 1 -- 172.21.15.19:0/2527282386 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f18840e1fd0 con 0x7f189c107df0 2024-05-17T09:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a26e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a26eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a268a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f6a196d90 con 0x563f6a0e9f40 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f6a186930 con 0x563f6a268620 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.448+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f6a18a970 con 0x563f6a1674d0 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a26eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a26eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40936/0 (socket says 172.21.15.19:40936) 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 -- 172.21.15.19:0/3854829544 learned_addr learned my addr 172.21.15.19:0/3854829544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f18a08c7700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18880b2880 0x7f18880b4d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f189400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f18a08c7700 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a268a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f189bfff700 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a26e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 -- 172.21.15.19:0/3854829544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 msgr2=0x563f6a26e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a26e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 -- 172.21.15.19:0/3854829544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 msgr2=0x563f6a268a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a268a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 -- 172.21.15.19:0/3854829544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a0d69d0 con 0x563f6a0e9f40 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f1893fff700 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a26eaa0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f18840a5260 tx=0x7f18840a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7fb2fea182a7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f18927fc700 1 -- 172.21.15.19:0/3854829544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884020070 con 0x563f6a0e9f40 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f18927fc700 1 -- 172.21.15.19:0/3854829544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18840ad590 con 0x563f6a0e9f40 2024-05-17T09:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.449+0000 7f18927fc700 1 -- 172.21.15.19:0/3854829544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840dfb80 con 0x563f6a0e9f40 2024-05-17T09:39:02.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/3854829544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 msgr2=0x563f6a26eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a26eaa0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f18840a5260 tx=0x7f18840a98e0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/3854829544 shutdown_connections 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a26e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a268a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/3854829544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a26eaa0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/3854829544 >> 172.21.15.19:0/3854829544 conn(0x563f6a128d80 msgr2=0x563f6a263520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/3854829544 shutdown_connections 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.450+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/3854829544 wait complete. 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 -- start start 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 Processor -- start 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a2720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 0x563f6a272620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a272b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f6a08d0a0 con 0x563f6a1674d0 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f6a1288c0 con 0x563f6a268620 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563f6a2693e0 con 0x563f6a0e9f40 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f18b0d3fb80 10 client.0 _ll_register_callbacks cb 0x563f6a121a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f1893fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 0x563f6a272620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f1893fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 0x563f6a272620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40938/0 (socket says 172.21.15.19:40938) 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.451+0000 7f1893fff700 1 -- 172.21.15.19:0/1539807421 learned_addr learned my addr 172.21.15.19:0/1539807421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f18a08c7700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a272b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f189bfff700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a2720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f1893fff700 1 -- 172.21.15.19:0/1539807421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 msgr2=0x563f6a2720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f1893fff700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a2720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f1893fff700 1 -- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 msgr2=0x563f6a272b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f1893fff700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a272b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f1893fff700 1 -- 172.21.15.19:0/1539807421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6a125790 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f1893fff700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 0x563f6a272620 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f18840109a0 tx=0x7f18840ad3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7ca0616019a93a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f187b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f187b7fe700 1 -- 172.21.15.19:0/1539807421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840c2020 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f187b7fe700 1 -- 172.21.15.19:0/1539807421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f18840b1dc0 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18840ad590 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f187b7fe700 1 -- 172.21.15.19:0/1539807421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1884020070 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f18b0d3fb80 10 client.18435 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.452+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563f6a27fa00 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f187b7fe700 1 -- 172.21.15.19:0/1539807421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18840ad590 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f187b7fe700 1 -- 172.21.15.19:0/1539807421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f18840a8440 con 0x563f6a1674d0 2024-05-17T09:39:02.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f18b0d3fb80 10 client.18435 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f18b0d3fb80 10 client.18435 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f18b0d3fb80 20 client.18435 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a281ed0 0x563f6a2842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f18b0d3fb80 4 client.18435 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f6a198ce0 con 0x563f6a281ed0 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.453+0000 7f189bfff700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a281ed0 0x563f6a2842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.454+0000 7f187b7fe700 10 client.18435 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.454+0000 7f189bfff700 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a281ed0 0x563f6a2842f0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9124c3cb44d41bc0 server_cookie=4c29440421bd09e0 in_seq=0 out_seq=0 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.454+0000 7f187b7fe700 1 -- 172.21.15.19:0/1539807421 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563f6a198ce0 con 0x563f6a281ed0 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.454+0000 7f187b7fe700 10 client.18435 handle_command_reply: tid=0 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.454+0000 7f18b0d3fb80 1 client.18435 shutdown 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 msgr2=0x563f6a272620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 0x563f6a272620 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f18840109a0 tx=0x7f18840ad3d0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 shutdown_connections 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a2720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a281ed0 0x563f6a2842f0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a272b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1539807421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a1674d0 0x563f6a272620 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f187b7fe700 0 client.18435 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 >> 172.21.15.19:0/1539807421 conn(0x563f6a128d80 msgr2=0x563f6a25d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 shutdown_connections 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1539807421 wait complete. 2024-05-17T09:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.455+0000 7f18b0d3fb80 20 client.18435 trim_cache size 0 max 16384 2024-05-17T09:39:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.475+0000 7f18b0d3fb80 1 Processor -- start 2024-05-17T09:39:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 -- start start 2024-05-17T09:39:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a272650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a1674d0 0x563f6a286f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 0x563f6a287440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f187c011400 con 0x563f6a268620 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f187c00ec90 con 0x563f6a1674d0 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f187c00cae0 con 0x563f6a0e9f40 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 0x563f6a287440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 0x563f6a287440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40944/0 (socket says 172.21.15.19:40944) 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 -- 172.21.15.19:0/1899521581 learned_addr learned my addr 172.21.15.19:0/1899521581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f1893fff700 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a1674d0 0x563f6a286f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f189bfff700 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a272650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 -- 172.21.15.19:0/1899521581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 msgr2=0x563f6a272650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a272650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 -- 172.21.15.19:0/1899521581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a1674d0 msgr2=0x563f6a286f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a1674d0 0x563f6a286f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 -- 172.21.15.19:0/1899521581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a263a60 con 0x563f6a268620 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f1893fff700 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a1674d0 0x563f6a286f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.476+0000 7f18a08c7700 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 0x563f6a287440 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f189400b0d0 tx=0x7f1894020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cb8f13770fb4bd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.477+0000 7f1891ffb700 1 -- 172.21.15.19:0/1899521581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894038070 con 0x563f6a268620 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.477+0000 7f1891ffb700 1 -- 172.21.15.19:0/1899521581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1894007c80 con 0x563f6a268620 2024-05-17T09:39:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.477+0000 7f1891ffb700 1 -- 172.21.15.19:0/1899521581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894022070 con 0x563f6a268620 2024-05-17T09:39:02.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1899521581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 msgr2=0x563f6a287440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 0x563f6a287440 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f189400b0d0 tx=0x7f1894020000 comp rx=0 tx=0).stop 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1899521581 shutdown_connections 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a0e9f40 0x563f6a272650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a268620 0x563f6a287440 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/1899521581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a1674d0 0x563f6a286f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1899521581 >> 172.21.15.19:0/1899521581 conn(0x563f6a128d80 msgr2=0x563f6a270c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1899521581 shutdown_connections 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.478+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/1899521581 wait complete. 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 -- start start 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 Processor -- start 2024-05-17T09:39:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a284ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a285530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a2eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f187c00d150 con 0x563f6a0e9f40 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f187c00e5c0 con 0x563f6a268620 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.479+0000 7f18b0d3fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f187c00d820 con 0x563f6a1674d0 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18b0d3fb80 10 client.0 _ll_register_callbacks cb 0x563f6a121a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f189bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a284ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a2eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a2eb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:40840/0 (socket says 172.21.15.19:40840) 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 -- 172.21.15.19:0/2015814234 learned_addr learned my addr 172.21.15.19:0/2015814234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f189bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a284ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40958/0 (socket says 172.21.15.19:40958) 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f1893fff700 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a285530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 -- 172.21.15.19:0/2015814234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 msgr2=0x563f6a285530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a285530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 -- 172.21.15.19:0/2015814234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 msgr2=0x563f6a284ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a284ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 -- 172.21.15.19:0/2015814234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6a285dc0 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f18a08c7700 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a2eb740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f189400b8a0 tx=0x7f1894028b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a62a3743227dfa00 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.480+0000 7f187b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f187b7fe700 1 -- 172.21.15.19:0/2015814234 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894041060 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894007c80 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f187b7fe700 1 -- 172.21.15.19:0/2015814234 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f189400f040 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f187b7fe700 1 -- 172.21.15.19:0/2015814234 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1894038030 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 10 client.27974 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563f6a2ecce0 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f187b7fe700 1 -- 172.21.15.19:0/2015814234 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1894007c80 con 0x563f6a268620 2024-05-17T09:39:02.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f187b7fe700 1 -- 172.21.15.19:0/2015814234 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1894032070 con 0x563f6a268620 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 10 client.27974 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 10 client.27974 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 20 client.27974 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a2ed230 0x563f6a2ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 4 client.27974 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "334c53bf", "release": true, "await": true}] 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.481+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "334c53bf", "release": true, "await": true}) -- 0x563f6a1924c0 con 0x563f6a2ed230 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.482+0000 7f189bfff700 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a2ed230 0x563f6a2ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.482+0000 7f187b7fe700 10 client.27974 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:02.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.482+0000 7f189bfff700 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a2ed230 0x563f6a2ef650 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19105b669ddd10 server_cookie=6284edef80300308 in_seq=0 out_seq=0 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.505+0000 7f187b7fe700 1 -- 172.21.15.19:0/2015814234 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563f6a1924c0 con 0x563f6a2ed230 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.505+0000 7f187b7fe700 10 client.27974 handle_command_reply: tid=0 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.505+0000 7f18b0d3fb80 1 client.27974 shutdown 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 msgr2=0x563f6a2eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a2eb740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f189400b8a0 tx=0x7f1894028b60 comp rx=0 tx=0).stop 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 shutdown_connections 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x563f6a2ed230 0x563f6a2ef650 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f6a268620 0x563f6a2eb740 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f6a0e9f40 0x563f6a284ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 --2- 172.21.15.19:0/2015814234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563f6a1674d0 0x563f6a285530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 >> 172.21.15.19:0/2015814234 conn(0x563f6a128d80 msgr2=0x563f6a25d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f187b7fe700 0 client.27974 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 shutdown_connections 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 1 -- 172.21.15.19:0/2015814234 wait complete. 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.506+0000 7f18b0d3fb80 20 client.27974 trim_cache size 0 max 16384 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:39:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 68, 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "334c53bf": { 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 68, 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.2, 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:39:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:39:02.509 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18880b2880 msgr2=0x7f18880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18880b2880 0x7f18880b4d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f189400b040 comp rx=0 tx=0).stop 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 msgr2=0x7f189c0ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 0x7f189c0ff930 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f188400d980 tx=0x7f188400dc90 comp rx=0 tx=0).stop 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 shutdown_connections 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f18880b2880 0x7f18880b4d90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f189c108740 0x7f189c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f189c1034c0 0x7f189c0ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 --2- 172.21.15.19:0/2527282386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f189c107df0 0x7f189c0ff930 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:02.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 >> 172.21.15.19:0/2527282386 conn(0x7f189c0fef10 msgr2=0x7f189c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:02.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 shutdown_connections 2024-05-17T09:39:02.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:39:02.507+0000 7f18a232a700 1 -- 172.21.15.19:0/2527282386 wait complete. 2024-05-17T09:39:02.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34113 5.84 MB/sec execute 179 sec latency 21066.799 ms 2024-05-17T09:39:02.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:02 smithi069 ceph-mon[35081]: pgmap v1016: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-05-17T09:39:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:02 smithi079 ceph-mon[35502]: pgmap v1016: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-05-17T09:39:03.271 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '334c53bf', total seconds elapsed: 28.3 2024-05-17T09:39:03.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34269 5.87 MB/sec execute 180 sec latency 59.642 ms 2024-05-17T09:39:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 -- 172.21.15.19:0/3493778193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310107090 msgr2=0x7f73101094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 --2- 172.21.15.19:0/3493778193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310107090 0x7f73101094b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7304009900 tx=0x7f730402a410 comp rx=0 tx=0).stop 2024-05-17T09:39:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 -- 172.21.15.19:0/3493778193 shutdown_connections 2024-05-17T09:39:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 --2- 172.21.15.19:0/3493778193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73101099f0 0x7f731010be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 --2- 172.21.15.19:0/3493778193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310107090 0x7f73101094b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 --2- 172.21.15.19:0/3493778193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310104730 0x7f7310106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 -- 172.21.15.19:0/3493778193 >> 172.21.15.19:0/3493778193 conn(0x7f73100fdd30 msgr2=0x7f73101001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 -- 172.21.15.19:0/3493778193 shutdown_connections 2024-05-17T09:39:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.991+0000 7f7315b00640 1 -- 172.21.15.19:0/3493778193 wait complete. 2024-05-17T09:39:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.992+0000 7f7315b00640 1 Processor -- start 2024-05-17T09:39:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.992+0000 7f7315b00640 1 -- start start 2024-05-17T09:39:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f7315b00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310104730 0x7f731019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f7315b00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 0x7f731019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f7315b00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73101099f0 0x7f73101a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f7315b00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f731010f620 con 0x7f73101099f0 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f7315b00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f731010f4a0 con 0x7f7310104730 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f7315b00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f731010f7a0 con 0x7f7310107090 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310104730 0x7f731019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 0x7f731019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73101099f0 0x7f73101a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 0x7f731019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41376/0 (socket says 172.21.15.19:41376) 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 -- 172.21.15.19:0/4006487558 learned_addr learned my addr 172.21.15.19:0/4006487558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 -- 172.21.15.19:0/4006487558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310104730 msgr2=0x7f731019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310104730 0x7f731019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 -- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73101099f0 msgr2=0x7f73101a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73101099f0 0x7f73101a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730effd640 1 -- 172.21.15.19:0/4006487558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73101a3f90 con 0x7f7310107090 2024-05-17T09:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.993+0000 7f730f7fe640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310104730 0x7f731019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.994+0000 7f730effd640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 0x7f731019c990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7304009230 tx=0x7f7304005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.994+0000 7f730cff9640 1 -- 172.21.15.19:0/4006487558 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730403f030 con 0x7f7310107090 2024-05-17T09:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.994+0000 7f730cff9640 1 -- 172.21.15.19:0/4006487558 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7304004350 con 0x7f7310107090 2024-05-17T09:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.994+0000 7f730cff9640 1 -- 172.21.15.19:0/4006487558 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304035070 con 0x7f7310107090 2024-05-17T09:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.994+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73101a4220 con 0x7f7310107090 2024-05-17T09:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.994+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7310194f40 con 0x7f7310107090 2024-05-17T09:39:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.997+0000 7f731c9b3740 1 Processor -- start 2024-05-17T09:39:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.997+0000 7f731c9b3740 1 -- start start 2024-05-17T09:39:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.997+0000 7f730cff9640 1 -- 172.21.15.19:0/4006487558 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7304042030 con 0x7f7310107090 2024-05-17T09:39:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664968f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x55664968f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x556649696820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f731c9b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556649601c30 con 0x5566492c52f0 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f731c9b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5566495f17d0 con 0x5566492ee1b0 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f731c9b3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5566495f5810 con 0x5566492cd4d0 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664968f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x55664968f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664968f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35754/0 (socket says 172.21.15.19:35754) 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 -- 172.21.15.19:0/100268292 learned_addr learned my addr 172.21.15.19:0/100268292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730effd640 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x556649696820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 -- 172.21.15.19:0/100268292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 msgr2=0x556649696820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x556649696820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 -- 172.21.15.19:0/100268292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 msgr2=0x55664968f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x55664968f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.998+0000 7f730ffff640 1 -- 172.21.15.19:0/100268292 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556649306210 con 0x5566492ee1b0 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f730ffff640 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664968f3a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7308009f90 tx=0x7f73080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ed9c8c35cc995e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f730cff9640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72f80b29f0 0x7f72f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f72f5ffb640 1 -- 172.21.15.19:0/100268292 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004b20 con 0x5566492ee1b0 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f72f5ffb640 1 -- 172.21.15.19:0/100268292 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308004cc0 con 0x5566492ee1b0 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f72f5ffb640 1 -- 172.21.15.19:0/100268292 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004860 con 0x5566492ee1b0 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f730f7fe640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72f80b29f0 0x7f72f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:03.999+0000 7f730cff9640 1 -- 172.21.15.19:0/4006487558 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7304102870 con 0x7f7310107090 2024-05-17T09:39:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 -- 172.21.15.19:0/100268292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 msgr2=0x55664968f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f730f7fe640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72f80b29f0 0x7f72f80b4f00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x556649690780 tx=0x7f7300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664968f3a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7308009f90 tx=0x7f73080118d0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 -- 172.21.15.19:0/100268292 shutdown_connections 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x556649696820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x55664968f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 --2- 172.21.15.19:0/100268292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664968f3a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 -- 172.21.15.19:0/100268292 >> 172.21.15.19:0/100268292 conn(0x5566492e2300 msgr2=0x55664968ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 -- 172.21.15.19:0/100268292 shutdown_connections 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.000+0000 7f731c9b3740 1 -- 172.21.15.19:0/100268292 wait complete. 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.001+0000 7f731c9b3740 1 -- start start 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.001+0000 7f731c9b3740 1 Processor -- start 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x556649698d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 0x556649699260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492ee1b0 0x5566496a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566492d7790 con 0x5566492c52f0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5566495ad890 con 0x5566492cd4d0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556649697270 con 0x5566492ee1b0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 0x556649699260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x556649698d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 0x556649699260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35770/0 (socket says 172.21.15.19:35770) 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 -- 172.21.15.19:0/654078567 learned_addr learned my addr 172.21.15.19:0/654078567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730ffff640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492ee1b0 0x5566496a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f731c9b3740 10 client.0 _ll_register_callbacks cb 0x5566495b0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 -- 172.21.15.19:0/654078567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492ee1b0 msgr2=0x5566496a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492ee1b0 0x5566496a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 -- 172.21.15.19:0/654078567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 msgr2=0x556649698d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x556649698d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730effd640 1 -- 172.21.15.19:0/654078567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566493d2620 con 0x5566492cd4d0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730ffff640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492ee1b0 0x5566496a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.002+0000 7f730f7fe640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x556649698d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f730effd640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 0x556649699260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7304003fd0 tx=0x7f73040c5a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef1aa409bfadb2f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f72eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f72eeffd640 1 -- 172.21.15.19:0/654078567 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304105e60 con 0x5566492cd4d0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308004cc0 con 0x5566492cd4d0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f72eeffd640 1 -- 172.21.15.19:0/654078567 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f73040c9d40 con 0x5566492cd4d0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f72eeffd640 1 -- 172.21.15.19:0/654078567 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73040c5270 con 0x5566492cd4d0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f731c9b3740 10 client.27986 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5566496a3090 con 0x5566492cd4d0 2024-05-17T09:39:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.003+0000 7f72eeffd640 1 -- 172.21.15.19:0/654078567 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308004cc0 con 0x5566492cd4d0 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f72eeffd640 1 -- 172.21.15.19:0/654078567 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f73040d24b0 con 0x5566492cd4d0 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f731c9b3740 10 client.27986 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f731c9b3740 10 client.27986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f731c9b3740 20 client.27986 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f731c9b3740 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492acd80 0x5566496a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f731c9b3740 4 client.27986 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556649603b80 con 0x5566492acd80 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.004+0000 7f730f7fe640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492acd80 0x5566496a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.005+0000 7f72eeffd640 10 client.27986 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.005+0000 7f730f7fe640 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492acd80 0x5566496a68a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d72ec8db2775c0 server_cookie=f4fb5ba670927f0e in_seq=0 out_seq=0 2024-05-17T09:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f72eeffd640 1 -- 172.21.15.19:0/654078567 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556649603b80 con 0x5566492acd80 2024-05-17T09:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f72eeffd640 10 client.27986 handle_command_reply: tid=0 2024-05-17T09:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 client.27986 shutdown 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 msgr2=0x556649699260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 0x556649699260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7304003fd0 tx=0x7f73040c5a60 comp rx=0 tx=0).stop 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 shutdown_connections 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492acd80 0x5566496a68a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492ee1b0 0x5566496a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492cd4d0 0x556649699260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 --2- 172.21.15.19:0/654078567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492c52f0 0x556649698d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 >> 172.21.15.19:0/654078567 conn(0x5566492e2300 msgr2=0x55664968dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.007+0000 7f72eeffd640 0 client.27986 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.008+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 shutdown_connections 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.008+0000 7f731c9b3740 1 -- 172.21.15.19:0/654078567 wait complete. 2024-05-17T09:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.008+0000 7f731c9b3740 20 client.27986 trim_cache size 0 max 16384 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 Processor -- start 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 -- start start 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492c52f0 0x5566496ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x5566496a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492ee1b0 0x5566496ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b58c0 con 0x5566492ee1b0 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b3190 con 0x5566492c52f0 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f731c9b3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b0fe0 con 0x5566492cd4d0 2024-05-17T09:39:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f730effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x5566496a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f730f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492c52f0 0x5566496ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f730effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x5566496a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41412/0 (socket says 172.21.15.19:41412) 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.031+0000 7f730effd640 1 -- 172.21.15.19:0/1151118759 learned_addr learned my addr 172.21.15.19:0/1151118759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730ffff640 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492ee1b0 0x5566496ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730effd640 1 -- 172.21.15.19:0/1151118759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492c52f0 msgr2=0x5566496ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730effd640 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492c52f0 0x5566496ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730effd640 1 -- 172.21.15.19:0/1151118759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492ee1b0 msgr2=0x5566496ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730effd640 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492ee1b0 0x5566496ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730effd640 1 -- 172.21.15.19:0/1151118759 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664968b2c0 con 0x5566492cd4d0 2024-05-17T09:39:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f730effd640 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x5566496a00e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f73040c5b60 tx=0x7f73040cdff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26184fa0f3c711ea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f72f57fa640 1 -- 172.21.15.19:0/1151118759 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304105e60 con 0x5566492cd4d0 2024-05-17T09:39:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f72f57fa640 1 -- 172.21.15.19:0/1151118759 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73040d2280 con 0x5566492cd4d0 2024-05-17T09:39:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.032+0000 7f72f57fa640 1 -- 172.21.15.19:0/1151118759 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73040d2990 con 0x5566492cd4d0 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 -- 172.21.15.19:0/1151118759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 msgr2=0x5566496a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x5566496a00e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f73040c5b60 tx=0x7f73040cdff0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 -- 172.21.15.19:0/1151118759 shutdown_connections 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492ee1b0 0x5566496ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492cd4d0 0x5566496a00e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 --2- 172.21.15.19:0/1151118759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492c52f0 0x5566496ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 -- 172.21.15.19:0/1151118759 >> 172.21.15.19:0/1151118759 conn(0x5566492e2300 msgr2=0x5566496927c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 -- 172.21.15.19:0/1151118759 shutdown_connections 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.033+0000 7f731c9b3740 1 -- 172.21.15.19:0/1151118759 wait complete. 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.034+0000 7f731c9b3740 1 -- start start 2024-05-17T09:39:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.034+0000 7f731c9b3740 1 Processor -- start 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.034+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 0x55664969d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.034+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492cd4d0 0x55664969d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f731c9b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664969e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f731c9b3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b1650 con 0x5566492cd4d0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f731c9b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b2ac0 con 0x5566492ee1b0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f731c9b3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b1d20 con 0x5566492c52f0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f731c9b3740 10 client.0 _ll_register_callbacks cb 0x7f7304009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 0x55664969d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492cd4d0 0x55664969d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664969e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 0x55664969d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:41422/0 (socket says 172.21.15.19:41422) 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730f7fe640 1 -- 172.21.15.19:0/4248783317 learned_addr learned my addr 172.21.15.19:0/4248783317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664969e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35790/0 (socket says 172.21.15.19:35790) 2024-05-17T09:39:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 -- 172.21.15.19:0/4248783317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 msgr2=0x55664969d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 0x55664969d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 -- 172.21.15.19:0/4248783317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492cd4d0 msgr2=0x55664969d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492cd4d0 0x55664969d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 -- 172.21.15.19:0/4248783317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664969ec50 con 0x5566492ee1b0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730f7fe640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 0x55664969d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730effd640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492cd4d0 0x55664969d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f730ffff640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664969e690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7308005c10 tx=0x7f7308004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a1b622a0f5048f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.035+0000 7f72eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f72eeffd640 1 -- 172.21.15.19:0/4248783317 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730802f060 con 0x5566492ee1b0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f72eeffd640 1 -- 172.21.15.19:0/4248783317 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7308005250 con 0x5566492ee1b0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566495fd360 con 0x5566492ee1b0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f72eeffd640 1 -- 172.21.15.19:0/4248783317 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7308036050 con 0x5566492ee1b0 2024-05-17T09:39:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f731c9b3740 10 client.27992 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55664969fee0 con 0x5566492ee1b0 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f72eeffd640 1 -- 172.21.15.19:0/4248783317 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5566495fd360 con 0x5566492ee1b0 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.036+0000 7f72eeffd640 1 -- 172.21.15.19:0/4248783317 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f730800fa10 con 0x5566492ee1b0 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f731c9b3740 10 client.27992 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f731c9b3740 10 client.27992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f731c9b3740 20 client.27992 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f731c9b3740 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492821c0 0x556649710f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f731c9b3740 4 client.27992 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566495775f0 con 0x5566492821c0 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f730f7fe640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492821c0 0x556649710f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f72eeffd640 10 client.27992 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.037+0000 7f730f7fe640 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492821c0 0x556649710f80 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f86d6edbd12d43 server_cookie=4bb853ba3621b608 in_seq=0 out_seq=0 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f72eeffd640 1 -- 172.21.15.19:0/4248783317 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5566495775f0 con 0x5566492821c0 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f72eeffd640 10 client.27992 handle_command_reply: tid=0 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 client.27992 shutdown 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 msgr2=0x55664969e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664969e690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7308005c10 tx=0x7f7308004d50 comp rx=0 tx=0).stop 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 shutdown_connections 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5566492821c0 0x556649710f80 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566492ee1b0 0x55664969e690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566492cd4d0 0x55664969d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f72eeffd640 0 client.27992 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 --2- 172.21.15.19:0/4248783317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5566492c52f0 0x55664969d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 >> 172.21.15.19:0/4248783317 conn(0x5566492e2300 msgr2=0x5566496931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 shutdown_connections 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.112+0000 7f731c9b3740 1 -- 172.21.15.19:0/4248783317 wait complete. 2024-05-17T09:39:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f731c9b3740 20 client.27992 trim_cache size 0 max 16384 2024-05-17T09:39:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:39:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:39:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:39:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72f80b29f0 msgr2=0x7f72f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72f80b29f0 0x7f72f80b4f00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x556649690780 tx=0x7f7300005f90 comp rx=0 tx=0).stop 2024-05-17T09:39:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 msgr2=0x7f731019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 0x7f731019c990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7304009230 tx=0x7f7304005dd0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 shutdown_connections 2024-05-17T09:39:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72f80b29f0 0x7f72f80b4f00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73101099f0 0x7f73101a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7310107090 0x7f731019c990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 --2- 172.21.15.19:0/4006487558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7310104730 0x7f731019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.113+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 >> 172.21.15.19:0/4006487558 conn(0x7f73100fdd30 msgr2=0x7f7310102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.114+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 shutdown_connections 2024-05-17T09:39:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:04.114+0000 7f7315b00640 1 -- 172.21.15.19:0/4006487558 wait complete. 2024-05-17T09:39:04.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34396 5.86 MB/sec execute 181 sec latency 92.165 ms 2024-05-17T09:39:04.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:04 smithi069 ceph-mon[35081]: pgmap v1017: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-05-17T09:39:04.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:04 smithi019 ceph-mon[31620]: pgmap v1017: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-05-17T09:39:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:04 smithi079 ceph-mon[35502]: pgmap v1017: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-05-17T09:39:05.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34540 5.85 MB/sec execute 182 sec latency 89.883 ms 2024-05-17T09:39:06.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34695 5.84 MB/sec execute 183 sec latency 129.573 ms 2024-05-17T09:39:06.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:06 smithi069 ceph-mon[35081]: pgmap v1018: 129 pgs: 129 active+clean; 391 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 96 KiB/s wr, 26 op/s 2024-05-17T09:39:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:06 smithi019 ceph-mon[31620]: pgmap v1018: 129 pgs: 129 active+clean; 391 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 96 KiB/s wr, 26 op/s 2024-05-17T09:39:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:06 smithi079 ceph-mon[35502]: pgmap v1018: 129 pgs: 129 active+clean; 391 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 96 KiB/s wr, 26 op/s 2024-05-17T09:39:07.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34845 5.85 MB/sec execute 184 sec latency 72.834 ms 2024-05-17T09:39:08.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 34997 5.84 MB/sec execute 185 sec latency 73.210 ms 2024-05-17T09:39:08.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:08 smithi069 ceph-mon[35081]: pgmap v1019: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 384 KiB/s wr, 47 op/s 2024-05-17T09:39:08.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:08 smithi019 ceph-mon[31620]: pgmap v1019: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 384 KiB/s wr, 47 op/s 2024-05-17T09:39:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:08 smithi079 ceph-mon[35502]: pgmap v1019: 129 pgs: 129 active+clean; 390 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 384 KiB/s wr, 47 op/s 2024-05-17T09:39:09.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 35185 5.91 MB/sec execute 186 sec latency 62.722 ms 2024-05-17T09:39:10.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 35323 5.91 MB/sec execute 187 sec latency 119.881 ms 2024-05-17T09:39:10.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:10 smithi069 ceph-mon[35081]: pgmap v1020: 129 pgs: 129 active+clean; 389 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-05-17T09:39:10.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:10 smithi019 ceph-mon[31620]: pgmap v1020: 129 pgs: 129 active+clean; 389 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-05-17T09:39:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:10 smithi079 ceph-mon[35502]: pgmap v1020: 129 pgs: 129 active+clean; 389 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-05-17T09:39:11.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 35491 5.92 MB/sec execute 188 sec latency 66.226 ms 2024-05-17T09:39:12.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 35672 5.92 MB/sec execute 189 sec latency 76.231 ms 2024-05-17T09:39:12.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:12 smithi069 ceph-mon[35081]: pgmap v1021: 129 pgs: 129 active+clean; 387 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-17T09:39:12.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:12 smithi019 ceph-mon[31620]: pgmap v1021: 129 pgs: 129 active+clean; 387 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-17T09:39:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:12 smithi079 ceph-mon[35502]: pgmap v1021: 129 pgs: 129 active+clean; 387 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-17T09:39:13.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 35842 5.92 MB/sec execute 190 sec latency 73.590 ms 2024-05-17T09:39:14.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 35997 5.93 MB/sec execute 191 sec latency 61.860 ms 2024-05-17T09:39:14.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:14 smithi069 ceph-mon[35081]: pgmap v1022: 129 pgs: 129 active+clean; 388 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-17T09:39:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:14 smithi019 ceph-mon[31620]: pgmap v1022: 129 pgs: 129 active+clean; 388 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-17T09:39:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:14 smithi079 ceph-mon[35502]: pgmap v1022: 129 pgs: 129 active+clean; 388 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-17T09:39:15.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 36157 5.95 MB/sec execute 192 sec latency 68.060 ms 2024-05-17T09:39:16.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 36327 5.99 MB/sec execute 193 sec latency 95.370 ms 2024-05-17T09:39:16.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:16 smithi069 ceph-mon[35081]: pgmap v1023: 129 pgs: 129 active+clean; 390 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-05-17T09:39:16.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:16 smithi019 ceph-mon[31620]: pgmap v1023: 129 pgs: 129 active+clean; 390 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-05-17T09:39:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:16 smithi079 ceph-mon[35502]: pgmap v1023: 129 pgs: 129 active+clean; 390 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-05-17T09:39:17.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 36513 6.05 MB/sec execute 194 sec latency 92.179 ms 2024-05-17T09:39:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 -- 172.21.15.19:0/790787150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c103d80 msgr2=0x7fdf9c104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 --2- 172.21.15.19:0/790787150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c103d80 0x7fdf9c104190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009860 tx=0x7fdf8c02a340 comp rx=0 tx=0).stop 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 -- 172.21.15.19:0/790787150 shutdown_connections 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 --2- 172.21.15.19:0/790787150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c108a80 0x7fdf9c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 --2- 172.21.15.19:0/790787150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c103d80 0x7fdf9c104190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 --2- 172.21.15.19:0/790787150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 -- 172.21.15.19:0/790787150 >> 172.21.15.19:0/790787150 conn(0x7fdf9c0fdd30 msgr2=0x7fdf9c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 -- 172.21.15.19:0/790787150 shutdown_connections 2024-05-17T09:39:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.776+0000 7fdfa3384640 1 -- 172.21.15.19:0/790787150 wait complete. 2024-05-17T09:39:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 Processor -- start 2024-05-17T09:39:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 -- start start 2024-05-17T09:39:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c103d80 0x7fdf9c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57698/0 (socket says 172.21.15.19:57698) 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa1120640 1 -- 172.21.15.19:0/1615404700 learned_addr learned my addr 172.21.15.19:0/1615404700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c108a80 0x7fdf9c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10e4f0 con 0x7fdf9c102b40 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10e370 con 0x7fdf9c103d80 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10e670 con 0x7fdf9c108a80 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.777+0000 7fdfa091f640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c103d80 0x7fdf9c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa1921640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c108a80 0x7fdf9c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa091f640 1 -- 172.21.15.19:0/1615404700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c108a80 msgr2=0x7fdf9c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa091f640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c108a80 0x7fdf9c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa091f640 1 -- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 msgr2=0x7fdf9c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa091f640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa091f640 1 -- 172.21.15.19:0/1615404700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1a1cf0 con 0x7fdf9c103d80 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa1120640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa1921640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c108a80 0x7fdf9c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa091f640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c103d80 0x7fdf9c19a6f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c02a860 tx=0x7fdf8c02ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdf927fc640 1 -- 172.21.15.19:0/1615404700 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c040030 con 0x7fdf9c103d80 2024-05-17T09:39:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdf927fc640 1 -- 172.21.15.19:0/1615404700 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf8c030ab0 con 0x7fdf9c103d80 2024-05-17T09:39:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.778+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c1a1f80 con 0x7fdf9c103d80 2024-05-17T09:39:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.779+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c1a23e0 con 0x7fdf9c103d80 2024-05-17T09:39:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.779+0000 7fdf927fc640 1 -- 172.21.15.19:0/1615404700 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c037040 con 0x7fdf9c103d80 2024-05-17T09:39:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 Processor -- start 2024-05-17T09:39:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 -- start start 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdf927fc640 1 -- 172.21.15.19:0/1615404700 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdf8c043070 con 0x7fdf9c103d80 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 0x55f300eac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300eacd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f300e1ac30 con 0x55f300ade2f0 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f300e0a7d0 con 0x55f300b071b0 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f300e0e810 con 0x55f300ae64d0 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfa1921640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 0x55f300eac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfa1921640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 0x55f300eac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:56126/0 (socket says 172.21.15.19:56126) 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfa1921640 1 -- 172.21.15.19:0/358090870 learned_addr learned my addr 172.21.15.19:0/358090870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.782+0000 7fdfa1120640 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300eacd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa091f640 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1921640 1 -- 172.21.15.19:0/358090870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 msgr2=0x55f300eb3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1921640 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1921640 1 -- 172.21.15.19:0/358090870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 msgr2=0x55f300eacd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1921640 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300eacd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1921640 1 -- 172.21.15.19:0/358090870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f300b1f210 con 0x55f300b071b0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1120640 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300eacd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdfa1921640 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 0x55f300eac7b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009fd0 tx=0x7fdf940108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4607394bf2effd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdf927fc640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdf880b2920 0x7fdf880b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.783+0000 7fdf877fe640 1 -- 172.21.15.19:0/358090870 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011760 con 0x55f300b071b0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.784+0000 7fdfa1120640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdf880b2920 0x7fdf880b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.784+0000 7fdf877fe640 1 -- 172.21.15.19:0/358090870 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf94011d90 con 0x55f300b071b0 2024-05-17T09:39:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.784+0000 7fdf927fc640 1 -- 172.21.15.19:0/1615404700 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdf8c1022a0 con 0x7fdf9c103d80 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.784+0000 7fdf877fe640 1 -- 172.21.15.19:0/358090870 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9401be50 con 0x55f300b071b0 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfa1120640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdf880b2920 0x7fdf880b4e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009fd0 tx=0x7fdf98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 -- 172.21.15.19:0/358090870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 msgr2=0x55f300eac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 0x55f300eac7b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009fd0 tx=0x7fdf940108e0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 -- 172.21.15.19:0/358090870 shutdown_connections 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300eacd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/358090870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300b071b0 0x55f300eac7b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 -- 172.21.15.19:0/358090870 >> 172.21.15.19:0/358090870 conn(0x55f300afb300 msgr2=0x55f300ea8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 -- 172.21.15.19:0/358090870 shutdown_connections 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 -- 172.21.15.19:0/358090870 wait complete. 2024-05-17T09:39:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 -- start start 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.785+0000 7fdfaa23b740 1 Processor -- start 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300eb6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300b071b0 0x55f300ebd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f300af0790 con 0x55f300b071b0 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f300dc6890 con 0x55f300ade2f0 2024-05-17T09:39:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f300eb46d0 con 0x55f300ae64d0 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfaa23b740 10 client.0 _ll_register_callbacks cb 0x55f300dc9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300eb6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300eb6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:56128/0 (socket says 172.21.15.19:56128) 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa1120640 1 -- 172.21.15.19:0/2348657570 learned_addr learned my addr 172.21.15.19:0/2348657570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa091f640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa1921640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300b071b0 0x55f300ebd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa091f640 1 -- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 msgr2=0x55f300eb6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa091f640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300eb6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa091f640 1 -- 172.21.15.19:0/2348657570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300b071b0 msgr2=0x55f300ebd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa091f640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300b071b0 0x55f300ebd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.786+0000 7fdfa091f640 1 -- 172.21.15.19:0/2348657570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f300beb620 con 0x55f300ae64d0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdfa091f640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb66b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c005e00 tx=0x7fdf8c0c94d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=636fd5c7f8bad20e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdf84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdf84ff9640 1 -- 172.21.15.19:0/2348657570 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0c9100 con 0x55f300ae64d0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf94011d90 con 0x55f300ae64d0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdf84ff9640 1 -- 172.21.15.19:0/2348657570 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdf8c0b10f0 con 0x55f300ae64d0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdf84ff9640 1 -- 172.21.15.19:0/2348657570 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c0c95f0 con 0x55f300ae64d0 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdfaa23b740 10 client.27991 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f300ec0540 con 0x55f300ae64d0 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.787+0000 7fdf84ff9640 1 -- 172.21.15.19:0/2348657570 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf94011d90 con 0x55f300ae64d0 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdf84ff9640 1 -- 172.21.15.19:0/2348657570 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdf8c04d060 con 0x55f300ae64d0 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfaa23b740 10 client.27991 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfaa23b740 10 client.27991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfaa23b740 20 client.27991 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300ac5d80 0x55f300ec3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfaa23b740 4 client.27991 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f300e1cb80 con 0x55f300ac5d80 2024-05-17T09:39:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.788+0000 7fdfa1120640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300ac5d80 0x55f300ec3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.789+0000 7fdf84ff9640 10 client.27991 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.789+0000 7fdfa1120640 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300ac5d80 0x55f300ec3a70 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4e4ac474e79bbe server_cookie=5c1c30ee8d319585 in_seq=0 out_seq=0 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdf84ff9640 1 -- 172.21.15.19:0/2348657570 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f300e1cb80 con 0x55f300ac5d80 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdf84ff9640 10 client.27991 handle_command_reply: tid=0 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 client.27991 shutdown 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 msgr2=0x55f300eb66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb66b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c005e00 tx=0x7fdf8c0c94d0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 shutdown_connections 2024-05-17T09:39:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300ac5d80 0x55f300ec3a70 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300b071b0 0x55f300ebd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdf84ff9640 0 client.27991 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eb66b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/2348657570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300eb6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.790+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 >> 172.21.15.19:0/2348657570 conn(0x55f300afb300 msgr2=0x55f300ea2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.791+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 shutdown_connections 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.791+0000 7fdfaa23b740 1 -- 172.21.15.19:0/2348657570 wait complete. 2024-05-17T09:39:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.791+0000 7fdfaa23b740 20 client.27991 trim_cache size 0 max 16384 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.813+0000 7fdfaa23b740 1 Processor -- start 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.813+0000 7fdfaa23b740 1 -- start start 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 0x55f300eb4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300ebd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1acf60 con 0x55f300ade2f0 2024-05-17T09:39:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1aa830 con 0x55f300ac5d80 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1a8680 con 0x55f300ae64d0 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 0x55f300eb4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 0x55f300eb4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:56142/0 (socket says 172.21.15.19:56142) 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa091f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300ebd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa1120640 1 -- 172.21.15.19:0/3374929413 learned_addr learned my addr 172.21.15.19:0/3374929413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa1921640 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa091f640 1 -- 172.21.15.19:0/3374929413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 msgr2=0x55f300eab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa091f640 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa091f640 1 -- 172.21.15.19:0/3374929413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 msgr2=0x55f300eb4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa091f640 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 0x55f300eb4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa091f640 1 -- 172.21.15.19:0/3374929413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f300e9dfd0 con 0x55f300ade2f0 2024-05-17T09:39:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa1120640 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 0x55f300eb4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.814+0000 7fdfa1921640 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.815+0000 7fdfa091f640 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300ebd530 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0c6860 tx=0x7fdf8c1058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f24127176085c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.815+0000 7fdf86ffd640 1 -- 172.21.15.19:0/3374929413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0c9100 con 0x55f300ade2f0 2024-05-17T09:39:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.815+0000 7fdf86ffd640 1 -- 172.21.15.19:0/3374929413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf8c0ffe50 con 0x55f300ade2f0 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.815+0000 7fdf86ffd640 1 -- 172.21.15.19:0/3374929413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0afdc0 con 0x55f300ade2f0 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 -- 172.21.15.19:0/3374929413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 msgr2=0x55f300ebd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300ebd530 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0c6860 tx=0x7fdf8c1058c0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 -- 172.21.15.19:0/3374929413 shutdown_connections 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300eab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ade2f0 0x55f300ebd530 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/3374929413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ac5d80 0x55f300eb4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 -- 172.21.15.19:0/3374929413 >> 172.21.15.19:0/3374929413 conn(0x55f300afb300 msgr2=0x55f300eaaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.816+0000 7fdfaa23b740 1 -- 172.21.15.19:0/3374929413 shutdown_connections 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 -- 172.21.15.19:0/3374929413 wait complete. 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 -- start start 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 Processor -- start 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 0x55f300eba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300ebac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300ebbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1a8cf0 con 0x55f300ac5d80 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1aa160 con 0x55f300ade2f0 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1a93c0 con 0x55f300ae64d0 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfaa23b740 10 client.0 _ll_register_callbacks cb 0x7fdf8c0c5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.817+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 0x55f300eba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa1120640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 0x55f300eba700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57742/0 (socket says 172.21.15.19:57742) 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa1120640 1 -- 172.21.15.19:0/442643865 learned_addr learned my addr 172.21.15.19:0/442643865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300ebac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa1921640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300ebbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 -- 172.21.15.19:0/442643865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 msgr2=0x55f300ebbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300ebbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 -- 172.21.15.19:0/442643865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 msgr2=0x55f300eba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 0x55f300eba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 -- 172.21.15.19:0/442643865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f300ebc130 con 0x55f300ade2f0 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa1120640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 0x55f300eba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfa091f640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300ebac40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0c5670 tx=0x7fdf8c0056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537d081e1d4d0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdf84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdf84ff9640 1 -- 172.21.15.19:0/442643865 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0c9100 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdf84ff9640 1 -- 172.21.15.19:0/442643865 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdf8c0c8850 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.818+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c0ffe50 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdf84ff9640 1 -- 172.21.15.19:0/442643865 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c004490 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 10 client.28010 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f300ebc130 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdf84ff9640 1 -- 172.21.15.19:0/442643865 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf8c0ffe50 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdf84ff9640 1 -- 172.21.15.19:0/442643865 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdf8c0ea390 con 0x55f300ade2f0 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 10 client.28010 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 10 client.28010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 20 client.28010 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300b071b0 0x55f300f2e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 4 client.28010 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f300e16360 con 0x55f300b071b0 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.819+0000 7fdfa1120640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300b071b0 0x55f300f2e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.820+0000 7fdf84ff9640 10 client.28010 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.820+0000 7fdfa1120640 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300b071b0 0x55f300f2e150 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8765317feb99745 server_cookie=764be33e073a2b82 in_seq=0 out_seq=0 2024-05-17T09:39:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdf84ff9640 1 -- 172.21.15.19:0/442643865 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f300e16360 con 0x55f300b071b0 2024-05-17T09:39:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdf84ff9640 10 client.28010 handle_command_reply: tid=0 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 client.28010 shutdown 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 msgr2=0x55f300ebac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300ebac40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0c5670 tx=0x7fdf8c0056a0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 shutdown_connections 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f300b071b0 0x55f300f2e150 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f300ae64d0 0x55f300ebbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f300ade2f0 0x55f300ebac40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdf84ff9640 0 client.28010 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 --2- 172.21.15.19:0/442643865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f300ac5d80 0x55f300eba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.930+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 >> 172.21.15.19:0/442643865 conn(0x55f300afb300 msgr2=0x55f300eab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 shutdown_connections 2024-05-17T09:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfaa23b740 1 -- 172.21.15.19:0/442643865 wait complete. 2024-05-17T09:39:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfaa23b740 20 client.28010 trim_cache size 0 max 16384 2024-05-17T09:39:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:39:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:39:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdf880b2920 msgr2=0x7fdf880b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdf880b2920 0x7fdf880b4e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009fd0 tx=0x7fdf98009260 comp rx=0 tx=0).stop 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c103d80 msgr2=0x7fdf9c19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.931+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c103d80 0x7fdf9c19a6f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c02a860 tx=0x7fdf8c02ad70 comp rx=0 tx=0).stop 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 shutdown_connections 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf9c108a80 0x7fdf9c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf9c103d80 0x7fdf9c19a6f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdf880b2920 0x7fdf880b4e30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 --2- 172.21.15.19:0/1615404700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9c102b40 0x7fdf9c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 >> 172.21.15.19:0/1615404700 conn(0x7fdf9c0fdd30 msgr2=0x7fdf9c0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 shutdown_connections 2024-05-17T09:39:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:17.932+0000 7fdfa3384640 1 -- 172.21.15.19:0/1615404700 wait complete. 2024-05-17T09:39:18.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 36698 6.07 MB/sec execute 195 sec latency 114.620 ms 2024-05-17T09:39:18.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:18 smithi069 ceph-mon[35081]: pgmap v1024: 129 pgs: 129 active+clean; 397 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-17T09:39:18.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:18 smithi019 ceph-mon[31620]: pgmap v1024: 129 pgs: 129 active+clean; 397 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-17T09:39:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:18 smithi079 ceph-mon[35502]: pgmap v1024: 129 pgs: 129 active+clean; 397 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-17T09:39:19.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 36846 6.08 MB/sec execute 196 sec latency 85.420 ms 2024-05-17T09:39:20.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 36980 6.08 MB/sec execute 197 sec latency 65.536 ms 2024-05-17T09:39:20.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:20 smithi069 ceph-mon[35081]: pgmap v1025: 129 pgs: 129 active+clean; 401 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-05-17T09:39:20.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:20 smithi019 ceph-mon[31620]: pgmap v1025: 129 pgs: 129 active+clean; 401 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-05-17T09:39:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:20 smithi079 ceph-mon[35502]: pgmap v1025: 129 pgs: 129 active+clean; 401 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-05-17T09:39:21.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 37146 6.11 MB/sec execute 198 sec latency 81.396 ms 2024-05-17T09:39:22.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 37288 6.12 MB/sec execute 199 sec latency 185.239 ms 2024-05-17T09:39:22.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:22 smithi069 ceph-mon[35081]: pgmap v1026: 129 pgs: 129 active+clean; 400 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-05-17T09:39:22.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:22 smithi019 ceph-mon[31620]: pgmap v1026: 129 pgs: 129 active+clean; 400 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-05-17T09:39:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:22 smithi079 ceph-mon[35502]: pgmap v1026: 129 pgs: 129 active+clean; 400 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-05-17T09:39:23.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 37436 6.14 MB/sec execute 200 sec latency 350.046 ms 2024-05-17T09:39:24.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 37559 6.13 MB/sec execute 201 sec latency 69.364 ms 2024-05-17T09:39:24.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:24 smithi019 ceph-mon[31620]: pgmap v1027: 129 pgs: 129 active+clean; 406 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-05-17T09:39:24.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:24 smithi069 ceph-mon[35081]: pgmap v1027: 129 pgs: 129 active+clean; 406 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-05-17T09:39:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:24 smithi079 ceph-mon[35502]: pgmap v1027: 129 pgs: 129 active+clean; 406 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-05-17T09:39:25.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 37705 6.12 MB/sec execute 202 sec latency 96.521 ms 2024-05-17T09:39:26.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 37870 6.13 MB/sec execute 203 sec latency 99.202 ms 2024-05-17T09:39:26.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:26 smithi069 ceph-mon[35081]: pgmap v1028: 129 pgs: 129 active+clean; 406 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-17T09:39:26.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:26 smithi019 ceph-mon[31620]: pgmap v1028: 129 pgs: 129 active+clean; 406 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-17T09:39:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:26 smithi079 ceph-mon[35502]: pgmap v1028: 129 pgs: 129 active+clean; 406 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-17T09:39:27.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38012 6.12 MB/sec execute 204 sec latency 74.748 ms 2024-05-17T09:39:28.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38156 6.12 MB/sec execute 205 sec latency 98.866 ms 2024-05-17T09:39:28.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:28 smithi069 ceph-mon[35081]: pgmap v1029: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 279 op/s 2024-05-17T09:39:28.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:28 smithi019 ceph-mon[31620]: pgmap v1029: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 279 op/s 2024-05-17T09:39:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:28 smithi079 ceph-mon[35502]: pgmap v1029: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 279 op/s 2024-05-17T09:39:29.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38316 6.13 MB/sec execute 206 sec latency 86.060 ms 2024-05-17T09:39:30.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38507 6.18 MB/sec execute 207 sec latency 74.102 ms 2024-05-17T09:39:30.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:30 smithi069 ceph-mon[35081]: pgmap v1030: 129 pgs: 129 active+clean; 403 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-05-17T09:39:30.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:30 smithi019 ceph-mon[31620]: pgmap v1030: 129 pgs: 129 active+clean; 403 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-05-17T09:39:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:30 smithi079 ceph-mon[35502]: pgmap v1030: 129 pgs: 129 active+clean; 403 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-05-17T09:39:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 -- 172.21.15.19:0/1002493217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798102b40 msgr2=0x7f4798103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 --2- 172.21.15.19:0/1002493217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798102b40 0x7f4798103840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4788009860 tx=0x7f478802a340 comp rx=0 tx=0).stop 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 -- 172.21.15.19:0/1002493217 shutdown_connections 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 --2- 172.21.15.19:0/1002493217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 0x7f479810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 --2- 172.21.15.19:0/1002493217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798103d80 0x7f4798104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 --2- 172.21.15.19:0/1002493217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798102b40 0x7f4798103840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 -- 172.21.15.19:0/1002493217 >> 172.21.15.19:0/1002493217 conn(0x7f47980fdd30 msgr2=0x7f47981001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 -- 172.21.15.19:0/1002493217 shutdown_connections 2024-05-17T09:39:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.155+0000 7f479d013640 1 -- 172.21.15.19:0/1002493217 wait complete. 2024-05-17T09:39:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 Processor -- start 2024-05-17T09:39:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 -- start start 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 0x7f47981a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 0x7f47981a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 0x7f47981a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f479810d3c0 con 0x7f4798102b40 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f479810d240 con 0x7f4798103d80 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479d013640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f479810d540 con 0x7f4798108a80 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 0x7f47981a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 0x7f47981a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 0x7f47981a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53838/0 (socket says 172.21.15.19:53838) 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 0x7f47981a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:57784/0 (socket says 172.21.15.19:57784) 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f4796d9d640 1 -- 172.21.15.19:0/2019457349 learned_addr learned my addr 172.21.15.19:0/2019457349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.156+0000 7f479759e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 0x7f47981a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479759e640 1 -- 172.21.15.19:0/2019457349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 msgr2=0x7f47981a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479759e640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 0x7f47981a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479759e640 1 -- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 msgr2=0x7f47981a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479759e640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 0x7f47981a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479759e640 1 -- 172.21.15.19:0/2019457349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47981a82f0 con 0x7f4798108a80 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479659c640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 0x7f47981a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f4796d9d640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 0x7f47981a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479759e640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 0x7f47981a7c20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4790009fd0 tx=0x7f479000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f477ffff640 1 -- 172.21.15.19:0/2019457349 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479000ce20 con 0x7f4798108a80 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47981a85e0 con 0x7f4798108a80 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f477ffff640 1 -- 172.21.15.19:0/2019457349 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4790005020 con 0x7f4798108a80 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.157+0000 7f477ffff640 1 -- 172.21.15.19:0/2019457349 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790010690 con 0x7f4798108a80 2024-05-17T09:39:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.158+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47981a8b30 con 0x7f4798108a80 2024-05-17T09:39:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.161+0000 7f477ffff640 1 -- 172.21.15.19:0/2019457349 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4790004a40 con 0x7f4798108a80 2024-05-17T09:39:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.162+0000 7f477ffff640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47800b2920 0x7f47800b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.162+0000 7f4796d9d640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47800b2920 0x7f47800b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.163+0000 7f477ffff640 1 -- 172.21.15.19:0/2019457349 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f47900e2ca0 con 0x7f4798108a80 2024-05-17T09:39:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.163+0000 7f4796d9d640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47800b2920 0x7f47800b4e30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4788005e00 tx=0x7f478802bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 Processor -- start 2024-05-17T09:39:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 -- start start 2024-05-17T09:39:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513828e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ec42f0 0x56513828e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x565138295820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565138200c30 con 0x565137eed1b0 2024-05-17T09:39:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5651381f07d0 con 0x565137ecc4d0 2024-05-17T09:39:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f47a3f9f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5651381f4810 con 0x565137ec42f0 2024-05-17T09:39:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f479659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ec42f0 0x56513828e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f479659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ec42f0 0x56513828e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58568/0 (socket says 172.21.15.19:58568) 2024-05-17T09:39:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.170+0000 7f479659c640 1 -- 172.21.15.19:0/315160828 learned_addr learned my addr 172.21.15.19:0/315160828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.171+0000 7f4796d9d640 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513828e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.172+0000 7f4796d9d640 1 -- 172.21.15.19:0/315160828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ec42f0 msgr2=0x56513828e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.172+0000 7f4796d9d640 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ec42f0 0x56513828e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.172+0000 7f4796d9d640 1 -- 172.21.15.19:0/315160828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 msgr2=0x565138295820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:39:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.172+0000 7f4796d9d640 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x565138295820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.172+0000 7f4796d9d640 1 -- 172.21.15.19:0/315160828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565137f05210 con 0x565137eed1b0 2024-05-17T09:39:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.172+0000 7f4796d9d640 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513828e3a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f478802a930 tx=0x7f478803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601b4b89bf8eea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.173+0000 7f477d7fa640 1 -- 172.21.15.19:0/315160828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788037040 con 0x565137eed1b0 2024-05-17T09:39:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.173+0000 7f477d7fa640 1 -- 172.21.15.19:0/315160828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f478804e070 con 0x565137eed1b0 2024-05-17T09:39:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.173+0000 7f477d7fa640 1 -- 172.21.15.19:0/315160828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788032a50 con 0x565137eed1b0 2024-05-17T09:39:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 -- 172.21.15.19:0/315160828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 msgr2=0x56513828e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513828e3a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f478802a930 tx=0x7f478803d000 comp rx=0 tx=0).stop 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 -- 172.21.15.19:0/315160828 shutdown_connections 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x565138295820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ec42f0 0x56513828e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/315160828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513828e3a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 -- 172.21.15.19:0/315160828 >> 172.21.15.19:0/315160828 conn(0x565137ee1300 msgr2=0x565138289d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 -- 172.21.15.19:0/315160828 shutdown_connections 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.175+0000 7f47a3f9f740 1 -- 172.21.15.19:0/315160828 wait complete. 2024-05-17T09:39:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.176+0000 7f47a3f9f740 1 -- start start 2024-05-17T09:39:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.176+0000 7f47a3f9f740 1 Processor -- start 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x565138297d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ecc4d0 0x5651382982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x565138297d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x565138297d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:57800/0 (socket says 172.21.15.19:57800) 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f4796d9d640 1 -- 172.21.15.19:0/2609945346 learned_addr learned my addr 172.21.15.19:0/2609945346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f479659c640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ecc4d0 0x5651382982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.177+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513829f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.178+0000 7f479759e640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513829f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.178+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565137ed6790 con 0x565137eed1b0 2024-05-17T09:39:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.178+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5651381ac890 con 0x565137ec42f0 2024-05-17T09:39:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.178+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565138296210 con 0x565137ecc4d0 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f4796d9d640 1 -- 172.21.15.19:0/2609945346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ecc4d0 msgr2=0x5651382982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f4796d9d640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ecc4d0 0x5651382982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f4796d9d640 1 -- 172.21.15.19:0/2609945346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 msgr2=0x56513829f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f4796d9d640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513829f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f4796d9d640 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565137fd1620 con 0x565137ec42f0 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f479659c640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ecc4d0 0x5651382982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f47a3f9f740 10 client.0 _ll_register_callbacks cb 0x5651381afd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478804e070 con 0x565137ec42f0 2024-05-17T09:39:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f479759e640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513829f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f4796d9d640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x565138297d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4788032a20 tx=0x7f4788004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee16c6847a33109 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f477aff5640 10 client.28019 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.179+0000 7f477aff5640 1 -- 172.21.15.19:0/2609945346 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788058020 con 0x565137ec42f0 2024-05-17T09:39:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.180+0000 7f477aff5640 1 -- 172.21.15.19:0/2609945346 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f47880037f0 con 0x565137ec42f0 2024-05-17T09:39:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.180+0000 7f477aff5640 1 -- 172.21.15.19:0/2609945346 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4788037040 con 0x565137ec42f0 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.180+0000 7f47a3f9f740 10 client.28019 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.180+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5651382a1c60 con 0x565137ec42f0 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f477aff5640 1 -- 172.21.15.19:0/2609945346 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4788003a30 con 0x565137ec42f0 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f47a3f9f740 10 client.28019 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f47a3f9f740 10 client.28019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f47a3f9f740 20 client.28019 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137eabd80 0x5651382a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f47a3f9f740 4 client.28019 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:39:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565138202b80 con 0x565137eabd80 2024-05-17T09:39:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.181+0000 7f479659c640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137eabd80 0x5651382a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.182+0000 7f477aff5640 10 client.28019 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.182+0000 7f479659c640 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137eabd80 0x5651382a4fc0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf610eb47c976f2c server_cookie=bf8122556fe87ed7 in_seq=0 out_seq=0 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f477aff5640 1 -- 172.21.15.19:0/2609945346 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565138202b80 con 0x565137eabd80 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f477aff5640 10 client.28019 handle_command_reply: tid=0 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 client.28019 shutdown 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 msgr2=0x565138297d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x565138297d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4788032a20 tx=0x7f4788004340 comp rx=0 tx=0).stop 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 shutdown_connections 2024-05-17T09:39:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137eabd80 0x5651382a4fc0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137eed1b0 0x56513829f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137ecc4d0 0x5651382982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/2609945346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x565138297d70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 >> 172.21.15.19:0/2609945346 conn(0x565137ee1300 msgr2=0x565138288d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f477aff5640 0 client.28019 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 shutdown_connections 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 1 -- 172.21.15.19:0/2609945346 wait complete. 2024-05-17T09:39:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.183+0000 7f47a3f9f740 20 client.28019 trim_cache size 0 max 16384 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 Processor -- start 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 -- start start 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x5651382929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 0x56513828cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b57f0 con 0x565137ecc4d0 2024-05-17T09:39:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b30c0 con 0x565137ec42f0 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.212+0000 7f47a3f9f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b0f10 con 0x565137eed1b0 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x5651382929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f479659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 0x56513828cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x5651382929d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:57808/0 (socket says 172.21.15.19:57808) 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f479659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 0x56513828cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53860/0 (socket says 172.21.15.19:53860) 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 -- 172.21.15.19:0/3788373610 learned_addr learned my addr 172.21.15.19:0/3788373610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f479759e640 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 -- 172.21.15.19:0/3788373610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 msgr2=0x56513829f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 -- 172.21.15.19:0/3788373610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 msgr2=0x56513828cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 0x56513828cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 -- 172.21.15.19:0/3788373610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565138296340 con 0x565137ec42f0 2024-05-17T09:39:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f479659c640 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 0x56513828cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.213+0000 7f4796d9d640 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x5651382929d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4788035040 tx=0x7f4788055000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a713174d10f184 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.214+0000 7f477cff9640 1 -- 172.21.15.19:0/3788373610 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788058040 con 0x565137ec42f0 2024-05-17T09:39:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.214+0000 7f477cff9640 1 -- 172.21.15.19:0/3788373610 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f478804e070 con 0x565137ec42f0 2024-05-17T09:39:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.214+0000 7f477cff9640 1 -- 172.21.15.19:0/3788373610 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788003850 con 0x565137ec42f0 2024-05-17T09:39:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 -- 172.21.15.19:0/3788373610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 msgr2=0x5651382929d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x5651382929d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4788035040 tx=0x7f4788055000 comp rx=0 tx=0).stop 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 -- 172.21.15.19:0/3788373610 shutdown_connections 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ecc4d0 0x56513828cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/3788373610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ec42f0 0x5651382929d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 -- 172.21.15.19:0/3788373610 >> 172.21.15.19:0/3788373610 conn(0x565137ee1300 msgr2=0x565138297d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 -- 172.21.15.19:0/3788373610 shutdown_connections 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 -- 172.21.15.19:0/3788373610 wait complete. 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.215+0000 7f47a3f9f740 1 -- start start 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 Processor -- start 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 0x56513829f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x56513829c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 0x56513829f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f4796d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 0x56513829f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53866/0 (socket says 172.21.15.19:53866) 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f4796d9d640 1 -- 172.21.15.19:0/1959353741 learned_addr learned my addr 172.21.15.19:0/1959353741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f479659c640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x56513829c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b1580 con 0x565137ec42f0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b29f0 con 0x565137ecc4d0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.216+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b1c50 con 0x565137eed1b0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f47a3f9f740 10 client.0 _ll_register_callbacks cb 0x7f4788004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 -- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 msgr2=0x56513829c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x56513829c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 -- 172.21.15.19:0/1959353741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 msgr2=0x56513829f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 0x56513829f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56513830b5f0 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f4796d9d640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 0x56513829f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479659c640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x56513829c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f479759e640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829c6c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47900e0390 tx=0x7f47900aa5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222e756ba78499ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f477aff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f477aff5640 1 -- 172.21.15.19:0/1959353741 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47900c9060 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.217+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478804e070 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f477aff5640 1 -- 172.21.15.19:0/1959353741 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f47900051d0 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f477aff5640 1 -- 172.21.15.19:0/1959353741 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f47900b2d10 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 10 client.28015 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56513830c820 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f477aff5640 1 -- 172.21.15.19:0/1959353741 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f478804e070 con 0x565137eed1b0 2024-05-17T09:39:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f477aff5640 1 -- 172.21.15.19:0/1959353741 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f47900a8b80 con 0x565137eed1b0 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 10 client.28015 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 10 client.28015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 20 client.28015 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137e811c0 0x56513830f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 4 client.28015 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651381fc360 con 0x565137e811c0 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.218+0000 7f4796d9d640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137e811c0 0x56513830f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.219+0000 7f477aff5640 10 client.28015 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.219+0000 7f4796d9d640 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137e811c0 0x56513830f0e0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb27cfacf8ca5c2c server_cookie=c63866d4a8efd2bd in_seq=0 out_seq=0 2024-05-17T09:39:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.312+0000 7f477aff5640 1 -- 172.21.15.19:0/1959353741 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5651381fc360 con 0x565137e811c0 2024-05-17T09:39:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.312+0000 7f477aff5640 10 client.28015 handle_command_reply: tid=0 2024-05-17T09:39:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.312+0000 7f47a3f9f740 1 client.28015 shutdown 2024-05-17T09:39:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 msgr2=0x56513829c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829c6c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47900e0390 tx=0x7f47900aa5b0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 shutdown_connections 2024-05-17T09:39:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565137e811c0 0x56513830f0e0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565137eed1b0 0x56513829c6c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565137ecc4d0 0x56513829c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 --2- 172.21.15.19:0/1959353741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565137ec42f0 0x56513829f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f477aff5640 0 client.28015 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 >> 172.21.15.19:0/1959353741 conn(0x565137ee1300 msgr2=0x5651382917c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 shutdown_connections 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 1 -- 172.21.15.19:0/1959353741 wait complete. 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.313+0000 7f47a3f9f740 20 client.28015 trim_cache size 0 max 16384 2024-05-17T09:39:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:39:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:39:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:39:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47800b2920 msgr2=0x7f47800b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47800b2920 0x7f47800b4e30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4788005e00 tx=0x7f478802bf90 comp rx=0 tx=0).stop 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 msgr2=0x7f47981a7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 0x7f47981a7c20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4790009fd0 tx=0x7f479000ef10 comp rx=0 tx=0).stop 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 shutdown_connections 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4798108a80 0x7f47981a7c20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4798103d80 0x7f47981a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f47800b2920 0x7f47800b4e30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 --2- 172.21.15.19:0/2019457349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4798102b40 0x7f47981a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.314+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 >> 172.21.15.19:0/2019457349 conn(0x7f47980fdd30 msgr2=0x7f4798104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.315+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 shutdown_connections 2024-05-17T09:39:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:31.315+0000 7f479d013640 1 -- 172.21.15.19:0/2019457349 wait complete. 2024-05-17T09:39:31.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38661 6.18 MB/sec execute 208 sec latency 109.321 ms 2024-05-17T09:39:32.581 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38854 6.19 MB/sec execute 209 sec latency 74.622 ms 2024-05-17T09:39:32.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:32 smithi069 ceph-mon[35081]: pgmap v1031: 129 pgs: 129 active+clean; 402 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-05-17T09:39:32.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:32 smithi019 ceph-mon[31620]: pgmap v1031: 129 pgs: 129 active+clean; 402 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-05-17T09:39:32.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:32 smithi079 ceph-mon[35502]: pgmap v1031: 129 pgs: 129 active+clean; 402 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-05-17T09:39:33.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 38996 6.19 MB/sec execute 210 sec latency 68.874 ms 2024-05-17T09:39:34.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 39150 6.18 MB/sec execute 211 sec latency 81.558 ms 2024-05-17T09:39:34.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:34 smithi069 ceph-mon[35081]: pgmap v1032: 129 pgs: 129 active+clean; 402 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-05-17T09:39:34.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:34 smithi019 ceph-mon[31620]: pgmap v1032: 129 pgs: 129 active+clean; 402 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-05-17T09:39:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:34 smithi079 ceph-mon[35502]: pgmap v1032: 129 pgs: 129 active+clean; 402 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-05-17T09:39:35.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 39334 6.21 MB/sec execute 212 sec latency 67.063 ms 2024-05-17T09:39:36.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 39501 6.22 MB/sec execute 213 sec latency 77.490 ms 2024-05-17T09:39:36.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:36 smithi069 ceph-mon[35081]: pgmap v1033: 129 pgs: 129 active+clean; 399 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-05-17T09:39:36.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:36 smithi019 ceph-mon[31620]: pgmap v1033: 129 pgs: 129 active+clean; 399 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-05-17T09:39:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:36 smithi079 ceph-mon[35502]: pgmap v1033: 129 pgs: 129 active+clean; 399 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-05-17T09:39:37.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 39646 6.22 MB/sec execute 214 sec latency 68.628 ms 2024-05-17T09:39:38.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 39833 6.27 MB/sec execute 215 sec latency 68.501 ms 2024-05-17T09:39:38.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:38 smithi069 ceph-mon[35081]: pgmap v1034: 129 pgs: 129 active+clean; 400 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2024-05-17T09:39:38.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:38 smithi019 ceph-mon[31620]: pgmap v1034: 129 pgs: 129 active+clean; 400 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2024-05-17T09:39:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:38 smithi079 ceph-mon[35502]: pgmap v1034: 129 pgs: 129 active+clean; 400 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2024-05-17T09:39:39.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40053 6.31 MB/sec execute 216 sec latency 78.854 ms 2024-05-17T09:39:40.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40193 6.31 MB/sec execute 217 sec latency 78.516 ms 2024-05-17T09:39:40.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:40 smithi069 ceph-mon[35081]: pgmap v1035: 129 pgs: 129 active+clean; 406 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.8 MiB/s wr, 276 op/s 2024-05-17T09:39:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:40 smithi019 ceph-mon[31620]: pgmap v1035: 129 pgs: 129 active+clean; 406 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.8 MiB/s wr, 276 op/s 2024-05-17T09:39:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:40 smithi079 ceph-mon[35502]: pgmap v1035: 129 pgs: 129 active+clean; 406 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.8 MiB/s wr, 276 op/s 2024-05-17T09:39:41.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40344 6.32 MB/sec execute 218 sec latency 78.608 ms 2024-05-17T09:39:42.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40497 6.35 MB/sec execute 219 sec latency 132.983 ms 2024-05-17T09:39:42.680 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:42 smithi069 ceph-mon[35081]: pgmap v1036: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-05-17T09:39:42.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:42 smithi019 ceph-mon[31620]: pgmap v1036: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-05-17T09:39:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:42 smithi079 ceph-mon[35502]: pgmap v1036: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-05-17T09:39:43.582 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40655 6.37 MB/sec execute 220 sec latency 69.819 ms 2024-05-17T09:39:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 -- 172.21.15.19:0/2340813482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc104590 msgr2=0x7fb0fc10ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 --2- 172.21.15.19:0/2340813482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc104590 0x7fb0fc10ace0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009ca0 tx=0x7fb0f402a8c0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 -- 172.21.15.19:0/2340813482 shutdown_connections 2024-05-17T09:39:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 --2- 172.21.15.19:0/2340813482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc104590 0x7fb0fc10ace0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 --2- 172.21.15.19:0/2340813482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 --2- 172.21.15.19:0/2340813482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc103050 0x7fb0fc103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 -- 172.21.15.19:0/2340813482 >> 172.21.15.19:0/2340813482 conn(0x7fb0fc0fd470 msgr2=0x7fb0fc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 -- 172.21.15.19:0/2340813482 shutdown_connections 2024-05-17T09:39:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.487+0000 7fb10334d640 1 -- 172.21.15.19:0/2340813482 wait complete. 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 Processor -- start 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 -- start start 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc103050 0x7fb0fc10a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc108d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc1092d0 0x7fb0fc1096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10e540 con 0x7fb0fc103050 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10e3c0 con 0x7fb0fc1092d0 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.488+0000 7fb10334d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10e6c0 con 0x7fb0fc103a30 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc103050 0x7fb0fc10a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52190/0 (socket says 172.21.15.19:52190) 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 -- 172.21.15.19:0/1618938462 learned_addr learned my addr 172.21.15.19:0/1618938462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1018ea640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc1092d0 0x7fb0fc1096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 -- 172.21.15.19:0/1618938462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc1092d0 msgr2=0x7fb0fc1096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc1092d0 0x7fb0fc1096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 -- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc103050 msgr2=0x7fb0fc10a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc103050 0x7fb0fc10a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 -- 172.21.15.19:0/1618938462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc1a3e60 con 0x7fb0fc103a30 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1010e9640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc103050 0x7fb0fc10a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.489+0000 7fb1008e8640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc108d90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000ea90 tx=0x7fb0f000ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.490+0000 7fb0ee7fc640 1 -- 172.21.15.19:0/1618938462 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000ce40 con 0x7fb0fc103a30 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.490+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc1a4150 con 0x7fb0fc103a30 2024-05-17T09:39:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.490+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc1a46f0 con 0x7fb0fc103a30 2024-05-17T09:39:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.490+0000 7fb0ee7fc640 1 -- 172.21.15.19:0/1618938462 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0f00035f0 con 0x7fb0fc103a30 2024-05-17T09:39:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.490+0000 7fb0ee7fc640 1 -- 172.21.15.19:0/1618938462 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0010690 con 0x7fb0fc103a30 2024-05-17T09:39:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb0ee7fc640 1 -- 172.21.15.19:0/1618938462 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb0f0003fb0 con 0x7fb0fc103a30 2024-05-17T09:39:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 Processor -- start 2024-05-17T09:39:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 -- start start 2024-05-17T09:39:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bc03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bc7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b462b32c30 con 0x55b4627fe4d0 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b462b227d0 con 0x55b46281f1b0 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.494+0000 7fb10a204740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b462b26810 con 0x55b4627f62f0 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bc03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bc03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:56952/0 (socket says 172.21.15.19:56952) 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 -- 172.21.15.19:0/2456994014 learned_addr learned my addr 172.21.15.19:0/2456994014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1008e8640 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bc7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 -- 172.21.15.19:0/2456994014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 msgr2=0x55b462bc0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 -- 172.21.15.19:0/2456994014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 msgr2=0x55b462bc7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bc7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 -- 172.21.15.19:0/2456994014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b462837210 con 0x55b46281f1b0 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1010e9640 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1008e8640 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bc7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb0ee7fc640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0e80b2920 0x7fb0e80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.495+0000 7fb1018ea640 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bc03a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4035000 tx=0x7fb0f4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c7d097e3837b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.496+0000 7fb1010e9640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0e80b2920 0x7fb0e80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.496+0000 7fb0ee7fc640 1 -- 172.21.15.19:0/1618938462 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0f00e29c0 con 0x7fb0fc103a30 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.496+0000 7fb0e77fe640 1 -- 172.21.15.19:0/2456994014 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4031a80 con 0x55b46281f1b0 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.496+0000 7fb0e77fe640 1 -- 172.21.15.19:0/2456994014 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0f4036030 con 0x55b46281f1b0 2024-05-17T09:39:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.496+0000 7fb0e77fe640 1 -- 172.21.15.19:0/2456994014 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40043a0 con 0x55b46281f1b0 2024-05-17T09:39:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 -- 172.21.15.19:0/2456994014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 msgr2=0x55b462bc03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bc03a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4035000 tx=0x7fb0f4009800 comp rx=0 tx=0).stop 2024-05-17T09:39:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb1010e9640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0e80b2920 0x7fb0e80b4e30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009fd0 tx=0x7fb0f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 -- 172.21.15.19:0/2456994014 shutdown_connections 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bc7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 --2- 172.21.15.19:0/2456994014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bc03a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 -- 172.21.15.19:0/2456994014 >> 172.21.15.19:0/2456994014 conn(0x55b462813300 msgr2=0x55b462bbbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 -- 172.21.15.19:0/2456994014 shutdown_connections 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 -- 172.21.15.19:0/2456994014 wait complete. 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 -- start start 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.497+0000 7fb10a204740 1 Processor -- start 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b462808790 con 0x55b4627fe4d0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b462ade890 con 0x55b46281f1b0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b462bc8090 con 0x55b4627f62f0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb10a204740 10 client.0 _ll_register_callbacks cb 0x55b462ae1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1008e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52194/0 (socket says 172.21.15.19:52194) 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1008e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42076/0 (socket says 172.21.15.19:42076) 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 -- 172.21.15.19:0/2819478564 learned_addr learned my addr 172.21.15.19:0/2819478564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1018ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 -- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 msgr2=0x55b462bd0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 -- 172.21.15.19:0/2819478564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 msgr2=0x55b462bca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.498+0000 7fb1010e9640 1 -- 172.21.15.19:0/2819478564 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b462903620 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb1008e8640 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb1010e9640 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc9bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8012da0 tx=0x7fb0f8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29663a89c781f602 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb0e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/2819478564 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8011940 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/2819478564 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0f8011ae0 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f4036030 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/2819478564 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0f8011ce0 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb10a204740 10 client.28027 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b462bd3ff0 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.499+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/2819478564 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0f4036030 con 0x55b4627f62f0 2024-05-17T09:39:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/2819478564 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb0f8027070 con 0x55b4627f62f0 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb10a204740 10 client.28027 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb10a204740 10 client.28027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb10a204740 20 client.28027 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb10a204740 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627ddd80 0x55b462bd7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb10a204740 4 client.28027 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b462b34b80 con 0x55b4627ddd80 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.500+0000 7fb1008e8640 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627ddd80 0x55b462bd7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.501+0000 7fb0e4ff9640 10 client.28027 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.501+0000 7fb1008e8640 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627ddd80 0x55b462bd7780 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce7ec5488fbfa6a server_cookie=562a8dcdaedec688 in_seq=0 out_seq=0 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/2819478564 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b462b34b80 con 0x55b4627ddd80 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb0e4ff9640 10 client.28027 handle_command_reply: tid=0 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 client.28027 shutdown 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 msgr2=0x55b462bc9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc9bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8012da0 tx=0x7fb0f8004730 comp rx=0 tx=0).stop 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 shutdown_connections 2024-05-17T09:39:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627ddd80 0x55b462bd7780 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627fe4d0 0x55b462bca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 --2- 172.21.15.19:0/2819478564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627f62f0 0x55b462bc9bc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb0e4ff9640 0 client.28027 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 >> 172.21.15.19:0/2819478564 conn(0x55b462813300 msgr2=0x55b462bbec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 shutdown_connections 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.502+0000 7fb10a204740 1 -- 172.21.15.19:0/2819478564 wait complete. 2024-05-17T09:39:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.503+0000 7fb10a204740 20 client.28027 trim_cache size 0 max 16384 2024-05-17T09:39:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.533+0000 7fb10a204740 1 Processor -- start 2024-05-17T09:39:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.533+0000 7fb10a204740 1 -- start start 2024-05-17T09:39:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb10a204740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 0x55b462bca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb10a204740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bc48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb10a204740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb10a204740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1b13f0 con 0x55b4627f62f0 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb10a204740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1aecc0 con 0x55b46281f1b0 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb10a204740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1acb10 con 0x55b4627fe4d0 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 0x55b462bca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1008e8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bc48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 0x55b462bca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42082/0 (socket says 172.21.15.19:42082) 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1010e9640 1 -- 172.21.15.19:0/991173379 learned_addr learned my addr 172.21.15.19:0/991173379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1018ea640 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1008e8640 1 -- 172.21.15.19:0/991173379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 msgr2=0x55b462bd0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1008e8640 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1008e8640 1 -- 172.21.15.19:0/991173379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 msgr2=0x55b462bca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1008e8640 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 0x55b462bca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1008e8640 1 -- 172.21.15.19:0/991173379 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b462bd14c0 con 0x55b4627fe4d0 2024-05-17T09:39:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.534+0000 7fb1010e9640 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 0x55b462bca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.535+0000 7fb1008e8640 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bc48f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0f001f000 tx=0x7fb0f00dff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0deb77bf848f4fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.535+0000 7fb0e6ffd640 1 -- 172.21.15.19:0/991173379 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00aa2d0 con 0x55b4627fe4d0 2024-05-17T09:39:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.535+0000 7fb0e6ffd640 1 -- 172.21.15.19:0/991173379 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0f00a9bd0 con 0x55b4627fe4d0 2024-05-17T09:39:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.535+0000 7fb0e6ffd640 1 -- 172.21.15.19:0/991173379 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0003830 con 0x55b4627fe4d0 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 -- 172.21.15.19:0/991173379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 msgr2=0x55b462bc48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bc48f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0f001f000 tx=0x7fb0f00dff80 comp rx=0 tx=0).stop 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 -- 172.21.15.19:0/991173379 shutdown_connections 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b46281f1b0 0x55b462bd0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bc48f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 --2- 172.21.15.19:0/991173379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4627f62f0 0x55b462bca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 -- 172.21.15.19:0/991173379 >> 172.21.15.19:0/991173379 conn(0x55b462813300 msgr2=0x55b462bbf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 -- 172.21.15.19:0/991173379 shutdown_connections 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.536+0000 7fb10a204740 1 -- 172.21.15.19:0/991173379 wait complete. 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 -- start start 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 Processor -- start 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 0x55b462bd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bd06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b46281f1b0 0x55b462c3df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1ad180 con 0x55b46281f1b0 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1ae5f0 con 0x55b4627f62f0 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1ad850 con 0x55b4627fe4d0 2024-05-17T09:39:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb10a204740 10 client.0 _ll_register_callbacks cb 0x7fb0f80124c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 0x55b462bd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb1010e9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 0x55b462bd0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:56976/0 (socket says 172.21.15.19:56976) 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb1010e9640 1 -- 172.21.15.19:0/3939304412 learned_addr learned my addr 172.21.15.19:0/3939304412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb1018ea640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b46281f1b0 0x55b462c3df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.537+0000 7fb1008e8640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bd06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1008e8640 1 -- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 msgr2=0x55b462bd0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1008e8640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 0x55b462bd0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1008e8640 1 -- 172.21.15.19:0/3939304412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b46281f1b0 msgr2=0x55b462c3df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1008e8640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b46281f1b0 0x55b462c3df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1008e8640 1 -- 172.21.15.19:0/3939304412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b462c3e530 con 0x55b4627fe4d0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1010e9640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 0x55b462bd0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb1008e8640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bd06a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00e2cb0 tx=0x7fb0f00e2c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5ae694c444f5d2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb0e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/3939304412 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0003810 con 0x55b4627fe4d0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/3939304412 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0f00a5e60 con 0x55b4627fe4d0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f00a9bd0 con 0x55b4627fe4d0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/3939304412 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0f00c3050 con 0x55b4627fe4d0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb10a204740 10 client.28033 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.538+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b462c3f7a0 con 0x55b4627fe4d0 2024-05-17T09:39:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/3939304412 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0f00aa2d0 con 0x55b4627fe4d0 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/3939304412 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb0f00a5040 con 0x55b4627fe4d0 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb10a204740 10 client.28033 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb10a204740 10 client.28033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb10a204740 20 client.28033 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb10a204740 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627b31c0 0x55b462c420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb10a204740 4 client.28033 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb0f00a9bd0 con 0x55b4627b31c0 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.539+0000 7fb1010e9640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627b31c0 0x55b462c420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.540+0000 7fb1010e9640 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627b31c0 0x55b462c420e0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bc6f8ead8fd730 server_cookie=422584dc51d6460a in_seq=0 out_seq=0 2024-05-17T09:39:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.540+0000 7fb0e4ff9640 10 client.28033 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:44.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40802 6.39 MB/sec execute 221 sec latency 110.248 ms 2024-05-17T09:39:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb0e4ff9640 1 -- 172.21.15.19:0/3939304412 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fb0f00a9bd0 con 0x55b4627b31c0 2024-05-17T09:39:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb0e4ff9640 10 client.28033 handle_command_reply: tid=0 2024-05-17T09:39:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 client.28033 shutdown 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 msgr2=0x55b462bd06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bd06a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00e2cb0 tx=0x7fb0f00e2c40 comp rx=0 tx=0).stop 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 shutdown_connections 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4627b31c0 0x55b462c420e0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b46281f1b0 0x55b462c3df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b4627fe4d0 0x55b462bd06a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb0e4ff9640 0 client.28033 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 --2- 172.21.15.19:0/3939304412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4627f62f0 0x55b462bd0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 >> 172.21.15.19:0/3939304412 conn(0x55b462813300 msgr2=0x55b462bc37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 shutdown_connections 2024-05-17T09:39:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 1 -- 172.21.15.19:0/3939304412 wait complete. 2024-05-17T09:39:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.641+0000 7fb10a204740 20 client.28033 trim_cache size 0 max 16384 2024-05-17T09:39:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:39:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:39:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:39:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0e80b2920 msgr2=0x7fb0e80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0e80b2920 0x7fb0e80b4e30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009fd0 tx=0x7fb0f8005f90 comp rx=0 tx=0).stop 2024-05-17T09:39:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 msgr2=0x7fb0fc108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc108d90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000ea90 tx=0x7fb0f000ef50 comp rx=0 tx=0).stop 2024-05-17T09:39:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 shutdown_connections 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc1092d0 0x7fb0fc1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0fc103a30 0x7fb0fc108d90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0e80b2920 0x7fb0e80b4e30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 --2- 172.21.15.19:0/1618938462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc103050 0x7fb0fc10a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 >> 172.21.15.19:0/1618938462 conn(0x7fb0fc0fd470 msgr2=0x7fb0fc0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 shutdown_connections 2024-05-17T09:39:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:44.642+0000 7fb10334d640 1 -- 172.21.15.19:0/1618938462 wait complete. 2024-05-17T09:39:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:44 smithi079 ceph-mon[35502]: pgmap v1037: 129 pgs: 129 active+clean; 414 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-05-17T09:39:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:44 smithi069 ceph-mon[35081]: pgmap v1037: 129 pgs: 129 active+clean; 414 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-05-17T09:39:44.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:44 smithi019 ceph-mon[31620]: pgmap v1037: 129 pgs: 129 active+clean; 414 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-05-17T09:39:45.583 INFO:tasks.workunit.client.0.smithi019.stdout: 10 40938 6.38 MB/sec execute 222 sec latency 128.001 ms 2024-05-17T09:39:46.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41092 6.39 MB/sec execute 223 sec latency 80.032 ms 2024-05-17T09:39:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:46 smithi079 ceph-mon[35502]: pgmap v1038: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 252 op/s 2024-05-17T09:39:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:46 smithi069 ceph-mon[35081]: pgmap v1038: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 252 op/s 2024-05-17T09:39:46.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:46 smithi019 ceph-mon[31620]: pgmap v1038: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 252 op/s 2024-05-17T09:39:47.583 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41220 6.37 MB/sec execute 224 sec latency 84.727 ms 2024-05-17T09:39:48.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41368 6.38 MB/sec execute 225 sec latency 143.540 ms 2024-05-17T09:39:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:48 smithi079 ceph-mon[35502]: pgmap v1039: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-05-17T09:39:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:48 smithi069 ceph-mon[35081]: pgmap v1039: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-05-17T09:39:48.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:48 smithi019 ceph-mon[31620]: pgmap v1039: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-05-17T09:39:49.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41507 6.38 MB/sec execute 226 sec latency 117.103 ms 2024-05-17T09:39:49.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:49 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:39:49.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:49 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:39:49.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:49 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:39:50.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41643 6.37 MB/sec execute 227 sec latency 76.474 ms 2024-05-17T09:39:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:50 smithi079 ceph-mon[35502]: pgmap v1040: 129 pgs: 129 active+clean; 417 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 278 op/s 2024-05-17T09:39:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:39:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:50 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:39:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:50 smithi069 ceph-mon[35081]: pgmap v1040: 129 pgs: 129 active+clean; 417 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 278 op/s 2024-05-17T09:39:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:39:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:50 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:39:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:50 smithi019 ceph-mon[31620]: pgmap v1040: 129 pgs: 129 active+clean; 417 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 278 op/s 2024-05-17T09:39:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:39:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:50 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:39:51.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41814 6.39 MB/sec execute 228 sec latency 78.160 ms 2024-05-17T09:39:52.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 41974 6.39 MB/sec execute 229 sec latency 79.373 ms 2024-05-17T09:39:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:52 smithi079 ceph-mon[35502]: pgmap v1041: 129 pgs: 129 active+clean; 417 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-17T09:39:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:52 smithi069 ceph-mon[35081]: pgmap v1041: 129 pgs: 129 active+clean; 417 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-17T09:39:52.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:52 smithi019 ceph-mon[31620]: pgmap v1041: 129 pgs: 129 active+clean; 417 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-05-17T09:39:53.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 42122 6.38 MB/sec execute 230 sec latency 94.700 ms 2024-05-17T09:39:54.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 42268 6.38 MB/sec execute 231 sec latency 152.053 ms 2024-05-17T09:39:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:54 smithi079 ceph-mon[35502]: pgmap v1042: 129 pgs: 129 active+clean; 413 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-05-17T09:39:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:54 smithi069 ceph-mon[35081]: pgmap v1042: 129 pgs: 129 active+clean; 413 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-05-17T09:39:54.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:54 smithi019 ceph-mon[31620]: pgmap v1042: 129 pgs: 129 active+clean; 413 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-05-17T09:39:55.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 42421 6.38 MB/sec execute 232 sec latency 82.530 ms 2024-05-17T09:39:56.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 42584 6.38 MB/sec execute 233 sec latency 68.473 ms 2024-05-17T09:39:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:56 smithi079 ceph-mon[35502]: pgmap v1043: 129 pgs: 129 active+clean; 411 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-05-17T09:39:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:56 smithi069 ceph-mon[35081]: pgmap v1043: 129 pgs: 129 active+clean; 411 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-05-17T09:39:56.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:56 smithi019 ceph-mon[31620]: pgmap v1043: 129 pgs: 129 active+clean; 411 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-05-17T09:39:57.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 42745 6.40 MB/sec execute 234 sec latency 79.939 ms 2024-05-17T09:39:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 -- 172.21.15.19:0/2126312563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 msgr2=0x7fe18c105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 --2- 172.21.15.19:0/2126312563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c105290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009900 tx=0x7fe17c02a400 comp rx=0 tx=0).stop 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 -- 172.21.15.19:0/2126312563 shutdown_connections 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 --2- 172.21.15.19:0/2126312563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c109b80 0x7fe18c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 --2- 172.21.15.19:0/2126312563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c105290 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 --2- 172.21.15.19:0/2126312563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c103c40 0x7fe18c104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.805+0000 7fe19338d640 1 -- 172.21.15.19:0/2126312563 >> 172.21.15.19:0/2126312563 conn(0x7fe18c0fee50 msgr2=0x7fe18c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.806+0000 7fe19338d640 1 -- 172.21.15.19:0/2126312563 shutdown_connections 2024-05-17T09:39:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.806+0000 7fe19338d640 1 -- 172.21.15.19:0/2126312563 wait complete. 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 Processor -- start 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 -- start start 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103c40 0x7fe18c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 0x7fe18c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10f5f0 con 0x7fe18c103c40 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10f470 con 0x7fe18c104e80 2024-05-17T09:39:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.807+0000 7fe19338d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10f770 con 0x7fe18c109b80 2024-05-17T09:39:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 0x7fe18c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe190928640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe190928640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50330/0 (socket says 172.21.15.19:50330) 2024-05-17T09:39:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe191129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103c40 0x7fe18c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 0x7fe18c1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42128/0 (socket says 172.21.15.19:42128) 2024-05-17T09:39:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe190928640 1 -- 172.21.15.19:0/3595934262 learned_addr learned my addr 172.21.15.19:0/3595934262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 -- 172.21.15.19:0/3595934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 msgr2=0x7fe18c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 -- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103c40 msgr2=0x7fe18c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103c40 0x7fe18c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 -- 172.21.15.19:0/3595934262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c1a1d70 con 0x7fe18c109b80 2024-05-17T09:39:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe190928640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe19192a640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 0x7fe18c1a1700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe184009fd0 tx=0x7fe18400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.808+0000 7fe1827fc640 1 -- 172.21.15.19:0/3595934262 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18400ce20 con 0x7fe18c109b80 2024-05-17T09:39:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.809+0000 7fe1827fc640 1 -- 172.21.15.19:0/3595934262 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe184005020 con 0x7fe18c109b80 2024-05-17T09:39:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.809+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c074f50 con 0x7fe18c109b80 2024-05-17T09:39:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.809+0000 7fe1827fc640 1 -- 172.21.15.19:0/3595934262 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184010690 con 0x7fe18c109b80 2024-05-17T09:39:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.809+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c0753f0 con 0x7fe18c109b80 2024-05-17T09:39:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.812+0000 7fe1827fc640 1 -- 172.21.15.19:0/3595934262 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe184010830 con 0x7fe18c109b80 2024-05-17T09:39:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 Processor -- start 2024-05-17T09:39:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 -- start start 2024-05-17T09:39:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd76130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5572ffcdfc30 con 0x5572ff9ab4d0 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5572ffccf7d0 con 0x5572ff9a32f0 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.813+0000 7fe19a247740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5572ffcd3810 con 0x5572ff9cc1b0 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd75bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42140/0 (socket says 172.21.15.19:42140) 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 -- 172.21.15.19:0/3465052626 learned_addr learned my addr 172.21.15.19:0/3465052626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe191129640 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd76130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 -- 172.21.15.19:0/3465052626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 msgr2=0x5572ffd76130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd76130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 -- 172.21.15.19:0/3465052626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 msgr2=0x5572ffd7d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 -- 172.21.15.19:0/3465052626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ff9e4210 con 0x5572ff9cc1b0 2024-05-17T09:39:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.814+0000 7fe190928640 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd75bb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009230 tx=0x7fe17c0318b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5836ccff8cd93 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.815+0000 7fe1777fe640 1 -- 172.21.15.19:0/3465052626 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c035040 con 0x5572ff9cc1b0 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.815+0000 7fe1777fe640 1 -- 172.21.15.19:0/3465052626 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe17c004030 con 0x5572ff9cc1b0 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.815+0000 7fe1777fe640 1 -- 172.21.15.19:0/3465052626 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c003850 con 0x5572ff9cc1b0 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.815+0000 7fe1827fc640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1780b28a0 0x7fe1780b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.815+0000 7fe191129640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1780b28a0 0x7fe1780b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe1827fc640 1 -- 172.21.15.19:0/3595934262 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe1840e2d70 con 0x7fe18c109b80 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 -- 172.21.15.19:0/3465052626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 msgr2=0x5572ffd75bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe191129640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1780b28a0 0x7fe1780b4db0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5572ffd76f90 tx=0x7fe188012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd75bb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009230 tx=0x7fe17c0318b0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 -- 172.21.15.19:0/3465052626 shutdown_connections 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd76130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 --2- 172.21.15.19:0/3465052626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd75bb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 -- 172.21.15.19:0/3465052626 >> 172.21.15.19:0/3465052626 conn(0x5572ff9c0300 msgr2=0x5572ffd71540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.816+0000 7fe19a247740 1 -- 172.21.15.19:0/3465052626 shutdown_connections 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.817+0000 7fe19a247740 1 -- 172.21.15.19:0/3465052626 wait complete. 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.817+0000 7fe19a247740 1 -- start start 2024-05-17T09:39:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.817+0000 7fe19a247740 1 Processor -- start 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd7f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 0x5572ffd869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5572ff9b5790 con 0x5572ff9a32f0 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5572ffc8b890 con 0x5572ff9cc1b0 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5572ffd7d9c0 con 0x5572ff9ab4d0 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19a247740 10 client.0 _ll_register_callbacks cb 0x5572ffc8ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe190928640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19192a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 0x5572ffd869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19192a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 0x5572ffd869c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50352/0 (socket says 172.21.15.19:50352) 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19192a640 1 -- 172.21.15.19:0/670942054 learned_addr learned my addr 172.21.15.19:0/670942054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe191129640 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd7f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe190928640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:42156/0 (socket says 172.21.15.19:42156) 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.818+0000 7fe19192a640 1 -- 172.21.15.19:0/670942054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9ab4d0 msgr2=0x5572ffd7fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19192a640 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19192a640 1 -- 172.21.15.19:0/670942054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 msgr2=0x5572ffd7f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19192a640 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd7f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19192a640 1 -- 172.21.15.19:0/670942054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ffab0620 con 0x5572ff9cc1b0 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19192a640 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 0x5572ffd869c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe184005c40 tx=0x7fe1840aa7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4c7947642c0c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe174ff9640 1 -- 172.21.15.19:0/670942054 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840b2de0 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe174ff9640 1 -- 172.21.15.19:0/670942054 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe1840051d0 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17c004030 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe174ff9640 1 -- 172.21.15.19:0/670942054 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe1840b2950 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19a247740 10 client.28055 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.819+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572ffd899c0 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe174ff9640 1 -- 172.21.15.19:0/670942054 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe17c004030 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe174ff9640 1 -- 172.21.15.19:0/670942054 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe1840a58f0 con 0x5572ff9cc1b0 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe19a247740 10 client.28055 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe19a247740 10 client.28055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe19a247740 20 client.28055 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe19a247740 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff98ad80 0x5572ffd8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe19a247740 4 client.28055 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:39:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572ffce1b80 con 0x5572ff98ad80 2024-05-17T09:39:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.820+0000 7fe191129640 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff98ad80 0x5572ffd8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.821+0000 7fe174ff9640 10 client.28055 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.821+0000 7fe191129640 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff98ad80 0x5572ffd8d600 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba21dad9a0c99d22 server_cookie=6ecde0cf618ee462 in_seq=0 out_seq=0 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.822+0000 7fe174ff9640 1 -- 172.21.15.19:0/670942054 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5572ffce1b80 con 0x5572ff98ad80 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.822+0000 7fe174ff9640 10 client.28055 handle_command_reply: tid=0 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.822+0000 7fe19a247740 1 client.28055 shutdown 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 msgr2=0x5572ffd869c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 0x5572ffd869c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe184005c40 tx=0x7fe1840aa7c0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 shutdown_connections 2024-05-17T09:39:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff98ad80 0x5572ffd8d600 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9cc1b0 0x5572ffd869c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9ab4d0 0x5572ffd7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 --2- 172.21.15.19:0/670942054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd7f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe174ff9640 0 client.28055 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 >> 172.21.15.19:0/670942054 conn(0x5572ff9c0300 msgr2=0x5572ffd74450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 shutdown_connections 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 1 -- 172.21.15.19:0/670942054 wait complete. 2024-05-17T09:39:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.823+0000 7fe19a247740 20 client.28055 trim_cache size 0 max 16384 2024-05-17T09:39:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.845+0000 7fe19a247740 1 Processor -- start 2024-05-17T09:39:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.845+0000 7fe19a247740 1 -- start start 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19a247740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd67a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19a247740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19a247740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd86e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19a247740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1ad120 con 0x5572ff9ab4d0 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19a247740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1aa9f0 con 0x5572ff9a32f0 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19a247740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1a8840 con 0x5572ff9cc1b0 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd67a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe190928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd67a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50368/0 (socket says 172.21.15.19:50368) 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 -- 172.21.15.19:0/2429366788 learned_addr learned my addr 172.21.15.19:0/2429366788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19192a640 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd86e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 -- 172.21.15.19:0/2429366788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 msgr2=0x5572ffd86e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd86e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 -- 172.21.15.19:0/2429366788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 msgr2=0x5572ffd868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe191129640 1 -- 172.21.15.19:0/2429366788 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ffd76770 con 0x5572ff9a32f0 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.846+0000 7fe19192a640 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.847+0000 7fe191129640 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd67a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe18c104840 tx=0x7fe188017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793ac4679d7fc2a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.847+0000 7fe176ffd640 1 -- 172.21.15.19:0/2429366788 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18801ed80 con 0x5572ff9a32f0 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.847+0000 7fe176ffd640 1 -- 172.21.15.19:0/2429366788 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe18802a8b0 con 0x5572ff9a32f0 2024-05-17T09:39:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.847+0000 7fe176ffd640 1 -- 172.21.15.19:0/2429366788 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18801c670 con 0x5572ff9a32f0 2024-05-17T09:39:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 -- 172.21.15.19:0/2429366788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 msgr2=0x5572ffd67a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd67a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe18c104840 tx=0x7fe188017000 comp rx=0 tx=0).stop 2024-05-17T09:39:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 -- 172.21.15.19:0/2429366788 shutdown_connections 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffd86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9ab4d0 0x5572ffd868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 --2- 172.21.15.19:0/2429366788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9a32f0 0x5572ffd67a40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 -- 172.21.15.19:0/2429366788 >> 172.21.15.19:0/2429366788 conn(0x5572ff9c0300 msgr2=0x5572ffd67330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 -- 172.21.15.19:0/2429366788 shutdown_connections 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.848+0000 7fe19a247740 1 -- 172.21.15.19:0/2429366788 wait complete. 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 -- start start 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 Processor -- start 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 0x5572ffd846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffdf3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1a8eb0 con 0x5572ff9a32f0 2024-05-17T09:39:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1aa320 con 0x5572ff9ab4d0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1a9580 con 0x5572ff9cc1b0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe190928640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 0x5572ffd846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe19a247740 10 client.0 _ll_register_callbacks cb 0x7fe1840a9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe191129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe190928640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 0x5572ffd846f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50380/0 (socket says 172.21.15.19:50380) 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.849+0000 7fe190928640 1 -- 172.21.15.19:0/1814395302 learned_addr learned my addr 172.21.15.19:0/1814395302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe19192a640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffdf3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe190928640 1 -- 172.21.15.19:0/1814395302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 msgr2=0x5572ffdf3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe190928640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffdf3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe190928640 1 -- 172.21.15.19:0/1814395302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 msgr2=0x5572ffd841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe190928640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd841b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe190928640 1 -- 172.21.15.19:0/1814395302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ffdf4230 con 0x5572ff9ab4d0 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe191129640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:39:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe190928640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 0x5572ffd846f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009a20 tx=0x7fe17c032cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6acac45ca49ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe174ff9640 1 -- 172.21.15.19:0/1814395302 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c04d020 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe174ff9640 1 -- 172.21.15.19:0/1814395302 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe17c004450 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.850+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18802a8b0 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe174ff9640 1 -- 172.21.15.19:0/1814395302 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe17c058050 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 10 client.28061 fetch_fsmap learned FSMap version 39 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572ffdf5460 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe174ff9640 1 -- 172.21.15.19:0/1814395302 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe18802a8b0 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe174ff9640 1 -- 172.21.15.19:0/1814395302 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe17c0544a0 con 0x5572ff9ab4d0 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 10 client.28061 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 10 client.28061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:39:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 20 client.28061 populate_metadata read hostname 'smithi019' 2024-05-17T09:39:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff9601c0 0x5572ffdf7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:39:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 4 client.28061 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:39:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572ffcdb360 con 0x5572ff9601c0 2024-05-17T09:39:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.851+0000 7fe191129640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff9601c0 0x5572ffdf7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:39:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.852+0000 7fe174ff9640 10 client.28061 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.852+0000 7fe191129640 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff9601c0 0x5572ffdf7970 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c8b7ff802ea215 server_cookie=496812957999319c in_seq=0 out_seq=0 2024-05-17T09:39:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe174ff9640 1 -- 172.21.15.19:0/1814395302 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5572ffcdb360 con 0x5572ff9601c0 2024-05-17T09:39:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe174ff9640 10 client.28061 handle_command_reply: tid=0 2024-05-17T09:39:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 client.28061 shutdown 2024-05-17T09:39:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 msgr2=0x5572ffd846f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 0x5572ffd846f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009a20 tx=0x7fe17c032cb0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 shutdown_connections 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5572ff9601c0 0x5572ffdf7970 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5572ff9cc1b0 0x5572ffdf3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5572ff9ab4d0 0x5572ffd846f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 --2- 172.21.15.19:0/1814395302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572ff9a32f0 0x5572ffd841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 >> 172.21.15.19:0/1814395302 conn(0x5572ff9c0300 msgr2=0x5572ffd74340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe174ff9640 0 client.28061 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 shutdown_connections 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.944+0000 7fe19a247740 1 -- 172.21.15.19:0/1814395302 wait complete. 2024-05-17T09:39:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.945+0000 7fe19a247740 20 client.28061 trim_cache size 0 max 16384 2024-05-17T09:39:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:39:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:39:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.945+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1780b28a0 msgr2=0x7fe1780b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.945+0000 7fe19338d640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1780b28a0 0x7fe1780b4db0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5572ffd76f90 tx=0x7fe188012040 comp rx=0 tx=0).stop 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 msgr2=0x7fe18c1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 0x7fe18c1a1700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe184009fd0 tx=0x7fe18400ef10 comp rx=0 tx=0).stop 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 shutdown_connections 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe18c109b80 0x7fe18c1a1700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe18c104e80 0x7fe18c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1780b28a0 0x7fe1780b4db0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 --2- 172.21.15.19:0/3595934262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18c103c40 0x7fe18c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:39:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 >> 172.21.15.19:0/3595934262 conn(0x7fe18c0fee50 msgr2=0x7fe18c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:39:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 shutdown_connections 2024-05-17T09:39:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:39:57.946+0000 7fe19338d640 1 -- 172.21.15.19:0/3595934262 wait complete. 2024-05-17T09:39:58.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 42907 6.42 MB/sec execute 235 sec latency 123.070 ms 2024-05-17T09:39:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:39:58 smithi079 ceph-mon[35502]: pgmap v1044: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-05-17T09:39:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:39:58 smithi069 ceph-mon[35081]: pgmap v1044: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-05-17T09:39:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:39:58 smithi019 ceph-mon[31620]: pgmap v1044: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-05-17T09:39:59.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 43109 6.46 MB/sec execute 236 sec latency 65.891 ms 2024-05-17T09:40:00.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:39:59.998+0000 7f49787cc700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:40:00.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 43293 6.46 MB/sec execute 237 sec latency 66.030 ms 2024-05-17T09:40:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:00 smithi079 ceph-mon[35502]: pgmap v1045: 129 pgs: 129 active+clean; 411 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-05-17T09:40:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:00 smithi079 ceph-mon[35502]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:40:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:00 smithi069 ceph-mon[35081]: pgmap v1045: 129 pgs: 129 active+clean; 411 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-05-17T09:40:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:00 smithi069 ceph-mon[35081]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:40:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:00 smithi019 ceph-mon[31620]: pgmap v1045: 129 pgs: 129 active+clean; 411 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-05-17T09:40:00.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:00 smithi019 ceph-mon[31620]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-17T09:40:01.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 43467 6.46 MB/sec execute 238 sec latency 73.658 ms 2024-05-17T09:40:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:40:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:40:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:01 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:40:01.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:40:01.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:40:01.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:01 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:40:01.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:40:01.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:40:01.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:01 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:40:02.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 43620 6.49 MB/sec execute 239 sec latency 92.236 ms 2024-05-17T09:40:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:02 smithi079 ceph-mon[35502]: pgmap v1046: 129 pgs: 129 active+clean; 412 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-05-17T09:40:02.830 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:02 smithi069 ceph-mon[35081]: pgmap v1046: 129 pgs: 129 active+clean; 412 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-05-17T09:40:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:02 smithi019 ceph-mon[31620]: pgmap v1046: 129 pgs: 129 active+clean; 412 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-05-17T09:40:03.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 43756 6.49 MB/sec execute 240 sec latency 61.097 ms 2024-05-17T09:40:04.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 43929 6.53 MB/sec execute 241 sec latency 71.094 ms 2024-05-17T09:40:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:04 smithi079 ceph-mon[35502]: pgmap v1047: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-17T09:40:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:04 smithi069 ceph-mon[35081]: pgmap v1047: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-17T09:40:04.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:04 smithi019 ceph-mon[31620]: pgmap v1047: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-17T09:40:05.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44103 6.56 MB/sec execute 242 sec latency 66.137 ms 2024-05-17T09:40:06.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44272 6.58 MB/sec execute 243 sec latency 65.205 ms 2024-05-17T09:40:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:06 smithi079 ceph-mon[35502]: pgmap v1048: 129 pgs: 129 active+clean; 423 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-05-17T09:40:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:06 smithi069 ceph-mon[35081]: pgmap v1048: 129 pgs: 129 active+clean; 423 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-05-17T09:40:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:06 smithi019 ceph-mon[31620]: pgmap v1048: 129 pgs: 129 active+clean; 423 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-05-17T09:40:07.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44423 6.57 MB/sec execute 244 sec latency 72.437 ms 2024-05-17T09:40:08.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44526 6.55 MB/sec execute 245 sec latency 106.577 ms 2024-05-17T09:40:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:08 smithi079 ceph-mon[35502]: pgmap v1049: 129 pgs: 129 active+clean; 428 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 4.5 MiB/s wr, 251 op/s 2024-05-17T09:40:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:08 smithi069 ceph-mon[35081]: pgmap v1049: 129 pgs: 129 active+clean; 428 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 4.5 MiB/s wr, 251 op/s 2024-05-17T09:40:08.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:08 smithi019 ceph-mon[31620]: pgmap v1049: 129 pgs: 129 active+clean; 428 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 4.5 MiB/s wr, 251 op/s 2024-05-17T09:40:09.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44658 6.55 MB/sec execute 246 sec latency 65.164 ms 2024-05-17T09:40:10.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44807 6.56 MB/sec execute 247 sec latency 105.669 ms 2024-05-17T09:40:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:10 smithi079 ceph-mon[35502]: pgmap v1050: 129 pgs: 129 active+clean; 432 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-05-17T09:40:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:10 smithi069 ceph-mon[35081]: pgmap v1050: 129 pgs: 129 active+clean; 432 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-05-17T09:40:10.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:10 smithi019 ceph-mon[31620]: pgmap v1050: 129 pgs: 129 active+clean; 432 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-05-17T09:40:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 -- 172.21.15.19:0/1401352896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 msgr2=0x7fed50103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 --2- 172.21.15.19:0/1401352896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed50103840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed40009900 tx=0x7fed4002a410 comp rx=0 tx=0).stop 2024-05-17T09:40:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 -- 172.21.15.19:0/1401352896 shutdown_connections 2024-05-17T09:40:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 --2- 172.21.15.19:0/1401352896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed50108a80 0x7fed5010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 --2- 172.21.15.19:0/1401352896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed50103d80 0x7fed50104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 --2- 172.21.15.19:0/1401352896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed50103840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 -- 172.21.15.19:0/1401352896 >> 172.21.15.19:0/1401352896 conn(0x7fed500fdd30 msgr2=0x7fed501001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 -- 172.21.15.19:0/1401352896 shutdown_connections 2024-05-17T09:40:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.120+0000 7fed557e3640 1 -- 172.21.15.19:0/1401352896 wait complete. 2024-05-17T09:40:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 Processor -- start 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 -- start start 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed5019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed50103d80 0x7fed5019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed50108a80 0x7fed501a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed5010e4d0 con 0x7fed50103d80 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed5010e350 con 0x7fed50102b40 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.121+0000 7fed557e3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed5010e650 con 0x7fed50108a80 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed5019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed5019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:32954/0 (socket says 172.21.15.19:32954) 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 -- 172.21.15.19:0/1959682732 learned_addr learned my addr 172.21.15.19:0/1959682732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed50103d80 0x7fed5019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed50108a80 0x7fed501a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 -- 172.21.15.19:0/1959682732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed50108a80 msgr2=0x7fed501a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed50108a80 0x7fed501a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 -- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed50103d80 msgr2=0x7fed5019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed50103d80 0x7fed5019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 -- 172.21.15.19:0/1959682732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed501a1d30 con 0x7fed50102b40 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.122+0000 7fed4effd640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed5019a1f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed40005760 tx=0x7fed40005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.123+0000 7fed37fff640 1 -- 172.21.15.19:0/1959682732 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4003f030 con 0x7fed50102b40 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.123+0000 7fed37fff640 1 -- 172.21.15.19:0/1959682732 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed400043d0 con 0x7fed50102b40 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.123+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed501a1fc0 con 0x7fed50102b40 2024-05-17T09:40:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.123+0000 7fed37fff640 1 -- 172.21.15.19:0/1959682732 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40035070 con 0x7fed50102b40 2024-05-17T09:40:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.123+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed501a2420 con 0x7fed50102b40 2024-05-17T09:40:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed37fff640 1 -- 172.21.15.19:0/1959682732 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fed40042030 con 0x7fed50102b40 2024-05-17T09:40:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 Processor -- start 2024-05-17T09:40:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 -- start start 2024-05-17T09:40:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x56035849c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 0x56035849c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580da4d0 0x5603584a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56035840ec30 con 0x5603580fb1b0 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5603583fe7d0 con 0x5603580da4d0 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.126+0000 7fed5c76e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560358402810 con 0x5603580d22f0 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x56035849c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 0x56035849c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x56035849c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37118/0 (socket says 172.21.15.19:37118) 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580da4d0 0x5603584a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 0x56035849c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46512/0 (socket says 172.21.15.19:46512) 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 -- 172.21.15.19:0/4240237364 learned_addr learned my addr 172.21.15.19:0/4240237364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 -- 172.21.15.19:0/4240237364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 msgr2=0x56035849c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 0x56035849c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 -- 172.21.15.19:0/4240237364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580da4d0 msgr2=0x5603584a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580da4d0 0x5603584a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 -- 172.21.15.19:0/4240237364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560358113210 con 0x5603580fb1b0 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4effd640 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 0x56035849c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580da4d0 0x5603584a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed37fff640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fed380b2920 0x7fed380b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.127+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x56035849c3a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fed48009fc0 tx=0x7fed48010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a6f608542bd9ee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.128+0000 7fed4effd640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fed380b2920 0x7fed380b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.128+0000 7fed37fff640 1 -- 172.21.15.19:0/1959682732 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fed40102910 con 0x7fed50102b40 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.128+0000 7fed357fa640 1 -- 172.21.15.19:0/4240237364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48003b70 con 0x5603580fb1b0 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.128+0000 7fed357fa640 1 -- 172.21.15.19:0/4240237364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed48003d10 con 0x5603580fb1b0 2024-05-17T09:40:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.128+0000 7fed357fa640 1 -- 172.21.15.19:0/4240237364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48011980 con 0x5603580fb1b0 2024-05-17T09:40:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 -- 172.21.15.19:0/4240237364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 msgr2=0x56035849c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x56035849c3a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fed48009fc0 tx=0x7fed48010860 comp rx=0 tx=0).stop 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed4effd640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fed380b2920 0x7fed380b4e30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed400c9820 tx=0x7fed40108530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 -- 172.21.15.19:0/4240237364 shutdown_connections 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580da4d0 0x5603584a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580d22f0 0x56035849c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 --2- 172.21.15.19:0/4240237364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x56035849c3a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 -- 172.21.15.19:0/4240237364 >> 172.21.15.19:0/4240237364 conn(0x5603580ef300 msgr2=0x560358497d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 -- 172.21.15.19:0/4240237364 shutdown_connections 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.129+0000 7fed5c76e740 1 -- 172.21.15.19:0/4240237364 wait complete. 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 -- start start 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 Processor -- start 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5603580e4790 con 0x5603580fb1b0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5603583ba890 con 0x5603580d22f0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5603584a4310 con 0x5603580da4d0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed5c76e740 10 client.0 _ll_register_callbacks cb 0x5603583bdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed4e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:32978/0 (socket says 172.21.15.19:32978) 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed4e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46520/0 (socket says 172.21.15.19:46520) 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed4f7fe640 1 -- 172.21.15.19:0/112925793 learned_addr learned my addr 172.21.15.19:0/112925793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.130+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed4e7fc640 1 -- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 msgr2=0x5603584a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed4e7fc640 1 -- 172.21.15.19:0/112925793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 msgr2=0x5603584ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed4e7fc640 1 -- 172.21.15.19:0/112925793 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603581df620 con 0x5603580da4d0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584a6300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fed5019b290 tx=0x7fed44012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b204c78be781165 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed32ff5640 1 -- 172.21.15.19:0/112925793 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44004030 con 0x5603580da4d0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed48003d10 con 0x5603580da4d0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.131+0000 7fed32ff5640 1 -- 172.21.15.19:0/112925793 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fed440041d0 con 0x5603580da4d0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed32ff5640 1 -- 172.21.15.19:0/112925793 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed44025050 con 0x5603580da4d0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 10 client.28063 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5603584b02e0 con 0x5603580da4d0 2024-05-17T09:40:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed32ff5640 1 -- 172.21.15.19:0/112925793 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed48003d10 con 0x5603580da4d0 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed32ff5640 1 -- 172.21.15.19:0/112925793 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fed440038a0 con 0x5603580da4d0 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 10 client.28063 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 10 client.28063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 20 client.28063 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5603580b9d80 0x5603584b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 4 client.28063 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.132+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560358410b80 con 0x5603580b9d80 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.133+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5603580b9d80 0x5603584b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.133+0000 7fed32ff5640 10 client.28063 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.133+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5603580b9d80 0x5603584b3ac0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e1658de221142b server_cookie=203f0a63148a4dfe in_seq=0 out_seq=0 2024-05-17T09:40:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed32ff5640 1 -- 172.21.15.19:0/112925793 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560358410b80 con 0x5603580b9d80 2024-05-17T09:40:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed32ff5640 10 client.28063 handle_command_reply: tid=0 2024-05-17T09:40:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 client.28063 shutdown 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 msgr2=0x5603584a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584a6300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fed5019b290 tx=0x7fed44012b60 comp rx=0 tx=0).stop 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 shutdown_connections 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5603580b9d80 0x5603584b3ac0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584a6300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 --2- 172.21.15.19:0/112925793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 >> 172.21.15.19:0/112925793 conn(0x5603580ef300 msgr2=0x56035849ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed32ff5640 0 client.28063 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 shutdown_connections 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 1 -- 172.21.15.19:0/112925793 wait complete. 2024-05-17T09:40:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.134+0000 7fed5c76e740 20 client.28063 trim_cache size 0 max 16384 2024-05-17T09:40:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 Processor -- start 2024-05-17T09:40:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 -- start start 2024-05-17T09:40:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 0x5603584a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x56035849b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580fb1b0 0x5603584ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed501af210 con 0x5603580d22f0 2024-05-17T09:40:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed501acae0 con 0x5603580fb1b0 2024-05-17T09:40:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed5c76e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed501aa930 con 0x5603580da4d0 2024-05-17T09:40:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 0x5603584a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 0x5603584a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37146/0 (socket says 172.21.15.19:37146) 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed4f7fe640 1 -- 172.21.15.19:0/3817538559 learned_addr learned my addr 172.21.15.19:0/3817538559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580fb1b0 0x5603584ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.159+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x56035849b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed4f7fe640 1 -- 172.21.15.19:0/3817538559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 msgr2=0x56035849b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x56035849b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed4f7fe640 1 -- 172.21.15.19:0/3817538559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580fb1b0 msgr2=0x5603584ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580fb1b0 0x5603584ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed4f7fe640 1 -- 172.21.15.19:0/3817538559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035849b780 con 0x5603580d22f0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 0x5603584a08b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fed48011010 tx=0x7fed48009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6806605b555fc40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed34ff9640 1 -- 172.21.15.19:0/3817538559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48024040 con 0x5603580d22f0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed34ff9640 1 -- 172.21.15.19:0/3817538559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed480116b0 con 0x5603580d22f0 2024-05-17T09:40:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.160+0000 7fed34ff9640 1 -- 172.21.15.19:0/3817538559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4802c690 con 0x5603580d22f0 2024-05-17T09:40:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.161+0000 7fed5c76e740 1 -- 172.21.15.19:0/3817538559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 msgr2=0x5603584a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.161+0000 7fed5c76e740 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 0x5603584a08b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fed48011010 tx=0x7fed48009500 comp rx=0 tx=0).stop 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 -- 172.21.15.19:0/3817538559 shutdown_connections 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580fb1b0 0x5603584ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x56035849b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 --2- 172.21.15.19:0/3817538559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580d22f0 0x5603584a08b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 -- 172.21.15.19:0/3817538559 >> 172.21.15.19:0/3817538559 conn(0x5603580ef300 msgr2=0x56035849ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 -- 172.21.15.19:0/3817538559 shutdown_connections 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 -- 172.21.15.19:0/3817538559 wait complete. 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 -- start start 2024-05-17T09:40:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.162+0000 7fed5c76e740 1 Processor -- start 2024-05-17T09:40:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed501aafa0 con 0x5603580fb1b0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed501ac410 con 0x5603580d22f0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed501ab670 con 0x5603580da4d0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed5c76e740 10 client.0 _ll_register_callbacks cb 0x7fed44009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:32992/0 (socket says 172.21.15.19:32992) 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 -- 172.21.15.19:0/663797168 learned_addr learned my addr 172.21.15.19:0/663797168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46536/0 (socket says 172.21.15.19:46536) 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 -- 172.21.15.19:0/663797168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 msgr2=0x5603584ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 -- 172.21.15.19:0/663797168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 msgr2=0x5603584ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.163+0000 7fed4f7fe640 1 -- 172.21.15.19:0/663797168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035851a270 con 0x5603580d22f0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed4f7fe640 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584ac3c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed48011fd0 tx=0x7fed48013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967d8561bb8c1a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed32ff5640 1 -- 172.21.15.19:0/663797168 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48011b70 con 0x5603580d22f0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed480116b0 con 0x5603580d22f0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed32ff5640 1 -- 172.21.15.19:0/663797168 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fed480050d0 con 0x5603580d22f0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed32ff5640 1 -- 172.21.15.19:0/663797168 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed480052d0 con 0x5603580d22f0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed5c76e740 10 client.28079 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.164+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56035851b4e0 con 0x5603580d22f0 2024-05-17T09:40:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed32ff5640 1 -- 172.21.15.19:0/663797168 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed480116b0 con 0x5603580d22f0 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed32ff5640 1 -- 172.21.15.19:0/663797168 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fed4802cb40 con 0x5603580d22f0 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed5c76e740 10 client.28079 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed5c76e740 10 client.28079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed5c76e740 20 client.28079 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed5c76e740 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035808f1c0 0x56035851dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed5c76e740 4 client.28079 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56035840a360 con 0x56035808f1c0 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.165+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035808f1c0 0x56035851dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.166+0000 7fed4e7fc640 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035808f1c0 0x56035851dce0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be811785206149c server_cookie=76448c37c392f1bf in_seq=0 out_seq=0 2024-05-17T09:40:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.166+0000 7fed32ff5640 10 client.28079 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.261+0000 7fed32ff5640 1 -- 172.21.15.19:0/663797168 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56035840a360 con 0x56035808f1c0 2024-05-17T09:40:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.261+0000 7fed32ff5640 10 client.28079 handle_command_reply: tid=0 2024-05-17T09:40:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.261+0000 7fed5c76e740 1 client.28079 shutdown 2024-05-17T09:40:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.264+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 msgr2=0x5603584ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.264+0000 7fed5c76e740 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584ac3c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed48011fd0 tx=0x7fed48013040 comp rx=0 tx=0).stop 2024-05-17T09:40:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.265+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 shutdown_connections 2024-05-17T09:40:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.265+0000 7fed5c76e740 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56035808f1c0 0x56035851dce0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.265+0000 7fed5c76e740 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603580fb1b0 0x5603584ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.265+0000 7fed5c76e740 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603580da4d0 0x5603584ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.265+0000 7fed5c76e740 1 --2- 172.21.15.19:0/663797168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603580d22f0 0x5603584ac3c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.265+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 >> 172.21.15.19:0/663797168 conn(0x5603580ef300 msgr2=0x56035849f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.266+0000 7fed32ff5640 0 client.28079 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.267+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 shutdown_connections 2024-05-17T09:40:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.267+0000 7fed5c76e740 1 -- 172.21.15.19:0/663797168 wait complete. 2024-05-17T09:40:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.268+0000 7fed5c76e740 20 client.28079 trim_cache size 0 max 16384 2024-05-17T09:40:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:40:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:40:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:40:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.271+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fed380b2920 msgr2=0x7fed380b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.271+0000 7fed557e3640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fed380b2920 0x7fed380b4e30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed400c9820 tx=0x7fed40108530 comp rx=0 tx=0).stop 2024-05-17T09:40:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.271+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 msgr2=0x7fed5019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.271+0000 7fed557e3640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed5019a1f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed40005760 tx=0x7fed40005e20 comp rx=0 tx=0).stop 2024-05-17T09:40:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.272+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 shutdown_connections 2024-05-17T09:40:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.272+0000 7fed557e3640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed50108a80 0x7fed501a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.272+0000 7fed557e3640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fed380b2920 0x7fed380b4e30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.272+0000 7fed557e3640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed50103d80 0x7fed5019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.272+0000 7fed557e3640 1 --2- 172.21.15.19:0/1959682732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed50102b40 0x7fed5019a1f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.272+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 >> 172.21.15.19:0/1959682732 conn(0x7fed500fdd30 msgr2=0x7fed50109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.274+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 shutdown_connections 2024-05-17T09:40:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:11.274+0000 7fed557e3640 1 -- 172.21.15.19:0/1959682732 wait complete. 2024-05-17T09:40:11.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 44949 6.58 MB/sec execute 248 sec latency 144.279 ms 2024-05-17T09:40:12.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45095 6.57 MB/sec execute 249 sec latency 101.315 ms 2024-05-17T09:40:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:12 smithi079 ceph-mon[35502]: pgmap v1051: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 230 op/s 2024-05-17T09:40:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:12 smithi069 ceph-mon[35081]: pgmap v1051: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 230 op/s 2024-05-17T09:40:12.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:12 smithi019 ceph-mon[31620]: pgmap v1051: 129 pgs: 129 active+clean; 435 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 230 op/s 2024-05-17T09:40:13.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45261 6.60 MB/sec execute 250 sec latency 86.397 ms 2024-05-17T09:40:14.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45371 6.58 MB/sec execute 251 sec latency 179.213 ms 2024-05-17T09:40:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:14 smithi069 ceph-mon[35081]: pgmap v1052: 129 pgs: 129 active+clean; 434 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-05-17T09:40:14.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:14 smithi019 ceph-mon[31620]: pgmap v1052: 129 pgs: 129 active+clean; 434 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-05-17T09:40:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:14 smithi079 ceph-mon[35502]: pgmap v1052: 129 pgs: 129 active+clean; 434 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-05-17T09:40:15.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45504 6.56 MB/sec execute 252 sec latency 87.216 ms 2024-05-17T09:40:16.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45667 6.57 MB/sec execute 253 sec latency 153.600 ms 2024-05-17T09:40:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:16 smithi069 ceph-mon[35081]: pgmap v1053: 129 pgs: 129 active+clean; 426 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 259 op/s 2024-05-17T09:40:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:16 smithi019 ceph-mon[31620]: pgmap v1053: 129 pgs: 129 active+clean; 426 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 259 op/s 2024-05-17T09:40:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:16 smithi079 ceph-mon[35502]: pgmap v1053: 129 pgs: 129 active+clean; 426 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 259 op/s 2024-05-17T09:40:17.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45833 6.57 MB/sec execute 254 sec latency 81.937 ms 2024-05-17T09:40:18.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 45981 6.57 MB/sec execute 255 sec latency 97.066 ms 2024-05-17T09:40:18.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:18 smithi069 ceph-mon[35081]: pgmap v1054: 129 pgs: 129 active+clean; 423 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-05-17T09:40:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:18 smithi019 ceph-mon[31620]: pgmap v1054: 129 pgs: 129 active+clean; 423 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-05-17T09:40:19.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:18 smithi079 ceph-mon[35502]: pgmap v1054: 129 pgs: 129 active+clean; 423 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-05-17T09:40:19.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 46121 6.56 MB/sec execute 256 sec latency 71.285 ms 2024-05-17T09:40:20.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 46280 6.57 MB/sec execute 257 sec latency 65.750 ms 2024-05-17T09:40:20.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:20 smithi069 ceph-mon[35081]: pgmap v1055: 129 pgs: 129 active+clean; 422 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-05-17T09:40:20.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:20 smithi019 ceph-mon[31620]: pgmap v1055: 129 pgs: 129 active+clean; 422 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-05-17T09:40:21.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:20 smithi079 ceph-mon[35502]: pgmap v1055: 129 pgs: 129 active+clean; 422 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-05-17T09:40:21.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 46474 6.60 MB/sec execute 258 sec latency 67.262 ms 2024-05-17T09:40:22.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 46667 6.62 MB/sec execute 259 sec latency 133.664 ms 2024-05-17T09:40:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:22 smithi069 ceph-mon[35081]: pgmap v1056: 129 pgs: 129 active+clean; 423 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-05-17T09:40:22.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:22 smithi019 ceph-mon[31620]: pgmap v1056: 129 pgs: 129 active+clean; 423 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-05-17T09:40:23.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:22 smithi079 ceph-mon[35502]: pgmap v1056: 129 pgs: 129 active+clean; 423 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.0 MiB/s wr, 247 op/s 2024-05-17T09:40:23.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 46836 6.62 MB/sec execute 260 sec latency 83.678 ms 2024-05-17T09:40:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 -- 172.21.15.19:0/1868378007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630102b40 msgr2=0x7f3630103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 --2- 172.21.15.19:0/1868378007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630102b40 0x7f3630103840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3620009900 tx=0x7f362002a400 comp rx=0 tx=0).stop 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 -- 172.21.15.19:0/1868378007 shutdown_connections 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 --2- 172.21.15.19:0/1868378007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630108a80 0x7f363010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 --2- 172.21.15.19:0/1868378007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3630103d80 0x7f3630104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 --2- 172.21.15.19:0/1868378007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630102b40 0x7f3630103840 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 -- 172.21.15.19:0/1868378007 >> 172.21.15.19:0/1868378007 conn(0x7f36300fdd30 msgr2=0x7f36301001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 -- 172.21.15.19:0/1868378007 shutdown_connections 2024-05-17T09:40:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.522+0000 7f36366e0640 1 -- 172.21.15.19:0/1868378007 wait complete. 2024-05-17T09:40:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 Processor -- start 2024-05-17T09:40:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 -- start start 2024-05-17T09:40:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 0x7f3630198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630103d80 0x7f3630198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3630108a80 0x7f363019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f363010d410 con 0x7f3630102b40 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f363010d290 con 0x7f3630108a80 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f36366e0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f363010d590 con 0x7f3630103d80 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f362ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 0x7f3630198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f362ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 0x7f3630198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45486/0 (socket says 172.21.15.19:45486) 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f362ffff640 1 -- 172.21.15.19:0/870366161 learned_addr learned my addr 172.21.15.19:0/870366161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f3634c7d640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3630108a80 0x7f363019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.523+0000 7f362f7fe640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630103d80 0x7f3630198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362ffff640 1 -- 172.21.15.19:0/870366161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630103d80 msgr2=0x7f3630198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362ffff640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630103d80 0x7f3630198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362ffff640 1 -- 172.21.15.19:0/870366161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3630108a80 msgr2=0x7f363019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362ffff640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3630108a80 0x7f363019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362ffff640 1 -- 172.21.15.19:0/870366161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363019fb20 con 0x7f3630102b40 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362ffff640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 0x7f3630198050 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3620005760 tx=0x7f3620005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362d7fa640 1 -- 172.21.15.19:0/870366161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362003f030 con 0x7f3630102b40 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363019fdb0 con 0x7f3630102b40 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362d7fa640 1 -- 172.21.15.19:0/870366161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36200043d0 con 0x7f3630102b40 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f362d7fa640 1 -- 172.21.15.19:0/870366161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620035070 con 0x7f3630102b40 2024-05-17T09:40:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.524+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36301a0200 con 0x7f3630102b40 2024-05-17T09:40:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 Processor -- start 2024-05-17T09:40:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 -- start start 2024-05-17T09:40:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f362d7fa640 1 -- 172.21.15.19:0/870366161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3620042030 con 0x7f3630102b40 2024-05-17T09:40:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5c82f0 0x55dc8a992920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a999820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a904c30 con 0x55dc8a5d04d0 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a8f47d0 con 0x55dc8a5c82f0 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.528+0000 7f363d58b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a8f8810 con 0x55dc8a5f11b0 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9923a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35738/0 (socket says 172.21.15.19:35738) 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 -- 172.21.15.19:0/646083344 learned_addr learned my addr 172.21.15.19:0/646083344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f362f7fe640 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a999820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 -- 172.21.15.19:0/646083344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5c82f0 msgr2=0x55dc8a992920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:40:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5c82f0 0x55dc8a992920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 -- 172.21.15.19:0/646083344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 msgr2=0x55dc8a999820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a999820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 -- 172.21.15.19:0/646083344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8a609210 con 0x55dc8a5f11b0 2024-05-17T09:40:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f362f7fe640 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a999820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f3634c7d640 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9923a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f362800c2d0 tx=0x7f3628009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d34e19230b73965a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f362d7fa640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36180b2920 0x7f36180b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.529+0000 7f36167fc640 1 -- 172.21.15.19:0/646083344 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628003c30 con 0x55dc8a5f11b0 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.530+0000 7f36167fc640 1 -- 172.21.15.19:0/646083344 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362801b070 con 0x55dc8a5f11b0 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.530+0000 7f36167fc640 1 -- 172.21.15.19:0/646083344 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628016650 con 0x55dc8a5f11b0 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.530+0000 7f362ffff640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36180b2920 0x7f36180b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.530+0000 7f362d7fa640 1 -- 172.21.15.19:0/870366161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3620102940 con 0x7f3630102b40 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 -- 172.21.15.19:0/646083344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 msgr2=0x55dc8a9923a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f362ffff640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36180b2920 0x7f36180b4e30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36200c9610 tx=0x7f36200c4140 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9923a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f362800c2d0 tx=0x7f3628009500 comp rx=0 tx=0).stop 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 -- 172.21.15.19:0/646083344 shutdown_connections 2024-05-17T09:40:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a999820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5c82f0 0x55dc8a992920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 --2- 172.21.15.19:0/646083344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9923a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 -- 172.21.15.19:0/646083344 >> 172.21.15.19:0/646083344 conn(0x55dc8a5e5300 msgr2=0x55dc8a98dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 -- 172.21.15.19:0/646083344 shutdown_connections 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 -- 172.21.15.19:0/646083344 wait complete. 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 -- start start 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.531+0000 7f363d58b740 1 Processor -- start 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5c82f0 0x55dc8a99bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a99c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a5da790 con 0x55dc8a5d04d0 2024-05-17T09:40:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a8b0890 con 0x55dc8a5f11b0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a99a270 con 0x55dc8a5c82f0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f363d58b740 10 client.0 _ll_register_callbacks cb 0x55dc8a8b3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a99c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a99c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45504/0 (socket says 172.21.15.19:45504) 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f362f7fe640 1 -- 172.21.15.19:0/3714051454 learned_addr learned my addr 172.21.15.19:0/3714051454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f3634c7d640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.532+0000 7f3634c7d640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5c82f0 0x55dc8a99bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f362f7fe640 1 -- 172.21.15.19:0/3714051454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5c82f0 msgr2=0x55dc8a99bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f362f7fe640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5c82f0 0x55dc8a99bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f362f7fe640 1 -- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5f11b0 msgr2=0x55dc8a9a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f362f7fe640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f362f7fe640 1 -- 172.21.15.19:0/3714051454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8a6d5620 con 0x55dc8a5d04d0 2024-05-17T09:40:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f3634c7d640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f362f7fe640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a99c250 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3624009840 tx=0x7f3624009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc175a9c9221faa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f360f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f360f7fe640 1 -- 172.21.15.19:0/3714051454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624004460 con 0x55dc8a5d04d0 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.533+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362801b070 con 0x55dc8a5d04d0 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.534+0000 7f360f7fe640 1 -- 172.21.15.19:0/3714051454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f362400b820 con 0x55dc8a5d04d0 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.534+0000 7f360f7fe640 1 -- 172.21.15.19:0/3714051454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f362400ba20 con 0x55dc8a5d04d0 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.534+0000 7f363d58b740 10 client.18558 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.534+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dc8a9a6170 con 0x55dc8a5d04d0 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.534+0000 7f360f7fe640 1 -- 172.21.15.19:0/3714051454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362801b070 con 0x55dc8a5d04d0 2024-05-17T09:40:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f360f7fe640 1 -- 172.21.15.19:0/3714051454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3624024450 con 0x55dc8a5d04d0 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f363d58b740 10 client.18558 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f363d58b740 10 client.18558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f363d58b740 20 client.18558 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f363d58b740 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5afd80 0x55dc8a9a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f363d58b740 4 client.18558 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8a906b80 con 0x55dc8a5afd80 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.535+0000 7f3634c7d640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5afd80 0x55dc8a9a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.536+0000 7f360f7fe640 10 client.18558 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.537+0000 7f3634c7d640 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5afd80 0x55dc8a9a9950 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c490daa271c3b497 server_cookie=5580ca759ed604c4 in_seq=0 out_seq=0 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f360f7fe640 1 -- 172.21.15.19:0/3714051454 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc8a906b80 con 0x55dc8a5afd80 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f360f7fe640 10 client.18558 handle_command_reply: tid=0 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 client.18558 shutdown 2024-05-17T09:40:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 msgr2=0x55dc8a99c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a99c250 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3624009840 tx=0x7f3624009d00 comp rx=0 tx=0).stop 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 shutdown_connections 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5afd80 0x55dc8a9a9950 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5f11b0 0x55dc8a9a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5d04d0 0x55dc8a99c250 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f360f7fe640 0 client.18558 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 --2- 172.21.15.19:0/3714051454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5c82f0 0x55dc8a99bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.538+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 >> 172.21.15.19:0/3714051454 conn(0x55dc8a5e5300 msgr2=0x55dc8a98cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.539+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 shutdown_connections 2024-05-17T09:40:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.539+0000 7f363d58b740 1 -- 172.21.15.19:0/3714051454 wait complete. 2024-05-17T09:40:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.539+0000 7f363d58b740 20 client.18558 trim_cache size 0 max 16384 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.566+0000 7f363d58b740 1 Processor -- start 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.566+0000 7f363d58b740 1 -- start start 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f363d58b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5afd80 0x55dc8a991240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f363d58b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 0x55dc8a991780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f363d58b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f363d58b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36301ad010 con 0x55dc8a5c82f0 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f363d58b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36301aa8e0 con 0x55dc8a5d04d0 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f363d58b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36301a8730 con 0x55dc8a5afd80 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 0x55dc8a991780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 0x55dc8a991780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45514/0 (socket says 172.21.15.19:45514) 2024-05-17T09:40:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 -- 172.21.15.19:0/508831402 learned_addr learned my addr 172.21.15.19:0/508831402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f3634c7d640 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f3634c7d640 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5afd80 0x55dc8a991240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 -- 172.21.15.19:0/508831402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5afd80 msgr2=0x55dc8a991240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5afd80 0x55dc8a991240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 -- 172.21.15.19:0/508831402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 msgr2=0x55dc8a9a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f362f7fe640 1 -- 172.21.15.19:0/508831402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8a989540 con 0x55dc8a5c82f0 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.567+0000 7f3634c7d640 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5afd80 0x55dc8a991240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.568+0000 7f362f7fe640 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 0x55dc8a991780 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3624024870 tx=0x7f362400b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af229843540f6b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.568+0000 7f3615ffb640 1 -- 172.21.15.19:0/508831402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624024450 con 0x55dc8a5c82f0 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.568+0000 7f3615ffb640 1 -- 172.21.15.19:0/508831402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362401f070 con 0x55dc8a5c82f0 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.568+0000 7f3615ffb640 1 -- 172.21.15.19:0/508831402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36240035f0 con 0x55dc8a5c82f0 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 -- 172.21.15.19:0/508831402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 msgr2=0x55dc8a991780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 0x55dc8a991780 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3624024870 tx=0x7f362400b9b0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 -- 172.21.15.19:0/508831402 shutdown_connections 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5c82f0 0x55dc8a991780 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 --2- 172.21.15.19:0/508831402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5afd80 0x55dc8a991240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 -- 172.21.15.19:0/508831402 >> 172.21.15.19:0/508831402 conn(0x55dc8a5e5300 msgr2=0x55dc8a990b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 -- 172.21.15.19:0/508831402 shutdown_connections 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 -- 172.21.15.19:0/508831402 wait complete. 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.569+0000 7f363d58b740 1 -- start start 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 Processor -- start 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5afd80 0x55dc8a9a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8aa0ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36301a8da0 con 0x55dc8a5afd80 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36301aa210 con 0x55dc8a5d04d0 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36301a9470 con 0x55dc8a5f11b0 2024-05-17T09:40:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f363d58b740 10 client.0 _ll_register_callbacks cb 0x7f3624010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f3634c7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5afd80 0x55dc8a9a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:57376/0 (socket says 172.21.15.19:57376) 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.570+0000 7f362f7fe640 1 -- 172.21.15.19:0/4166010543 learned_addr learned my addr 172.21.15.19:0/4166010543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f3634c7d640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8aa0ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f362f7fe640 1 -- 172.21.15.19:0/4166010543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 msgr2=0x55dc8aa0ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f362f7fe640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8aa0ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f362f7fe640 1 -- 172.21.15.19:0/4166010543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5afd80 msgr2=0x55dc8a9a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f362f7fe640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5afd80 0x55dc8a9a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f362f7fe640 1 -- 172.21.15.19:0/4166010543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8a9a3c00 con 0x55dc8a5d04d0 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f3634c7d640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8aa0ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f362f7fe640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a0c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f362400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4206cb7e93ef3d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f360f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:40:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f360f7fe640 1 -- 172.21.15.19:0/4166010543 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362403e070 con 0x55dc8a5d04d0 2024-05-17T09:40:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f360f7fe640 1 -- 172.21.15.19:0/4166010543 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f36240054d0 con 0x55dc8a5d04d0 2024-05-17T09:40:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.571+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362401f070 con 0x55dc8a5d04d0 2024-05-17T09:40:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f360f7fe640 1 -- 172.21.15.19:0/4166010543 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3624024660 con 0x55dc8a5d04d0 2024-05-17T09:40:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 10 client.28094 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dc8aa114e0 con 0x55dc8a5d04d0 2024-05-17T09:40:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f360f7fe640 1 -- 172.21.15.19:0/4166010543 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362401f070 con 0x55dc8a5d04d0 2024-05-17T09:40:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f360f7fe640 1 -- 172.21.15.19:0/4166010543 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f362402c070 con 0x55dc8a5d04d0 2024-05-17T09:40:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 10 client.28094 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 10 client.28094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 20 client.28094 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5851c0 0x55dc8aa13df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 4 client.28094 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:40:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc8a900360 con 0x55dc8a5851c0 2024-05-17T09:40:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.572+0000 7f3634c7d640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5851c0 0x55dc8aa13df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.573+0000 7f360f7fe640 10 client.28094 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.573+0000 7f3634c7d640 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5851c0 0x55dc8aa13df0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96640512a784964d server_cookie=ce3bb2b6a9b87049 in_seq=0 out_seq=0 2024-05-17T09:40:24.584 INFO:tasks.workunit.client.0.smithi019.stdout: 10 46954 6.62 MB/sec execute 261 sec latency 154.393 ms 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f360f7fe640 1 -- 172.21.15.19:0/4166010543 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc8a900360 con 0x55dc8a5851c0 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f360f7fe640 10 client.28094 handle_command_reply: tid=0 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 client.28094 shutdown 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 msgr2=0x55dc8a9a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a0c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f362400bfd0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 shutdown_connections 2024-05-17T09:40:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55dc8a5851c0 0x55dc8aa13df0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc8a5f11b0 0x55dc8aa0ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc8a5d04d0 0x55dc8a9a0c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f360f7fe640 0 client.28094 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 --2- 172.21.15.19:0/4166010543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc8a5afd80 0x55dc8a9a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 >> 172.21.15.19:0/4166010543 conn(0x55dc8a5e5300 msgr2=0x55dc8a991560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 shutdown_connections 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.692+0000 7f363d58b740 1 -- 172.21.15.19:0/4166010543 wait complete. 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.693+0000 7f363d58b740 20 client.28094 trim_cache size 0 max 16384 2024-05-17T09:40:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:40:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:40:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.693+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36180b2920 msgr2=0x7f36180b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.693+0000 7f36366e0640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36180b2920 0x7f36180b4e30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36200c9610 tx=0x7f36200c4140 comp rx=0 tx=0).stop 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.693+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 msgr2=0x7f3630198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.693+0000 7f36366e0640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 0x7f3630198050 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3620005760 tx=0x7f3620005e20 comp rx=0 tx=0).stop 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 shutdown_connections 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3630108a80 0x7f363019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3630103d80 0x7f3630198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f36180b2920 0x7f36180b4e30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 --2- 172.21.15.19:0/870366161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3630102b40 0x7f3630198050 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 >> 172.21.15.19:0/870366161 conn(0x7f36300fdd30 msgr2=0x7f3630104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 shutdown_connections 2024-05-17T09:40:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:24.694+0000 7f36366e0640 1 -- 172.21.15.19:0/870366161 wait complete. 2024-05-17T09:40:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:24 smithi069 ceph-mon[35081]: pgmap v1057: 129 pgs: 129 active+clean; 426 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-05-17T09:40:24.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:24 smithi019 ceph-mon[31620]: pgmap v1057: 129 pgs: 129 active+clean; 426 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-05-17T09:40:25.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:24 smithi079 ceph-mon[35502]: pgmap v1057: 129 pgs: 129 active+clean; 426 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-05-17T09:40:25.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 47110 6.64 MB/sec execute 262 sec latency 122.996 ms 2024-05-17T09:40:25.729 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.3 2024-05-17T09:40:25.730 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:40:26.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 47295 6.68 MB/sec execute 263 sec latency 78.461 ms 2024-05-17T09:40:26.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:26 smithi019 ceph-mon[31620]: pgmap v1058: 129 pgs: 129 active+clean; 427 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-05-17T09:40:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:26 smithi069 ceph-mon[35081]: pgmap v1058: 129 pgs: 129 active+clean; 427 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-05-17T09:40:27.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:26 smithi079 ceph-mon[35502]: pgmap v1058: 129 pgs: 129 active+clean; 427 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-05-17T09:40:27.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 47452 6.68 MB/sec execute 264 sec latency 64.695 ms 2024-05-17T09:40:28.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 47592 6.68 MB/sec execute 265 sec latency 61.052 ms 2024-05-17T09:40:28.610 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:40:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:28 smithi069 ceph-mon[35081]: pgmap v1059: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-05-17T09:40:29.075 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:28 smithi079 ceph-mon[35502]: pgmap v1059: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-05-17T09:40:29.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:28 smithi019 ceph-mon[31620]: pgmap v1059: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-05-17T09:40:29.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 47739 6.68 MB/sec execute 266 sec latency 83.447 ms 2024-05-17T09:40:30.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 47890 6.69 MB/sec execute 267 sec latency 105.410 ms 2024-05-17T09:40:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:30 smithi069 ceph-mon[35081]: pgmap v1060: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-05-17T09:40:30.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:30 smithi019 ceph-mon[31620]: pgmap v1060: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-05-17T09:40:31.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:30 smithi079 ceph-mon[35502]: pgmap v1060: 129 pgs: 129 active+clean; 431 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-05-17T09:40:31.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48037 6.69 MB/sec execute 268 sec latency 77.031 ms 2024-05-17T09:40:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 -- 172.21.15.19:0/2768747908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 msgr2=0x7fe154113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2768747908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 0x7fe154113340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009f90 tx=0x7fe14c00b330 comp rx=0 tx=0).stop 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 -- 172.21.15.19:0/2768747908 shutdown_connections 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2768747908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 0x7fe154113340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2768747908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe15410c5a0 0x7fe15410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2768747908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe154106560 0x7fe154106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 -- 172.21.15.19:0/2768747908 >> 172.21.15.19:0/2768747908 conn(0x7fe1541020c0 msgr2=0x7fe154104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.139+0000 7fe15c16a700 1 -- 172.21.15.19:0/2768747908 shutdown_connections 2024-05-17T09:40:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.140+0000 7fe15c16a700 1 -- 172.21.15.19:0/2768747908 wait complete. 2024-05-17T09:40:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 Processor -- start 2024-05-17T09:40:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 -- start start 2024-05-17T09:40:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 0x7fe154077cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 0x7fe154078200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15410c5a0 0x7fe154078740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541165d0 con 0x7fe154106560 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe154116450 con 0x7fe15410c5a0 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe15c16a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe154116750 con 0x7fe154106eb0 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe159f06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 0x7fe154077cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe159f06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 0x7fe154077cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45554/0 (socket says 172.21.15.19:45554) 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.141+0000 7fe159f06700 1 -- 172.21.15.19:0/2242776750 learned_addr learned my addr 172.21.15.19:0/2242776750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.142+0000 7fe15a707700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15410c5a0 0x7fe154078740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.142+0000 7fe159f06700 1 -- 172.21.15.19:0/2242776750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 msgr2=0x7fe154078200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.142+0000 7fe159f06700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 0x7fe154078200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.142+0000 7fe159f06700 1 -- 172.21.15.19:0/2242776750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15410c5a0 msgr2=0x7fe154078740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.142+0000 7fe159f06700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15410c5a0 0x7fe154078740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.142+0000 7fe159f06700 1 -- 172.21.15.19:0/2242776750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1541ad560 con 0x7fe154106560 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.143+0000 7fe159f06700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 0x7fe154077cc0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe15000d950 tx=0x7fe15000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.143+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150016070 con 0x7fe154106560 2024-05-17T09:40:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.143+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe15000de30 con 0x7fe154106560 2024-05-17T09:40:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.143+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150005240 con 0x7fe154106560 2024-05-17T09:40:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.144+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1541ad7f0 con 0x7fe154106560 2024-05-17T09:40:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.144+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541adce0 con 0x7fe154106560 2024-05-17T09:40:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.145+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe154107c80 con 0x7fe154106560 2024-05-17T09:40:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.148+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe1500049a0 con 0x7fe154106560 2024-05-17T09:40:32.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.149+0000 7fe14affd700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2900 0x7fe1400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:32.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.149+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe1500e2110 con 0x7fe154106560 2024-05-17T09:40:32.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.150+0000 7fe159705700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2900 0x7fe1400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.151+0000 7fe159705700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2900 0x7fe1400b4e10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe144005950 tx=0x7fe1440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:32.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.156+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe15001f070 con 0x7fe154106560 2024-05-17T09:40:32.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48167 6.69 MB/sec execute 269 sec latency 125.521 ms 2024-05-17T09:40:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.625+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe154079220 con 0x7fe154106560 2024-05-17T09:40:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.626+0000 7fe14affd700 1 -- 172.21.15.19:0/2242776750 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fe154079220 con 0x7fe154106560 2024-05-17T09:40:32.626 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:40:32.627 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2900 msgr2=0x7fe1400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2900 0x7fe1400b4e10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe144005950 tx=0x7fe1440058e0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 msgr2=0x7fe154077cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 0x7fe154077cc0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe15000d950 tx=0x7fe15000dd10 comp rx=0 tx=0).stop 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 shutdown_connections 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe15410c5a0 0x7fe154078740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe154106560 0x7fe154077cc0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe1400b2900 0x7fe1400b4e10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 --2- 172.21.15.19:0/2242776750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe154106eb0 0x7fe154078200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 >> 172.21.15.19:0/2242776750 conn(0x7fe1541020c0 msgr2=0x7fe154104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 shutdown_connections 2024-05-17T09:40:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:32.630+0000 7fe15c16a700 1 -- 172.21.15.19:0/2242776750 wait complete. 2024-05-17T09:40:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:40:32.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:32 smithi079 ceph-mon[35502]: pgmap v1061: 129 pgs: 129 active+clean; 435 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-05-17T09:40:32.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:32 smithi069 ceph-mon[35081]: pgmap v1061: 129 pgs: 129 active+clean; 435 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-05-17T09:40:32.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:32 smithi019 ceph-mon[31620]: pgmap v1061: 129 pgs: 129 active+clean; 435 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-05-17T09:40:33.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48316 6.70 MB/sec execute 270 sec latency 124.547 ms 2024-05-17T09:40:33.765 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 131.3 --await-for 45.0' 2024-05-17T09:40:33.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 131.3 --await-for 45.0 2024-05-17T09:40:33.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:33 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2242776750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:40:33.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:33 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2242776750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:40:34.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:33 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2242776750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:40:34.156 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:40:34.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48490 6.72 MB/sec execute 271 sec latency 98.356 ms 2024-05-17T09:40:34.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:34 smithi019 ceph-mon[31620]: pgmap v1062: 129 pgs: 129 active+clean; 437 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 256 op/s 2024-05-17T09:40:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:34 smithi069 ceph-mon[35081]: pgmap v1062: 129 pgs: 129 active+clean; 437 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 256 op/s 2024-05-17T09:40:35.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:34 smithi079 ceph-mon[35502]: pgmap v1062: 129 pgs: 129 active+clean; 437 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 256 op/s 2024-05-17T09:40:35.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48639 6.73 MB/sec execute 272 sec latency 87.756 ms 2024-05-17T09:40:36.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48761 6.73 MB/sec execute 273 sec latency 113.194 ms 2024-05-17T09:40:36.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 -- 172.21.15.19:0/3062703701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc103a40 msgr2=0x7f43dc110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:36.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3062703701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc103a40 0x7f43dc110760 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098f0 tx=0x7f43c4009c00 comp rx=0 tx=0).stop 2024-05-17T09:40:36.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 -- 172.21.15.19:0/3062703701 shutdown_connections 2024-05-17T09:40:36.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3062703701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43dc1030f0 0x7f43dc103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3062703701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 0x7f43dc1130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3062703701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc103a40 0x7f43dc110760 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.676+0000 7f43e26ab700 1 -- 172.21.15.19:0/3062703701 >> 172.21.15.19:0/3062703701 conn(0x7f43dc0fedb0 msgr2=0x7f43dc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:36.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.677+0000 7f43e26ab700 1 -- 172.21.15.19:0/3062703701 shutdown_connections 2024-05-17T09:40:36.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.677+0000 7f43e26ab700 1 -- 172.21.15.19:0/3062703701 wait complete. 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.677+0000 7f43e26ab700 1 Processor -- start 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 -- start start 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 0x7f43dc19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43dc103a40 0x7f43dc19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 0x7f43dc1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc116500 con 0x7f43dc1030f0 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc116380 con 0x7f43dc110ca0 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e26ab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc116680 con 0x7f43dc103a40 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43dbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 0x7f43dc19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43dbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 0x7f43dc19cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52138/0 (socket says 172.21.15.19:52138) 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43dbfff700 1 -- 172.21.15.19:0/3289211449 learned_addr learned my addr 172.21.15.19:0/3289211449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:36.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43d3fff700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43dc103a40 0x7f43dc19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.678+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 0x7f43dc1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e0c48700 1 -- 172.21.15.19:0/3289211449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43dc103a40 msgr2=0x7f43dc19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:36.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43dc103a40 0x7f43dc19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e0c48700 1 -- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 msgr2=0x7f43dc19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 0x7f43dc19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e0c48700 1 -- 172.21.15.19:0/3289211449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc1a1e40 con 0x7f43dc110ca0 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 0x7f43dc19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 0x7f43dc1a17a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43d400efc0 tx=0x7f43d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43d9ffb700 1 -- 172.21.15.19:0/3289211449 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40092e0 con 0x7f43dc110ca0 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43d9ffb700 1 -- 172.21.15.19:0/3289211449 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43d4004bb0 con 0x7f43dc110ca0 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43d9ffb700 1 -- 172.21.15.19:0/3289211449 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4004020 con 0x7f43dc110ca0 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc1a20d0 con 0x7f43dc110ca0 2024-05-17T09:40:36.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.679+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc1a25e0 con 0x7f43dc110ca0 2024-05-17T09:40:36.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.683+0000 7f43d9ffb700 1 -- 172.21.15.19:0/3289211449 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f43d401b030 con 0x7f43dc110ca0 2024-05-17T09:40:36.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.684+0000 7f43d9ffb700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f43c80b2980 0x7f43c80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.684+0000 7f43d9ffb700 1 -- 172.21.15.19:0/3289211449 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f43d40e1ff0 con 0x7f43dc110ca0 2024-05-17T09:40:36.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.684+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f43c80b2980 0x7f43c80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 Processor -- start 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 -- start start 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f43c80b2980 0x7f43c80b4e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009750 tx=0x7f43cc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea5e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea58ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1fe986e20 con 0x55b1fea586b0 2024-05-17T09:40:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b1fe9769c0 con 0x55b1fe8d9fd0 2024-05-17T09:40:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b1fe97aa00 con 0x55b1fe957560 2024-05-17T09:40:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54142/0 (socket says 172.21.15.19:54142) 2024-05-17T09:40:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43e0c48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea58ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.685+0000 7f43d3fff700 1 -- 172.21.15.19:0/4008613847 learned_addr learned my addr 172.21.15.19:0/4008613847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43dbfff700 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea5e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43e0c48700 1 -- 172.21.15.19:0/4008613847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 msgr2=0x55b1fea5e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43e0c48700 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea5e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43e0c48700 1 -- 172.21.15.19:0/4008613847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 msgr2=0x55b1fea5eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43e0c48700 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43e0c48700 1 -- 172.21.15.19:0/4008613847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1fe8c6a60 con 0x55b1fea586b0 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43d3fff700 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.686+0000 7f43e0c48700 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea58ac0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f43d40df6e0 tx=0x7f43d40a8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6007552402b1b327 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.687+0000 7f43d27fc700 1 -- 172.21.15.19:0/4008613847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40a9290 con 0x55b1fea586b0 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.687+0000 7f43d27fc700 1 -- 172.21.15.19:0/4008613847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43d40ad640 con 0x55b1fea586b0 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.687+0000 7f43d27fc700 1 -- 172.21.15.19:0/4008613847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40dfe00 con 0x55b1fea586b0 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 -- 172.21.15.19:0/4008613847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 msgr2=0x55b1fea58ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea58ac0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f43d40df6e0 tx=0x7f43d40a8b20 comp rx=0 tx=0).stop 2024-05-17T09:40:36.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 -- 172.21.15.19:0/4008613847 shutdown_connections 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea5e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea58ac0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/4008613847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 -- 172.21.15.19:0/4008613847 >> 172.21.15.19:0/4008613847 conn(0x55b1fe918e10 msgr2=0x55b1fea535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 -- 172.21.15.19:0/4008613847 shutdown_connections 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 -- 172.21.15.19:0/4008613847 wait complete. 2024-05-17T09:40:36.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.688+0000 7f43f10c0b80 1 -- start start 2024-05-17T09:40:36.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43f10c0b80 1 Processor -- start 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43dbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43dbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea62310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54158/0 (socket says 172.21.15.19:54158) 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.689+0000 7f43dbfff700 1 -- 172.21.15.19:0/3986202292 learned_addr learned my addr 172.21.15.19:0/3986202292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1fe87d130 con 0x55b1fea586b0 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43d3fff700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b1fe918950 con 0x55b1fe8d9fd0 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b1fea6e460 con 0x55b1fe957560 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43f10c0b80 10 client.0 _ll_register_callbacks cb 0x55b1fe911ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43e0c48700 1 -- 172.21.15.19:0/3986202292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 msgr2=0x55b1fea62850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43e0c48700 1 -- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 msgr2=0x55b1fea62310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.690+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea62310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.691+0000 7f43e0c48700 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1fe915820 con 0x55b1fea586b0 2024-05-17T09:40:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.691+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:36.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.691+0000 7f43e0c48700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea62d90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f43d40a90d0 tx=0x7f43d4009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e6187fe9101048 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:36.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.692+0000 7f43bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:36.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.692+0000 7f43bb7fe700 1 -- 172.21.15.19:0/3986202292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40c2070 con 0x55b1fea586b0 2024-05-17T09:40:36.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.692+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d40ad640 con 0x55b1fea586b0 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.692+0000 7f43bb7fe700 1 -- 172.21.15.19:0/3986202292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f43d40dfdb0 con 0x55b1fea586b0 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43bb7fe700 1 -- 172.21.15.19:0/3986202292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f43d40ad3d0 con 0x55b1fea586b0 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 10 client.18582 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b1fea6fdc0 con 0x55b1fea586b0 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43bb7fe700 1 -- 172.21.15.19:0/3986202292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43d40ad640 con 0x55b1fea586b0 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43bb7fe700 1 -- 172.21.15.19:0/3986202292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f43d40a4c00 con 0x55b1fea586b0 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 10 client.18582 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 10 client.18582 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 20 client.18582 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1fea72470 0x55b1fea748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 4 client.18582 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.693+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1fe988d70 con 0x55b1fea72470 2024-05-17T09:40:36.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.694+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1fea72470 0x55b1fea748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.694+0000 7f43bb7fe700 10 client.18582 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.694+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1fea72470 0x55b1fea748b0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75cd8d3c3cb28a12 server_cookie=dc8db0cdc271f845 in_seq=0 out_seq=0 2024-05-17T09:40:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.697+0000 7f43bb7fe700 1 -- 172.21.15.19:0/3986202292 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b1fe988d70 con 0x55b1fea72470 2024-05-17T09:40:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.697+0000 7f43bb7fe700 10 client.18582 handle_command_reply: tid=0 2024-05-17T09:40:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.697+0000 7f43f10c0b80 1 client.18582 shutdown 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 msgr2=0x55b1fea62d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea62d90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f43d40a90d0 tx=0x7f43d4009480 comp rx=0 tx=0).stop 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 shutdown_connections 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1fea72470 0x55b1fea748b0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fea586b0 0x55b1fea62d90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3986202292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 >> 172.21.15.19:0/3986202292 conn(0x55b1fe918e10 msgr2=0x55b1fea4d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43bb7fe700 0 client.18582 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 shutdown_connections 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3986202292 wait complete. 2024-05-17T09:40:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.698+0000 7f43f10c0b80 20 client.18582 trim_cache size 0 max 16384 2024-05-17T09:40:36.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:36 smithi019 ceph-mon[31620]: pgmap v1063: 129 pgs: 129 active+clean; 440 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.0 MiB/s wr, 231 op/s 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.736+0000 7f43f10c0b80 1 Processor -- start 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.736+0000 7f43f10c0b80 1 -- start start 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea5c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b4e20 con 0x55b1fe8d9fd0 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b26b0 con 0x55b1fea586b0 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b0500 con 0x55b1fe957560 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43e0c48700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea5c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43e0c48700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea5c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54170/0 (socket says 172.21.15.19:54170) 2024-05-17T09:40:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43e0c48700 1 -- 172.21.15.19:0/3966746099 learned_addr learned my addr 172.21.15.19:0/3966746099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.737+0000 7f43dbfff700 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.738+0000 7f43d3fff700 1 -- 172.21.15.19:0/3966746099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 msgr2=0x55b1fea5c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.738+0000 7f43d3fff700 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea5c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.738+0000 7f43d3fff700 1 -- 172.21.15.19:0/3966746099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 msgr2=0x55b1fea5da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.738+0000 7f43d3fff700 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.738+0000 7f43d3fff700 1 -- 172.21.15.19:0/3966746099 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1fea60880 con 0x55b1fe957560 2024-05-17T09:40:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.738+0000 7f43d3fff700 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62bd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098c0 tx=0x7f43c4009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb227209586a8a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.739+0000 7f43d1ffb700 1 -- 172.21.15.19:0/3966746099 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4026070 con 0x55b1fe957560 2024-05-17T09:40:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.739+0000 7f43d1ffb700 1 -- 172.21.15.19:0/3966746099 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c401d040 con 0x55b1fe957560 2024-05-17T09:40:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.739+0000 7f43d1ffb700 1 -- 172.21.15.19:0/3966746099 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4017400 con 0x55b1fe957560 2024-05-17T09:40:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.740+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3966746099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 msgr2=0x55b1fea62bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.740+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62bd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098c0 tx=0x7f43c4009590 comp rx=0 tx=0).stop 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3966746099 shutdown_connections 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea5c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea5da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/3966746099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fe957560 0x55b1fea62bd0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098c0 tx=0x7f43c4009590 comp rx=0 tx=0).stop 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3966746099 >> 172.21.15.19:0/3966746099 conn(0x55b1fe918e10 msgr2=0x55b1fea5bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3966746099 shutdown_connections 2024-05-17T09:40:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.741+0000 7f43f10c0b80 1 -- 172.21.15.19:0/3966746099 wait complete. 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 -- start start 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 Processor -- start 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea6a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fea67390 0x55b1fea677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b0b70 con 0x55b1fe8d9fd0 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b1fe0 con 0x55b1fea586b0 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.742+0000 7f43f10c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b1240 con 0x55b1fea67390 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43f10c0b80 10 client.0 _ll_register_callbacks cb 0x55b1fe911ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea6a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43e0c48700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fea67390 0x55b1fea677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43dbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea6a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54174/0 (socket says 172.21.15.19:54174) 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 -- 172.21.15.19:0/333833452 learned_addr learned my addr 172.21.15.19:0/333833452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 -- 172.21.15.19:0/333833452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fea67390 msgr2=0x55b1fea677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fea67390 0x55b1fea677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 -- 172.21.15.19:0/333833452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 msgr2=0x55b1fea6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea6a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 -- 172.21.15.19:0/333833452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1fea6af40 con 0x55b1fea586b0 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43dbfff700 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43d3fff700 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea6a900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c010 tx=0x7f43c400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77212c076920531c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.743+0000 7f43bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:40:36.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43bb7fe700 1 -- 172.21.15.19:0/333833452 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4026070 con 0x55b1fea586b0 2024-05-17T09:40:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43bb7fe700 1 -- 172.21.15.19:0/333833452 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f43c4005450 con 0x55b1fea586b0 2024-05-17T09:40:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c401d040 con 0x55b1fea586b0 2024-05-17T09:40:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43bb7fe700 1 -- 172.21.15.19:0/333833452 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f43c4038050 con 0x55b1fea586b0 2024-05-17T09:40:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43f10c0b80 10 client.28118 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b1feae6080 con 0x55b1fea586b0 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.744+0000 7f43bb7fe700 1 -- 172.21.15.19:0/333833452 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c4027040 con 0x55b1fea586b0 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43bb7fe700 1 -- 172.21.15.19:0/333833452 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f43c4033030 con 0x55b1fea586b0 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43f10c0b80 10 client.28118 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43f10c0b80 10 client.28118 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43f10c0b80 20 client.28118 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1feae66e0 0x55b1feae8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43f10c0b80 4 client.28118 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.3, "await_for": 45.0}] 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.3, "await_for": 45.0}) -- 0x7f43c401d040 con 0x55b1feae66e0 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43dbfff700 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1feae66e0 0x55b1feae8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.745+0000 7f43bb7fe700 10 client.28118 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:36.746+0000 7f43dbfff700 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1feae66e0 0x55b1feae8b00 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8d731636cb3cc7 server_cookie=546467dec62e80a0 in_seq=0 out_seq=0 2024-05-17T09:40:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:36 smithi069 ceph-mon[35081]: pgmap v1063: 129 pgs: 129 active+clean; 440 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.0 MiB/s wr, 231 op/s 2024-05-17T09:40:37.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:36 smithi079 ceph-mon[35502]: pgmap v1063: 129 pgs: 129 active+clean; 440 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.0 MiB/s wr, 231 op/s 2024-05-17T09:40:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.149+0000 7f43bb7fe700 1 -- 172.21.15.19:0/333833452 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f43c401d040 con 0x55b1feae66e0 2024-05-17T09:40:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.149+0000 7f43bb7fe700 10 client.28118 handle_command_reply: tid=0 2024-05-17T09:40:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 client.28118 shutdown 2024-05-17T09:40:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 msgr2=0x55b1fea6a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea6a900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c010 tx=0x7f43c400f790 comp rx=0 tx=0).stop 2024-05-17T09:40:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 shutdown_connections 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1fea67390 0x55b1fea677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b1feae66e0 0x55b1feae8b00 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1fe8d9fd0 0x55b1fea6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 --2- 172.21.15.19:0/333833452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1fea586b0 0x55b1fea6a900 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 >> 172.21.15.19:0/333833452 conn(0x55b1fe918e10 msgr2=0x55b1fea4daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43bb7fe700 0 client.28118 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 shutdown_connections 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 1 -- 172.21.15.19:0/333833452 wait complete. 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.150+0000 7f43f10c0b80 20 client.28118 trim_cache size 0 max 16384 2024-05-17T09:40:37.151 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 70, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "6308053e": { 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 70, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:40:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 131.3, 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:37.153 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f43c80b2980 msgr2=0x7f43c80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f43c80b2980 0x7f43c80b4e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009750 tx=0x7f43cc009310 comp rx=0 tx=0).stop 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 msgr2=0x7f43dc1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 0x7f43dc1a17a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43d400efc0 tx=0x7f43d400c3a0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 shutdown_connections 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43dc110ca0 0x7f43dc1a17a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43dc1030f0 0x7f43dc19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f43c80b2980 0x7f43c80b4e90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 --2- 172.21.15.19:0/3289211449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f43dc103a40 0x7f43dc19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 >> 172.21.15.19:0/3289211449 conn(0x7f43dc0fedb0 msgr2=0x7f43dc101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 shutdown_connections 2024-05-17T09:40:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:37.151+0000 7f43e26ab700 1 -- 172.21.15.19:0/3289211449 wait complete. 2024-05-17T09:40:37.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48810 6.71 MB/sec execute 274 sec latency 853.842 ms 2024-05-17T09:40:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 -- 172.21.15.19:0/4121950269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038103d60 msgr2=0x7f5038104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 --2- 172.21.15.19:0/4121950269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038103d60 0x7f5038104170 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f502c009900 tx=0x7f502c02a410 comp rx=0 tx=0).stop 2024-05-17T09:40:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 -- 172.21.15.19:0/4121950269 shutdown_connections 2024-05-17T09:40:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 --2- 172.21.15.19:0/4121950269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038108a60 0x7f503810ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 --2- 172.21.15.19:0/4121950269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038103d60 0x7f5038104170 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 --2- 172.21.15.19:0/4121950269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038102b20 0x7f5038103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 -- 172.21.15.19:0/4121950269 >> 172.21.15.19:0/4121950269 conn(0x7f50380fdd10 msgr2=0x7f50381001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.805+0000 7f503dd76640 1 -- 172.21.15.19:0/4121950269 shutdown_connections 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 -- 172.21.15.19:0/4121950269 wait complete. 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 Processor -- start 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 -- start start 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038102b20 0x7f5038199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038103d60 0x7f5038199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 0x7f50381a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f503810e4f0 con 0x7f5038108a60 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f503810e370 con 0x7f5038102b20 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.806+0000 7f503dd76640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f503810e670 con 0x7f5038103d60 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 0x7f50381a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f50377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038102b20 0x7f5038199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f50377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038102b20 0x7f5038199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:54194/0 (socket says 172.21.15.19:54194) 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f50377fe640 1 -- 172.21.15.19:0/1601097346 learned_addr learned my addr 172.21.15.19:0/1601097346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 0x7f50381a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52194/0 (socket says 172.21.15.19:52194) 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038103d60 0x7f5038199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 -- 172.21.15.19:0/1601097346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038103d60 msgr2=0x7f5038199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038103d60 0x7f5038199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 -- 172.21.15.19:0/1601097346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038102b20 msgr2=0x7f5038199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038102b20 0x7f5038199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 -- 172.21.15.19:0/1601097346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50381a0c40 con 0x7f5038108a60 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038103d60 0x7f5038199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5037fff640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 0x7f50381a05d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f503000da10 tx=0x7f503000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5034ff9640 1 -- 172.21.15.19:0/1601097346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503000bd70 con 0x7f5038108a60 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5034ff9640 1 -- 172.21.15.19:0/1601097346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5030003e00 con 0x7f5038108a60 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f5034ff9640 1 -- 172.21.15.19:0/1601097346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300106c0 con 0x7f5038108a60 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50381a0f30 con 0x7f5038108a60 2024-05-17T09:40:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.807+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a1390 con 0x7f5038108a60 2024-05-17T09:40:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.810+0000 7f5044c2c740 1 Processor -- start 2024-05-17T09:40:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5034ff9640 1 -- 172.21.15.19:0/1601097346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5030003970 con 0x7f5038108a60 2024-05-17T09:40:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 -- start start 2024-05-17T09:40:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 0x5629b8d0c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d0cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d13a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629b8c7cc30 con 0x5629b89691b0 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5629b8c6c7d0 con 0x5629b89402f0 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5044c2c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5629b8c70810 con 0x5629b89484d0 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 0x5629b8d0c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 0x5629b8d0c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52206/0 (socket says 172.21.15.19:52206) 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5036ffd640 1 -- 172.21.15.19:0/1594959559 learned_addr learned my addr 172.21.15.19:0/1594959559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f50377fe640 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d0cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.811+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d13a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5036ffd640 1 -- 172.21.15.19:0/1594959559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 msgr2=0x5629b8d13a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d13a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5036ffd640 1 -- 172.21.15.19:0/1594959559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 msgr2=0x5629b8d0cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d0cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5036ffd640 1 -- 172.21.15.19:0/1594959559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b8981210 con 0x5629b89691b0 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f50377fe640 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d0cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5034ff9640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50200b2920 0x7f50200b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 0x5629b8d0c5b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f502c000c00 tx=0x7f502c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbd34cd7eb9e1e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f50377fe640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50200b2920 0x7f50200b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f5034ff9640 1 -- 172.21.15.19:0/1601097346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f50300e29d0 con 0x7f5038108a60 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f501dffb640 1 -- 172.21.15.19:0/1594959559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c035070 con 0x5629b89691b0 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f501dffb640 1 -- 172.21.15.19:0/1594959559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f502c045070 con 0x5629b89691b0 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.812+0000 7f501dffb640 1 -- 172.21.15.19:0/1594959559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c003ca0 con 0x5629b89691b0 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 -- 172.21.15.19:0/1594959559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 msgr2=0x5629b8d0c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 0x5629b8d0c5b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f502c000c00 tx=0x7f502c009560 comp rx=0 tx=0).stop 2024-05-17T09:40:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f50377fe640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50200b2920 0x7f50200b4e30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5629b8d0d990 tx=0x7f502800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 -- 172.21.15.19:0/1594959559 shutdown_connections 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d13a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d0cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1594959559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89691b0 0x5629b8d0c5b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 -- 172.21.15.19:0/1594959559 >> 172.21.15.19:0/1594959559 conn(0x5629b895d300 msgr2=0x5629b8d05d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 -- 172.21.15.19:0/1594959559 shutdown_connections 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 -- 172.21.15.19:0/1594959559 wait complete. 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 -- start start 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.813+0000 7f5044c2c740 1 Processor -- start 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 0x5629b8d15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d16430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89691b0 0x5629b8d1d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629b8952790 con 0x5629b89402f0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5629b8c28890 con 0x5629b89484d0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5629b8d14420 con 0x5629b89691b0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 10 client.0 _ll_register_callbacks cb 0x5629b8c2bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 0x5629b8d15ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 0x5629b8d15ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52212/0 (socket says 172.21.15.19:52212) 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 -- 172.21.15.19:0/2431302123 learned_addr learned my addr 172.21.15.19:0/2431302123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5036ffd640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d16430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5037fff640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89691b0 0x5629b8d1d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 -- 172.21.15.19:0/2431302123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89691b0 msgr2=0x5629b8d1d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89691b0 0x5629b8d1d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 -- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 msgr2=0x5629b8d16430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d16430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 -- 172.21.15.19:0/2431302123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b8a4d620 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5037fff640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89691b0 0x5629b8d1d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f50377fe640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 0x5629b8d15ef0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5028010890 tx=0x7f50280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8381f8cb861b409 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5016ffd640 1 -- 172.21.15.19:0/2431302123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028023070 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5016ffd640 1 -- 172.21.15.19:0/2431302123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f502800fbd0 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.814+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c045070 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5016ffd640 1 -- 172.21.15.19:0/2431302123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f502802f050 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 10 client.18600 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629b8d20390 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5016ffd640 1 -- 172.21.15.19:0/2431302123 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f502c045070 con 0x5629b89402f0 2024-05-17T09:40:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5016ffd640 1 -- 172.21.15.19:0/2431302123 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f50280273b0 con 0x5629b89402f0 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 10 client.18600 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 10 client.18600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 20 client.18600 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b8927d80 0x5629b8d23b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 4 client.18600 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b8c7eb80 con 0x5629b8927d80 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.815+0000 7f5036ffd640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b8927d80 0x5629b8d23b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.816+0000 7f5016ffd640 10 client.18600 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.816+0000 7f5036ffd640 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b8927d80 0x5629b8d23b50 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb8e447fe45432a server_cookie=a082a11fbe386370 in_seq=0 out_seq=0 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.816+0000 7f5016ffd640 1 -- 172.21.15.19:0/2431302123 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629b8c7eb80 con 0x5629b8927d80 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.816+0000 7f5016ffd640 10 client.18600 handle_command_reply: tid=0 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.816+0000 7f5044c2c740 1 client.18600 shutdown 2024-05-17T09:40:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 msgr2=0x5629b8d15ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 0x5629b8d15ef0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5028010890 tx=0x7f50280046c0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 shutdown_connections 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b8927d80 0x5629b8d23b50 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89691b0 0x5629b8d1d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d16430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5016ffd640 0 client.18600 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 --2- 172.21.15.19:0/2431302123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b89402f0 0x5629b8d15ef0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 >> 172.21.15.19:0/2431302123 conn(0x5629b895d300 msgr2=0x5629b8d08c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 shutdown_connections 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 1 -- 172.21.15.19:0/2431302123 wait complete. 2024-05-17T09:40:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.817+0000 7f5044c2c740 20 client.18600 trim_cache size 0 max 16384 2024-05-17T09:40:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.833+0000 7f5044c2c740 1 Processor -- start 2024-05-17T09:40:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 -- start start 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89402f0 0x5629b8d10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ad080 con 0x5629b8927d80 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aa950 con 0x5629b89484d0 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5044c2c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a87a0 con 0x5629b89402f0 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d10520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52214/0 (socket says 172.21.15.19:52214) 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 -- 172.21.15.19:0/4253009805 learned_addr learned my addr 172.21.15.19:0/4253009805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5037fff640 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f5036ffd640 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89402f0 0x5629b8d10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 -- 172.21.15.19:0/4253009805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89402f0 msgr2=0x5629b8d10a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89402f0 0x5629b8d10a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 -- 172.21.15.19:0/4253009805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 msgr2=0x5629b8d1d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.834+0000 7f50377fe640 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d1d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.835+0000 7f50377fe640 1 -- 172.21.15.19:0/4253009805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b8d0d310 con 0x5629b8927d80 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.835+0000 7f50377fe640 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d10520 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f5028027750 tx=0x7f5028005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7ed9245bcbe3b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.835+0000 7f501d7fa640 1 -- 172.21.15.19:0/4253009805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028037020 con 0x5629b8927d80 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.835+0000 7f501d7fa640 1 -- 172.21.15.19:0/4253009805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5028012d00 con 0x5629b8927d80 2024-05-17T09:40:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.835+0000 7f501d7fa640 1 -- 172.21.15.19:0/4253009805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028023040 con 0x5629b8927d80 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 -- 172.21.15.19:0/4253009805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 msgr2=0x5629b8d10520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d10520 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f5028027750 tx=0x7f5028005740 comp rx=0 tx=0).stop 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 -- 172.21.15.19:0/4253009805 shutdown_connections 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89484d0 0x5629b8d1d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89402f0 0x5629b8d10a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 --2- 172.21.15.19:0/4253009805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d10520 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 -- 172.21.15.19:0/4253009805 >> 172.21.15.19:0/4253009805 conn(0x5629b895d300 msgr2=0x5629b8d152c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 -- 172.21.15.19:0/4253009805 shutdown_connections 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 -- 172.21.15.19:0/4253009805 wait complete. 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.836+0000 7f5044c2c740 1 -- start start 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 Processor -- start 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d1cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d8a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a8e10 con 0x5629b8927d80 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aa280 con 0x5629b89402f0 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a94e0 con 0x5629b89484d0 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5044c2c740 10 client.0 _ll_register_callbacks cb 0x7f5028010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d1cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d1cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52226/0 (socket says 172.21.15.19:52226) 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f50377fe640 1 -- 172.21.15.19:0/1490357731 learned_addr learned my addr 172.21.15.19:0/1490357731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.837+0000 7f5037fff640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d8a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f50377fe640 1 -- 172.21.15.19:0/1490357731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 msgr2=0x5629b8d8a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f50377fe640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d8a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f50377fe640 1 -- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 msgr2=0x5629b8d1d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f50377fe640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d1d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f50377fe640 1 -- 172.21.15.19:0/1490357731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b8d8adc0 con 0x5629b8927d80 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d1d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f50377fe640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d1cf00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5028005d00 tx=0x7f50280275c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10f8003dd007417 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5016ffd640 1 -- 172.21.15.19:0/1490357731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028041060 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5016ffd640 1 -- 172.21.15.19:0/1490357731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f502800fbd0 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5028012d00 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5016ffd640 1 -- 172.21.15.19:0/1490357731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5028048050 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5044c2c740 10 client.18606 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629b8d8c050 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.838+0000 7f5016ffd640 1 -- 172.21.15.19:0/1490357731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5028012d00 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5016ffd640 1 -- 172.21.15.19:0/1490357731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5028028440 con 0x5629b8927d80 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5044c2c740 10 client.18606 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5044c2c740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5044c2c740 20 client.18606 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b89691b0 0x5629b8d8e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5044c2c740 4 client.18606 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:40:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629b8c78360 con 0x5629b89691b0 2024-05-17T09:40:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b89691b0 0x5629b8d8e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5016ffd640 10 client.18606 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.839+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b89691b0 0x5629b8d8e930 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613af3cac5ccc403 server_cookie=e5d83eb4989c3b93 in_seq=0 out_seq=0 2024-05-17T09:40:37.854 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6308053e', quiesce took 12.1 seconds. Will release after: -0.7999999999999989 2024-05-17T09:40:37.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:40:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5016ffd640 1 -- 172.21.15.19:0/1490357731 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629b8c78360 con 0x5629b89691b0 2024-05-17T09:40:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5016ffd640 10 client.18606 handle_command_reply: tid=0 2024-05-17T09:40:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 client.18606 shutdown 2024-05-17T09:40:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 msgr2=0x5629b8d1cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d1cf00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5028005d00 tx=0x7f50280275c0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 shutdown_connections 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5629b89691b0 0x5629b8d8e930 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b89484d0 0x5629b8d8a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5016ffd640 0 client.18606 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5629b89402f0 0x5629b8d1d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 --2- 172.21.15.19:0/1490357731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629b8927d80 0x5629b8d1cf00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 >> 172.21.15.19:0/1490357731 conn(0x5629b895d300 msgr2=0x5629b8d152c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 shutdown_connections 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.925+0000 7f5044c2c740 1 -- 172.21.15.19:0/1490357731 wait complete. 2024-05-17T09:40:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f5044c2c740 20 client.18606 trim_cache size 0 max 16384 2024-05-17T09:40:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:40:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:40:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:40:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50200b2920 msgr2=0x7f50200b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50200b2920 0x7f50200b4e30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5629b8d0d990 tx=0x7f502800b410 comp rx=0 tx=0).stop 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 msgr2=0x7f50381a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 0x7f50381a05d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f503000da10 tx=0x7f503000ded0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 shutdown_connections 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f50200b2920 0x7f50200b4e30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108a60 0x7f50381a05d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5038103d60 0x7f5038199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 --2- 172.21.15.19:0/1601097346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038102b20 0x7f5038199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 >> 172.21.15.19:0/1601097346 conn(0x7f50380fdd10 msgr2=0x7f5038104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 shutdown_connections 2024-05-17T09:40:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:37.926+0000 7f503dd76640 1 -- 172.21.15.19:0/1601097346 wait complete. 2024-05-17T09:40:38.188 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:40:38.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48810 6.69 MB/sec execute 275 sec latency 1853.906 ms 2024-05-17T09:40:38.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:38 smithi019 ceph-mon[31620]: pgmap v1064: 129 pgs: 129 active+clean; 437 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-05-17T09:40:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:38 smithi069 ceph-mon[35081]: pgmap v1064: 129 pgs: 129 active+clean; 437 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-05-17T09:40:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:38 smithi079 ceph-mon[35502]: pgmap v1064: 129 pgs: 129 active+clean; 437 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-05-17T09:40:39.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48810 6.66 MB/sec execute 276 sec latency 2853.952 ms 2024-05-17T09:40:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 -- 172.21.15.19:0/3851952101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 msgr2=0x7fce3c105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 --2- 172.21.15.19:0/3851952101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c105880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009990 tx=0x7fce2c009ca0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 -- 172.21.15.19:0/3851952101 shutdown_connections 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 --2- 172.21.15.19:0/3851952101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce3c10e8e0 0x7fce3c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 --2- 172.21.15.19:0/3851952101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c105880 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 --2- 172.21.15.19:0/3851952101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3c105dc0 0x7fce3c10e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 -- 172.21.15.19:0/3851952101 >> 172.21.15.19:0/3851952101 conn(0x7fce3c0fef20 msgr2=0x7fce3c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 -- 172.21.15.19:0/3851952101 shutdown_connections 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 -- 172.21.15.19:0/3851952101 wait complete. 2024-05-17T09:40:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 Processor -- start 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.032+0000 7fce42971700 1 -- start start 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce42971700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce42971700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce3c105dc0 0x7fce3c1036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce42971700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3c10e8e0 0x7fce3c103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce42971700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1142e0 con 0x7fce3c105470 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce42971700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c114160 con 0x7fce3c10e8e0 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce42971700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c114460 con 0x7fce3c105dc0 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52248/0 (socket says 172.21.15.19:52248) 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 -- 172.21.15.19:0/2719261955 learned_addr learned my addr 172.21.15.19:0/2719261955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3b7fe700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce3c105dc0 0x7fce3c1036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce40f0e700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3c10e8e0 0x7fce3c103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 -- 172.21.15.19:0/2719261955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce3c105dc0 msgr2=0x7fce3c1036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce3c105dc0 0x7fce3c1036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 -- 172.21.15.19:0/2719261955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3c10e8e0 msgr2=0x7fce3c103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3c10e8e0 0x7fce3c103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 -- 172.21.15.19:0/2719261955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c1a90c0 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.033+0000 7fce3bfff700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c103160 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fce2c000c00 tx=0x7fce2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.034+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c027070 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.034+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce2c00fb80 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.034+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c1a9350 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.034+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c022d00 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.034+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c1a97a0 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.035+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c0512b0 con 0x7fce3c105470 2024-05-17T09:40:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.036+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce2c02a070 con 0x7fce3c105470 2024-05-17T09:40:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.037+0000 7fce397fa700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fce240b2880 0x7fce240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.037+0000 7fce3b7fe700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fce240b2880 0x7fce240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.037+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fce2c0e38b0 con 0x7fce3c105470 2024-05-17T09:40:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.038+0000 7fce3b7fe700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fce240b2880 0x7fce240b4d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce3c1111e0 tx=0x7fce3000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.041+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce2c031080 con 0x7fce3c105470 2024-05-17T09:40:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.309+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce3c02e5e0 con 0x7fce3c105470 2024-05-17T09:40:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.310+0000 7fce397fa700 1 -- 172.21.15.19:0/2719261955 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fce2c013070 con 0x7fce3c105470 2024-05-17T09:40:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:40:40.310 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fce240b2880 msgr2=0x7fce240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fce240b2880 0x7fce240b4d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce3c1111e0 tx=0x7fce3000a300 comp rx=0 tx=0).stop 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 msgr2=0x7fce3c103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c103160 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fce2c000c00 tx=0x7fce2c0047b0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 shutdown_connections 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3c10e8e0 0x7fce3c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce3c105470 0x7fce3c103160 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fce240b2880 0x7fce240b4d90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 --2- 172.21.15.19:0/2719261955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce3c105dc0 0x7fce3c1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 >> 172.21.15.19:0/2719261955 conn(0x7fce3c0fef20 msgr2=0x7fce3c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 shutdown_connections 2024-05-17T09:40:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:40.313+0000 7fce42971700 1 -- 172.21.15.19:0/2719261955 wait complete. 2024-05-17T09:40:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:40:40.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48810 6.64 MB/sec execute 277 sec latency 3854.046 ms 2024-05-17T09:40:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:40 smithi019 ceph-mon[31620]: pgmap v1065: 129 pgs: 129 active+clean; 435 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-05-17T09:40:40.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:40 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2719261955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:40:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:40 smithi069 ceph-mon[35081]: pgmap v1065: 129 pgs: 129 active+clean; 435 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-05-17T09:40:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:40 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2719261955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:40:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:40 smithi079 ceph-mon[35502]: pgmap v1065: 129 pgs: 129 active+clean; 435 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-05-17T09:40:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:40 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2719261955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:40:41.046 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 6308053e --release --await' 2024-05-17T09:40:41.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 6308053e --release --await 2024-05-17T09:40:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:40:41.585 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48810 6.62 MB/sec execute 278 sec latency 4854.091 ms 2024-05-17T09:40:42.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48810 6.59 MB/sec execute 279 sec latency 5854.174 ms 2024-05-17T09:40:42.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:42 smithi019 ceph-mon[31620]: pgmap v1066: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-05-17T09:40:42.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:42 smithi069 ceph-mon[35081]: pgmap v1066: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-05-17T09:40:43.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:42 smithi079 ceph-mon[35502]: pgmap v1066: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-05-17T09:40:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 -- 172.21.15.19:0/4086127392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c108750 msgr2=0x7f075c1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 --2- 172.21.15.19:0/4086127392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c108750 0x7f075c1100b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f075400b0d0 tx=0x7f075400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 -- 172.21.15.19:0/4086127392 shutdown_connections 2024-05-17T09:40:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 --2- 172.21.15.19:0/4086127392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c1034d0 0x7f075c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 --2- 172.21.15.19:0/4086127392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c108750 0x7f075c1100b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 --2- 172.21.15.19:0/4086127392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f075c107e00 0x7f075c108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 -- 172.21.15.19:0/4086127392 >> 172.21.15.19:0/4086127392 conn(0x7f075c0fef40 msgr2=0x7f075c101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 -- 172.21.15.19:0/4086127392 shutdown_connections 2024-05-17T09:40:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.198+0000 7f0764645700 1 -- 172.21.15.19:0/4086127392 wait complete. 2024-05-17T09:40:43.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 Processor -- start 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 -- start start 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 0x7f075c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c107e00 0x7f075c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f075c108750 0x7f075c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f075c112110 con 0x7f075c1034d0 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f075c111f90 con 0x7f075c108750 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0764645700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f075c112290 con 0x7f075c107e00 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f07623e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 0x7f075c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f07623e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 0x7f075c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52258/0 (socket says 172.21.15.19:52258) 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f07623e1700 1 -- 172.21.15.19:0/2685325859 learned_addr learned my addr 172.21.15.19:0/2685325859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0761be0700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c107e00 0x7f075c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.199+0000 7f0762be2700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f075c108750 0x7f075c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07623e1700 1 -- 172.21.15.19:0/2685325859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c107e00 msgr2=0x7f075c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07623e1700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c107e00 0x7f075c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07623e1700 1 -- 172.21.15.19:0/2685325859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f075c108750 msgr2=0x7f075c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07623e1700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f075c108750 0x7f075c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07623e1700 1 -- 172.21.15.19:0/2685325859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a1d80 con 0x7f075c1034d0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f0761be0700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c107e00 0x7f075c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07623e1700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 0x7f075c19ceb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f075800d950 tx=0x7f075800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07537fe700 1 -- 172.21.15.19:0/2685325859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758016070 con 0x7f075c1034d0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a2070 con 0x7f075c1034d0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07537fe700 1 -- 172.21.15.19:0/2685325859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0758009dd0 con 0x7f075c1034d0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f07537fe700 1 -- 172.21.15.19:0/2685325859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075800f3c0 con 0x7f075c1034d0 2024-05-17T09:40:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.200+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a9990 con 0x7f075c1034d0 2024-05-17T09:40:43.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.203+0000 7f07537fe700 1 -- 172.21.15.19:0/2685325859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0758019070 con 0x7f075c1034d0 2024-05-17T09:40:43.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.203+0000 7f07537fe700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f07480b2880 0x7f07480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f0761be0700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f07480b2880 0x7f07480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f07537fe700 1 -- 172.21.15.19:0/2685325859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f07580e2250 con 0x7f075c1034d0 2024-05-17T09:40:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f077305ab80 1 Processor -- start 2024-05-17T09:40:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f0761be0700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f07480b2880 0x7f07480b4d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f075c1026c0 tx=0x7f074c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f077305ab80 1 -- start start 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f077305ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.204+0000 7f077305ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f077305ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f077305ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0fea72d90 con 0x55d0fe9c5f40 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f077305ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0fea62930 con 0x55d0feb4ce30 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f077305ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d0fea66970 con 0x55d0fea434d0 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52268/0 (socket says 172.21.15.19:52268) 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 -- 172.21.15.19:0/3211867082 learned_addr learned my addr 172.21.15.19:0/3211867082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f07623e1700 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0762be2700 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 -- 172.21.15.19:0/3211867082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 msgr2=0x55d0feb52d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb52d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 -- 172.21.15.19:0/3211867082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 msgr2=0x55d0feb4d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb4d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 -- 172.21.15.19:0/3211867082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fe9b29d0 con 0x55d0fe9c5f40 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0762be2700 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.205+0000 7f0761be0700 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb532b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f074c010e90 tx=0x7f074c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e751b818aed77ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.206+0000 7f0750ff9700 1 -- 172.21.15.19:0/3211867082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c01e620 con 0x55d0fe9c5f40 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.206+0000 7f0750ff9700 1 -- 172.21.15.19:0/3211867082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f074c01ec50 con 0x55d0fe9c5f40 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.206+0000 7f0750ff9700 1 -- 172.21.15.19:0/3211867082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c01ca70 con 0x55d0fe9c5f40 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.206+0000 7f077305ab80 1 -- 172.21.15.19:0/3211867082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 msgr2=0x55d0feb532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.206+0000 7f077305ab80 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb532b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f074c010e90 tx=0x7f074c013000 comp rx=0 tx=0).stop 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 -- 172.21.15.19:0/3211867082 shutdown_connections 2024-05-17T09:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb52d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 --2- 172.21.15.19:0/3211867082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb532b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 -- 172.21.15.19:0/3211867082 >> 172.21.15.19:0/3211867082 conn(0x55d0fea04d80 msgr2=0x55d0feb3f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 -- 172.21.15.19:0/3211867082 shutdown_connections 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 -- 172.21.15.19:0/3211867082 wait complete. 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 -- start start 2024-05-17T09:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.207+0000 7f077305ab80 1 Processor -- start 2024-05-17T09:40:43.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb56f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe9690a0 con 0x55d0fe9c5f40 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0fea048c0 con 0x55d0feb4ce30 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d0feb62be0 con 0x55d0fea434d0 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f077305ab80 10 client.0 _ll_register_callbacks cb 0x55d0fe9fda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb56a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52276/0 (socket says 172.21.15.19:52276) 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 -- 172.21.15.19:0/3787638676 learned_addr learned my addr 172.21.15.19:0/3787638676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f0761be0700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb56f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f0762be2700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb57490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 -- 172.21.15.19:0/3787638676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 msgr2=0x55d0feb56f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb56f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 -- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 msgr2=0x55d0feb57490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb57490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 -- 172.21.15.19:0/3787638676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fea01790 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f0762be2700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.208+0000 7f07623e1700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb56a10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f07580a5540 tx=0x7f07580a9bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c86de5262fa1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f0745ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f0745ffb700 1 -- 172.21.15.19:0/3787638676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07580dfe60 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f0745ffb700 1 -- 172.21.15.19:0/3787638676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f07580a9720 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c01ec50 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f0745ffb700 1 -- 172.21.15.19:0/3787638676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0758020040 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 10 client.18624 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d0feb64550 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f0745ffb700 1 -- 172.21.15.19:0/3787638676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f074c01ec50 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f0745ffb700 1 -- 172.21.15.19:0/3787638676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f07580b1e90 con 0x55d0fe9c5f40 2024-05-17T09:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 10 client.18624 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 10 client.18624 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 20 client.18624 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0feb66be0 0x55d0feb69000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 4 client.18624 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.209+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fea74ce0 con 0x55d0feb66be0 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.210+0000 7f0761be0700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0feb66be0 0x55d0feb69000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.210+0000 7f0761be0700 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0feb66be0 0x55d0feb69000 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fba6e949291ee5 server_cookie=2314243ff25fa022 in_seq=0 out_seq=0 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.210+0000 7f0745ffb700 10 client.18624 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f0745ffb700 1 -- 172.21.15.19:0/3787638676 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d0fea74ce0 con 0x55d0feb66be0 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f0745ffb700 10 client.18624 handle_command_reply: tid=0 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 client.18624 shutdown 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 msgr2=0x55d0feb56a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb56a10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f07580a5540 tx=0x7f07580a9bc0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 shutdown_connections 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0feb66be0 0x55d0feb69000 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb56a10 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f0745ffb700 0 client.18624 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 --2- 172.21.15.19:0/3787638676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb56f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.211+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 >> 172.21.15.19:0/3787638676 conn(0x55d0fea04d80 msgr2=0x55d0feb39280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.212+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 shutdown_connections 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.212+0000 7f077305ab80 1 -- 172.21.15.19:0/3787638676 wait complete. 2024-05-17T09:40:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.212+0000 7f077305ab80 20 client.18624 trim_cache size 0 max 16384 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 Processor -- start 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 -- start start 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fe9c5f40 0x55d0feb572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb6e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 0x55d0feb6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1b09b0 con 0x55d0feb4ce30 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ae240 con 0x55d0fe9c5f40 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.232+0000 7f077305ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ac090 con 0x55d0fea434d0 2024-05-17T09:40:43.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 0x55d0feb6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 0x55d0feb6ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52292/0 (socket says 172.21.15.19:52292) 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 -- 172.21.15.19:0/3872491197 learned_addr learned my addr 172.21.15.19:0/3872491197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f07623e1700 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fe9c5f40 0x55d0feb572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0761be0700 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb6e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 -- 172.21.15.19:0/3872491197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 msgr2=0x55d0feb6e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb6e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 -- 172.21.15.19:0/3872491197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fe9c5f40 msgr2=0x55d0feb572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fe9c5f40 0x55d0feb572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 -- 172.21.15.19:0/3872491197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0feb54fa0 con 0x55d0feb4ce30 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0762be2700 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 0x55d0feb6ecd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0754009390 tx=0x7f075400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6ab044650b5542 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.233+0000 7f0747fff700 1 -- 172.21.15.19:0/3872491197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075400baa0 con 0x55d0feb4ce30 2024-05-17T09:40:43.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f0747fff700 1 -- 172.21.15.19:0/3872491197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f075400bc40 con 0x55d0feb4ce30 2024-05-17T09:40:43.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f0747fff700 1 -- 172.21.15.19:0/3872491197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754007c20 con 0x55d0feb4ce30 2024-05-17T09:40:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 -- 172.21.15.19:0/3872491197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 msgr2=0x55d0feb6ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 0x55d0feb6ecd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0754009390 tx=0x7f075400b040 comp rx=0 tx=0).stop 2024-05-17T09:40:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 -- 172.21.15.19:0/3872491197 shutdown_connections 2024-05-17T09:40:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0feb4ce30 0x55d0feb6ecd0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0fe9c5f40 0x55d0feb572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 --2- 172.21.15.19:0/3872491197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb6e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 -- 172.21.15.19:0/3872491197 >> 172.21.15.19:0/3872491197 conn(0x55d0fea04d80 msgr2=0x55d0feb50cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 -- 172.21.15.19:0/3872491197 shutdown_connections 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.234+0000 7f077305ab80 1 -- 172.21.15.19:0/3872491197 wait complete. 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 -- start start 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 Processor -- start 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb5f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb5bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ac700 con 0x55d0fe9c5f40 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1adb70 con 0x55d0feb4ce30 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1acdd0 con 0x55d0fea434d0 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f077305ab80 10 client.0 _ll_register_callbacks cb 0x55d0fe9fda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f07623e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb5f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f07623e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb5f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52298/0 (socket says 172.21.15.19:52298) 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.235+0000 7f07623e1700 1 -- 172.21.15.19:0/681563285 learned_addr learned my addr 172.21.15.19:0/681563285 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0762be2700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0761be0700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb5bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f07623e1700 1 -- 172.21.15.19:0/681563285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 msgr2=0x55d0feb5bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f07623e1700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb5bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f07623e1700 1 -- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 msgr2=0x55d0feb5c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f07623e1700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb5c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f07623e1700 1 -- 172.21.15.19:0/681563285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0feb5c6f0 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0762be2700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb5c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f07623e1700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb5f190 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f07580df9f0 tx=0x7f07580e2450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71210b859c746160 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0745ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0745ffb700 1 -- 172.21.15.19:0/681563285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07580dfe60 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0745ffb700 1 -- 172.21.15.19:0/681563285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0758009940 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f075400bc40 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0745ffb700 1 -- 172.21.15.19:0/681563285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f07580a9290 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f077305ab80 10 client.18630 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d0febd24c0 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.236+0000 7f0745ffb700 1 -- 172.21.15.19:0/681563285 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f075400bc40 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f0745ffb700 1 -- 172.21.15.19:0/681563285 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f07580ca300 con 0x55d0fe9c5f40 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f077305ab80 10 client.18630 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:43.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f077305ab80 10 client.18630 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f077305ab80 20 client.18630 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f077305ab80 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0febd2bf0 0x55d0febd5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f077305ab80 4 client.18630 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6308053e", "release": true, "await": true}] 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6308053e", "release": true, "await": true}) -- 0x55d0fea6e4c0 con 0x55d0febd2bf0 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f0761be0700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0febd2bf0 0x55d0febd5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f0761be0700 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0febd2bf0 0x55d0febd5010 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f7d117bda2132d server_cookie=f90b01ff9f386598 in_seq=0 out_seq=0 2024-05-17T09:40:43.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.237+0000 7f0745ffb700 10 client.18630 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.263+0000 7f0745ffb700 1 -- 172.21.15.19:0/681563285 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d0fea6e4c0 con 0x55d0febd2bf0 2024-05-17T09:40:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.263+0000 7f0745ffb700 10 client.18630 handle_command_reply: tid=0 2024-05-17T09:40:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.263+0000 7f077305ab80 1 client.18630 shutdown 2024-05-17T09:40:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.263+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 msgr2=0x55d0feb5f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.263+0000 7f077305ab80 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb5f190 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f07580df9f0 tx=0x7f07580e2450 comp rx=0 tx=0).stop 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 shutdown_connections 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55d0febd2bf0 0x55d0febd5010 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0feb4ce30 0x55d0feb5c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0fe9c5f40 0x55d0feb5f190 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 --2- 172.21.15.19:0/681563285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0fea434d0 0x55d0feb5bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 >> 172.21.15.19:0/681563285 conn(0x55d0fea04d80 msgr2=0x55d0feb4be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f0745ffb700 0 client.18630 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 shutdown_connections 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 1 -- 172.21.15.19:0/681563285 wait complete. 2024-05-17T09:40:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f077305ab80 20 client.18630 trim_cache size 0 max 16384 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 72, 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "6308053e": { 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 72, 2024-05-17T09:40:43.266 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 131.3, 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:40:43.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:40:43.268 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f07480b2880 msgr2=0x7f07480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f0764645700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f07480b2880 0x7f07480b4d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f075c1026c0 tx=0x7f074c009260 comp rx=0 tx=0).stop 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 msgr2=0x7f075c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.264+0000 7f0764645700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 0x7f075c19ceb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f075800d950 tx=0x7f075800dc60 comp rx=0 tx=0).stop 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 shutdown_connections 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f075c108750 0x7f075c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f075c1034d0 0x7f075c19ceb0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f07480b2880 0x7f07480b4d90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 --2- 172.21.15.19:0/2685325859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c107e00 0x7f075c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 >> 172.21.15.19:0/2685325859 conn(0x7f075c0fef40 msgr2=0x7f075c105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 shutdown_connections 2024-05-17T09:40:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:40:43.265+0000 7f0764645700 1 -- 172.21.15.19:0/2685325859 wait complete. 2024-05-17T09:40:43.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 48856 6.57 MB/sec execute 280 sec latency 6530.356 ms 2024-05-17T09:40:44.217 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6308053e', total seconds elapsed: 18.5 2024-05-17T09:40:44.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49024 6.57 MB/sec execute 281 sec latency 72.374 ms 2024-05-17T09:40:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:44 smithi069 ceph-mon[35081]: pgmap v1067: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.8 MiB/s wr, 219 op/s 2024-05-17T09:40:44.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:44 smithi019 ceph-mon[31620]: pgmap v1067: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.8 MiB/s wr, 219 op/s 2024-05-17T09:40:45.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:44 smithi079 ceph-mon[35502]: pgmap v1067: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.8 MiB/s wr, 219 op/s 2024-05-17T09:40:45.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49184 6.58 MB/sec execute 282 sec latency 70.282 ms 2024-05-17T09:40:46.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49331 6.57 MB/sec execute 283 sec latency 79.706 ms 2024-05-17T09:40:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:46 smithi069 ceph-mon[35081]: pgmap v1068: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-05-17T09:40:46.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:46 smithi019 ceph-mon[31620]: pgmap v1068: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-05-17T09:40:47.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:46 smithi079 ceph-mon[35502]: pgmap v1068: 129 pgs: 129 active+clean; 434 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-05-17T09:40:47.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49477 6.56 MB/sec execute 284 sec latency 79.941 ms 2024-05-17T09:40:48.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49619 6.55 MB/sec execute 285 sec latency 83.513 ms 2024-05-17T09:40:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:48 smithi069 ceph-mon[35081]: pgmap v1069: 129 pgs: 129 active+clean; 436 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-17T09:40:48.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:48 smithi019 ceph-mon[31620]: pgmap v1069: 129 pgs: 129 active+clean; 436 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-17T09:40:49.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:48 smithi079 ceph-mon[35502]: pgmap v1069: 129 pgs: 129 active+clean; 436 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-17T09:40:49.586 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49788 6.56 MB/sec execute 286 sec latency 100.834 ms 2024-05-17T09:40:50.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 49998 6.60 MB/sec execute 287 sec latency 86.167 ms 2024-05-17T09:40:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:50 smithi069 ceph-mon[35081]: pgmap v1070: 129 pgs: 129 active+clean; 438 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-05-17T09:40:50.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:50 smithi019 ceph-mon[31620]: pgmap v1070: 129 pgs: 129 active+clean; 438 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-05-17T09:40:51.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:50 smithi079 ceph-mon[35502]: pgmap v1070: 129 pgs: 129 active+clean; 438 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-05-17T09:40:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 -- 172.21.15.19:0/1314983167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 msgr2=0x7fd118104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/1314983167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 0x7fd118104230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009900 tx=0x7fd10c02a3f0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 -- 172.21.15.19:0/1314983167 shutdown_connections 2024-05-17T09:40:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/1314983167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 0x7fd11810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/1314983167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 0x7fd118104230 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/1314983167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd1181038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 -- 172.21.15.19:0/1314983167 >> 172.21.15.19:0/1314983167 conn(0x7fd1180fddb0 msgr2=0x7fd118100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 -- 172.21.15.19:0/1314983167 shutdown_connections 2024-05-17T09:40:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.103+0000 7fd11cf8f640 1 -- 172.21.15.19:0/1314983167 wait complete. 2024-05-17T09:40:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.104+0000 7fd11cf8f640 1 Processor -- start 2024-05-17T09:40:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.104+0000 7fd11cf8f640 1 -- start start 2024-05-17T09:40:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11cf8f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd11806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11cf8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 0x7fd11819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11cf8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 0x7fd1181a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11cf8f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd11810e5b0 con 0x7fd118108b20 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11cf8f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd11810e430 con 0x7fd118103e20 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11cf8f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd11810e730 con 0x7fd118102be0 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd11806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd11806bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:34628/0 (socket says 172.21.15.19:34628) 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 -- 172.21.15.19:0/194802979 learned_addr learned my addr 172.21.15.19:0/194802979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd115d9b640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 0x7fd11819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd116d9d640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 0x7fd1181a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 -- 172.21.15.19:0/194802979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 msgr2=0x7fd11819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 0x7fd11819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 -- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 msgr2=0x7fd1181a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 0x7fd1181a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd11659c640 1 -- 172.21.15.19:0/194802979 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1181a1de0 con 0x7fd118102be0 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.105+0000 7fd116d9d640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 0x7fd1181a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.106+0000 7fd11659c640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd11806bd40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd10800e9f0 tx=0x7fd10800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.106+0000 7fd1077fe640 1 -- 172.21.15.19:0/194802979 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10800ccc0 con 0x7fd118102be0 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.106+0000 7fd1077fe640 1 -- 172.21.15.19:0/194802979 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1080035f0 con 0x7fd118102be0 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.106+0000 7fd1077fe640 1 -- 172.21.15.19:0/194802979 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10800ce60 con 0x7fd118102be0 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.106+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1181a20d0 con 0x7fd118102be0 2024-05-17T09:40:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.106+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1181a2620 con 0x7fd118102be0 2024-05-17T09:40:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 Processor -- start 2024-05-17T09:40:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd1077fe640 1 -- 172.21.15.19:0/194802979 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd108004000 con 0x7fd118102be0 2024-05-17T09:40:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 -- start start 2024-05-17T09:40:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df83773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8377920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df837e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556df82e9c30 con 0x556df7fd61b0 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556df82d97d0 con 0x556df7fad2f0 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd123e45740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556df82dd810 con 0x556df7fb54d0 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df83773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df83773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35376/0 (socket says 172.21.15.19:35376) 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd116d9d640 1 -- 172.21.15.19:0/1273382238 learned_addr learned my addr 172.21.15.19:0/1273382238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.110+0000 7fd115d9b640 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df837e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd11659c640 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8377920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd116d9d640 1 -- 172.21.15.19:0/1273382238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 msgr2=0x556df837e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd116d9d640 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df837e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd116d9d640 1 -- 172.21.15.19:0/1273382238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 msgr2=0x556df8377920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd116d9d640 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8377920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd116d9d640 1 -- 172.21.15.19:0/1273382238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df7fee210 con 0x556df7fd61b0 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd1077fe640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1000b2920 0x7fd1000b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd116d9d640 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df83773a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd110009fd0 tx=0x7fd1100108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61e1a7cec455d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd1077fe640 1 -- 172.21.15.19:0/194802979 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd1080e2bb0 con 0x7fd118102be0 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.111+0000 7fd104ff9640 1 -- 172.21.15.19:0/1273382238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110011760 con 0x556df7fd61b0 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.112+0000 7fd11659c640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1000b2920 0x7fd1000b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.112+0000 7fd104ff9640 1 -- 172.21.15.19:0/1273382238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd110011d90 con 0x556df7fd61b0 2024-05-17T09:40:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.112+0000 7fd104ff9640 1 -- 172.21.15.19:0/1273382238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11001be50 con 0x556df7fd61b0 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 -- 172.21.15.19:0/1273382238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 msgr2=0x556df83773a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd11659c640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1000b2920 0x7fd1000b4e30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd1080095c0 tx=0x7fd1080e65f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df83773a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd110009fd0 tx=0x7fd1100108e0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 -- 172.21.15.19:0/1273382238 shutdown_connections 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df837e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8377920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 --2- 172.21.15.19:0/1273382238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df83773a0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 -- 172.21.15.19:0/1273382238 >> 172.21.15.19:0/1273382238 conn(0x556df7fca300 msgr2=0x556df8372d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 -- 172.21.15.19:0/1273382238 shutdown_connections 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.113+0000 7fd123e45740 1 -- 172.21.15.19:0/1273382238 wait complete. 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 -- start start 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 Processor -- start 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 0x556df8380cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 0x556df8381230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556df7fbf790 con 0x556df7fb54d0 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556df8295890 con 0x556df7fd61b0 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.114+0000 7fd123e45740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556df837f270 con 0x556df7fad2f0 2024-05-17T09:40:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd123e45740 10 client.0 _ll_register_callbacks cb 0x556df8298d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 0x556df8380cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd115d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 0x556df8381230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd115d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 0x556df8381230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35392/0 (socket says 172.21.15.19:35392) 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 0x556df8380cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:34642/0 (socket says 172.21.15.19:34642) 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd115d9b640 1 -- 172.21.15.19:0/4053037490 learned_addr learned my addr 172.21.15.19:0/4053037490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 -- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 msgr2=0x556df8388190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 -- 172.21.15.19:0/4053037490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 msgr2=0x556df8381230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 0x556df8381230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 -- 172.21.15.19:0/4053037490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df80ba620 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd115d9b640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 0x556df8381230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd116d9d640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 0x556df8380cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd110000f80 tx=0x7fd110005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1defd1955f927613 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.115+0000 7fd0fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd0fdffb640 1 -- 172.21.15.19:0/4053037490 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002f060 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110011d90 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd0fdffb640 1 -- 172.21.15.19:0/4053037490 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd110023070 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd0fdffb640 1 -- 172.21.15.19:0/4053037490 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd11001b630 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd123e45740 10 client.28156 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556df838af00 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd0fdffb640 1 -- 172.21.15.19:0/4053037490 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd110011d90 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.116+0000 7fd0fdffb640 1 -- 172.21.15.19:0/4053037490 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd110029040 con 0x556df7fad2f0 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd123e45740 10 client.28156 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd123e45740 10 client.28156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd123e45740 20 client.28156 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd123e45740 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f94d80 0x556df838e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd123e45740 4 client.28156 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556df82ebb80 con 0x556df7f94d80 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd115d9b640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f94d80 0x556df838e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.117+0000 7fd0fdffb640 10 client.28156 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.118+0000 7fd115d9b640 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f94d80 0x556df838e380 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a8a4ef59279b38 server_cookie=bdcc883386d48943 in_seq=0 out_seq=0 2024-05-17T09:40:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.118+0000 7fd0fdffb640 1 -- 172.21.15.19:0/4053037490 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556df82ebb80 con 0x556df7f94d80 2024-05-17T09:40:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.118+0000 7fd0fdffb640 10 client.28156 handle_command_reply: tid=0 2024-05-17T09:40:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.118+0000 7fd123e45740 1 client.28156 shutdown 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 msgr2=0x556df8380cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 0x556df8380cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd110000f80 tx=0x7fd110005c80 comp rx=0 tx=0).stop 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 shutdown_connections 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f94d80 0x556df838e380 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fb54d0 0x556df8381230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 --2- 172.21.15.19:0/4053037490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fad2f0 0x556df8380cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd0fdffb640 0 client.28156 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 >> 172.21.15.19:0/4053037490 conn(0x556df7fca300 msgr2=0x556df8375c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 shutdown_connections 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 1 -- 172.21.15.19:0/4053037490 wait complete. 2024-05-17T09:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.119+0000 7fd123e45740 20 client.28156 trim_cache size 0 max 16384 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.147+0000 7fd123e45740 1 Processor -- start 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.147+0000 7fd123e45740 1 -- start start 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd123e45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fad2f0 0x556df8393a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd123e45740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8393f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd123e45740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd123e45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181ad1b0 con 0x556df7fad2f0 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd123e45740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181aaa80 con 0x556df7fd61b0 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd123e45740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181a88d0 con 0x556df7fb54d0 2024-05-17T09:40:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fad2f0 0x556df8393a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8393f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8393f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:34652/0 (socket says 172.21.15.19:34652) 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 -- 172.21.15.19:0/3076774765 learned_addr learned my addr 172.21.15.19:0/3076774765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd116d9d640 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 -- 172.21.15.19:0/3076774765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 msgr2=0x556df8388120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 -- 172.21.15.19:0/3076774765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fad2f0 msgr2=0x556df8393a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fad2f0 0x556df8393a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.148+0000 7fd115d9b640 1 -- 172.21.15.19:0/3076774765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df7f94fb0 con 0x556df7fb54d0 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.149+0000 7fd115d9b640 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8393f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd10c02a910 tx=0x7fd10c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50fc0467df1bd7bb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.149+0000 7fd0fffff640 1 -- 172.21.15.19:0/3076774765 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c04f020 con 0x556df7fb54d0 2024-05-17T09:40:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.149+0000 7fd0fffff640 1 -- 172.21.15.19:0/3076774765 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd10c031cd0 con 0x556df7fb54d0 2024-05-17T09:40:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.149+0000 7fd0fffff640 1 -- 172.21.15.19:0/3076774765 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c035040 con 0x556df7fb54d0 2024-05-17T09:40:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 -- 172.21.15.19:0/3076774765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 msgr2=0x556df8393f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8393f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd10c02a910 tx=0x7fd10c000bc0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 -- 172.21.15.19:0/3076774765 shutdown_connections 2024-05-17T09:40:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fd61b0 0x556df8388120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8393f70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 --2- 172.21.15.19:0/3076774765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fad2f0 0x556df8393a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 -- 172.21.15.19:0/3076774765 >> 172.21.15.19:0/3076774765 conn(0x556df7fca300 msgr2=0x556df837a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 -- 172.21.15.19:0/3076774765 shutdown_connections 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.150+0000 7fd123e45740 1 -- 172.21.15.19:0/3076774765 wait complete. 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 -- start start 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 Processor -- start 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8387310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8387850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df8387d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181a8f40 con 0x556df7fd61b0 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181aa3b0 con 0x556df7fad2f0 2024-05-17T09:40:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181a9610 con 0x556df7fb54d0 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df8387d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd123e45740 10 client.0 _ll_register_callbacks cb 0x7fd110004c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd116d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df8387d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35416/0 (socket says 172.21.15.19:35416) 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd116d9d640 1 -- 172.21.15.19:0/2877918838 learned_addr learned my addr 172.21.15.19:0/2877918838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd115d9b640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8387850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.151+0000 7fd116d9d640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8387310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd116d9d640 1 -- 172.21.15.19:0/2877918838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 msgr2=0x556df8387850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd116d9d640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8387850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd116d9d640 1 -- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 msgr2=0x556df8387310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd116d9d640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8387310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd116d9d640 1 -- 172.21.15.19:0/2877918838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df83f4a20 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd116d9d640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df8387d90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd11001b880 tx=0x7fd1100108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d37c41a9b9d195 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd0fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.152+0000 7fd0fdffb640 1 -- 172.21.15.19:0/2877918838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002f040 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd0fdffb640 1 -- 172.21.15.19:0/2877918838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd110002030 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd10c031cd0 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd0fdffb640 1 -- 172.21.15.19:0/2877918838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd11004b050 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd123e45740 10 client.18648 fetch_fsmap learned FSMap version 39 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556df83f5cb0 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd0fdffb640 1 -- 172.21.15.19:0/2877918838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd10c031cd0 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.153+0000 7fd0fdffb640 1 -- 172.21.15.19:0/2877918838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd110011770 con 0x556df7fd61b0 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd123e45740 10 client.18648 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:40:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd123e45740 10 client.18648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:40:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd123e45740 20 client.18648 populate_metadata read hostname 'smithi019' 2024-05-17T09:40:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd123e45740 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f6a1c0 0x556df83f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:40:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd123e45740 4 client.18648 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:40:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556df82e5360 con 0x556df7f6a1c0 2024-05-17T09:40:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd115d9b640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f6a1c0 0x556df83f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:40:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd0fdffb640 10 client.18648 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.154+0000 7fd115d9b640 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f6a1c0 0x556df83f8610 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4dd2dfc567afbbc server_cookie=97250bfe25db19a6 in_seq=0 out_seq=0 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.230+0000 7fd0fdffb640 1 -- 172.21.15.19:0/2877918838 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556df82e5360 con 0x556df7f6a1c0 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.230+0000 7fd0fdffb640 10 client.18648 handle_command_reply: tid=0 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.230+0000 7fd123e45740 1 client.18648 shutdown 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 msgr2=0x556df8387d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df8387d90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd11001b880 tx=0x7fd1100108e0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 shutdown_connections 2024-05-17T09:40:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x556df7f6a1c0 0x556df83f8610 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd0fdffb640 0 client.18648 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556df7fd61b0 0x556df8387d90 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556df7fb54d0 0x556df8387850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 --2- 172.21.15.19:0/2877918838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556df7fad2f0 0x556df8387310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 >> 172.21.15.19:0/2877918838 conn(0x556df7fca300 msgr2=0x556df837b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 shutdown_connections 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 1 -- 172.21.15.19:0/2877918838 wait complete. 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.231+0000 7fd123e45740 20 client.18648 trim_cache size 0 max 16384 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:40:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:40:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:40:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.232+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1000b2920 msgr2=0x7fd1000b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.232+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1000b2920 0x7fd1000b4e30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd1080095c0 tx=0x7fd1080e65f0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.232+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 msgr2=0x7fd11806bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:40:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.232+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd11806bd40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd10800e9f0 tx=0x7fd10800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 shutdown_connections 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fd1000b2920 0x7fd1000b4e30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd118108b20 0x7fd1181a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd118103e20 0x7fd11819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 --2- 172.21.15.19:0/194802979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd118102be0 0x7fd11806bd40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 >> 172.21.15.19:0/194802979 conn(0x7fd1180fddb0 msgr2=0x7fd118104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 shutdown_connections 2024-05-17T09:40:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:40:51.233+0000 7fd11cf8f640 1 -- 172.21.15.19:0/194802979 wait complete. 2024-05-17T09:40:51.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 50184 6.61 MB/sec execute 288 sec latency 96.068 ms 2024-05-17T09:40:52.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 50309 6.62 MB/sec execute 289 sec latency 71.686 ms 2024-05-17T09:40:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:52 smithi069 ceph-mon[35081]: pgmap v1071: 129 pgs: 129 active+clean; 439 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-05-17T09:40:52.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:52 smithi019 ceph-mon[31620]: pgmap v1071: 129 pgs: 129 active+clean; 439 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-05-17T09:40:53.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:52 smithi079 ceph-mon[35502]: pgmap v1071: 129 pgs: 129 active+clean; 439 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-05-17T09:40:53.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 50474 6.64 MB/sec execute 290 sec latency 70.833 ms 2024-05-17T09:40:54.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 50667 6.67 MB/sec execute 291 sec latency 100.621 ms 2024-05-17T09:40:54.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:54 smithi019 ceph-mon[31620]: pgmap v1072: 129 pgs: 129 active+clean; 440 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-05-17T09:40:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:54 smithi069 ceph-mon[35081]: pgmap v1072: 129 pgs: 129 active+clean; 440 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-05-17T09:40:55.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:54 smithi079 ceph-mon[35502]: pgmap v1072: 129 pgs: 129 active+clean; 440 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-05-17T09:40:55.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 50811 6.67 MB/sec execute 292 sec latency 95.598 ms 2024-05-17T09:40:56.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 50919 6.66 MB/sec execute 293 sec latency 87.247 ms 2024-05-17T09:40:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:56 smithi069 ceph-mon[35081]: pgmap v1073: 129 pgs: 129 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-05-17T09:40:56.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:56 smithi019 ceph-mon[31620]: pgmap v1073: 129 pgs: 129 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-05-17T09:40:57.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:56 smithi079 ceph-mon[35502]: pgmap v1073: 129 pgs: 129 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-05-17T09:40:57.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51068 6.66 MB/sec execute 294 sec latency 68.370 ms 2024-05-17T09:40:58.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51184 6.65 MB/sec execute 295 sec latency 136.221 ms 2024-05-17T09:40:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:58 smithi069 ceph-mon[35081]: pgmap v1074: 129 pgs: 129 active+clean; 442 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-17T09:40:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:58 smithi019 ceph-mon[31620]: pgmap v1074: 129 pgs: 129 active+clean; 442 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-17T09:40:59.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:58 smithi079 ceph-mon[35502]: pgmap v1074: 129 pgs: 129 active+clean; 442 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-17T09:40:59.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51318 6.64 MB/sec execute 296 sec latency 88.723 ms 2024-05-17T09:40:59.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:40:59 smithi069 ceph-mon[35081]: pgmap v1075: 129 pgs: 129 active+clean; 445 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.1 MiB/s wr, 264 op/s 2024-05-17T09:40:59.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:40:59 smithi019 ceph-mon[31620]: pgmap v1075: 129 pgs: 129 active+clean; 445 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.1 MiB/s wr, 264 op/s 2024-05-17T09:41:00.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:40:59 smithi079 ceph-mon[35502]: pgmap v1075: 129 pgs: 129 active+clean; 445 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.1 MiB/s wr, 264 op/s 2024-05-17T09:41:00.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51494 6.66 MB/sec execute 297 sec latency 90.589 ms 2024-05-17T09:41:01.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:00 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:41:01.190 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:00 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:41:01.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:00 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:41:01.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51645 6.65 MB/sec execute 298 sec latency 113.903 ms 2024-05-17T09:41:02.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:01 smithi079 ceph-mon[35502]: pgmap v1076: 129 pgs: 129 active+clean; 445 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-05-17T09:41:02.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:01 smithi069 ceph-mon[35081]: pgmap v1076: 129 pgs: 129 active+clean; 445 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-05-17T09:41:02.190 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:01 smithi019 ceph-mon[31620]: pgmap v1076: 129 pgs: 129 active+clean; 445 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-05-17T09:41:02.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51777 6.65 MB/sec execute 299 sec latency 83.665 ms 2024-05-17T09:41:03.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:41:03.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:41:03.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:41:03.190 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:41:03.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:41:03.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:41:03.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:41:03.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:41:03.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:41:03.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 51906 6.66 MB/sec execute 300 sec latency 230.955 ms 2024-05-17T09:41:04.179 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:03 smithi069 ceph-mon[35081]: pgmap v1077: 129 pgs: 129 active+clean; 448 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-05-17T09:41:04.191 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:03 smithi019 ceph-mon[31620]: pgmap v1077: 129 pgs: 129 active+clean; 448 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-05-17T09:41:04.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:03 smithi079 ceph-mon[35502]: pgmap v1077: 129 pgs: 129 active+clean; 448 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.423+0000 7f86d8213640 1 -- 172.21.15.19:0/3821690009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 msgr2=0x7f86d0103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.423+0000 7f86d8213640 1 --2- 172.21.15.19:0/3821690009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d0103830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02a410 comp rx=0 tx=0).stop 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.423+0000 7f86d8213640 1 -- 172.21.15.19:0/3821690009 shutdown_connections 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.423+0000 7f86d8213640 1 --2- 172.21.15.19:0/3821690009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86d0108a70 0x7f86d010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.424+0000 7f86d8213640 1 --2- 172.21.15.19:0/3821690009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86d0103d70 0x7f86d0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.424+0000 7f86d8213640 1 --2- 172.21.15.19:0/3821690009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d0103830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.424+0000 7f86d8213640 1 -- 172.21.15.19:0/3821690009 >> 172.21.15.19:0/3821690009 conn(0x7f86d00fdd20 msgr2=0x7f86d01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.424+0000 7f86d8213640 1 -- 172.21.15.19:0/3821690009 shutdown_connections 2024-05-17T09:41:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.424+0000 7f86d8213640 1 -- 172.21.15.19:0/3821690009 wait complete. 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 Processor -- start 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 -- start start 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d006be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86d0103d70 0x7f86d019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86d0108a70 0x7f86d01a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010e4c0 con 0x7f86d0103d70 2024-05-17T09:41:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010e340 con 0x7f86d0108a70 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d8213640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010e640 con 0x7f86d0102b30 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d006be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d006be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:44318/0 (socket says 172.21.15.19:44318) 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d5faf640 1 -- 172.21.15.19:0/1101059040 learned_addr learned my addr 172.21.15.19:0/1101059040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d57ae640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86d0103d70 0x7f86d019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.426+0000 7f86d67b0640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86d0108a70 0x7f86d01a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d5faf640 1 -- 172.21.15.19:0/1101059040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86d0108a70 msgr2=0x7f86d01a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d5faf640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86d0108a70 0x7f86d01a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d5faf640 1 -- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86d0103d70 msgr2=0x7f86d019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d5faf640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86d0103d70 0x7f86d019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d5faf640 1 -- 172.21.15.19:0/1101059040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d01a8310 con 0x7f86d0102b30 2024-05-17T09:41:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d5faf640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d006be30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005760 tx=0x7f86cc005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86c6ffd640 1 -- 172.21.15.19:0/1101059040 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc03f030 con 0x7f86d0102b30 2024-05-17T09:41:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86c6ffd640 1 -- 172.21.15.19:0/1101059040 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86cc0043d0 con 0x7f86d0102b30 2024-05-17T09:41:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86c6ffd640 1 -- 172.21.15.19:0/1101059040 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc035070 con 0x7f86d0102b30 2024-05-17T09:41:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.427+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d01a85a0 con 0x7f86d0102b30 2024-05-17T09:41:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.428+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d01a8980 con 0x7f86d0102b30 2024-05-17T09:41:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86c6ffd640 1 -- 172.21.15.19:0/1101059040 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f86cc042030 con 0x7f86d0102b30 2024-05-17T09:41:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 Processor -- start 2024-05-17T09:41:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 -- start start 2024-05-17T09:41:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 0x55e4a1bb63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e4a1b28c30 con 0x55e4a17f44d0 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e4a1b187d0 con 0x55e4a18151b0 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.431+0000 7f86df19d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e4a1b1c810 con 0x55e4a17ec2f0 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d67b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 0x55e4a1bb63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d67b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 0x55e4a1bb63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33454/0 (socket says 172.21.15.19:33454) 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d67b0640 1 -- 172.21.15.19:0/613288372 learned_addr learned my addr 172.21.15.19:0/613288372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d57ae640 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86c6ffd640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86bc0b2920 0x7f86bc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86c6ffd640 1 -- 172.21.15.19:0/1101059040 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86cc102960 con 0x7f86d0102b30 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 -- 172.21.15.19:0/613288372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 msgr2=0x55e4a1bb63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 0x55e4a1bb63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 -- 172.21.15.19:0/613288372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17f44d0 msgr2=0x55e4a1bbd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 -- 172.21.15.19:0/613288372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a182d210 con 0x55e4a17ec2f0 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d67b0640 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 0x55e4a1bb63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d67b0640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86bc0b2920 0x7f86bc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d57ae640 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86d5faf640 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb6920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0c6de0 tx=0x7f86cc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aefec11872761b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.432+0000 7f86bbfff640 1 -- 172.21.15.19:0/613288372 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc049070 con 0x55e4a17ec2f0 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.433+0000 7f86bbfff640 1 -- 172.21.15.19:0/613288372 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86cc108070 con 0x55e4a17ec2f0 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.433+0000 7f86bbfff640 1 -- 172.21.15.19:0/613288372 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0c9b70 con 0x55e4a17ec2f0 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 -- 172.21.15.19:0/613288372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17ec2f0 msgr2=0x55e4a1bb6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb6920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0c6de0 tx=0x7f86cc009260 comp rx=0 tx=0).stop 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86d67b0640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86bc0b2920 0x7f86bc0b4e30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55e4a1bb7570 tx=0x7f86c8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 -- 172.21.15.19:0/613288372 shutdown_connections 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb6920 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 --2- 172.21.15.19:0/613288372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a18151b0 0x55e4a1bb63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 -- 172.21.15.19:0/613288372 >> 172.21.15.19:0/613288372 conn(0x55e4a1809300 msgr2=0x55e4a1bb1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 -- 172.21.15.19:0/613288372 shutdown_connections 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 -- 172.21.15.19:0/613288372 wait complete. 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 -- start start 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.434+0000 7f86df19d740 1 Processor -- start 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bbf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a18151b0 0x55e4a1bc6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e4a17fe790 con 0x55e4a17ec2f0 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e4a1ad4890 con 0x55e4a17f44d0 2024-05-17T09:41:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e4a1b1c810 con 0x55e4a18151b0 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86df19d740 10 client.0 _ll_register_callbacks cb 0x55e4a1ad7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bbf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33466/0 (socket says 172.21.15.19:33466) 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bbf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48192/0 (socket says 172.21.15.19:48192) 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 -- 172.21.15.19:0/2508093562 learned_addr learned my addr 172.21.15.19:0/2508093562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d67b0640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a18151b0 0x55e4a1bc6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 -- 172.21.15.19:0/2508093562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a18151b0 msgr2=0x55e4a1bc6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a18151b0 0x55e4a1bc6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 -- 172.21.15.19:0/2508093562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 msgr2=0x55e4a1bbf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bbf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d57ae640 1 -- 172.21.15.19:0/2508093562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a18f9620 con 0x55e4a17f44d0 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.435+0000 7f86d5faf640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bbf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86d57ae640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbfe90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86c0012720 tx=0x7f86c0012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5dc2e437d70c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86b97fa640 1 -- 172.21.15.19:0/2508093562 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004030 con 0x55e4a17f44d0 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86cc108070 con 0x55e4a17f44d0 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86b97fa640 1 -- 172.21.15.19:0/2508093562 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86c00041d0 con 0x55e4a17f44d0 2024-05-17T09:41:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86b97fa640 1 -- 172.21.15.19:0/2508093562 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f86c0025050 con 0x55e4a17f44d0 2024-05-17T09:41:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86df19d740 10 client.28187 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.436+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e4a1bc9e40 con 0x55e4a17f44d0 2024-05-17T09:41:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86b97fa640 1 -- 172.21.15.19:0/2508093562 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86cc108070 con 0x55e4a17f44d0 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86b97fa640 1 -- 172.21.15.19:0/2508093562 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f86c00038d0 con 0x55e4a17f44d0 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86df19d740 10 client.28187 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86df19d740 10 client.28187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86df19d740 20 client.28187 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86df19d740 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a17d3d80 0x55e4a1bcd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86df19d740 4 client.28187 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a1b2ab80 con 0x55e4a17d3d80 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.437+0000 7f86d5faf640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a17d3d80 0x55e4a1bcd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.438+0000 7f86d5faf640 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a17d3d80 0x55e4a1bcd540 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2753c8993e0a07d server_cookie=ca1b31bf56547e02 in_seq=0 out_seq=0 2024-05-17T09:41:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.438+0000 7f86b97fa640 10 client.28187 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.439+0000 7f86b97fa640 1 -- 172.21.15.19:0/2508093562 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e4a1b2ab80 con 0x55e4a17d3d80 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.439+0000 7f86b97fa640 10 client.28187 handle_command_reply: tid=0 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.439+0000 7f86df19d740 1 client.28187 shutdown 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.439+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 msgr2=0x55e4a1bbfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.439+0000 7f86df19d740 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbfe90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86c0012720 tx=0x7f86c0012be0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 shutdown_connections 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a17d3d80 0x55e4a1bcd540 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a18151b0 0x55e4a1bc6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86b97fa640 0 client.28187 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17f44d0 0x55e4a1bbfe90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 --2- 172.21.15.19:0/2508093562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bbf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 >> 172.21.15.19:0/2508093562 conn(0x55e4a1809300 msgr2=0x55e4a1bb4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 shutdown_connections 2024-05-17T09:41:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 1 -- 172.21.15.19:0/2508093562 wait complete. 2024-05-17T09:41:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.440+0000 7f86df19d740 20 client.28187 trim_cache size 0 max 16384 2024-05-17T09:41:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 Processor -- start 2024-05-17T09:41:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 -- start start 2024-05-17T09:41:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17d3d80 0x55e4a1bba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b58e0 con 0x55e4a17d3d80 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b31b0 con 0x55e4a17ec2f0 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86df19d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b1000 con 0x55e4a17f44d0 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86d57ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86d57ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33478/0 (socket says 172.21.15.19:33478) 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86d67b0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86d57ae640 1 -- 172.21.15.19:0/1500317992 learned_addr learned my addr 172.21.15.19:0/1500317992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.468+0000 7f86d5faf640 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17d3d80 0x55e4a1bba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d57ae640 1 -- 172.21.15.19:0/1500317992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 msgr2=0x55e4a1bc6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d57ae640 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d57ae640 1 -- 172.21.15.19:0/1500317992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17d3d80 msgr2=0x55e4a1bba8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d57ae640 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17d3d80 0x55e4a1bba8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d57ae640 1 -- 172.21.15.19:0/1500317992 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a1bc9c60 con 0x55e4a17ec2f0 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d5faf640 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17d3d80 0x55e4a1bba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86d57ae640 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb53d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86c0003ca0 tx=0x7f86c00106c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba1633c1a6ea0826 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86bb7fe640 1 -- 172.21.15.19:0/1500317992 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c002e070 con 0x55e4a17ec2f0 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86bb7fe640 1 -- 172.21.15.19:0/1500317992 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86c0003a80 con 0x55e4a17ec2f0 2024-05-17T09:41:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.469+0000 7f86bb7fe640 1 -- 172.21.15.19:0/1500317992 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0022040 con 0x55e4a17ec2f0 2024-05-17T09:41:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 -- 172.21.15.19:0/1500317992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 msgr2=0x55e4a1bb53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb53d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86c0003ca0 tx=0x7f86c00106c0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 -- 172.21.15.19:0/1500317992 shutdown_connections 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bb53d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 --2- 172.21.15.19:0/1500317992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17d3d80 0x55e4a1bba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 -- 172.21.15.19:0/1500317992 >> 172.21.15.19:0/1500317992 conn(0x55e4a1809300 msgr2=0x55e4a1bbec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 -- 172.21.15.19:0/1500317992 shutdown_connections 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.470+0000 7f86df19d740 1 -- 172.21.15.19:0/1500317992 wait complete. 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 -- start start 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 Processor -- start 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 0x55e4a1bc5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bc61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b1670 con 0x55e4a17ec2f0 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b2ae0 con 0x55e4a17d3d80 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01b1d40 con 0x55e4a17f44d0 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86df19d740 10 client.0 _ll_register_callbacks cb 0x7f86c0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86d57ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bc61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 0x55e4a1bc5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86d5faf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 0x55e4a1bc5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33490/0 (socket says 172.21.15.19:33490) 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.471+0000 7f86d5faf640 1 -- 172.21.15.19:0/347228293 learned_addr learned my addr 172.21.15.19:0/347228293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d67b0640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d5faf640 1 -- 172.21.15.19:0/347228293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 msgr2=0x55e4a1bc66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d5faf640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d5faf640 1 -- 172.21.15.19:0/347228293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 msgr2=0x55e4a1bc61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d5faf640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bc61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d5faf640 1 -- 172.21.15.19:0/347228293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a1bc7920 con 0x55e4a17d3d80 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d67b0640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86d5faf640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 0x55e4a1bc5c60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0c5bf0 tx=0x7f86cc108260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c111698bb80e3b5a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86b97fa640 1 -- 172.21.15.19:0/347228293 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0e2070 con 0x55e4a17d3d80 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86b97fa640 1 -- 172.21.15.19:0/347228293 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f86cc003c80 con 0x55e4a17d3d80 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c0003a80 con 0x55e4a17d3d80 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86b97fa640 1 -- 172.21.15.19:0/347228293 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f86cc108040 con 0x55e4a17d3d80 2024-05-17T09:41:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.472+0000 7f86df19d740 10 client.28193 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e4a1c34ce0 con 0x55e4a17d3d80 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86b97fa640 1 -- 172.21.15.19:0/347228293 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86c0003a80 con 0x55e4a17d3d80 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86b97fa640 1 -- 172.21.15.19:0/347228293 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f86cc0c9dd0 con 0x55e4a17d3d80 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 10 client.28193 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 10 client.28193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 20 client.28193 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a18151b0 0x55e4a1c374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 4 client.28193 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4a1b24360 con 0x55e4a18151b0 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.473+0000 7f86d57ae640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a18151b0 0x55e4a1c374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.474+0000 7f86b97fa640 10 client.28193 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.474+0000 7f86d57ae640 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a18151b0 0x55e4a1c374f0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4b397bec29d2fd server_cookie=824f48403ce5f3b5 in_seq=0 out_seq=0 2024-05-17T09:41:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86b97fa640 1 -- 172.21.15.19:0/347228293 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4a1b24360 con 0x55e4a18151b0 2024-05-17T09:41:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86b97fa640 10 client.28193 handle_command_reply: tid=0 2024-05-17T09:41:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 client.28193 shutdown 2024-05-17T09:41:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 msgr2=0x55e4a1bc5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 0x55e4a1bc5c60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0c5bf0 tx=0x7f86cc108260 comp rx=0 tx=0).stop 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 shutdown_connections 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e4a18151b0 0x55e4a1c374f0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e4a17f44d0 0x55e4a1bc66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a17ec2f0 0x55e4a1bc61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 --2- 172.21.15.19:0/347228293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4a17d3d80 0x55e4a1bc5c60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 >> 172.21.15.19:0/347228293 conn(0x55e4a1809300 msgr2=0x55e4a1bbec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86b97fa640 0 client.28193 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 shutdown_connections 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.545+0000 7f86df19d740 1 -- 172.21.15.19:0/347228293 wait complete. 2024-05-17T09:41:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.546+0000 7f86df19d740 20 client.28193 trim_cache size 0 max 16384 2024-05-17T09:41:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:41:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:41:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86bc0b2920 msgr2=0x7f86bc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86bc0b2920 0x7f86bc0b4e30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55e4a1bb7570 tx=0x7f86c8004ab0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 msgr2=0x7f86d006be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d006be30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005760 tx=0x7f86cc005e20 comp rx=0 tx=0).stop 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 shutdown_connections 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86d0108a70 0x7f86d01a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f86bc0b2920 0x7f86bc0b4e30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86d0103d70 0x7f86d019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 --2- 172.21.15.19:0/1101059040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d0102b30 0x7f86d006be30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 >> 172.21.15.19:0/1101059040 conn(0x7f86d00fdd20 msgr2=0x7f86d0104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.547+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 shutdown_connections 2024-05-17T09:41:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:04.548+0000 7f86d8213640 1 -- 172.21.15.19:0/1101059040 wait complete. 2024-05-17T09:41:04.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52061 6.69 MB/sec execute 301 sec latency 117.374 ms 2024-05-17T09:41:05.203 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.6 2024-05-17T09:41:05.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:41:05.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52230 6.70 MB/sec execute 302 sec latency 101.192 ms 2024-05-17T09:41:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:41:06.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52367 6.69 MB/sec execute 303 sec latency 128.087 ms 2024-05-17T09:41:06.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:06 smithi069 ceph-mon[35081]: pgmap v1078: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 247 op/s 2024-05-17T09:41:06.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:06 smithi019 ceph-mon[31620]: pgmap v1078: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 247 op/s 2024-05-17T09:41:06.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:06 smithi079 ceph-mon[35502]: pgmap v1078: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.9 MiB/s wr, 247 op/s 2024-05-17T09:41:07.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52507 6.68 MB/sec execute 304 sec latency 79.399 ms 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 -- 172.21.15.19:0/3588833398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f158810e8f0 msgr2=0x7f1588110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 --2- 172.21.15.19:0/3588833398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f158810e8f0 0x7f1588110db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1580009f90 tx=0x7f158000b330 comp rx=0 tx=0).stop 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 -- 172.21.15.19:0/3588833398 shutdown_connections 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 --2- 172.21.15.19:0/3588833398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1588105510 0x7f1588105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 --2- 172.21.15.19:0/3588833398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f158810e8f0 0x7f1588110db0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 --2- 172.21.15.19:0/3588833398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105e60 0x7f158810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 -- 172.21.15.19:0/3588833398 >> 172.21.15.19:0/3588833398 conn(0x7f15880fedb0 msgr2=0x7f15881011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 -- 172.21.15.19:0/3588833398 shutdown_connections 2024-05-17T09:41:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.864+0000 7f15900b0700 1 -- 172.21.15.19:0/3588833398 wait complete. 2024-05-17T09:41:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.865+0000 7f15900b0700 1 Processor -- start 2024-05-17T09:41:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.865+0000 7f15900b0700 1 -- start start 2024-05-17T09:41:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f15900b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105510 0x7f15881a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f15900b0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588105e60 0x7f15881a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f15900b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 0x7f158819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f15900b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15881142e0 con 0x7f1588105510 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f15900b0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1588114160 con 0x7f1588105e60 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f15900b0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1588114460 con 0x7f158810e8f0 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 0x7f158819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 0x7f158819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:44360/0 (socket says 172.21.15.19:44360) 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 -- 172.21.15.19:0/905403607 learned_addr learned my addr 172.21.15.19:0/905403607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158d64b700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588105e60 0x7f15881a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158de4c700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105510 0x7f15881a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 -- 172.21.15.19:0/905403607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588105e60 msgr2=0x7f15881a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588105e60 0x7f15881a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 -- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105510 msgr2=0x7f15881a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105510 0x7f15881a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158e64d700 1 -- 172.21.15.19:0/905403607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158819b130 con 0x7f158810e8f0 2024-05-17T09:41:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158d64b700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588105e60 0x7f15881a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.866+0000 7f158de4c700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105510 0x7f15881a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.867+0000 7f158e64d700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 0x7f158819aa80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1580009790 tx=0x7f158000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.867+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580021070 con 0x7f158810e8f0 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.867+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158819b3c0 con 0x7f158810e8f0 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.867+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1580007ac0 con 0x7f158810e8f0 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.867+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800052a0 con 0x7f158810e8f0 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.867+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15881a76f0 con 0x7f158810e8f0 2024-05-17T09:41:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.868+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15880512b0 con 0x7f158810e8f0 2024-05-17T09:41:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.870+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1580024070 con 0x7f158810e8f0 2024-05-17T09:41:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.871+0000 7f157effd700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f15740b2900 0x7f15740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.871+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f158000f020 con 0x7f158810e8f0 2024-05-17T09:41:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.871+0000 7f158de4c700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f15740b2900 0x7f15740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.872+0000 7f158de4c700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f15740b2900 0x7f15740b4e10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1584009750 tx=0x7f1584009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:07.876+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f158002b020 con 0x7f158810e8f0 2024-05-17T09:41:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.335+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158819bfb0 con 0x7f158810e8f0 2024-05-17T09:41:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.336+0000 7f157effd700 1 -- 172.21.15.19:0/905403607 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f15800ab3c0 con 0x7f158810e8f0 2024-05-17T09:41:08.335 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:41:08.335 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:41:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f15740b2900 msgr2=0x7f15740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f15740b2900 0x7f15740b4e10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1584009750 tx=0x7f1584009310 comp rx=0 tx=0).stop 2024-05-17T09:41:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 msgr2=0x7f158819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 0x7f158819aa80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1580009790 tx=0x7f158000cc30 comp rx=0 tx=0).stop 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 shutdown_connections 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f15740b2900 0x7f15740b4e10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f158810e8f0 0x7f158819aa80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588105510 0x7f15881a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 --2- 172.21.15.19:0/905403607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1588105e60 0x7f15881a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 >> 172.21.15.19:0/905403607 conn(0x7f15880fedb0 msgr2=0x7f15880ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 shutdown_connections 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:08.340+0000 7f15900b0700 1 -- 172.21.15.19:0/905403607 wait complete. 2024-05-17T09:41:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:41:08.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52653 6.68 MB/sec execute 305 sec latency 95.860 ms 2024-05-17T09:41:08.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:08 smithi069 ceph-mon[35081]: pgmap v1079: 129 pgs: 129 active+clean; 446 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 4.2 MiB/s wr, 253 op/s 2024-05-17T09:41:08.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:08 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/905403607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:41:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:08 smithi019 ceph-mon[31620]: pgmap v1079: 129 pgs: 129 active+clean; 446 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 4.2 MiB/s wr, 253 op/s 2024-05-17T09:41:08.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:08 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/905403607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:41:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:08 smithi079 ceph-mon[35502]: pgmap v1079: 129 pgs: 129 active+clean; 446 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 4.2 MiB/s wr, 253 op/s 2024-05-17T09:41:08.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:08 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/905403607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:41:09.083 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 137.6 --await-for 45.0' 2024-05-17T09:41:09.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 137.6 --await-for 45.0 2024-05-17T09:41:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:41:09.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52790 6.68 MB/sec execute 306 sec latency 126.536 ms 2024-05-17T09:41:10.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 52917 6.67 MB/sec execute 307 sec latency 109.695 ms 2024-05-17T09:41:10.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:10 smithi019 ceph-mon[31620]: pgmap v1080: 129 pgs: 129 active+clean; 446 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-05-17T09:41:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:10 smithi079 ceph-mon[35502]: pgmap v1080: 129 pgs: 129 active+clean; 446 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-05-17T09:41:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:10 smithi069 ceph-mon[35081]: pgmap v1080: 129 pgs: 129 active+clean; 446 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-05-17T09:41:11.587 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53055 6.66 MB/sec execute 308 sec latency 109.019 ms 2024-05-17T09:41:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 -- 172.21.15.19:0/262846893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 msgr2=0x7ffa98110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/262846893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa98110090 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b0d0 tx=0x7ffa9000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 -- 172.21.15.19:0/262846893 shutdown_connections 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/262846893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa98110090 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/262846893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa981034b0 0x7ffa981038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/262846893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 0x7ffa981081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 -- 172.21.15.19:0/262846893 >> 172.21.15.19:0/262846893 conn(0x7ffa980fedb0 msgr2=0x7ffa981011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 -- 172.21.15.19:0/262846893 shutdown_connections 2024-05-17T09:41:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.754+0000 7ffaa02e7700 1 -- 172.21.15.19:0/262846893 wait complete. 2024-05-17T09:41:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.755+0000 7ffaa02e7700 1 Processor -- start 2024-05-17T09:41:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.755+0000 7ffaa02e7700 1 -- start start 2024-05-17T09:41:11.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.755+0000 7ffaa02e7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa981034b0 0x7ffa981a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.755+0000 7ffaa02e7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 0x7ffa981a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.755+0000 7ffaa02e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa981a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffaa02e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa981121c0 con 0x7ffa98108730 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffaa02e7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98112040 con 0x7ffa981034b0 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffaa02e7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98112340 con 0x7ffa98107de0 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9e884700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa981a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 0x7ffa981a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9e884700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa981a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48244/0 (socket says 172.21.15.19:48244) 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9e884700 1 -- 172.21.15.19:0/1540122033 learned_addr learned my addr 172.21.15.19:0/1540122033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 -- 172.21.15.19:0/1540122033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa981034b0 msgr2=0x7ffa981a1300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa981034b0 0x7ffa981a1300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 -- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 msgr2=0x7ffa981a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa981a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 -- 172.21.15.19:0/1540122033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa981a61d0 con 0x7ffa98107de0 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.756+0000 7ffa9d882700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 0x7ffa981a18a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800bfd0 tx=0x7ffa88007c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.757+0000 7ffa8f7fe700 1 -- 172.21.15.19:0/1540122033 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88019020 con 0x7ffa98107de0 2024-05-17T09:41:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.757+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa981a64c0 con 0x7ffa98107de0 2024-05-17T09:41:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.757+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa981adde0 con 0x7ffa98107de0 2024-05-17T09:41:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.757+0000 7ffa8f7fe700 1 -- 172.21.15.19:0/1540122033 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa8800fe60 con 0x7ffa98107de0 2024-05-17T09:41:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.757+0000 7ffa8f7fe700 1 -- 172.21.15.19:0/1540122033 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88015e00 con 0x7ffa98107de0 2024-05-17T09:41:11.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.760+0000 7ffa8f7fe700 1 -- 172.21.15.19:0/1540122033 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffa8801c020 con 0x7ffa98107de0 2024-05-17T09:41:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.761+0000 7ffa8f7fe700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffa840b2900 0x7ffa840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.761+0000 7ffa9e083700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffa840b2900 0x7ffa840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.761+0000 7ffa8f7fe700 1 -- 172.21.15.19:0/1540122033 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffa880e3680 con 0x7ffa98107de0 2024-05-17T09:41:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.778+0000 7ffa9e083700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffa840b2900 0x7ffa840b4e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400df00 tx=0x7ffa9400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.780+0000 7ffaaecfcb80 1 Processor -- start 2024-05-17T09:41:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.780+0000 7ffaaecfcb80 1 -- start start 2024-05-17T09:41:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.780+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b0025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.780+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b002b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4affc6b0 0x560a4affcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a4af2ae20 con 0x560a4ae7dfd0 2024-05-17T09:41:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b002b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9e083700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b0025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a4af1a9c0 con 0x560a4affc6b0 2024-05-17T09:41:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9e083700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b0025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:44376/0 (socket says 172.21.15.19:44376) 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b002b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48246/0 (socket says 172.21.15.19:48246) 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9e083700 1 -- 172.21.15.19:0/1784991985 learned_addr learned my addr 172.21.15.19:0/1784991985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/1784991985 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560a4af1ea00 con 0x560a4aefb560 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9e884700 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4affc6b0 0x560a4affcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 -- 172.21.15.19:0/1784991985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 msgr2=0x560a4b0025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b0025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 -- 172.21.15.19:0/1784991985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4affc6b0 msgr2=0x560a4affcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4affc6b0 0x560a4affcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9d882700 1 -- 172.21.15.19:0/1784991985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4ae6aa60 con 0x560a4ae7dfd0 2024-05-17T09:41:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9e884700 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4affc6b0 0x560a4affcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.781+0000 7ffa9e083700 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b0025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.782+0000 7ffa9d882700 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b002b30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ffa880aa590 tx=0x7ffa880aa790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d0b96efb3cd5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.782+0000 7ffa8cd47700 1 -- 172.21.15.19:0/1784991985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880aaf90 con 0x560a4ae7dfd0 2024-05-17T09:41:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.782+0000 7ffa8cd47700 1 -- 172.21.15.19:0/1784991985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa880aa890 con 0x560a4ae7dfd0 2024-05-17T09:41:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.782+0000 7ffa8cd47700 1 -- 172.21.15.19:0/1784991985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880aaaf0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/1784991985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 msgr2=0x560a4b002b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b002b30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ffa880aa590 tx=0x7ffa880aa790 comp rx=0 tx=0).stop 2024-05-17T09:41:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/1784991985 shutdown_connections 2024-05-17T09:41:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b0025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4affc6b0 0x560a4affcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/1784991985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b002b30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.783+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/1784991985 >> 172.21.15.19:0/1784991985 conn(0x560a4aebce10 msgr2=0x560a4aff75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.784+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/1784991985 shutdown_connections 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.784+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/1784991985 wait complete. 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.784+0000 7ffaaecfcb80 1 -- start start 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.784+0000 7ffaaecfcb80 1 Processor -- start 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b0067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffa9e083700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffa9e083700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b006290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48258/0 (socket says 172.21.15.19:48258) 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffa9e083700 1 -- 172.21.15.19:0/3023324087 learned_addr learned my addr 172.21.15.19:0/3023324087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4affc6b0 0x560a4b006d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b0067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a4ae21130 con 0x560a4ae7dfd0 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a4aebc950 con 0x560a4aefb560 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560a4b0092b0 con 0x560a4affc6b0 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffa9e884700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4affc6b0 0x560a4b006d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.785+0000 7ffaaecfcb80 10 client.0 _ll_register_callbacks cb 0x560a4aeb5ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9e083700 1 -- 172.21.15.19:0/3023324087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4affc6b0 msgr2=0x560a4b006d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9e083700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4affc6b0 0x560a4b006d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9e083700 1 -- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 msgr2=0x560a4b0067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9e083700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b0067d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9e083700 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4aeb9820 con 0x560a4ae7dfd0 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b0067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa9e083700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b006290 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ffa9401b700 tx=0x7ffa940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b983fdad2af9e823 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3023324087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94014070 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3023324087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffa94009050 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa880aa890 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.786+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3023324087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa9401bcd0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 10 client.18684 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a4b013cc0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3023324087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa940188d0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3023324087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffa94025650 con 0x560a4ae7dfd0 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 10 client.18684 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 10 client.18684 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 20 client.18684 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:11.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b018820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 4 client.18684 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa880aa890 con 0x560a4b016370 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.787+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b018820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.788+0000 7ffa81ffb700 10 client.18684 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.788+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b018820 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592eef8dcc284b20 server_cookie=54788eb91be349b8 in_seq=0 out_seq=0 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3023324087 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7ffa880aa890 con 0x560a4b016370 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffa81ffb700 10 client.18684 handle_command_reply: tid=0 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 client.18684 shutdown 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 msgr2=0x560a4b006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b006290 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ffa9401b700 tx=0x7ffa940047c0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 shutdown_connections 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4affc6b0 0x560a4b006d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b018820 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4ae7dfd0 0x560a4b006290 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.789+0000 7ffa81ffb700 0 client.18684 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.790+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3023324087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b0067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.790+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 >> 172.21.15.19:0/3023324087 conn(0x560a4aebce10 msgr2=0x560a4aff1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.790+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 shutdown_connections 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.790+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3023324087 wait complete. 2024-05-17T09:41:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.790+0000 7ffaaecfcb80 20 client.18684 trim_cache size 0 max 16384 2024-05-17T09:41:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 Processor -- start 2024-05-17T09:41:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 -- start start 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4ae7dfd0 0x560a4affc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b012310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b01dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c011400 con 0x560a4affc6b0 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c00ec90 con 0x560a4ae7dfd0 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.822+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c00cae0 con 0x560a4aefb560 2024-05-17T09:41:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9e884700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b01dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b012310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9e884700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b01dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48268/0 (socket says 172.21.15.19:48268) 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9e884700 1 -- 172.21.15.19:0/2843284961 learned_addr learned my addr 172.21.15.19:0/2843284961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9e083700 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4ae7dfd0 0x560a4affc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 -- 172.21.15.19:0/2843284961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4ae7dfd0 msgr2=0x560a4affc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4ae7dfd0 0x560a4affc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 -- 172.21.15.19:0/2843284961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 msgr2=0x560a4b01dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b01dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 -- 172.21.15.19:0/2843284961 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4b004800 con 0x560a4aefb560 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9e083700 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4ae7dfd0 0x560a4affc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.823+0000 7ffa9d882700 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b012310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x560a4affd100 tx=0x7ffa880b3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9592d3855eb0da0f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.824+0000 7ffa83fff700 1 -- 172.21.15.19:0/2843284961 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880aa890 con 0x560a4aefb560 2024-05-17T09:41:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.824+0000 7ffa83fff700 1 -- 172.21.15.19:0/2843284961 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa880c6070 con 0x560a4aefb560 2024-05-17T09:41:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.824+0000 7ffa83fff700 1 -- 172.21.15.19:0/2843284961 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880b3260 con 0x560a4aefb560 2024-05-17T09:41:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/2843284961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 msgr2=0x560a4b012310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b012310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x560a4affd100 tx=0x7ffa880b3040 comp rx=0 tx=0).stop 2024-05-17T09:41:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/2843284961 shutdown_connections 2024-05-17T09:41:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b01dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4ae7dfd0 0x560a4affc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/2843284961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4aefb560 0x560a4b012310 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/2843284961 >> 172.21.15.19:0/2843284961 conn(0x560a4aebce10 msgr2=0x560a4b0051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/2843284961 shutdown_connections 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/2843284961 wait complete. 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.825+0000 7ffaaecfcb80 1 -- start start 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 Processor -- start 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 0x560a4b00eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b00b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c00d150 con 0x560a4affc6b0 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c00e5c0 con 0x560a4aefb560 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c00d820 con 0x560a4ae7dfd0 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffa9e083700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 0x560a4b00eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffaaecfcb80 10 client.0 _ll_register_callbacks cb 0x560a4aeb5ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffa9e884700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffa9e083700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 0x560a4b00eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:44408/0 (socket says 172.21.15.19:44408) 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffa9e884700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b00ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48272/0 (socket says 172.21.15.19:48272) 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.826+0000 7ffa9e083700 1 -- 172.21.15.19:0/3826561430 learned_addr learned my addr 172.21.15.19:0/3826561430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b00b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e083700 1 -- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 msgr2=0x560a4b00b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e083700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b00b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e083700 1 -- 172.21.15.19:0/3826561430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 msgr2=0x560a4b00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e083700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e083700 1 -- 172.21.15.19:0/3826561430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4b00c080 con 0x560a4ae7dfd0 2024-05-17T09:41:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e884700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa9e083700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 0x560a4b00eaf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa94000c00 tx=0x7ffa94004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a09f7a460a8545 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:41:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.827+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3826561430 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94004ce0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3826561430 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffa94009040 con 0x560a4ae7dfd0 2024-05-17T09:41:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa880c6070 con 0x560a4ae7dfd0 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3826561430 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa94005040 con 0x560a4ae7dfd0 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffaaecfcb80 10 client.28204 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a4b081ca0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3826561430 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa880c6070 con 0x560a4ae7dfd0 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.828+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3826561430 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffa9401a9e0 con 0x560a4ae7dfd0 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffaaecfcb80 10 client.28204 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffaaecfcb80 10 client.28204 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffaaecfcb80 20 client.28204 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b0844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffaaecfcb80 4 client.28204 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.6, "await_for": 45.0}] 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.6, "await_for": 45.0}) -- 0x560a4af2cd70 con 0x560a4b016370 2024-05-17T09:41:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.829+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b0844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:11.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.830+0000 7ffa9d882700 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b0844b0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37891bac36cc01a1 server_cookie=819741f8dc7d8bcd in_seq=0 out_seq=0 2024-05-17T09:41:11.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:11.830+0000 7ffa81ffb700 10 client.28204 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffa81ffb700 1 -- 172.21.15.19:0/3826561430 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560a4af2cd70 con 0x560a4b016370 2024-05-17T09:41:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffa81ffb700 10 client.28204 handle_command_reply: tid=0 2024-05-17T09:41:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 client.28204 shutdown 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 msgr2=0x560a4b00eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 0x560a4b00eaf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa94000c00 tx=0x7ffa94004a40 comp rx=0 tx=0).stop 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 shutdown_connections 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a4ae7dfd0 0x560a4b00eaf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a4b016370 0x560a4b0844b0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a4affc6b0 0x560a4b00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 --2- 172.21.15.19:0/3826561430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a4aefb560 0x560a4b00b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffa81ffb700 0 client.28204 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 >> 172.21.15.19:0/3826561430 conn(0x560a4aebce10 msgr2=0x560a4b0051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.341+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 shutdown_connections 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaaecfcb80 1 -- 172.21.15.19:0/3826561430 wait complete. 2024-05-17T09:41:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaaecfcb80 20 client.28204 trim_cache size 0 max 16384 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 74, 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "b7a271fa": { 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 74, 2024-05-17T09:41:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 137.6, 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:41:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:41:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffa840b2900 msgr2=0x7ffa840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffa840b2900 0x7ffa840b4e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400df00 tx=0x7ffa9400f040 comp rx=0 tx=0).stop 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 msgr2=0x7ffa981a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 0x7ffa981a18a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800bfd0 tx=0x7ffa88007c50 comp rx=0 tx=0).stop 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 shutdown_connections 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa98108730 0x7ffa981a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa981034b0 0x7ffa981a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ffa840b2900 0x7ffa840b4e10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 --2- 172.21.15.19:0/1540122033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa98107de0 0x7ffa981a18a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.342+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 >> 172.21.15.19:0/1540122033 conn(0x7ffa980fedb0 msgr2=0x7ffa98109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.343+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 shutdown_connections 2024-05-17T09:41:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:12.343+0000 7ffaa02e7700 1 -- 172.21.15.19:0/1540122033 wait complete. 2024-05-17T09:41:12.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.67 MB/sec execute 309 sec latency 767.147 ms 2024-05-17T09:41:12.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:12 smithi019 ceph-mon[31620]: pgmap v1081: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-05-17T09:41:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:12 smithi079 ceph-mon[35502]: pgmap v1081: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-05-17T09:41:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:12 smithi069 ceph-mon[35081]: pgmap v1081: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-05-17T09:41:13.089 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b7a271fa', quiesce took 7.9 seconds. Will release after: 9.700000000000001 2024-05-17T09:41:13.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.64 MB/sec execute 310 sec latency 1767.515 ms 2024-05-17T09:41:14.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.62 MB/sec execute 311 sec latency 2767.645 ms 2024-05-17T09:41:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:14 smithi019 ceph-mon[31620]: pgmap v1082: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-05-17T09:41:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:14 smithi079 ceph-mon[35502]: pgmap v1082: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-05-17T09:41:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:14 smithi069 ceph-mon[35081]: pgmap v1082: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-05-17T09:41:15.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.60 MB/sec execute 312 sec latency 3767.761 ms 2024-05-17T09:41:16.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.58 MB/sec execute 313 sec latency 4767.858 ms 2024-05-17T09:41:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:16 smithi019 ceph-mon[31620]: pgmap v1083: 129 pgs: 129 active+clean; 448 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2024-05-17T09:41:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:16 smithi079 ceph-mon[35502]: pgmap v1083: 129 pgs: 129 active+clean; 448 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2024-05-17T09:41:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:16 smithi069 ceph-mon[35081]: pgmap v1083: 129 pgs: 129 active+clean; 448 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2024-05-17T09:41:17.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.56 MB/sec execute 314 sec latency 5767.911 ms 2024-05-17T09:41:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 -- 172.21.15.19:0/1626268926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c102b20 msgr2=0x7fad5c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 --2- 172.21.15.19:0/1626268926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c102b20 0x7fad5c103820 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad5802a410 comp rx=0 tx=0).stop 2024-05-17T09:41:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 -- 172.21.15.19:0/1626268926 shutdown_connections 2024-05-17T09:41:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 --2- 172.21.15.19:0/1626268926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5c108a60 0x7fad5c10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 --2- 172.21.15.19:0/1626268926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad5c103d60 0x7fad5c104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 --2- 172.21.15.19:0/1626268926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c102b20 0x7fad5c103820 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 -- 172.21.15.19:0/1626268926 >> 172.21.15.19:0/1626268926 conn(0x7fad5c0fdd10 msgr2=0x7fad5c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 -- 172.21.15.19:0/1626268926 shutdown_connections 2024-05-17T09:41:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 -- 172.21.15.19:0/1626268926 wait complete. 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 Processor -- start 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.677+0000 7fad64100640 1 -- start start 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad64100640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5c102b20 0x7fad5c0759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad64100640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad5c103d60 0x7fad5c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad64100640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 0x7fad5c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad64100640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c10e4b0 con 0x7fad5c108a60 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad64100640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c10e330 con 0x7fad5c103d60 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad64100640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c10e630 con 0x7fad5c102b20 2024-05-17T09:41:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 0x7fad5c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5c102b20 0x7fad5c0759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 0x7fad5c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43336/0 (socket says 172.21.15.19:43336) 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 -- 172.21.15.19:0/3257224587 learned_addr learned my addr 172.21.15.19:0/3257224587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6169b640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad5c103d60 0x7fad5c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 -- 172.21.15.19:0/3257224587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5c102b20 msgr2=0x7fad5c0759c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5c102b20 0x7fad5c0759c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 -- 172.21.15.19:0/3257224587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad5c103d60 msgr2=0x7fad5c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad5c103d60 0x7fad5c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 -- 172.21.15.19:0/3257224587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c1a1d40 con 0x7fad5c108a60 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.678+0000 7fad6269d640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 0x7fad5c1a1520 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fad54009fd0 tx=0x7fad5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.679+0000 7fad52ffd640 1 -- 172.21.15.19:0/3257224587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54017070 con 0x7fad5c108a60 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.679+0000 7fad52ffd640 1 -- 172.21.15.19:0/3257224587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad54003e00 con 0x7fad5c108a60 2024-05-17T09:41:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.679+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c1a1fd0 con 0x7fad5c108a60 2024-05-17T09:41:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.679+0000 7fad52ffd640 1 -- 172.21.15.19:0/3257224587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54010460 con 0x7fad5c108a60 2024-05-17T09:41:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.679+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c1a2440 con 0x7fad5c108a60 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.681+0000 7fad6afb2740 1 Processor -- start 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.681+0000 7fad52ffd640 1 -- 172.21.15.19:0/3257224587 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fad54003970 con 0x7fad5c108a60 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 -- start start 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5df33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b9f5d65c30 con 0x55b9f5a292f0 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9f5d557d0 con 0x55b9f5a314d0 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6afb2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b9f5d59810 con 0x55b9f5a521b0 2024-05-17T09:41:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43346/0 (socket says 172.21.15.19:43346) 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 -- 172.21.15.19:0/2880373487 learned_addr learned my addr 172.21.15.19:0/2880373487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6169b640 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5df33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6169b640 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 -- 172.21.15.19:0/2880373487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 msgr2=0x55b9f5df33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5df33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 -- 172.21.15.19:0/2880373487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a314d0 msgr2=0x55b9f5dfa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad61e9c640 1 -- 172.21.15.19:0/2880373487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f5a6a210 con 0x55b9f5a292f0 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad6169b640 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5df33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.682+0000 7fad52ffd640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad480b28a0 0x7fad480b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.683+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df3920 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fad580057c0 tx=0x7fad58005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264c190382ccb018 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.683+0000 7fad6169b640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad480b28a0 0x7fad480b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.683+0000 7fad52ffd640 1 -- 172.21.15.19:0/3257224587 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fad540e2690 con 0x7fad5c108a60 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.683+0000 7fad47fff640 1 -- 172.21.15.19:0/2880373487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58035030 con 0x55b9f5a292f0 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.683+0000 7fad47fff640 1 -- 172.21.15.19:0/2880373487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad58040070 con 0x55b9f5a292f0 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.683+0000 7fad47fff640 1 -- 172.21.15.19:0/2880373487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58003850 con 0x55b9f5a292f0 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 -- 172.21.15.19:0/2880373487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 msgr2=0x55b9f5df3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df3920 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fad580057c0 tx=0x7fad58005fd0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6169b640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad480b28a0 0x7fad480b4db0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55b9f5df4570 tx=0x7fad4c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 -- 172.21.15.19:0/2880373487 shutdown_connections 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df3920 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2880373487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5df33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 -- 172.21.15.19:0/2880373487 >> 172.21.15.19:0/2880373487 conn(0x55b9f5a46300 msgr2=0x55b9f5deed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 -- 172.21.15.19:0/2880373487 shutdown_connections 2024-05-17T09:41:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 -- 172.21.15.19:0/2880373487 wait complete. 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 -- start start 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.684+0000 7fad6afb2740 1 Processor -- start 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a292f0 0x55b9f5dfccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5e04150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b9f5a3b790 con 0x55b9f5a314d0 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9f5d11890 con 0x55b9f5a292f0 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b9f5dfb210 con 0x55b9f5a521b0 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 10 client.0 _ll_register_callbacks cb 0x55b9f5d14d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43356/0 (socket says 172.21.15.19:43356) 2024-05-17T09:41:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 -- 172.21.15.19:0/262226689 learned_addr learned my addr 172.21.15.19:0/262226689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad61e9c640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a292f0 0x55b9f5dfccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6269d640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5e04150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 -- 172.21.15.19:0/262226689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 msgr2=0x55b9f5e04150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5e04150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 -- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a292f0 msgr2=0x55b9f5dfccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a292f0 0x55b9f5dfccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 -- 172.21.15.19:0/262226689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f5b36620 con 0x55b9f5a314d0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6169b640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfd1f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fad4c003f30 tx=0x7fad4c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b43d8ddf99d00a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad457fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad457fa640 1 -- 172.21.15.19:0/262226689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c01e600 con 0x55b9f5a314d0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad457fa640 1 -- 172.21.15.19:0/262226689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fad4c01e7a0 con 0x55b9f5a314d0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.685+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad58040070 con 0x55b9f5a314d0 2024-05-17T09:41:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad457fa640 1 -- 172.21.15.19:0/262226689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fad4c031050 con 0x55b9f5a314d0 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 10 client.18702 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9f5e071a0 con 0x55b9f5a314d0 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad457fa640 1 -- 172.21.15.19:0/262226689 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad58040070 con 0x55b9f5a314d0 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad457fa640 1 -- 172.21.15.19:0/262226689 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fad4c01f7f0 con 0x55b9f5a314d0 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 10 client.18702 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 10 client.18702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 20 client.18702 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f5a10d80 0x55b9f5e0a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 4 client.18702 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9f5d67b80 con 0x55b9f5a10d80 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.686+0000 7fad61e9c640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f5a10d80 0x55b9f5e0a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.687+0000 7fad457fa640 10 client.18702 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.687+0000 7fad61e9c640 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f5a10d80 0x55b9f5e0a9b0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53f12fceaf65f5c server_cookie=454156b4f442fcf5 in_seq=0 out_seq=0 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad457fa640 1 -- 172.21.15.19:0/262226689 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b9f5d67b80 con 0x55b9f5a10d80 2024-05-17T09:41:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad457fa640 10 client.18702 handle_command_reply: tid=0 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 client.18702 shutdown 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 msgr2=0x55b9f5dfd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfd1f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fad4c003f30 tx=0x7fad4c005c90 comp rx=0 tx=0).stop 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 shutdown_connections 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f5a10d80 0x55b9f5e0a9b0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a521b0 0x55b9f5e04150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a314d0 0x55b9f5dfd1f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad457fa640 0 client.18702 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 --2- 172.21.15.19:0/262226689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a292f0 0x55b9f5dfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 >> 172.21.15.19:0/262226689 conn(0x55b9f5a46300 msgr2=0x55b9f5df1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 shutdown_connections 2024-05-17T09:41:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 1 -- 172.21.15.19:0/262226689 wait complete. 2024-05-17T09:41:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.688+0000 7fad6afb2740 20 client.18702 trim_cache size 0 max 16384 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.703+0000 7fad6afb2740 1 Processor -- start 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.703+0000 7fad6afb2740 1 -- start start 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.703+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.703+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.703+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad6afb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1af260 con 0x55b9f5a292f0 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad6afb2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1acb30 con 0x55b9f5a521b0 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad6afb2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1aa980 con 0x55b9f5a314d0 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43372/0 (socket says 172.21.15.19:43372) 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 -- 172.21.15.19:0/2783890094 learned_addr learned my addr 172.21.15.19:0/2783890094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad6269d640 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 -- 172.21.15.19:0/2783890094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 msgr2=0x55b9f5e04070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 -- 172.21.15.19:0/2783890094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 msgr2=0x55b9f5e045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 -- 172.21.15.19:0/2783890094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f5df2eb0 con 0x55b9f5a292f0 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df2240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fad58031970 tx=0x7fad5802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cde23bb04a3098 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad477fe640 1 -- 172.21.15.19:0/2783890094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58053070 con 0x55b9f5a292f0 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad477fe640 1 -- 172.21.15.19:0/2783890094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad58040050 con 0x55b9f5a292f0 2024-05-17T09:41:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.704+0000 7fad477fe640 1 -- 172.21.15.19:0/2783890094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58035040 con 0x55b9f5a292f0 2024-05-17T09:41:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 -- 172.21.15.19:0/2783890094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 msgr2=0x55b9f5df2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df2240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fad58031970 tx=0x7fad5802a410 comp rx=0 tx=0).stop 2024-05-17T09:41:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 -- 172.21.15.19:0/2783890094 shutdown_connections 2024-05-17T09:41:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e04070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2783890094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5df2240 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 -- 172.21.15.19:0/2783890094 >> 172.21.15.19:0/2783890094 conn(0x55b9f5a46300 msgr2=0x55b9f5df1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 -- 172.21.15.19:0/2783890094 shutdown_connections 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 -- 172.21.15.19:0/2783890094 wait complete. 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.705+0000 7fad6afb2740 1 -- start start 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 Processor -- start 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5e01770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e01cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e71030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1aaff0 con 0x55b9f5a292f0 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1ac460 con 0x55b9f5a521b0 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1ab6c0 con 0x55b9f5a314d0 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6afb2740 10 client.0 _ll_register_callbacks cb 0x7fad4c003be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5e01770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5e01770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43374/0 (socket says 172.21.15.19:43374) 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 -- 172.21.15.19:0/2454990599 learned_addr learned my addr 172.21.15.19:0/2454990599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6169b640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e01cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6269d640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e71030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 -- 172.21.15.19:0/2454990599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 msgr2=0x55b9f5e01cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e01cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 -- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 msgr2=0x55b9f5e71030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e71030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 -- 172.21.15.19:0/2454990599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f5e022f0 con 0x55b9f5a292f0 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6269d640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e71030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad6169b640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e01cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad61e9c640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5e01770 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fad5802a930 tx=0x7fad58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1229a98c91f6c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.706+0000 7fad457fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:41:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad457fa640 1 -- 172.21.15.19:0/2454990599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5805c060 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad457fa640 1 -- 172.21.15.19:0/2454990599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fad5804d650 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad58040050 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad457fa640 1 -- 172.21.15.19:0/2454990599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fad58063050 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 10 client.18708 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9f5e725c0 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad457fa640 1 -- 172.21.15.19:0/2454990599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad58040050 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad457fa640 1 -- 172.21.15.19:0/2454990599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fad5803cb20 con 0x55b9f5a292f0 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 10 client.18708 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 10 client.18708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 20 client.18708 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f59e61c0 0x55b9f5e74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 4 client.18708 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:41:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.707+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9f5d61360 con 0x55b9f59e61c0 2024-05-17T09:41:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.708+0000 7fad6169b640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f59e61c0 0x55b9f5e74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.708+0000 7fad457fa640 10 client.18708 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.708+0000 7fad6169b640 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f59e61c0 0x55b9f5e74b60 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f7f8f660167d0d server_cookie=7a177648cc515195 in_seq=0 out_seq=0 2024-05-17T09:41:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad457fa640 1 -- 172.21.15.19:0/2454990599 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b9f5d61360 con 0x55b9f59e61c0 2024-05-17T09:41:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad457fa640 10 client.18708 handle_command_reply: tid=0 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 client.18708 shutdown 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 msgr2=0x55b9f5e01770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5e01770 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fad5802a930 tx=0x7fad58000bc0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 shutdown_connections 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b9f59e61c0 0x55b9f5e74b60 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9f5a521b0 0x55b9f5e71030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad457fa640 0 client.18708 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9f5a314d0 0x55b9f5e01cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 --2- 172.21.15.19:0/2454990599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b9f5a292f0 0x55b9f5e01770 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.782+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 >> 172.21.15.19:0/2454990599 conn(0x55b9f5a46300 msgr2=0x55b9f5df71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 shutdown_connections 2024-05-17T09:41:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad6afb2740 1 -- 172.21.15.19:0/2454990599 wait complete. 2024-05-17T09:41:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad6afb2740 20 client.18708 trim_cache size 0 max 16384 2024-05-17T09:41:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:41:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:41:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad480b28a0 msgr2=0x7fad480b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad64100640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad480b28a0 0x7fad480b4db0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55b9f5df4570 tx=0x7fad4c00b390 comp rx=0 tx=0).stop 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 msgr2=0x7fad5c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.783+0000 7fad64100640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 0x7fad5c1a1520 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fad54009fd0 tx=0x7fad5400b260 comp rx=0 tx=0).stop 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 shutdown_connections 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fad480b28a0 0x7fad480b4db0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c108a60 0x7fad5c1a1520 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad5c103d60 0x7fad5c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 --2- 172.21.15.19:0/3257224587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5c102b20 0x7fad5c0759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 >> 172.21.15.19:0/3257224587 conn(0x7fad5c0fdd10 msgr2=0x7fad5c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 shutdown_connections 2024-05-17T09:41:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:17.784+0000 7fad64100640 1 -- 172.21.15.19:0/3257224587 wait complete. 2024-05-17T09:41:18.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.54 MB/sec execute 315 sec latency 6768.020 ms 2024-05-17T09:41:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:18 smithi079 ceph-mon[35502]: pgmap v1084: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-05-17T09:41:18.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:18 smithi069 ceph-mon[35081]: pgmap v1084: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-05-17T09:41:18.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:18 smithi019 ceph-mon[31620]: pgmap v1084: 129 pgs: 129 active+clean; 447 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-05-17T09:41:19.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.52 MB/sec execute 316 sec latency 7768.097 ms 2024-05-17T09:41:20.588 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.50 MB/sec execute 317 sec latency 8768.153 ms 2024-05-17T09:41:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:20 smithi079 ceph-mon[35502]: pgmap v1085: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-17T09:41:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:20 smithi069 ceph-mon[35081]: pgmap v1085: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-17T09:41:20.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:20 smithi019 ceph-mon[31620]: pgmap v1085: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-17T09:41:21.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.48 MB/sec execute 318 sec latency 9768.224 ms 2024-05-17T09:41:22.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.46 MB/sec execute 319 sec latency 10768.366 ms 2024-05-17T09:41:22.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:22 smithi019 ceph-mon[31620]: pgmap v1086: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-17T09:41:22.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:22 smithi079 ceph-mon[35502]: pgmap v1086: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-17T09:41:22.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:41:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:22 smithi069 ceph-mon[35081]: pgmap v1086: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-17T09:41:23.061 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:41:23.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.44 MB/sec execute 320 sec latency 11768.431 ms 2024-05-17T09:41:24.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.42 MB/sec execute 321 sec latency 12768.518 ms 2024-05-17T09:41:24.637 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:24 smithi019 ceph-mon[31620]: pgmap v1087: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-05-17T09:41:24.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:24 smithi079 ceph-mon[35502]: pgmap v1087: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-05-17T09:41:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:24 smithi069 ceph-mon[35081]: pgmap v1087: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-05-17T09:41:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 -- 172.21.15.19:0/1525521330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec1034b0 msgr2=0x7fbeec1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 --2- 172.21.15.19:0/1525521330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec1034b0 0x7fbeec1038c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbee80099c0 tx=0x7fbee8009cd0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 -- 172.21.15.19:0/1525521330 shutdown_connections 2024-05-17T09:41:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 --2- 172.21.15.19:0/1525521330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec108730 0x7fbeec110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 --2- 172.21.15.19:0/1525521330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec1034b0 0x7fbeec1038c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 --2- 172.21.15.19:0/1525521330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeec107de0 0x7fbeec1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 -- 172.21.15.19:0/1525521330 >> 172.21.15.19:0/1525521330 conn(0x7fbeec0fede0 msgr2=0x7fbeec101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 -- 172.21.15.19:0/1525521330 shutdown_connections 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.560+0000 7fbef3aba700 1 -- 172.21.15.19:0/1525521330 wait complete. 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 Processor -- start 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 -- start start 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeec1034b0 0x7fbeec19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec107de0 0x7fbeec19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 0x7fbeec19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec113260 con 0x7fbeec108730 2024-05-17T09:41:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1130e0 con 0x7fbeec1034b0 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef3aba700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1133e0 con 0x7fbeec107de0 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef2057700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 0x7fbeec19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef2057700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 0x7fbeec19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55286/0 (socket says 172.21.15.19:55286) 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.561+0000 7fbef2057700 1 -- 172.21.15.19:0/3347572603 learned_addr learned my addr 172.21.15.19:0/3347572603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef1856700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeec1034b0 0x7fbeec19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef1055700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec107de0 0x7fbeec19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef2057700 1 -- 172.21.15.19:0/3347572603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec107de0 msgr2=0x7fbeec19c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef2057700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec107de0 0x7fbeec19c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef2057700 1 -- 172.21.15.19:0/3347572603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeec1034b0 msgr2=0x7fbeec19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef2057700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeec1034b0 0x7fbeec19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef2057700 1 -- 172.21.15.19:0/3347572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1a6f30 con 0x7fbeec108730 2024-05-17T09:41:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef2057700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 0x7fbeec19c720 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbee4007d70 tx=0x7fbee400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4016020 con 0x7fbeec108730 2024-05-17T09:41:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbee400ac90 con 0x7fbeec108730 2024-05-17T09:41:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40115e0 con 0x7fbeec108730 2024-05-17T09:41:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1a7220 con 0x7fbeec108730 2024-05-17T09:41:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.562+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1a7760 con 0x7fbeec108730 2024-05-17T09:41:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.563+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec0512b0 con 0x7fbeec108730 2024-05-17T09:41:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.565+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbee4019020 con 0x7fbeec108730 2024-05-17T09:41:25.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.566+0000 7fbedeffd700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbed80b2900 0x7fbed80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:25.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.566+0000 7fbef1856700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbed80b2900 0x7fbed80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:25.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.566+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbee40e29b0 con 0x7fbeec108730 2024-05-17T09:41:25.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.566+0000 7fbef1856700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbed80b2900 0x7fbed80b4e10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbee8000c00 tx=0x7fbee8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.569+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbee4020020 con 0x7fbeec108730 2024-05-17T09:41:25.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.40 MB/sec execute 322 sec latency 13768.588 ms 2024-05-17T09:41:25.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.840+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeec075fa0 con 0x7fbeec108730 2024-05-17T09:41:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.840+0000 7fbedeffd700 1 -- 172.21.15.19:0/3347572603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fbee40aa320 con 0x7fbeec108730 2024-05-17T09:41:25.839 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:41:25.839 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:41:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbed80b2900 msgr2=0x7fbed80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbed80b2900 0x7fbed80b4e10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbee8000c00 tx=0x7fbee8005fb0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 msgr2=0x7fbeec19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 0x7fbeec19c720 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbee4007d70 tx=0x7fbee400a340 comp rx=0 tx=0).stop 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 shutdown_connections 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeec108730 0x7fbeec19c720 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeec1034b0 0x7fbeec19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbed80b2900 0x7fbed80b4e10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 --2- 172.21.15.19:0/3347572603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec107de0 0x7fbeec19c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 >> 172.21.15.19:0/3347572603 conn(0x7fbeec0fede0 msgr2=0x7fbeec101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 shutdown_connections 2024-05-17T09:41:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:25.843+0000 7fbef3aba700 1 -- 172.21.15.19:0/3347572603 wait complete. 2024-05-17T09:41:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:41:26.501 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id b7a271fa --release --await' 2024-05-17T09:41:26.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id b7a271fa --release --await 2024-05-17T09:41:26.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.38 MB/sec execute 323 sec latency 14768.626 ms 2024-05-17T09:41:26.757 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:41:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:26 smithi079 ceph-mon[35502]: pgmap v1088: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 587 KiB/s wr, 72 op/s 2024-05-17T09:41:26.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:26 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3347572603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:41:26.879 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:26 smithi019 ceph-mon[31620]: pgmap v1088: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 587 KiB/s wr, 72 op/s 2024-05-17T09:41:26.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:26 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3347572603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:41:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:26 smithi069 ceph-mon[35081]: pgmap v1088: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 587 KiB/s wr, 72 op/s 2024-05-17T09:41:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:26 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3347572603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:41:27.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53127 6.36 MB/sec execute 324 sec latency 15768.657 ms 2024-05-17T09:41:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 -- 172.21.15.19:0/1855269001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230104640 msgr2=0x7f4230104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 --2- 172.21.15.19:0/1855269001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230104640 0x7f4230104a50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f42200099c0 tx=0x7f4220009cd0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 -- 172.21.15.19:0/1855269001 shutdown_connections 2024-05-17T09:41:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 --2- 172.21.15.19:0/1855269001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4230109870 0x7f42301111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 --2- 172.21.15.19:0/1855269001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230104640 0x7f4230104a50 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 --2- 172.21.15.19:0/1855269001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230108f20 0x7f4230109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 -- 172.21.15.19:0/1855269001 >> 172.21.15.19:0/1855269001 conn(0x7f4230079ef0 msgr2=0x7f423007a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 -- 172.21.15.19:0/1855269001 shutdown_connections 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.478+0000 7f42364dd700 1 -- 172.21.15.19:0/1855269001 wait complete. 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 Processor -- start 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 -- start start 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230104640 0x7f423019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 0x7f423019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4230109870 0x7f423019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4230114440 con 0x7f4230108f20 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42301142c0 con 0x7f4230109870 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.479+0000 7f42364dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42301145c0 con 0x7f4230104640 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 0x7f423019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 0x7f423019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55300/0 (socket says 172.21.15.19:55300) 2024-05-17T09:41:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 -- 172.21.15.19:0/2130833053 learned_addr learned my addr 172.21.15.19:0/2130833053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4234a7a700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4230109870 0x7f423019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f422ffff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230104640 0x7f423019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.480 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:28 smithi019 ceph-mon[31620]: pgmap v1089: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-05-17T09:41:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 -- 172.21.15.19:0/2130833053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230104640 msgr2=0x7f423019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230104640 0x7f423019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 -- 172.21.15.19:0/2130833053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4230109870 msgr2=0x7f423019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4230109870 0x7f423019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 -- 172.21.15.19:0/2130833053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423019fcd0 con 0x7f4230108f20 2024-05-17T09:41:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f422ffff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230104640 0x7f423019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:41:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f4227fff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 0x7f423019b3f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f421800b500 tx=0x7f421800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.480+0000 7f422dffb700 1 -- 172.21.15.19:0/2130833053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218015020 con 0x7f4230108f20 2024-05-17T09:41:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.481+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423019ffc0 con 0x7f4230108f20 2024-05-17T09:41:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.481+0000 7f422dffb700 1 -- 172.21.15.19:0/2130833053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4218011cc0 con 0x7f4230108f20 2024-05-17T09:41:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.481+0000 7f422dffb700 1 -- 172.21.15.19:0/2130833053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218010c80 con 0x7f4230108f20 2024-05-17T09:41:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.481+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42301a78e0 con 0x7f4230108f20 2024-05-17T09:41:28.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.484+0000 7f422dffb700 1 -- 172.21.15.19:0/2130833053 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4218018020 con 0x7f4230108f20 2024-05-17T09:41:28.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.484+0000 7f422dffb700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f421c0b2750 0x7f421c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.485+0000 7f422ffff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f421c0b2750 0x7f421c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.485+0000 7f422dffb700 1 -- 172.21.15.19:0/2130833053 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f42180e1c90 con 0x7f4230108f20 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.485+0000 7f422ffff700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f421c0b2750 0x7f421c0b4c60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4220000c00 tx=0x7f4220005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.485+0000 7f4244ef2b80 1 Processor -- start 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.485+0000 7f4244ef2b80 1 -- start start 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 0x55ced8ba1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a1cf40 0x55ced8ba2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b9c020 0x55ced8b9c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ced8ac9e70 con 0x55ced8b08530 2024-05-17T09:41:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ced8ab9a10 con 0x55ced8a1cf40 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ced8abda50 con 0x55ced8b9c020 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 0x55ced8ba1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 0x55ced8ba1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55312/0 (socket says 172.21.15.19:55312) 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 -- 172.21.15.19:0/4267935737 learned_addr learned my addr 172.21.15.19:0/4267935737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4227fff700 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a1cf40 0x55ced8ba2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f4234a7a700 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b9c020 0x55ced8b9c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 -- 172.21.15.19:0/4267935737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b9c020 msgr2=0x55ced8b9c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b9c020 0x55ced8b9c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 -- 172.21.15.19:0/4267935737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a1cf40 msgr2=0x55ced8ba2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a1cf40 0x55ced8ba2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 -- 172.21.15.19:0/4267935737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced8a630b0 con 0x55ced8b08530 2024-05-17T09:41:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.486+0000 7f422ffff700 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 0x55ced8ba1e90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f42200099c0 tx=0x7f422001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c8c423b06987b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.487+0000 7f42267fc700 1 -- 172.21.15.19:0/4267935737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422000bd60 con 0x55ced8b08530 2024-05-17T09:41:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.487+0000 7f42267fc700 1 -- 172.21.15.19:0/4267935737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4220004cc0 con 0x55ced8b08530 2024-05-17T09:41:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.487+0000 7f42267fc700 1 -- 172.21.15.19:0/4267935737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220005310 con 0x55ced8b08530 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.487+0000 7f4244ef2b80 1 -- 172.21.15.19:0/4267935737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 msgr2=0x55ced8ba1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.487+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 0x55ced8ba1e90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f42200099c0 tx=0x7f422001d000 comp rx=0 tx=0).stop 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.487+0000 7f4244ef2b80 1 -- 172.21.15.19:0/4267935737 shutdown_connections 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b9c020 0x55ced8b9c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b08530 0x55ced8ba1e90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/4267935737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a1cf40 0x55ced8ba2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 -- 172.21.15.19:0/4267935737 >> 172.21.15.19:0/4267935737 conn(0x55ced8a7d2c0 msgr2=0x55ced8b96cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 -- 172.21.15.19:0/4267935737 shutdown_connections 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 -- 172.21.15.19:0/4267935737 wait complete. 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 -- start start 2024-05-17T09:41:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.488+0000 7f4244ef2b80 1 Processor -- start 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a1cf40 0x55ced8ba5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8ba60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 0x55ced8ba6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ced89c00a0 con 0x55ced8b9c020 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ced8a21ea0 con 0x55ced8b08530 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ced8ba8b20 con 0x55ced8a1cf40 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4244ef2b80 10 client.0 _ll_register_callbacks cb 0x55ced8a54a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4234a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 0x55ced8ba6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4234a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 0x55ced8ba6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55318/0 (socket says 172.21.15.19:55318) 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4234a7a700 1 -- 172.21.15.19:0/3428066224 learned_addr learned my addr 172.21.15.19:0/3428066224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8ba60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f422ffff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a1cf40 0x55ced8ba5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 -- 172.21.15.19:0/3428066224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a1cf40 msgr2=0x55ced8ba5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a1cf40 0x55ced8ba5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 -- 172.21.15.19:0/3428066224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 msgr2=0x55ced8ba6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 0x55ced8ba6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 -- 172.21.15.19:0/3428066224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced8a099d0 con 0x55ced8b08530 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4234a7a700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 0x55ced8ba6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f4227fff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8ba60c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f42180df750 tx=0x7f4218010600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8c1b68f28a1a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.489+0000 7f420f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f420f7fe700 1 -- 172.21.15.19:0/3428066224 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180a8a10 con 0x55ced8b08530 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4220004cc0 con 0x55ced8b08530 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f420f7fe700 1 -- 172.21.15.19:0/3428066224 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f42180b1ae0 con 0x55ced8b08530 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f420f7fe700 1 -- 172.21.15.19:0/3428066224 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f42180a9600 con 0x55ced8b08530 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f4244ef2b80 10 client.28241 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ced8a73860 con 0x55ced8b08530 2024-05-17T09:41:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.490+0000 7f420f7fe700 1 -- 172.21.15.19:0/3428066224 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4220004cc0 con 0x55ced8b08530 2024-05-17T09:41:28.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f420f7fe700 1 -- 172.21.15.19:0/3428066224 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f42180a4870 con 0x55ced8b08530 2024-05-17T09:41:28.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f4244ef2b80 10 client.28241 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:28.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f4244ef2b80 10 client.28241 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f4244ef2b80 20 client.28241 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8bb5870 0x55ced8bb7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f4244ef2b80 4 client.28241 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ced8b67fd0 con 0x55ced8bb5870 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.491+0000 7f422ffff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8bb5870 0x55ced8bb7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.492+0000 7f422ffff700 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8bb5870 0x55ced8bb7ce0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d393a651b9fe78 server_cookie=fd974b1ab92e5d10 in_seq=0 out_seq=0 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.492+0000 7f420f7fe700 10 client.28241 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f420f7fe700 1 -- 172.21.15.19:0/3428066224 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ced8b67fd0 con 0x55ced8bb5870 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f420f7fe700 10 client.28241 handle_command_reply: tid=0 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 client.28241 shutdown 2024-05-17T09:41:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 msgr2=0x55ced8ba60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8ba60c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f42180df750 tx=0x7f4218010600 comp rx=0 tx=0).stop 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 shutdown_connections 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a1cf40 0x55ced8ba5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8bb5870 0x55ced8bb7ce0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8b9c020 0x55ced8ba6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/3428066224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8ba60c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 >> 172.21.15.19:0/3428066224 conn(0x55ced8a7d2c0 msgr2=0x55ced8b9b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f420f7fe700 0 client.28241 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 shutdown_connections 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 1 -- 172.21.15.19:0/3428066224 wait complete. 2024-05-17T09:41:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.493+0000 7f4244ef2b80 20 client.28241 trim_cache size 0 max 16384 2024-05-17T09:41:28.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 Processor -- start 2024-05-17T09:41:28.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 -- start start 2024-05-17T09:41:28.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a7d2c0 0x55ced8ba8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8b912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8b9b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b28a0 con 0x55ced8a1cf40 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b0130 con 0x55ced8b08530 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42301ae030 con 0x55ced8a7d2c0 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8b912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8b912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55322/0 (socket says 172.21.15.19:55322) 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 -- 172.21.15.19:0/1388537955 learned_addr learned my addr 172.21.15.19:0/1388537955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f422ffff700 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a7d2c0 0x55ced8ba8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 -- 172.21.15.19:0/1388537955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a7d2c0 msgr2=0x55ced8ba8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a7d2c0 0x55ced8ba8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 -- 172.21.15.19:0/1388537955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 msgr2=0x55ced8b9b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4234a7a700 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8b9b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8b9b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f4227fff700 1 -- 172.21.15.19:0/1388537955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced8ba6970 con 0x55ced8a1cf40 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.514+0000 7f422ffff700 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a7d2c0 0x55ced8ba8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:41:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.515+0000 7f4227fff700 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8b912d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f42180a8e90 tx=0x7f42180df430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79af8f75832a5d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.515+0000 7f4225ffb700 1 -- 172.21.15.19:0/1388537955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180a8f00 con 0x55ced8a1cf40 2024-05-17T09:41:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.515+0000 7f4225ffb700 1 -- 172.21.15.19:0/1388537955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42180ad2b0 con 0x55ced8a1cf40 2024-05-17T09:41:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.515+0000 7f4225ffb700 1 -- 172.21.15.19:0/1388537955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180a7bd0 con 0x55ced8a1cf40 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 -- 172.21.15.19:0/1388537955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 msgr2=0x55ced8b912d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8b912d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f42180a8e90 tx=0x7f42180df430 comp rx=0 tx=0).stop 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 -- 172.21.15.19:0/1388537955 shutdown_connections 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8a7d2c0 0x55ced8ba8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8b08530 0x55ced8b9b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/1388537955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8b912d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 -- 172.21.15.19:0/1388537955 >> 172.21.15.19:0/1388537955 conn(0x55ced8b96ed0 msgr2=0x55ced8b90bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 -- 172.21.15.19:0/1388537955 shutdown_connections 2024-05-17T09:41:28.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 -- 172.21.15.19:0/1388537955 wait complete. 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 -- start start 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.516+0000 7f4244ef2b80 1 Processor -- start 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8badd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a7d2c0 0x55ced8bae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b08530 0x55ced8baab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42301ae5f0 con 0x55ced8a1cf40 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42301afa60 con 0x55ced8a7d2c0 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42301aecc0 con 0x55ced8b08530 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4244ef2b80 10 client.0 _ll_register_callbacks cb 0x55ced8a54a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8badd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8badd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55334/0 (socket says 172.21.15.19:55334) 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 -- 172.21.15.19:0/2428137258 learned_addr learned my addr 172.21.15.19:0/2428137258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4234a7a700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b08530 0x55ced8baab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f4227fff700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a7d2c0 0x55ced8bae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 -- 172.21.15.19:0/2428137258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b08530 msgr2=0x55ced8baab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b08530 0x55ced8baab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 -- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a7d2c0 msgr2=0x55ced8bae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a7d2c0 0x55ced8bae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 -- 172.21.15.19:0/2428137258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced8bab260 con 0x55ced8a1cf40 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f422ffff700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8badd90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f422002f040 tx=0x7f422000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48341f8b8b36d17c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.517+0000 7f420f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f420f7fe700 1 -- 172.21.15.19:0/2428137258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422000fb20 con 0x55ced8a1cf40 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f420f7fe700 1 -- 172.21.15.19:0/2428137258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f422000fcc0 con 0x55ced8a1cf40 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42180ad2b0 con 0x55ced8a1cf40 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f420f7fe700 1 -- 172.21.15.19:0/2428137258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f422004d050 con 0x55ced8a1cf40 2024-05-17T09:41:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 10 client.18732 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ced8c250f0 con 0x55ced8a1cf40 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f420f7fe700 1 -- 172.21.15.19:0/2428137258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42180ad2b0 con 0x55ced8a1cf40 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f420f7fe700 1 -- 172.21.15.19:0/2428137258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f422002c030 con 0x55ced8a1cf40 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 10 client.18732 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 10 client.18732 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 20 client.18732 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8b9c020 0x55ced8c27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 4 client.18732 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b7a271fa", "release": true, "await": true}] 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.518+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b7a271fa", "release": true, "await": true}) -- 0x55ced8acbdc0 con 0x55ced8b9c020 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.519+0000 7f4227fff700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8b9c020 0x55ced8c27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.519+0000 7f4227fff700 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8b9c020 0x55ced8c27330 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ef85a17e9f927e server_cookie=b7dbc160997d112d in_seq=0 out_seq=0 2024-05-17T09:41:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.519+0000 7f420f7fe700 10 client.18732 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.557+0000 7f420f7fe700 1 -- 172.21.15.19:0/2428137258 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ced8acbdc0 con 0x55ced8b9c020 2024-05-17T09:41:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.557+0000 7f420f7fe700 10 client.18732 handle_command_reply: tid=0 2024-05-17T09:41:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.557+0000 7f4244ef2b80 1 client.18732 shutdown 2024-05-17T09:41:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.557+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 msgr2=0x55ced8badd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.557+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8badd90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f422002f040 tx=0x7f422000b7e0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 shutdown_connections 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ced8b08530 0x55ced8baab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ced8b9c020 0x55ced8c27330 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ced8a1cf40 0x55ced8badd90 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 --2- 172.21.15.19:0/2428137258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ced8a7d2c0 0x55ced8bae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 >> 172.21.15.19:0/2428137258 conn(0x55ced8b96ed0 msgr2=0x55ced8b9b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f420f7fe700 0 client.18732 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 shutdown_connections 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 1 -- 172.21.15.19:0/2428137258 wait complete. 2024-05-17T09:41:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.558+0000 7f4244ef2b80 20 client.18732 trim_cache size 0 max 16384 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 76, 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "b7a271fa": { 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 76, 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:41:28.558 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 137.6, 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:41:28.559 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:41:28.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:28.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:28.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:28.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:28.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:41:28.560 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f421c0b2750 msgr2=0x7f421c0b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f421c0b2750 0x7f421c0b4c60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4220000c00 tx=0x7f4220005fb0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 msgr2=0x7f423019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 0x7f423019b3f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f421800b500 tx=0x7f421800b810 comp rx=0 tx=0).stop 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 shutdown_connections 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f421c0b2750 0x7f421c0b4c60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4230104640 0x7f423019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4230109870 0x7f423019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 --2- 172.21.15.19:0/2130833053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4230108f20 0x7f423019b3f0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 >> 172.21.15.19:0/2130833053 conn(0x7f4230079ef0 msgr2=0x7f4230102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 shutdown_connections 2024-05-17T09:41:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:41:28.559+0000 7f42364dd700 1 -- 172.21.15.19:0/2130833053 wait complete. 2024-05-17T09:41:28.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53136 6.34 MB/sec execute 325 sec latency 16705.003 ms 2024-05-17T09:41:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:28 smithi079 ceph-mon[35502]: pgmap v1089: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-05-17T09:41:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:28 smithi069 ceph-mon[35081]: pgmap v1089: 129 pgs: 129 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-05-17T09:41:29.331 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b7a271fa', total seconds elapsed: 24.1 2024-05-17T09:41:29.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53313 6.36 MB/sec execute 326 sec latency 69.698 ms 2024-05-17T09:41:30.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53527 6.39 MB/sec execute 327 sec latency 62.356 ms 2024-05-17T09:41:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:30 smithi079 ceph-mon[35502]: pgmap v1090: 129 pgs: 129 active+clean; 444 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 196 KiB/s wr, 28 op/s 2024-05-17T09:41:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 -- 172.21.15.19:0/4165987672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c103ab0 msgr2=0x7f7e2c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/4165987672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c1047b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009860 tx=0x7f7e1c02a340 comp rx=0 tx=0).stop 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 -- 172.21.15.19:0/4165987672 shutdown_connections 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/4165987672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2c1099f0 0x7f7e2c10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/4165987672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104cf0 0x7f7e2c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/4165987672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c1047b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 -- 172.21.15.19:0/4165987672 >> 172.21.15.19:0/4165987672 conn(0x7f7e2c0feca0 msgr2=0x7f7e2c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 -- 172.21.15.19:0/4165987672 shutdown_connections 2024-05-17T09:41:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.928+0000 7f7e2b59e640 1 -- 172.21.15.19:0/4165987672 wait complete. 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 Processor -- start 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 -- start start 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c104cf0 0x7f7e2c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2c1099f0 0x7f7e2c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c10f4e0 con 0x7f7e2c1099f0 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c10f360 con 0x7f7e2c103ab0 2024-05-17T09:41:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.929+0000 7f7e2b59e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c10f660 con 0x7f7e2c104cf0 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e2a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2c1099f0 0x7f7e2c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e2a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33272/0 (socket says 172.21.15.19:33272) 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c104cf0 0x7f7e2c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e2a59c640 1 -- 172.21.15.19:0/3973785060 learned_addr learned my addr 172.21.15.19:0/3973785060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 -- 172.21.15.19:0/3973785060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c103ab0 msgr2=0x7f7e2c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 -- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2c1099f0 msgr2=0x7f7e2c1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2c1099f0 0x7f7e2c1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 -- 172.21.15.19:0/3973785060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c074ef0 con 0x7f7e2c104cf0 2024-05-17T09:41:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.930+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c104cf0 0x7f7e2c19a5e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005fd0 tx=0x7f7e2000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.931+0000 7f7e1b7fe640 1 -- 172.21.15.19:0/3973785060 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000cd00 con 0x7f7e2c104cf0 2024-05-17T09:41:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.931+0000 7f7e1b7fe640 1 -- 172.21.15.19:0/3973785060 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e200044b0 con 0x7f7e2c104cf0 2024-05-17T09:41:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.931+0000 7f7e1b7fe640 1 -- 172.21.15.19:0/3973785060 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20010430 con 0x7f7e2c104cf0 2024-05-17T09:41:30.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:30 smithi069 ceph-mon[35081]: pgmap v1090: 129 pgs: 129 active+clean; 444 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 196 KiB/s wr, 28 op/s 2024-05-17T09:41:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.931+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c0751e0 con 0x7f7e2c104cf0 2024-05-17T09:41:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.931+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c0756e0 con 0x7f7e2c104cf0 2024-05-17T09:41:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.934+0000 7f7e3798b740 1 Processor -- start 2024-05-17T09:41:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 -- start start 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e1b7fe640 1 -- 172.21.15.19:0/3973785060 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e200105d0 con 0x7f7e2c104cf0 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edeb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edae62f0 0x5622edeb92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edaee4d0 0x5622edec01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5622ede22c30 con 0x5622edb0f1b0 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5622ede127d0 con 0x5622edae62f0 2024-05-17T09:41:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e3798b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5622ede16810 con 0x5622edaee4d0 2024-05-17T09:41:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edeb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edeb8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55378/0 (socket says 172.21.15.19:55378) 2024-05-17T09:41:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e2ad9d640 1 -- 172.21.15.19:0/3935798579 learned_addr learned my addr 172.21.15.19:0/3935798579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.935+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edae62f0 0x5622edeb92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edaee4d0 0x5622edec01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2ad9d640 1 -- 172.21.15.19:0/3935798579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edaee4d0 msgr2=0x5622edec01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2ad9d640 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edaee4d0 0x5622edec01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2ad9d640 1 -- 172.21.15.19:0/3935798579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edae62f0 msgr2=0x5622edeb92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2ad9d640 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edae62f0 0x5622edeb92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2ad9d640 1 -- 172.21.15.19:0/3935798579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622edb27210 con 0x5622edb0f1b0 2024-05-17T09:41:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edae62f0 0x5622edeb92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.936+0000 7f7e2ad9d640 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edeb8d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7e240039f0 tx=0x7f7e24003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820b5b94e727c75c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.937+0000 7f7e1b7fe640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e140b2920 0x7f7e140b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.937+0000 7f7e18ff9640 1 -- 172.21.15.19:0/3935798579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2401a470 con 0x5622edb0f1b0 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.937+0000 7f7e18ff9640 1 -- 172.21.15.19:0/3935798579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e24015070 con 0x5622edb0f1b0 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.937+0000 7f7e1b7fe640 1 -- 172.21.15.19:0/3973785060 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7e200e27e0 con 0x7f7e2c104cf0 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.937+0000 7f7e18ff9640 1 -- 172.21.15.19:0/3935798579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24010770 con 0x5622edb0f1b0 2024-05-17T09:41:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.937+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e140b2920 0x7f7e140b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 -- 172.21.15.19:0/3935798579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 msgr2=0x5622edeb8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e140b2920 0x7f7e140b4e30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c005e00 tx=0x7f7e1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edeb8d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7e240039f0 tx=0x7f7e24003eb0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 -- 172.21.15.19:0/3935798579 shutdown_connections 2024-05-17T09:41:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edaee4d0 0x5622edec01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edae62f0 0x5622edeb92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 --2- 172.21.15.19:0/3935798579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edeb8d50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.938+0000 7f7e3798b740 1 -- 172.21.15.19:0/3935798579 >> 172.21.15.19:0/3935798579 conn(0x5622edb03300 msgr2=0x5622edeb46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 -- 172.21.15.19:0/3935798579 shutdown_connections 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 -- 172.21.15.19:0/3935798579 wait complete. 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 -- start start 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 Processor -- start 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.939+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edec9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e3798b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5622edaf8790 con 0x5622edb0f1b0 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edec9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33292/0 (socket says 172.21.15.19:33292) 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e29d9b640 1 -- 172.21.15.19:0/2867333827 learned_addr learned my addr 172.21.15.19:0/2867333827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edec9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55392/0 (socket says 172.21.15.19:55392) 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5622eddce890 con 0x5622edaee4d0 2024-05-17T09:41:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5622edec0c20 con 0x5622edae62f0 2024-05-17T09:41:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e3798b740 10 client.0 _ll_register_callbacks cb 0x5622eddd1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 -- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 msgr2=0x5622edec2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 -- 172.21.15.19:0/2867333827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 msgr2=0x5622edec9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edec9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 -- 172.21.15.19:0/2867333827 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622edbf3620 con 0x5622edae62f0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec26a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c02a930 tx=0x7f7e1c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ebdde538ba7a178 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.940+0000 7f7e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e11ffb640 1 -- 172.21.15.19:0/2867333827 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c037040 con 0x5622edae62f0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e24015070 con 0x5622edae62f0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e11ffb640 1 -- 172.21.15.19:0/2867333827 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7e1c005250 con 0x5622edae62f0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e11ffb640 1 -- 172.21.15.19:0/2867333827 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e1c057050 con 0x5622edae62f0 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 10 client.28249 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5622edeccb10 con 0x5622edae62f0 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e11ffb640 1 -- 172.21.15.19:0/2867333827 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e24015070 con 0x5622edae62f0 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e11ffb640 1 -- 172.21.15.19:0/2867333827 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7e1c052030 con 0x5622edae62f0 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 10 client.28249 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 10 client.28249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 20 client.28249 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edacdd80 0x5622eded0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 4 client.28249 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.941+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ede24b80 con 0x5622edacdd80 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.942+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edacdd80 0x5622eded0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.942+0000 7f7e11ffb640 10 client.28249 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.942+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edacdd80 0x5622eded0210 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf422054bfd12a1 server_cookie=f8fa99aab516bdf5 in_seq=0 out_seq=0 2024-05-17T09:41:30.949 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:30 smithi019 ceph-mon[31620]: pgmap v1090: 129 pgs: 129 active+clean; 444 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 196 KiB/s wr, 28 op/s 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e11ffb640 1 -- 172.21.15.19:0/2867333827 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5622ede24b80 con 0x5622edacdd80 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e11ffb640 10 client.28249 handle_command_reply: tid=0 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 client.28249 shutdown 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 msgr2=0x5622edec26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec26a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c02a930 tx=0x7f7e1c02a340 comp rx=0 tx=0).stop 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 shutdown_connections 2024-05-17T09:41:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edacdd80 0x5622eded0210 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edb0f1b0 0x5622edec9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e11ffb640 0 client.28249 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 --2- 172.21.15.19:0/2867333827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec26a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 >> 172.21.15.19:0/2867333827 conn(0x5622edb03300 msgr2=0x5622edeaafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.943+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 shutdown_connections 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.944+0000 7f7e3798b740 1 -- 172.21.15.19:0/2867333827 wait complete. 2024-05-17T09:41:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.944+0000 7f7e3798b740 20 client.28249 trim_cache size 0 max 16384 2024-05-17T09:41:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.970+0000 7f7e3798b740 1 Processor -- start 2024-05-17T09:41:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 -- start start 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 0x5622edeaac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 0x5622edec9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edeca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c1acf10 con 0x5622edae62f0 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c1aa7e0 con 0x5622edaee4d0 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e3798b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c1a8630 con 0x5622edacdd80 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 0x5622edec9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 0x5622edeaac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 0x5622edec9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55402/0 (socket says 172.21.15.19:55402) 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e29d9b640 1 -- 172.21.15.19:0/1411800271 learned_addr learned my addr 172.21.15.19:0/1411800271 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 0x5622edeaac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:60308/0 (socket says 172.21.15.19:60308) 2024-05-17T09:41:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edeca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 -- 172.21.15.19:0/1411800271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 msgr2=0x5622edeca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edeca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 -- 172.21.15.19:0/1411800271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 msgr2=0x5622edec9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 0x5622edec9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2a59c640 1 -- 172.21.15.19:0/1411800271 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622edec0d60 con 0x5622edacdd80 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 0x5622edec9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.971+0000 7f7e2ad9d640 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edeca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.972+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 0x5622edeaac90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c004f20 tx=0x7f7e1c02bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5fd510f1bf1090 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.972+0000 7f7e13fff640 1 -- 172.21.15.19:0/1411800271 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c05f020 con 0x5622edacdd80 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.972+0000 7f7e13fff640 1 -- 172.21.15.19:0/1411800271 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e1c005000 con 0x5622edacdd80 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.972+0000 7f7e13fff640 1 -- 172.21.15.19:0/1411800271 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c037050 con 0x5622edacdd80 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 -- 172.21.15.19:0/1411800271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 msgr2=0x5622edeaac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 0x5622edeaac90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c004f20 tx=0x7f7e1c02bf90 comp rx=0 tx=0).stop 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 -- 172.21.15.19:0/1411800271 shutdown_connections 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edeca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edae62f0 0x5622edec9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 --2- 172.21.15.19:0/1411800271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edacdd80 0x5622edeaac90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 -- 172.21.15.19:0/1411800271 >> 172.21.15.19:0/1411800271 conn(0x5622edb03300 msgr2=0x5622edeaa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 -- 172.21.15.19:0/1411800271 shutdown_connections 2024-05-17T09:41:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.973+0000 7f7e3798b740 1 -- 172.21.15.19:0/1411800271 wait complete. 2024-05-17T09:41:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 -- start start 2024-05-17T09:41:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 Processor -- start 2024-05-17T09:41:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edacdd80 0x5622edec7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c1a8ca0 con 0x5622edacdd80 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c1aa110 con 0x5622edaee4d0 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c1a9370 con 0x5622edae62f0 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e3798b740 10 client.0 _ll_register_callbacks cb 0x7f7e1c030de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e2a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edacdd80 0x5622edec7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.974+0000 7f7e29d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:60318/0 (socket says 172.21.15.19:60318) 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 -- 172.21.15.19:0/809975554 learned_addr learned my addr 172.21.15.19:0/809975554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e2ad9d640 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 -- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 msgr2=0x5622edec8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 -- 172.21.15.19:0/809975554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edacdd80 msgr2=0x5622edec7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edacdd80 0x5622edec7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 -- 172.21.15.19:0/809975554 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622edec9150 con 0x5622edae62f0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e29d9b640 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec7c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e200a6cc0 tx=0x7f7e200e8550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4720b76fcd546449 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e11ffb640 1 -- 172.21.15.19:0/809975554 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200a5040 con 0x5622edae62f0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e11ffb640 1 -- 172.21.15.19:0/809975554 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7e200a5ce0 con 0x5622edae62f0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1c005000 con 0x5622edae62f0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.975+0000 7f7e11ffb640 1 -- 172.21.15.19:0/809975554 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e200a9560 con 0x5622edae62f0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 10 client.28255 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5622edeca450 con 0x5622edae62f0 2024-05-17T09:41:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e11ffb640 1 -- 172.21.15.19:0/809975554 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e1c005000 con 0x5622edae62f0 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e11ffb640 1 -- 172.21.15.19:0/809975554 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7e200a8cb0 con 0x5622edae62f0 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 10 client.28255 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 10 client.28255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 20 client.28255 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edb0f1b0 0x5622edf3aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 4 client.28255 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622ede1e360 con 0x5622edb0f1b0 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.976+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edb0f1b0 0x5622edf3aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.977+0000 7f7e11ffb640 10 client.28255 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:30.977+0000 7f7e2a59c640 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edb0f1b0 0x5622edf3aee0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0c5d1b0d39c704 server_cookie=74fb965d8ead0fc4 in_seq=0 out_seq=0 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.104+0000 7f7e11ffb640 1 -- 172.21.15.19:0/809975554 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5622ede1e360 con 0x5622edb0f1b0 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.104+0000 7f7e11ffb640 10 client.28255 handle_command_reply: tid=0 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.104+0000 7f7e3798b740 1 client.28255 shutdown 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.104+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 msgr2=0x5622edec7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.104+0000 7f7e3798b740 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec7c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e200a6cc0 tx=0x7f7e200e8550 comp rx=0 tx=0).stop 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 shutdown_connections 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5622edb0f1b0 0x5622edf3aee0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5622edaee4d0 0x5622edec8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e11ffb640 0 client.28255 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5622edae62f0 0x5622edec7c60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 --2- 172.21.15.19:0/809975554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622edacdd80 0x5622edec7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 >> 172.21.15.19:0/809975554 conn(0x5622edb03300 msgr2=0x5622edeb7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 shutdown_connections 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 1 -- 172.21.15.19:0/809975554 wait complete. 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.105+0000 7f7e3798b740 20 client.28255 trim_cache size 0 max 16384 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:41:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e140b2920 msgr2=0x7f7e140b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e140b2920 0x7f7e140b4e30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c005e00 tx=0x7f7e1c000bc0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c104cf0 msgr2=0x7f7e2c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c104cf0 0x7f7e2c19a5e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005fd0 tx=0x7f7e2000ee50 comp rx=0 tx=0).stop 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 shutdown_connections 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f7e140b2920 0x7f7e140b4e30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2c1099f0 0x7f7e2c1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e2c104cf0 0x7f7e2c19a5e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 --2- 172.21.15.19:0/3973785060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c103ab0 0x7f7e2c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 >> 172.21.15.19:0/3973785060 conn(0x7f7e2c0feca0 msgr2=0x7f7e2c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 shutdown_connections 2024-05-17T09:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:31.106+0000 7f7e2b59e640 1 -- 172.21.15.19:0/3973785060 wait complete. 2024-05-17T09:41:31.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53666 6.39 MB/sec execute 328 sec latency 94.445 ms 2024-05-17T09:41:32.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53826 6.41 MB/sec execute 329 sec latency 77.343 ms 2024-05-17T09:41:32.731 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:32 smithi069 ceph-mon[35081]: pgmap v1091: 129 pgs: 129 active+clean; 445 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 319 KiB/s wr, 22 op/s 2024-05-17T09:41:32.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:32 smithi079 ceph-mon[35502]: pgmap v1091: 129 pgs: 129 active+clean; 445 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 319 KiB/s wr, 22 op/s 2024-05-17T09:41:32.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:32 smithi019 ceph-mon[31620]: pgmap v1091: 129 pgs: 129 active+clean; 445 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 319 KiB/s wr, 22 op/s 2024-05-17T09:41:33.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 53983 6.42 MB/sec execute 330 sec latency 79.280 ms 2024-05-17T09:41:34.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54128 6.42 MB/sec execute 331 sec latency 82.529 ms 2024-05-17T09:41:34.748 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:34 smithi019 ceph-mon[31620]: pgmap v1092: 129 pgs: 129 active+clean; 452 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-05-17T09:41:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:34 smithi079 ceph-mon[35502]: pgmap v1092: 129 pgs: 129 active+clean; 452 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-05-17T09:41:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:34 smithi069 ceph-mon[35081]: pgmap v1092: 129 pgs: 129 active+clean; 452 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-05-17T09:41:35.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54276 6.43 MB/sec execute 332 sec latency 112.501 ms 2024-05-17T09:41:36.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54403 6.42 MB/sec execute 333 sec latency 70.562 ms 2024-05-17T09:41:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:36 smithi079 ceph-mon[35502]: pgmap v1093: 129 pgs: 129 active+clean; 456 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-17T09:41:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:36 smithi069 ceph-mon[35081]: pgmap v1093: 129 pgs: 129 active+clean; 456 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-17T09:41:36.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:36 smithi019 ceph-mon[31620]: pgmap v1093: 129 pgs: 129 active+clean; 456 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-17T09:41:37.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54558 6.43 MB/sec execute 334 sec latency 75.503 ms 2024-05-17T09:41:38.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54701 6.43 MB/sec execute 335 sec latency 85.305 ms 2024-05-17T09:41:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:38 smithi079 ceph-mon[35502]: pgmap v1094: 129 pgs: 129 active+clean; 456 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-05-17T09:41:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:38 smithi069 ceph-mon[35081]: pgmap v1094: 129 pgs: 129 active+clean; 456 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-05-17T09:41:38.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:38 smithi019 ceph-mon[31620]: pgmap v1094: 129 pgs: 129 active+clean; 456 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-05-17T09:41:39.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54859 6.44 MB/sec execute 336 sec latency 133.660 ms 2024-05-17T09:41:40.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 54987 6.43 MB/sec execute 337 sec latency 138.358 ms 2024-05-17T09:41:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:40 smithi079 ceph-mon[35502]: pgmap v1095: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-05-17T09:41:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:40 smithi069 ceph-mon[35081]: pgmap v1095: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-05-17T09:41:40.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:40 smithi019 ceph-mon[31620]: pgmap v1095: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-05-17T09:41:41.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 55138 6.43 MB/sec execute 338 sec latency 77.288 ms 2024-05-17T09:41:42.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 55332 6.46 MB/sec execute 339 sec latency 88.911 ms 2024-05-17T09:41:42.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:42 smithi079 ceph-mon[35502]: pgmap v1096: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-05-17T09:41:42.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:42 smithi069 ceph-mon[35081]: pgmap v1096: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-05-17T09:41:42.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:42 smithi019 ceph-mon[31620]: pgmap v1096: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-05-17T09:41:43.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 55476 6.45 MB/sec execute 340 sec latency 82.789 ms 2024-05-17T09:41:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 -- 172.21.15.19:0/3358024818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83981043d0 msgr2=0x7f83981067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 --2- 172.21.15.19:0/3358024818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83981043d0 0x7f83981067f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402a3f0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 -- 172.21.15.19:0/3358024818 shutdown_connections 2024-05-17T09:41:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 --2- 172.21.15.19:0/3358024818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83981098d0 0x7f839810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 --2- 172.21.15.19:0/3358024818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8398106dc0 0x7f8398109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 --2- 172.21.15.19:0/3358024818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83981043d0 0x7f83981067f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 -- 172.21.15.19:0/3358024818 >> 172.21.15.19:0/3358024818 conn(0x7f83980fdd30 msgr2=0x7f83981001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 -- 172.21.15.19:0/3358024818 shutdown_connections 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.273+0000 7f83a058d640 1 -- 172.21.15.19:0/3358024818 wait complete. 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 Processor -- start 2024-05-17T09:41:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 -- start start 2024-05-17T09:41:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83981043d0 0x7f839819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 0x7f839819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83981098d0 0x7f83981a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f839810f620 con 0x7f83981098d0 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f839810f4a0 con 0x7f8398106dc0 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f83a058d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f839810f7a0 con 0x7f83981043d0 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f839e329640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83981043d0 0x7f839819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f839db28640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 0x7f839819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f839db28640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 0x7f839819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37374/0 (socket says 172.21.15.19:37374) 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f839e329640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83981043d0 0x7f839819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47900/0 (socket says 172.21.15.19:47900) 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f839db28640 1 -- 172.21.15.19:0/4270531111 learned_addr learned my addr 172.21.15.19:0/4270531111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.274+0000 7f839eb2a640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83981098d0 0x7f83981a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f839db28640 1 -- 172.21.15.19:0/4270531111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83981043d0 msgr2=0x7f839819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f839db28640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83981043d0 0x7f839819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f839db28640 1 -- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83981098d0 msgr2=0x7f83981a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f839db28640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83981098d0 0x7f83981a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f839db28640 1 -- 172.21.15.19:0/4270531111 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83981a1d30 con 0x7f8398106dc0 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f839db28640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 0x7f839819a730 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9f0 tx=0x7f838800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f838f7fe640 1 -- 172.21.15.19:0/4270531111 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838800cd80 con 0x7f8398106dc0 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f838f7fe640 1 -- 172.21.15.19:0/4270531111 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83880108c0 con 0x7f8398106dc0 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f838f7fe640 1 -- 172.21.15.19:0/4270531111 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83880187c0 con 0x7f8398106dc0 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8398102220 con 0x7f8398106dc0 2024-05-17T09:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.275+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8398102720 con 0x7f8398106dc0 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.278+0000 7f83a7449740 1 Processor -- start 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 -- start start 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6bb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5624de626c30 con 0x5624de3131b0 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624de6167d0 con 0x5624de2ea2f0 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f83a7449740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5624de61a810 con 0x5624de2f24d0 2024-05-17T09:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f839e329640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f839eb2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f839e329640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37386/0 (socket says 172.21.15.19:37386) 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.279+0000 7f839e329640 1 -- 172.21.15.19:0/1890337925 learned_addr learned my addr 172.21.15.19:0/1890337925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f838f7fe640 1 -- 172.21.15.19:0/4270531111 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f83880189a0 con 0x7f8398106dc0 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f839e329640 1 -- 172.21.15.19:0/1890337925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 msgr2=0x5624de6bb820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f839e329640 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6bb820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f839e329640 1 -- 172.21.15.19:0/1890337925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 msgr2=0x5624de6b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f839e329640 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f839e329640 1 -- 172.21.15.19:0/1890337925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624de32b210 con 0x5624de2ea2f0 2024-05-17T09:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.280+0000 7f839e329640 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6b4920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f839806afb0 tx=0x7f8394004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ae5380771f91ac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.281+0000 7f838f7fe640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f83840b2920 0x7f83840b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.281+0000 7f839eb2a640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f83840b2920 0x7f83840b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.281+0000 7f838f7fe640 1 -- 172.21.15.19:0/4270531111 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f83880e2910 con 0x7f8398106dc0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.281+0000 7f838cbfe640 1 -- 172.21.15.19:0/1890337925 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394035030 con 0x5624de2ea2f0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.281+0000 7f838cbfe640 1 -- 172.21.15.19:0/1890337925 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8394040070 con 0x5624de2ea2f0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.281+0000 7f838cbfe640 1 -- 172.21.15.19:0/1890337925 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940317f0 con 0x5624de2ea2f0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 -- 172.21.15.19:0/1890337925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 msgr2=0x5624de6b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f839eb2a640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f83840b2920 0x7f83840b4e30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5624de6b5570 tx=0x7f8390009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6b4920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f839806afb0 tx=0x7f8394004040 comp rx=0 tx=0).stop 2024-05-17T09:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 -- 172.21.15.19:0/1890337925 shutdown_connections 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6bb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6b4920 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 --2- 172.21.15.19:0/1890337925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 -- 172.21.15.19:0/1890337925 >> 172.21.15.19:0/1890337925 conn(0x5624de307300 msgr2=0x5624de6afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 -- 172.21.15.19:0/1890337925 shutdown_connections 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.282+0000 7f83a7449740 1 -- 172.21.15.19:0/1890337925 wait complete. 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 -- start start 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 Processor -- start 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5624de2fc790 con 0x5624de3131b0 2024-05-17T09:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5624de5d2890 con 0x5624de2ea2f0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f83a7449740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5624de6bc210 con 0x5624de2f24d0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.283+0000 7f839db28640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839db28640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6be1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:47908/0 (socket says 172.21.15.19:47908) 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839db28640 1 -- 172.21.15.19:0/4039629149 learned_addr learned my addr 172.21.15.19:0/4039629149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839eb2a640 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f83a7449740 10 client.0 _ll_register_callbacks cb 0x5624de5d5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 -- 172.21.15.19:0/4039629149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 msgr2=0x5624de6be1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6be1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 -- 172.21.15.19:0/4039629149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 msgr2=0x5624de6c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 -- 172.21.15.19:0/4039629149 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624de3f7620 con 0x5624de2ea2f0 2024-05-17T09:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f839e329640 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6bdcb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f839402a910 tx=0x7f8394004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5bd31fd997ca63 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f8381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f8381ffb640 1 -- 172.21.15.19:0/4039629149 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394053060 con 0x5624de2ea2f0 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f8381ffb640 1 -- 172.21.15.19:0/4039629149 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8394003ac0 con 0x5624de2ea2f0 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.284+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8394040070 con 0x5624de2ea2f0 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f8381ffb640 1 -- 172.21.15.19:0/4039629149 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f839405a050 con 0x5624de2ea2f0 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 10 client.28274 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624de6c8080 con 0x5624de2ea2f0 2024-05-17T09:41:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f8381ffb640 1 -- 172.21.15.19:0/4039629149 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8394040070 con 0x5624de2ea2f0 2024-05-17T09:41:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f8381ffb640 1 -- 172.21.15.19:0/4039629149 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f839404d030 con 0x5624de2ea2f0 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 10 client.28274 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 10 client.28274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 20 client.28274 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de2d1d80 0x5624de6cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 4 client.28274 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624de628b80 con 0x5624de2d1d80 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.285+0000 7f839db28640 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de2d1d80 0x5624de6cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.286+0000 7f8381ffb640 10 client.28274 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.286+0000 7f839db28640 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de2d1d80 0x5624de6cb3e0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8466ee3df1aceb47 server_cookie=894f85cd62772e4b in_seq=0 out_seq=0 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f8381ffb640 1 -- 172.21.15.19:0/4039629149 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5624de628b80 con 0x5624de2d1d80 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f8381ffb640 10 client.28274 handle_command_reply: tid=0 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 client.28274 shutdown 2024-05-17T09:41:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 msgr2=0x5624de6bdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6bdcb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f839402a910 tx=0x7f8394004520 comp rx=0 tx=0).stop 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 shutdown_connections 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de2d1d80 0x5624de6cb3e0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f8381ffb640 0 client.28274 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de3131b0 0x5624de6c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6be1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 --2- 172.21.15.19:0/4039629149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6bdcb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 >> 172.21.15.19:0/4039629149 conn(0x5624de307300 msgr2=0x5624de6b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 shutdown_connections 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 1 -- 172.21.15.19:0/4039629149 wait complete. 2024-05-17T09:41:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.288+0000 7f83a7449740 20 client.28274 trim_cache size 0 max 16384 2024-05-17T09:41:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 Processor -- start 2024-05-17T09:41:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 -- start start 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 0x5624de6b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 0x5624de6c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83981ad0a0 con 0x5624de2ea2f0 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83981aa970 con 0x5624de2d1d80 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f83a7449740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83981a87c0 con 0x5624de2f24d0 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f839db28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 0x5624de6c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f839e329640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 0x5624de6b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f839db28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 0x5624de6c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48646/0 (socket says 172.21.15.19:48646) 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f839e329640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 0x5624de6b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37394/0 (socket says 172.21.15.19:37394) 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f839db28640 1 -- 172.21.15.19:0/359450640 learned_addr learned my addr 172.21.15.19:0/359450640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.312+0000 7f839eb2a640 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839e329640 1 -- 172.21.15.19:0/359450640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 msgr2=0x5624de6cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839e329640 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839e329640 1 -- 172.21.15.19:0/359450640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 msgr2=0x5624de6c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839e329640 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 0x5624de6c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839e329640 1 -- 172.21.15.19:0/359450640 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624de6b3f60 con 0x5624de2d1d80 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839db28640 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 0x5624de6c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f839e329640 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 0x5624de6b88b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83940057c0 tx=0x7f839402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5f8c6542779a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f8383fff640 1 -- 172.21.15.19:0/359450640 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394053040 con 0x5624de2d1d80 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f8383fff640 1 -- 172.21.15.19:0/359450640 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8394040070 con 0x5624de2d1d80 2024-05-17T09:41:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.313+0000 7f8383fff640 1 -- 172.21.15.19:0/359450640 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394035030 con 0x5624de2d1d80 2024-05-17T09:41:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 -- 172.21.15.19:0/359450640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 msgr2=0x5624de6b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 0x5624de6b88b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83940057c0 tx=0x7f839402a3f0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 -- 172.21.15.19:0/359450640 shutdown_connections 2024-05-17T09:41:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2f24d0 0x5624de6cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2ea2f0 0x5624de6c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 --2- 172.21.15.19:0/359450640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2d1d80 0x5624de6b88b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.314+0000 7f83a7449740 1 -- 172.21.15.19:0/359450640 >> 172.21.15.19:0/359450640 conn(0x5624de307300 msgr2=0x5624de6b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 -- 172.21.15.19:0/359450640 shutdown_connections 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 -- 172.21.15.19:0/359450640 wait complete. 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 -- start start 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 Processor -- start 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2d1d80 0x5624de6c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2f24d0 0x5624de731900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83981a8e30 con 0x5624de2f24d0 2024-05-17T09:41:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83981aa2a0 con 0x5624de2ea2f0 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83981a9500 con 0x5624de2d1d80 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.315+0000 7f83a7449740 10 client.0 _ll_register_callbacks cb 0x7f8394005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:37398/0 (socket says 172.21.15.19:37398) 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 -- 172.21.15.19:0/2350115228 learned_addr learned my addr 172.21.15.19:0/2350115228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839eb2a640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2f24d0 0x5624de731900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839e329640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2d1d80 0x5624de6c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 -- 172.21.15.19:0/2350115228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2d1d80 msgr2=0x5624de6c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2d1d80 0x5624de6c1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 -- 172.21.15.19:0/2350115228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2f24d0 msgr2=0x5624de731900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2f24d0 0x5624de731900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 -- 172.21.15.19:0/2350115228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624de6bc380 con 0x5624de2ea2f0 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839eb2a640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2f24d0 0x5624de731900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f839db28640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6c2220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f838800c6a0 tx=0x7f8388014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdbea13fe5cb7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f8381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f8381ffb640 1 -- 172.21.15.19:0/2350115228 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83880b29e0 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.316+0000 7f8381ffb640 1 -- 172.21.15.19:0/2350115228 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8388010d50 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8394040070 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f8381ffb640 1 -- 172.21.15.19:0/2350115228 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83880a5ec0 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 10 client.28280 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624de732ba0 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f8381ffb640 1 -- 172.21.15.19:0/2350115228 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8394040070 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f8381ffb640 1 -- 172.21.15.19:0/2350115228 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f83880a8e90 con 0x5624de2ea2f0 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 10 client.28280 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 10 client.28280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 20 client.28280 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de3131b0 0x5624de735540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 4 client.28280 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:41:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624de622360 con 0x5624de3131b0 2024-05-17T09:41:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.317+0000 7f839e329640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de3131b0 0x5624de735540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.318+0000 7f8381ffb640 10 client.28280 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.318+0000 7f839e329640 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de3131b0 0x5624de735540 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3cd083a64a410a server_cookie=27887a3643fe24d in_seq=0 out_seq=0 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f8381ffb640 1 -- 172.21.15.19:0/2350115228 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5624de622360 con 0x5624de3131b0 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f8381ffb640 10 client.28280 handle_command_reply: tid=0 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 client.28280 shutdown 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 msgr2=0x5624de6c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6c2220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f838800c6a0 tx=0x7f8388014000 comp rx=0 tx=0).stop 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 shutdown_connections 2024-05-17T09:41:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5624de3131b0 0x5624de735540 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624de2f24d0 0x5624de731900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624de2ea2f0 0x5624de6c2220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 --2- 172.21.15.19:0/2350115228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624de2d1d80 0x5624de6c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 >> 172.21.15.19:0/2350115228 conn(0x5624de307300 msgr2=0x5624de6b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f8381ffb640 0 client.28280 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 shutdown_connections 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 1 -- 172.21.15.19:0/2350115228 wait complete. 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.479+0000 7f83a7449740 20 client.28280 trim_cache size 0 max 16384 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:41:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:41:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:41:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f83840b2920 msgr2=0x7f83840b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f83840b2920 0x7f83840b4e30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5624de6b5570 tx=0x7f8390009500 comp rx=0 tx=0).stop 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 msgr2=0x7f839819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 0x7f839819a730 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9f0 tx=0x7f838800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 shutdown_connections 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f83840b2920 0x7f83840b4e30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83981098d0 0x7f83981a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8398106dc0 0x7f839819a730 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 --2- 172.21.15.19:0/4270531111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83981043d0 0x7f839819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 >> 172.21.15.19:0/4270531111 conn(0x7f83980fdd30 msgr2=0x7f8398104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 shutdown_connections 2024-05-17T09:41:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:44.480+0000 7f83a058d640 1 -- 172.21.15.19:0/4270531111 wait complete. 2024-05-17T09:41:44.589 INFO:tasks.workunit.client.0.smithi019.stdout: 10 55628 6.45 MB/sec execute 341 sec latency 159.309 ms 2024-05-17T09:41:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:44 smithi079 ceph-mon[35502]: pgmap v1097: 129 pgs: 129 active+clean; 457 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-17T09:41:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:44 smithi069 ceph-mon[35081]: pgmap v1097: 129 pgs: 129 active+clean; 457 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-17T09:41:44.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:44 smithi019 ceph-mon[31620]: pgmap v1097: 129 pgs: 129 active+clean; 457 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 255 op/s 2024-05-17T09:41:45.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 55757 6.44 MB/sec execute 342 sec latency 110.322 ms 2024-05-17T09:41:46.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 55930 6.44 MB/sec execute 343 sec latency 108.113 ms 2024-05-17T09:41:46.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:46 smithi079 ceph-mon[35502]: pgmap v1098: 129 pgs: 129 active+clean; 456 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-05-17T09:41:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:46 smithi069 ceph-mon[35081]: pgmap v1098: 129 pgs: 129 active+clean; 456 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-05-17T09:41:46.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:46 smithi019 ceph-mon[31620]: pgmap v1098: 129 pgs: 129 active+clean; 456 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-05-17T09:41:47.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 56099 6.45 MB/sec execute 344 sec latency 108.428 ms 2024-05-17T09:41:48.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 56231 6.44 MB/sec execute 345 sec latency 93.205 ms 2024-05-17T09:41:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:48 smithi079 ceph-mon[35502]: pgmap v1099: 129 pgs: 129 active+clean; 453 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-05-17T09:41:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:48 smithi069 ceph-mon[35081]: pgmap v1099: 129 pgs: 129 active+clean; 453 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-05-17T09:41:48.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:48 smithi019 ceph-mon[31620]: pgmap v1099: 129 pgs: 129 active+clean; 453 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-05-17T09:41:49.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 56388 6.44 MB/sec execute 346 sec latency 97.065 ms 2024-05-17T09:41:50.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 56636 6.49 MB/sec execute 347 sec latency 67.383 ms 2024-05-17T09:41:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:50 smithi079 ceph-mon[35502]: pgmap v1100: 129 pgs: 129 active+clean; 456 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 287 op/s 2024-05-17T09:41:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:50 smithi069 ceph-mon[35081]: pgmap v1100: 129 pgs: 129 active+clean; 456 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 287 op/s 2024-05-17T09:41:50.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:50 smithi019 ceph-mon[31620]: pgmap v1100: 129 pgs: 129 active+clean; 456 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 287 op/s 2024-05-17T09:41:51.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 56860 6.51 MB/sec execute 348 sec latency 101.116 ms 2024-05-17T09:41:52.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57028 6.51 MB/sec execute 349 sec latency 91.932 ms 2024-05-17T09:41:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:52 smithi079 ceph-mon[35502]: pgmap v1101: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-05-17T09:41:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:52 smithi069 ceph-mon[35081]: pgmap v1101: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-05-17T09:41:52.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:52 smithi019 ceph-mon[31620]: pgmap v1101: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-05-17T09:41:53.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57147 6.52 MB/sec execute 350 sec latency 75.351 ms 2024-05-17T09:41:54.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57309 6.54 MB/sec execute 351 sec latency 108.612 ms 2024-05-17T09:41:54.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:54 smithi019 ceph-mon[31620]: pgmap v1102: 129 pgs: 129 active+clean; 463 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 273 op/s 2024-05-17T09:41:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:54 smithi079 ceph-mon[35502]: pgmap v1102: 129 pgs: 129 active+clean; 463 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 273 op/s 2024-05-17T09:41:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:54 smithi069 ceph-mon[35081]: pgmap v1102: 129 pgs: 129 active+clean; 463 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 273 op/s 2024-05-17T09:41:55.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57491 6.57 MB/sec execute 352 sec latency 94.761 ms 2024-05-17T09:41:56.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57644 6.57 MB/sec execute 353 sec latency 75.538 ms 2024-05-17T09:41:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:56 smithi069 ceph-mon[35081]: pgmap v1103: 129 pgs: 129 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-17T09:41:56.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:56 smithi019 ceph-mon[31620]: pgmap v1103: 129 pgs: 129 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-17T09:41:57.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:56 smithi079 ceph-mon[35502]: pgmap v1103: 129 pgs: 129 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-05-17T09:41:57.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57762 6.57 MB/sec execute 354 sec latency 73.274 ms 2024-05-17T09:41:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 -- 172.21.15.19:0/1035904014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0103c50 msgr2=0x7f60a0104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 --2- 172.21.15.19:0/1035904014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0103c50 0x7f60a0104950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f609c009900 tx=0x7f609c02a410 comp rx=0 tx=0).stop 2024-05-17T09:41:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 -- 172.21.15.19:0/1035904014 shutdown_connections 2024-05-17T09:41:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 --2- 172.21.15.19:0/1035904014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0109b90 0x7f60a010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 --2- 172.21.15.19:0/1035904014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 0x7f60a01052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 --2- 172.21.15.19:0/1035904014 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0103c50 0x7f60a0104950 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.650+0000 7f60a778d640 1 -- 172.21.15.19:0/1035904014 >> 172.21.15.19:0/1035904014 conn(0x7f60a00fee60 msgr2=0x7f60a01012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.651+0000 7f60a778d640 1 -- 172.21.15.19:0/1035904014 shutdown_connections 2024-05-17T09:41:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.651+0000 7f60a778d640 1 -- 172.21.15.19:0/1035904014 wait complete. 2024-05-17T09:41:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.651+0000 7f60a778d640 1 Processor -- start 2024-05-17T09:41:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 -- start start 2024-05-17T09:41:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 0x7f60a019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 0x7f60a019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0109b90 0x7f60a01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a5529640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 0x7f60a019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a5529640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 0x7f60a019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58836/0 (socket says 172.21.15.19:58836) 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a4d28640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 0x7f60a019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a5529640 1 -- 172.21.15.19:0/1515146333 learned_addr learned my addr 172.21.15.19:0/1515146333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60a010f5e0 con 0x7f60a0103c50 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60a010f460 con 0x7f60a0104e90 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60a010f760 con 0x7f60a0109b90 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.652+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0109b90 0x7f60a01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a5529640 1 -- 172.21.15.19:0/1515146333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0109b90 msgr2=0x7f60a01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a5529640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0109b90 0x7f60a01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a5529640 1 -- 172.21.15.19:0/1515146333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 msgr2=0x7f60a019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a5529640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 0x7f60a019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a5529640 1 -- 172.21.15.19:0/1515146333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a01a1cf0 con 0x7f60a0103c50 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a4d28640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 0x7f60a019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a5529640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 0x7f60a019a1b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f609c005760 tx=0x7f609c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60967fc640 1 -- 172.21.15.19:0/1515146333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c03f030 con 0x7f60a0103c50 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60967fc640 1 -- 172.21.15.19:0/1515146333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f609c0043d0 con 0x7f60a0103c50 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a01a1f80 con 0x7f60a0103c50 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.653+0000 7f60967fc640 1 -- 172.21.15.19:0/1515146333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c035070 con 0x7f60a0103c50 2024-05-17T09:41:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.654+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a01a2380 con 0x7f60a0103c50 2024-05-17T09:41:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 Processor -- start 2024-05-17T09:41:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 -- start start 2024-05-17T09:41:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fff1127c30 con 0x55fff0deb2f0 2024-05-17T09:41:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fff11177d0 con 0x55fff0e141b0 2024-05-17T09:41:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60ae64a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fff111b810 con 0x55fff0df34d0 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59492/0 (socket says 172.21.15.19:59492) 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60a5d2a640 1 -- 172.21.15.19:0/2111769978 learned_addr learned my addr 172.21.15.19:0/2111769978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.657+0000 7f60a4d28640 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60967fc640 1 -- 172.21.15.19:0/1515146333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f609c042030 con 0x7f60a0103c50 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a5d2a640 1 -- 172.21.15.19:0/2111769978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 msgr2=0x55fff11c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a5d2a640 1 -- 172.21.15.19:0/2111769978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 msgr2=0x55fff11b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a5d2a640 1 -- 172.21.15.19:0/2111769978 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fff0e2c210 con 0x55fff0e141b0 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a4d28640 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11b97b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6098009fd0 tx=0x7f60980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3a79f0e9516993 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f608b7fe640 1 -- 172.21.15.19:0/2111769978 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098004d20 con 0x55fff0e141b0 2024-05-17T09:41:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.658+0000 7f608b7fe640 1 -- 172.21.15.19:0/2111769978 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6098004050 con 0x55fff0e141b0 2024-05-17T09:41:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.659+0000 7f608b7fe640 1 -- 172.21.15.19:0/2111769978 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098005240 con 0x55fff0e141b0 2024-05-17T09:41:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.659+0000 7f60967fc640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f608c0b28a0 0x7f608c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.659+0000 7f60a5529640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f608c0b28a0 0x7f608c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.659+0000 7f60967fc640 1 -- 172.21.15.19:0/1515146333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f609c1028d0 con 0x7f60a0103c50 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 -- 172.21.15.19:0/2111769978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 msgr2=0x55fff11b97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11b97b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6098009fd0 tx=0x7f60980118d0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60a5529640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f608c0b28a0 0x7f608c0b4db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f609c0c9b80 tx=0x7f609c104040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 -- 172.21.15.19:0/2111769978 shutdown_connections 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2111769978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11b97b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 -- 172.21.15.19:0/2111769978 >> 172.21.15.19:0/2111769978 conn(0x55fff0e08300 msgr2=0x55fff11b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 -- 172.21.15.19:0/2111769978 shutdown_connections 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 -- 172.21.15.19:0/2111769978 wait complete. 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 -- start start 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.660+0000 7f60ae64a740 1 Processor -- start 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fff0dfd790 con 0x55fff0deb2f0 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fff10d3890 con 0x55fff0e141b0 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fff11c1680 con 0x55fff0df34d0 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58850/0 (socket says 172.21.15.19:58850) 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a5d2a640 1 -- 172.21.15.19:0/2924168929 learned_addr learned my addr 172.21.15.19:0/2924168929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60ae64a740 10 client.0 _ll_register_callbacks cb 0x55fff10d6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a4d28640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a4d28640 1 -- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 msgr2=0x55fff11ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a4d28640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a4d28640 1 -- 172.21.15.19:0/2924168929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 msgr2=0x55fff11c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a4d28640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a4d28640 1 -- 172.21.15.19:0/2924168929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fff0ef8620 con 0x55fff0df34d0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.661+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f60a4d28640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c3630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60a019b250 tx=0x7f609000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf08eba0e92ae465 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f6088ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f6088ff9640 1 -- 172.21.15.19:0/2924168929 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090012110 con 0x55fff0df34d0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f6088ff9640 1 -- 172.21.15.19:0/2924168929 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6090004050 con 0x55fff0df34d0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6098004050 con 0x55fff0df34d0 2024-05-17T09:41:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f6088ff9640 1 -- 172.21.15.19:0/2924168929 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f60900122b0 con 0x55fff0df34d0 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f60ae64a740 10 client.28282 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.662+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fff11cd590 con 0x55fff0df34d0 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f6088ff9640 1 -- 172.21.15.19:0/2924168929 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6098004050 con 0x55fff0df34d0 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f6088ff9640 1 -- 172.21.15.19:0/2924168929 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6090010430 con 0x55fff0df34d0 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60ae64a740 10 client.28282 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60ae64a740 10 client.28282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60ae64a740 20 client.28282 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0dd2d80 0x55fff11d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60ae64a740 4 client.28282 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fff1129b80 con 0x55fff0dd2d80 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.663+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0dd2d80 0x55fff11d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.664+0000 7f6088ff9640 10 client.28282 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.664+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0dd2d80 0x55fff11d0d70 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fc99877f956f44 server_cookie=fc73f4404f8677e in_seq=0 out_seq=0 2024-05-17T09:41:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.667+0000 7f6088ff9640 1 -- 172.21.15.19:0/2924168929 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fff1129b80 con 0x55fff0dd2d80 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.667+0000 7f6088ff9640 10 client.28282 handle_command_reply: tid=0 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.667+0000 7f60ae64a740 1 client.28282 shutdown 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 msgr2=0x55fff11c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c3630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60a019b250 tx=0x7f609000f790 comp rx=0 tx=0).stop 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 shutdown_connections 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0dd2d80 0x55fff11d0d70 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0e141b0 0x55fff11ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c3630 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 --2- 172.21.15.19:0/2924168929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0deb2f0 0x55fff11c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 >> 172.21.15.19:0/2924168929 conn(0x55fff0e08300 msgr2=0x55fff11afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f6088ff9640 0 client.28282 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 shutdown_connections 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 1 -- 172.21.15.19:0/2924168929 wait complete. 2024-05-17T09:41:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.668+0000 7f60ae64a740 20 client.28282 trim_cache size 0 max 16384 2024-05-17T09:41:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.699+0000 7f60ae64a740 1 Processor -- start 2024-05-17T09:41:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.699+0000 7f60ae64a740 1 -- start start 2024-05-17T09:41:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60ae64a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01ad090 con 0x55fff0e141b0 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60ae64a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01aa960 con 0x55fff0deb2f0 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60ae64a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01a87b0 con 0x55fff0df34d0 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59510/0 (socket says 172.21.15.19:59510) 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a5d2a640 1 -- 172.21.15.19:0/940794758 learned_addr learned my addr 172.21.15.19:0/940794758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a4d28640 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a4d28640 1 -- 172.21.15.19:0/940794758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 msgr2=0x55fff11c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a4d28640 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a4d28640 1 -- 172.21.15.19:0/940794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 msgr2=0x55fff11ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a4d28640 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.700+0000 7f60a4d28640 1 -- 172.21.15.19:0/940794758 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fff0dd2fb0 con 0x55fff0df34d0 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.701+0000 7f60a4d28640 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11ca4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6090010850 tx=0x7f60900057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2dc7f8108fea2f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.701+0000 7f608affd640 1 -- 172.21.15.19:0/940794758 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090010b80 con 0x55fff0df34d0 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.701+0000 7f608affd640 1 -- 172.21.15.19:0/940794758 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6090005540 con 0x55fff0df34d0 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.701+0000 7f608affd640 1 -- 172.21.15.19:0/940794758 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090003ad0 con 0x55fff0df34d0 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 -- 172.21.15.19:0/940794758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 msgr2=0x55fff11ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11ca4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6090010850 tx=0x7f60900057d0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 -- 172.21.15.19:0/940794758 shutdown_connections 2024-05-17T09:41:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11ca4b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 --2- 172.21.15.19:0/940794758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 -- 172.21.15.19:0/940794758 >> 172.21.15.19:0/940794758 conn(0x55fff0e08300 msgr2=0x55fff11b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 -- 172.21.15.19:0/940794758 shutdown_connections 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 -- 172.21.15.19:0/940794758 wait complete. 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 -- start start 2024-05-17T09:41:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.702+0000 7f60ae64a740 1 Processor -- start 2024-05-17T09:41:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01a8e20 con 0x55fff0e141b0 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01aa290 con 0x55fff0deb2f0 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01a94f0 con 0x55fff0df34d0 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60ae64a740 10 client.0 _ll_register_callbacks cb 0x7f6090012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a4d28640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a5d2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:59514/0 (socket says 172.21.15.19:59514) 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a5d2a640 1 -- 172.21.15.19:0/4233060385 learned_addr learned my addr 172.21.15.19:0/4233060385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:41:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a4d28640 1 -- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 msgr2=0x55fff11cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a4d28640 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a4d28640 1 -- 172.21.15.19:0/4233060385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 msgr2=0x55fff11c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a4d28640 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.703+0000 7f60a4d28640 1 -- 172.21.15.19:0/4233060385 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fff12373a0 con 0x55fff0df34d0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f60a4d28640 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c7790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f60900108d0 tx=0x7f60900106e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7122b1d6f50a2971 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f6088ff9640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f6088ff9640 1 -- 172.21.15.19:0/4233060385 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090036020 con 0x55fff0df34d0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6090005540 con 0x55fff0df34d0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f6088ff9640 1 -- 172.21.15.19:0/4233060385 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6090018040 con 0x55fff0df34d0 2024-05-17T09:41:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f6088ff9640 1 -- 172.21.15.19:0/4233060385 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f609003e050 con 0x55fff0df34d0 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f60ae64a740 10 client.28288 fetch_fsmap learned FSMap version 39 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fff1238570 con 0x55fff0df34d0 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.704+0000 7f6088ff9640 1 -- 172.21.15.19:0/4233060385 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6090005540 con 0x55fff0df34d0 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f6088ff9640 1 -- 172.21.15.19:0/4233060385 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f60900297b0 con 0x55fff0df34d0 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60ae64a740 10 client.28288 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60ae64a740 10 client.28288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60ae64a740 20 client.28288 populate_metadata read hostname 'smithi019' 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60ae64a740 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0da81c0 0x55fff123ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60ae64a740 4 client.28288 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fff1123360 con 0x55fff0da81c0 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0da81c0 0x55fff123ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f6088ff9640 10 client.28288 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.705+0000 7f60a5d2a640 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0da81c0 0x55fff123ae40 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4d6e4523d2f271 server_cookie=f44a3aa7997c77a2 in_seq=0 out_seq=0 2024-05-17T09:41:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f6088ff9640 1 -- 172.21.15.19:0/4233060385 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fff1123360 con 0x55fff0da81c0 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f6088ff9640 10 client.28288 handle_command_reply: tid=0 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 client.28288 shutdown 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 msgr2=0x55fff11c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c7790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f60900108d0 tx=0x7f60900106e0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 shutdown_connections 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55fff0da81c0 0x55fff123ae40 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fff0e141b0 0x55fff11c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fff0df34d0 0x55fff11c7790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f60ae64a740 1 --2- 172.21.15.19:0/4233060385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fff0deb2f0 0x55fff11cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.823+0000 7f6088ff9640 0 client.28288 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.824+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 >> 172.21.15.19:0/4233060385 conn(0x55fff0e08300 msgr2=0x55fff11b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.824+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 shutdown_connections 2024-05-17T09:41:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.824+0000 7f60ae64a740 1 -- 172.21.15.19:0/4233060385 wait complete. 2024-05-17T09:41:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.824+0000 7f60ae64a740 20 client.28288 trim_cache size 0 max 16384 2024-05-17T09:41:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:41:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:41:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:41:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f608c0b28a0 msgr2=0x7f608c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f608c0b28a0 0x7f608c0b4db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f609c0c9b80 tx=0x7f609c104040 comp rx=0 tx=0).stop 2024-05-17T09:41:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 msgr2=0x7f60a019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:41:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 0x7f60a019a1b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f609c005760 tx=0x7f609c005e20 comp rx=0 tx=0).stop 2024-05-17T09:41:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 shutdown_connections 2024-05-17T09:41:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60a0109b90 0x7f60a01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.825+0000 7f60a778d640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60a0104e90 0x7f60a019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.826+0000 7f60a778d640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f608c0b28a0 0x7f608c0b4db0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.826+0000 7f60a778d640 1 --2- 172.21.15.19:0/1515146333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60a0103c50 0x7f60a019a1b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:41:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.826+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 >> 172.21.15.19:0/1515146333 conn(0x7f60a00fee60 msgr2=0x7f60a0105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:41:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.826+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 shutdown_connections 2024-05-17T09:41:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:41:57.826+0000 7f60a778d640 1 -- 172.21.15.19:0/1515146333 wait complete. 2024-05-17T09:41:58.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 57909 6.57 MB/sec execute 355 sec latency 128.724 ms 2024-05-17T09:41:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:41:58 smithi069 ceph-mon[35081]: pgmap v1104: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.4 MiB/s wr, 260 op/s 2024-05-17T09:41:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:41:58 smithi019 ceph-mon[31620]: pgmap v1104: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.4 MiB/s wr, 260 op/s 2024-05-17T09:41:59.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:41:58 smithi079 ceph-mon[35502]: pgmap v1104: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.4 MiB/s wr, 260 op/s 2024-05-17T09:41:59.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58050 6.57 MB/sec execute 356 sec latency 93.740 ms 2024-05-17T09:42:00.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58184 6.56 MB/sec execute 357 sec latency 96.867 ms 2024-05-17T09:42:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:00 smithi069 ceph-mon[35081]: pgmap v1105: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-05-17T09:42:00.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:00 smithi019 ceph-mon[31620]: pgmap v1105: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-05-17T09:42:01.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:00 smithi079 ceph-mon[35502]: pgmap v1105: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-05-17T09:42:01.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58314 6.56 MB/sec execute 358 sec latency 131.314 ms 2024-05-17T09:42:02.018 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-17T09:42:02.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:42:02.573 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:02.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58445 6.56 MB/sec execute 359 sec latency 94.110 ms 2024-05-17T09:42:02.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:02 smithi079 ceph-mon[35502]: pgmap v1106: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 222 op/s 2024-05-17T09:42:02.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:02 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:42:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:02 smithi069 ceph-mon[35081]: pgmap v1106: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 222 op/s 2024-05-17T09:42:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:02 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:42:02.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:02 smithi019 ceph-mon[31620]: pgmap v1106: 129 pgs: 129 active+clean; 472 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 4.4 MiB/s wr, 222 op/s 2024-05-17T09:42:02.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:02 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:42:03.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58568 6.55 MB/sec execute 360 sec latency 97.223 ms 2024-05-17T09:42:04.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58721 6.57 MB/sec execute 361 sec latency 111.702 ms 2024-05-17T09:42:04.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:04 smithi019 ceph-mon[31620]: pgmap v1107: 129 pgs: 129 active+clean; 472 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 4.8 MiB/s wr, 270 op/s 2024-05-17T09:42:04.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:42:04.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:42:04.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:42:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:04 smithi069 ceph-mon[35081]: pgmap v1107: 129 pgs: 129 active+clean; 472 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 4.8 MiB/s wr, 270 op/s 2024-05-17T09:42:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:42:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:42:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:42:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:04 smithi079 ceph-mon[35502]: pgmap v1107: 129 pgs: 129 active+clean; 472 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 4.8 MiB/s wr, 270 op/s 2024-05-17T09:42:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:42:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:42:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 -- 172.21.15.19:0/1038503407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 msgr2=0x7f1edc110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 --2- 172.21.15.19:0/1038503407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc110ba0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400b0a0 tx=0x7f1ed400b3b0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 -- 172.21.15.19:0/1038503407 shutdown_connections 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 --2- 172.21.15.19:0/1038503407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc110ba0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 --2- 172.21.15.19:0/1038503407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1edc1052e0 0x7f1edc1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 --2- 172.21.15.19:0/1038503407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 -- 172.21.15.19:0/1038503407 >> 172.21.15.19:0/1038503407 conn(0x7f1edc078aa0 msgr2=0x7f1edc078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 -- 172.21.15.19:0/1038503407 shutdown_connections 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.248+0000 7f1edb59e700 1 -- 172.21.15.19:0/1038503407 wait complete. 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.249+0000 7f1edb59e700 1 Processor -- start 2024-05-17T09:42:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.249+0000 7f1edb59e700 1 -- start start 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1edc1052e0 0x7f1edc1a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edb59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc1a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edb59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc1adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edb59e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc122f20 con 0x7f1edc1052e0 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edb59e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc122da0 con 0x7f1edc105c30 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edb59e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1230a0 con 0x7f1edc10e6c0 2024-05-17T09:42:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc1adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1ed9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc1a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1ed9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc1a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58662/0 (socket says 172.21.15.19:58662) 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1ed9d9b700 1 -- 172.21.15.19:0/3655576611 learned_addr learned my addr 172.21.15.19:0/3655576611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.250+0000 7f1edad9d700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc1adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46598/0 (socket says 172.21.15.19:46598) 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1eda59c700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1edc1052e0 0x7f1edc1a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ed9d9b700 1 -- 172.21.15.19:0/3655576611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 msgr2=0x7f1edc1adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ed9d9b700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc1adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ed9d9b700 1 -- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1edc1052e0 msgr2=0x7f1edc1a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ed9d9b700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1edc1052e0 0x7f1edc1a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ed9d9b700 1 -- 172.21.15.19:0/3655576611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc1ae5e0 con 0x7f1edc105c30 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ed9d9b700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc1a9cb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000eb30 tx=0x7f1ed000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000cc90 con 0x7f1edc105c30 2024-05-17T09:42:05.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.251+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc1ae870 con 0x7f1edc105c30 2024-05-17T09:42:05.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.252+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ed000ce30 con 0x7f1edc105c30 2024-05-17T09:42:05.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.252+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003e30 con 0x7f1edc105c30 2024-05-17T09:42:05.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.252+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc1b61f0 con 0x7f1edc105c30 2024-05-17T09:42:05.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.253+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc0512b0 con 0x7f1edc105c30 2024-05-17T09:42:05.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.255+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ed0018470 con 0x7f1edc105c30 2024-05-17T09:42:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.256+0000 7f1ecb7fe700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ec40b2940 0x7f1ec40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.256+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1ed00e2790 con 0x7f1edc105c30 2024-05-17T09:42:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.256+0000 7f1eda59c700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ec40b2940 0x7f1ec40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.258+0000 7f1eda59c700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ec40b2940 0x7f1ec40b4e50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00ddc0 tx=0x7f1ecc00f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.262+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ed00aa100 con 0x7f1edc105c30 2024-05-17T09:42:05.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58846 6.56 MB/sec execute 362 sec latency 123.324 ms 2024-05-17T09:42:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.760+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1edc1b63c0 con 0x7f1edc105c30 2024-05-17T09:42:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.761+0000 7f1ecb7fe700 1 -- 172.21.15.19:0/3655576611 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f1ed00a9a00 con 0x7f1edc105c30 2024-05-17T09:42:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:42:05.761 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:42:05.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.764+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ec40b2940 msgr2=0x7f1ec40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:05.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.764+0000 7f1edb59e700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ec40b2940 0x7f1ec40b4e50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00ddc0 tx=0x7f1ecc00f040 comp rx=0 tx=0).stop 2024-05-17T09:42:05.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.764+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 msgr2=0x7f1edc1a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:05.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.764+0000 7f1edb59e700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc1a9cb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000eb30 tx=0x7f1ed000ee40 comp rx=0 tx=0).stop 2024-05-17T09:42:05.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 shutdown_connections 2024-05-17T09:42:05.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1ec40b2940 0x7f1ec40b4e50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1edc10e6c0 0x7f1edc1adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1edc1052e0 0x7f1edc1a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 --2- 172.21.15.19:0/3655576611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1edc105c30 0x7f1edc1a9cb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 >> 172.21.15.19:0/3655576611 conn(0x7f1edc078aa0 msgr2=0x7f1edc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 shutdown_connections 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:05.765+0000 7f1edb59e700 1 -- 172.21.15.19:0/3655576611 wait complete. 2024-05-17T09:42:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:42:06.523 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-17T09:42:06.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-17T09:42:06.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 58989 6.55 MB/sec execute 363 sec latency 112.011 ms 2024-05-17T09:42:06.886 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:06 smithi069 ceph-mon[35081]: pgmap v1108: 129 pgs: 129 active+clean; 470 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 273 op/s 2024-05-17T09:42:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:06 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3655576611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:06.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:06 smithi019 ceph-mon[31620]: pgmap v1108: 129 pgs: 129 active+clean; 470 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 273 op/s 2024-05-17T09:42:06.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3655576611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:06 smithi079 ceph-mon[35502]: pgmap v1108: 129 pgs: 129 active+clean; 470 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 273 op/s 2024-05-17T09:42:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:06 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3655576611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:07.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59149 6.55 MB/sec execute 364 sec latency 83.148 ms 2024-05-17T09:42:08.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59289 6.55 MB/sec execute 365 sec latency 114.882 ms 2024-05-17T09:42:08.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:08 smithi019 ceph-mon[31620]: pgmap v1109: 129 pgs: 129 active+clean; 468 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-05-17T09:42:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:08 smithi069 ceph-mon[35081]: pgmap v1109: 129 pgs: 129 active+clean; 468 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-05-17T09:42:09.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:08 smithi079 ceph-mon[35502]: pgmap v1109: 129 pgs: 129 active+clean; 468 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-05-17T09:42:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 -- 172.21.15.19:0/2258047518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201042a0 msgr2=0x7fe520110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 --2- 172.21.15.19:0/2258047518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201042a0 0x7fe520110060 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe518009f90 tx=0x7fe51800b330 comp rx=0 tx=0).stop 2024-05-17T09:42:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 -- 172.21.15.19:0/2258047518 shutdown_connections 2024-05-17T09:42:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 --2- 172.21.15.19:0/2258047518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201042a0 0x7fe520110060 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 --2- 172.21.15.19:0/2258047518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5201081f0 0x7fe520108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 --2- 172.21.15.19:0/2258047518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe520103950 0x7fe520103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 -- 172.21.15.19:0/2258047518 >> 172.21.15.19:0/2258047518 conn(0x7fe5200fe510 msgr2=0x7fe520100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.103+0000 7fe527fc5700 1 -- 172.21.15.19:0/2258047518 shutdown_connections 2024-05-17T09:42:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.104+0000 7fe527fc5700 1 -- 172.21.15.19:0/2258047518 wait complete. 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 Processor -- start 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 -- start start 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe520103950 0x7fe5201a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5201042a0 0x7fe5201a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 0x7fe52019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe520103610 con 0x7fe5201042a0 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe520103490 con 0x7fe520103950 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe527fc5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe520103790 con 0x7fe5201081f0 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe525560700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5201042a0 0x7fe5201a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe526562700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 0x7fe52019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe526562700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 0x7fe52019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46612/0 (socket says 172.21.15.19:46612) 2024-05-17T09:42:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.105+0000 7fe526562700 1 -- 172.21.15.19:0/3991902490 learned_addr learned my addr 172.21.15.19:0/3991902490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe526562700 1 -- 172.21.15.19:0/3991902490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe520103950 msgr2=0x7fe5201a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe526562700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe520103950 0x7fe5201a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe526562700 1 -- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5201042a0 msgr2=0x7fe5201a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe526562700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5201042a0 0x7fe5201a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe526562700 1 -- 172.21.15.19:0/3991902490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52019b0e0 con 0x7fe5201081f0 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe526562700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 0x7fe52019aa30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b6c0 tx=0x7fe51800bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe516ffd700 1 -- 172.21.15.19:0/3991902490 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51801f030 con 0x7fe5201081f0 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe516ffd700 1 -- 172.21.15.19:0/3991902490 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe518005040 con 0x7fe5201081f0 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe516ffd700 1 -- 172.21.15.19:0/3991902490 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51801b690 con 0x7fe5201081f0 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52019b370 con 0x7fe5201081f0 2024-05-17T09:42:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.106+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52019b800 con 0x7fe5201081f0 2024-05-17T09:42:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.110+0000 7fe516ffd700 1 -- 172.21.15.19:0/3991902490 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe518022030 con 0x7fe5201081f0 2024-05-17T09:42:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 Processor -- start 2024-05-17T09:42:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 -- start start 2024-05-17T09:42:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 0x559f8a018f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89e93fd0 0x559f8a0194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a013580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe516ffd700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe50c0b2900 0x7fe50c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f89f40f00 con 0x559f8a0130b0 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f89f30aa0 con 0x559f89f7f5c0 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.111+0000 7fe5369dab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f89f34ae0 con 0x559f89e93fd0 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 0x559f8a018f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 0x559f8a018f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58676/0 (socket says 172.21.15.19:58676) 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 -- 172.21.15.19:0/266562697 learned_addr learned my addr 172.21.15.19:0/266562697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe526562700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe50c0b2900 0x7fe50c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a013580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525d61700 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89e93fd0 0x559f8a0194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 -- 172.21.15.19:0/266562697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89e93fd0 msgr2=0x559f8a0194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89e93fd0 0x559f8a0194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 -- 172.21.15.19:0/266562697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 msgr2=0x559f8a013580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a013580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 -- 172.21.15.19:0/266562697 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f89eda140 con 0x559f89f7f5c0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe516ffd700 1 -- 172.21.15.19:0/3991902490 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe51800f020 con 0x7fe5201081f0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe525560700 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 0x559f8a018f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5100095f0 tx=0x7fe5100058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d6e364747d9fab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe526562700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe50c0b2900 0x7fe50c0b4e10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe518004fd0 tx=0x7fe518092040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.112+0000 7fe50bfff700 1 -- 172.21.15.19:0/266562697 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510023a60 con 0x559f89f7f5c0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.113+0000 7fe50bfff700 1 -- 172.21.15.19:0/266562697 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5100044c0 con 0x559f89f7f5c0 2024-05-17T09:42:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.113+0000 7fe50bfff700 1 -- 172.21.15.19:0/266562697 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510004d10 con 0x559f89f7f5c0 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 -- 172.21.15.19:0/266562697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 msgr2=0x559f8a018f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 0x559f8a018f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5100095f0 tx=0x7fe5100058e0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 -- 172.21.15.19:0/266562697 shutdown_connections 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a013580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89f7f5c0 0x559f8a018f20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 --2- 172.21.15.19:0/266562697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89e93fd0 0x559f8a0194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 -- 172.21.15.19:0/266562697 >> 172.21.15.19:0/266562697 conn(0x559f89ef4350 msgr2=0x559f8a00dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 -- 172.21.15.19:0/266562697 shutdown_connections 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 -- 172.21.15.19:0/266562697 wait complete. 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 -- start start 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.114+0000 7fe5369dab80 1 Processor -- start 2024-05-17T09:42:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a01cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a01d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f89e37130 con 0x559f8a0130b0 2024-05-17T09:42:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f89e98f30 con 0x559f89e93fd0 2024-05-17T09:42:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f8a01fbb0 con 0x559f89f7f5c0 2024-05-17T09:42:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe5369dab80 10 client.0 _ll_register_callbacks cb 0x559f89ecbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe525560700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a01cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe525560700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a01cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:58680/0 (socket says 172.21.15.19:58680) 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe525560700 1 -- 172.21.15.19:0/1565500181 learned_addr learned my addr 172.21.15.19:0/1565500181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe525d61700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.115+0000 7fe525560700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a01d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525560700 1 -- 172.21.15.19:0/1565500181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 msgr2=0x559f8a01d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525560700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525560700 1 -- 172.21.15.19:0/1565500181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 msgr2=0x559f8a01d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525560700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a01d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525560700 1 -- 172.21.15.19:0/1565500181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f89e80a60 con 0x559f89e93fd0 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525d61700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe525560700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a01cc40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe510004840 tx=0x7fe510009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ddeedb4564229d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe5097fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe5097fa700 1 -- 172.21.15.19:0/1565500181 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510032480 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe5097fa700 1 -- 172.21.15.19:0/1565500181 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe510032620 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5100044c0 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.116+0000 7fe5097fa700 1 -- 172.21.15.19:0/1565500181 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe51003e050 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 10 client.28313 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559f89eea8f0 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5097fa700 1 -- 172.21.15.19:0/1565500181 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe510032cb0 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5097fa700 1 -- 172.21.15.19:0/1565500181 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe51002f820 con 0x559f89e93fd0 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 10 client.28313 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 10 client.28313 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 20 client.28313 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a02cdc0 0x559f8a02f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 4 client.28313 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe5100044c0 con 0x559f8a02cdc0 2024-05-17T09:42:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.117+0000 7fe525d61700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a02cdc0 0x559f8a02f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.118+0000 7fe5097fa700 10 client.28313 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.118+0000 7fe525d61700 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a02cdc0 0x559f8a02f210 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68696d3ca1613c55 server_cookie=3c9930a8682f0baa in_seq=0 out_seq=0 2024-05-17T09:42:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.119+0000 7fe5097fa700 1 -- 172.21.15.19:0/1565500181 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fe5100044c0 con 0x559f8a02cdc0 2024-05-17T09:42:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.119+0000 7fe5097fa700 10 client.28313 handle_command_reply: tid=0 2024-05-17T09:42:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.119+0000 7fe5369dab80 1 client.28313 shutdown 2024-05-17T09:42:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.119+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 msgr2=0x559f8a01cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.119+0000 7fe5369dab80 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a01cc40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe510004840 tx=0x7fe510009830 comp rx=0 tx=0).stop 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 shutdown_connections 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a02cdc0 0x559f8a02f210 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a01d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a01cc40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 --2- 172.21.15.19:0/1565500181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 >> 172.21.15.19:0/1565500181 conn(0x559f89ef4350 msgr2=0x559f8a0120a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5097fa700 0 client.28313 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 shutdown_connections 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 1 -- 172.21.15.19:0/1565500181 wait complete. 2024-05-17T09:42:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.120+0000 7fe5369dab80 20 client.28313 trim_cache size 0 max 16384 2024-05-17T09:42:09.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.160+0000 7fe5369dab80 1 Processor -- start 2024-05-17T09:42:09.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.160+0000 7fe5369dab80 1 -- start start 2024-05-17T09:42:09.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f89e93fd0 0x559f8a028890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f8a0130b0 0x559f8a034a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe5369dab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201b0830 con 0x559f89e93fd0 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe5369dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ae0c0 con 0x559f8a0130b0 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe5369dab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201abf10 con 0x559f89f7f5c0 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe525d61700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe525d61700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46646/0 (socket says 172.21.15.19:46646) 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe525d61700 1 -- 172.21.15.19:0/846262903 learned_addr learned my addr 172.21.15.19:0/846262903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe525560700 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f89e93fd0 0x559f8a028890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.161+0000 7fe525560700 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f8a0130b0 0x559f8a034a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe525d61700 1 -- 172.21.15.19:0/846262903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f8a0130b0 msgr2=0x559f8a034a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe525d61700 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f8a0130b0 0x559f8a034a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe525d61700 1 -- 172.21.15.19:0/846262903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f89e93fd0 msgr2=0x559f8a028890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe525d61700 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f89e93fd0 0x559f8a028890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe525d61700 1 -- 172.21.15.19:0/846262903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8a00ed60 con 0x559f89f7f5c0 2024-05-17T09:42:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe525d61700 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01f0e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe52006ce40 tx=0x7fe51c0124c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=745ea617125b8742 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe50b7fe700 1 -- 172.21.15.19:0/846262903 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c013ce0 con 0x559f89f7f5c0 2024-05-17T09:42:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe50b7fe700 1 -- 172.21.15.19:0/846262903 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe51c004cc0 con 0x559f89f7f5c0 2024-05-17T09:42:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.162+0000 7fe50b7fe700 1 -- 172.21.15.19:0/846262903 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c01f3c0 con 0x559f89f7f5c0 2024-05-17T09:42:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 -- 172.21.15.19:0/846262903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 msgr2=0x559f8a01f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01f0e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe52006ce40 tx=0x7fe51c0124c0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 -- 172.21.15.19:0/846262903 shutdown_connections 2024-05-17T09:42:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f8a0130b0 0x559f8a034a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f89e93fd0 0x559f8a028890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 --2- 172.21.15.19:0/846262903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a01f0e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 -- 172.21.15.19:0/846262903 >> 172.21.15.19:0/846262903 conn(0x559f89ef4350 msgr2=0x559f8a018270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 -- 172.21.15.19:0/846262903 shutdown_connections 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.164+0000 7fe5369dab80 1 -- 172.21.15.19:0/846262903 wait complete. 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 -- start start 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 Processor -- start 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a025200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a027940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a027e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ac580 con 0x559f8a0130b0 2024-05-17T09:42:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201ad9f0 con 0x559f89e93fd0 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe5369dab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201acc50 con 0x559f89f7f5c0 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe525d61700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a027940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe525d61700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a027940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46656/0 (socket says 172.21.15.19:46656) 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.165+0000 7fe525d61700 1 -- 172.21.15.19:0/367994115 learned_addr learned my addr 172.21.15.19:0/367994115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525560700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a025200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525560700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a027e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe5369dab80 10 client.0 _ll_register_callbacks cb 0x559f89ecbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525d61700 1 -- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 msgr2=0x559f8a025200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525d61700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a025200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525d61700 1 -- 172.21.15.19:0/367994115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 msgr2=0x559f8a027e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525d61700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a027e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525d61700 1 -- 172.21.15.19:0/367994115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8a0285b0 con 0x559f89f7f5c0 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525560700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a025200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe525d61700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a027940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe51c004a90 tx=0x7fe51c004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaab774093993666 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe5097fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.166+0000 7fe5097fa700 1 -- 172.21.15.19:0/367994115 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c013ce0 con 0x559f89f7f5c0 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.167+0000 7fe5097fa700 1 -- 172.21.15.19:0/367994115 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe51c005020 con 0x559f89f7f5c0 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.167+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c004cc0 con 0x559f89f7f5c0 2024-05-17T09:42:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.167+0000 7fe5097fa700 1 -- 172.21.15.19:0/367994115 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe51c039050 con 0x559f89f7f5c0 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.167+0000 7fe5369dab80 10 client.28312 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.167+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559f8a097760 con 0x559f89f7f5c0 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.167+0000 7fe5097fa700 1 -- 172.21.15.19:0/367994115 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe51c02c030 con 0x559f89f7f5c0 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5097fa700 1 -- 172.21.15.19:0/367994115 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe51c01f8f0 con 0x559f89f7f5c0 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5369dab80 10 client.28312 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5369dab80 10 client.28312 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5369dab80 20 client.28312 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5369dab80 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a097dd0 0x559f8a09a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5369dab80 4 client.28312 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7fe51c004cc0 con 0x559f8a097dd0 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.168+0000 7fe525560700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a097dd0 0x559f8a09a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.169+0000 7fe5097fa700 10 client.28312 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.169+0000 7fe525560700 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a097dd0 0x559f8a09a1f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d6bcacc6a071b5 server_cookie=51807b024e17306c in_seq=0 out_seq=0 2024-05-17T09:42:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5097fa700 1 -- 172.21.15.19:0/367994115 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe51c004cc0 con 0x559f8a097dd0 2024-05-17T09:42:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5097fa700 10 client.28312 handle_command_reply: tid=0 2024-05-17T09:42:09.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 client.28312 shutdown 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 msgr2=0x559f8a027940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a027940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe51c004a90 tx=0x7fe51c004ac0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 shutdown_connections 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559f8a097dd0 0x559f8a09a1f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f8a0130b0 0x559f8a027e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f89e93fd0 0x559f8a025200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 --2- 172.21.15.19:0/367994115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f89f7f5c0 0x559f8a027940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5097fa700 0 client.28312 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 >> 172.21.15.19:0/367994115 conn(0x559f89ef4350 msgr2=0x559f8a007c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 shutdown_connections 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.530+0000 7fe5369dab80 1 -- 172.21.15.19:0/367994115 wait complete. 2024-05-17T09:42:09.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe5369dab80 20 client.28312 trim_cache size 0 max 16384 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 78, 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "151dc9b1": { 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 78, 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:09.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 130.0, 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:09.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:09.534 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:09.534 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:42:09.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe50c0b2900 msgr2=0x7fe50c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe50c0b2900 0x7fe50c0b4e10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe518004fd0 tx=0x7fe518092040 comp rx=0 tx=0).stop 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 msgr2=0x7fe52019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 0x7fe52019aa30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b6c0 tx=0x7fe51800bca0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 shutdown_connections 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fe50c0b2900 0x7fe50c0b4e10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5201081f0 0x7fe52019aa30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe520103950 0x7fe5201a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 --2- 172.21.15.19:0/3991902490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5201042a0 0x7fe5201a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.531+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 >> 172.21.15.19:0/3991902490 conn(0x7fe5200fe510 msgr2=0x7fe520109680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.532+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 shutdown_connections 2024-05-17T09:42:09.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:09.532+0000 7fe527fc5700 1 -- 172.21.15.19:0/3991902490 wait complete. 2024-05-17T09:42:09.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.54 MB/sec execute 366 sec latency 417.628 ms 2024-05-17T09:42:10.234 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '151dc9b1', quiesce took 8.2 seconds. Will release after: 1.8000000000000007 2024-05-17T09:42:10.590 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.52 MB/sec execute 367 sec latency 1417.727 ms 2024-05-17T09:42:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:10 smithi069 ceph-mon[35081]: pgmap v1110: 129 pgs: 129 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-05-17T09:42:10.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:10 smithi019 ceph-mon[31620]: pgmap v1110: 129 pgs: 129 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3130559090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178104e90 msgr2=0x7fc1781052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3130559090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178104e90 0x7fc1781052a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc168009900 tx=0x7fc16802a410 comp rx=0 tx=0).stop 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3130559090 shutdown_connections 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3130559090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc178109b90 0x7fc17810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3130559090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178104e90 0x7fc1781052a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3130559090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc178103c50 0x7fc178104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3130559090 >> 172.21.15.19:0/3130559090 conn(0x7fc1780fee60 msgr2=0x7fc1781012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3130559090 shutdown_connections 2024-05-17T09:42:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.963+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3130559090 wait complete. 2024-05-17T09:42:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 Processor -- start 2024-05-17T09:42:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 -- start start 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 0x7fc178077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc178104e90 0x7fc178076120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc178109b90 0x7fc178076660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810f650 con 0x7fc178103c50 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810f4d0 con 0x7fc178109b90 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17fbc7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810f7d0 con 0x7fc178104e90 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 0x7fc178077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 0x7fc178077a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52172/0 (socket says 172.21.15.19:52172) 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 -- 172.21.15.19:0/3451397186 learned_addr learned my addr 172.21.15.19:0/3451397186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17e164640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc178109b90 0x7fc178076660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d162640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc178104e90 0x7fc178076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 -- 172.21.15.19:0/3451397186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc178104e90 msgr2=0x7fc178076120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc178104e90 0x7fc178076120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 -- 172.21.15.19:0/3451397186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc178109b90 msgr2=0x7fc178076660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc178109b90 0x7fc178076660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.964+0000 7fc17d963640 1 -- 172.21.15.19:0/3451397186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc178076d90 con 0x7fc178103c50 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.965+0000 7fc17d963640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 0x7fc178077a80 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc174005fd0 tx=0x7fc17400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.965+0000 7fc16effd640 1 -- 172.21.15.19:0/3451397186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174017070 con 0x7fc178103c50 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.965+0000 7fc16effd640 1 -- 172.21.15.19:0/3451397186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc174009c30 con 0x7fc178103c50 2024-05-17T09:42:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.965+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1781a4260 con 0x7fc178103c50 2024-05-17T09:42:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.965+0000 7fc16effd640 1 -- 172.21.15.19:0/3451397186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174003c90 con 0x7fc178103c50 2024-05-17T09:42:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.965+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1781a4750 con 0x7fc178103c50 2024-05-17T09:42:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 Processor -- start 2024-05-17T09:42:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 -- start start 2024-05-17T09:42:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc16effd640 1 -- 172.21.15.19:0/3451397186 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc174003e30 con 0x7fc178103c50 2024-05-17T09:42:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95761b0 0x55b7c99173a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9917920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 0x55b7c991e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9889c30 con 0x55b7c95554d0 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b7c98797d0 con 0x55b7c954d2f0 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc186a78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b7c987d810 con 0x55b7c95761b0 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc17d162640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 0x55b7c991e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc17d162640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 0x55b7c991e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52176/0 (socket says 172.21.15.19:52176) 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc17d162640 1 -- 172.21.15.19:0/676365918 learned_addr learned my addr 172.21.15.19:0/676365918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc17e164640 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95761b0 0x55b7c99173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.968+0000 7fc17d963640 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9917920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17e164640 1 -- 172.21.15.19:0/676365918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 msgr2=0x55b7c9917920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17e164640 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9917920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17e164640 1 -- 172.21.15.19:0/676365918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 msgr2=0x55b7c991e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17e164640 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 0x55b7c991e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17e164640 1 -- 172.21.15.19:0/676365918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c958e210 con 0x55b7c95761b0 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17d162640 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 0x55b7c991e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc16effd640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc1640b2920 0x7fc1640b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17e164640 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95761b0 0x55b7c99173a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc170009eb0 tx=0x7fc170010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cab47702f11f98d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc17d963640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc1640b2920 0x7fc1640b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc163fff640 1 -- 172.21.15.19:0/676365918 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170016070 con 0x55b7c95761b0 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc163fff640 1 -- 172.21.15.19:0/676365918 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc170004d20 con 0x55b7c95761b0 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc163fff640 1 -- 172.21.15.19:0/676365918 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700126b0 con 0x55b7c95761b0 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.969+0000 7fc16effd640 1 -- 172.21.15.19:0/3451397186 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc1740e2240 con 0x7fc178103c50 2024-05-17T09:42:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 -- 172.21.15.19:0/676365918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95761b0 msgr2=0x55b7c99173a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95761b0 0x55b7c99173a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc170009eb0 tx=0x7fc170010740 comp rx=0 tx=0).stop 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc17d963640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc1640b2920 0x7fc1640b4e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55b7c9918780 tx=0x7fc174091040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 -- 172.21.15.19:0/676365918 shutdown_connections 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c95554d0 0x55b7c991e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9917920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 --2- 172.21.15.19:0/676365918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95761b0 0x55b7c99173a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 -- 172.21.15.19:0/676365918 >> 172.21.15.19:0/676365918 conn(0x55b7c956a300 msgr2=0x55b7c9912d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 -- 172.21.15.19:0/676365918 shutdown_connections 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.970+0000 7fc186a78740 1 -- 172.21.15.19:0/676365918 wait complete. 2024-05-17T09:42:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 -- start start 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 Processor -- start 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9920d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9921280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c95761b0 0x55b7c99281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b7c955f790 con 0x55b7c954d2f0 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9835890 con 0x55b7c95761b0 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b7c991f270 con 0x55b7c95554d0 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc186a78740 10 client.0 _ll_register_callbacks cb 0x55b7c9838d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc17e164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9920d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc17e164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9920d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52188/0 (socket says 172.21.15.19:52188) 2024-05-17T09:42:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc17e164640 1 -- 172.21.15.19:0/172809907 learned_addr learned my addr 172.21.15.19:0/172809907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc17e164640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c95761b0 0x55b7c99281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.971+0000 7fc17d162640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9921280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc17e164640 1 -- 172.21.15.19:0/172809907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 msgr2=0x55b7c9921280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc17e164640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9921280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc17e164640 1 -- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c95761b0 msgr2=0x55b7c99281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc17e164640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c95761b0 0x55b7c99281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc17e164640 1 -- 172.21.15.19:0/172809907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c965a620 con 0x55b7c954d2f0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc17e164640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9920d40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc1700098b0 tx=0x7fc170005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4982eff740e10f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc1617fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc1617fa640 1 -- 172.21.15.19:0/172809907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170016070 con 0x55b7c954d2f0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc1617fa640 1 -- 172.21.15.19:0/172809907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc170012450 con 0x55b7c954d2f0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc170004d20 con 0x55b7c954d2f0 2024-05-17T09:42:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc1617fa640 1 -- 172.21.15.19:0/172809907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc170012650 con 0x55b7c954d2f0 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc186a78740 10 client.18819 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b7c992b110 con 0x55b7c954d2f0 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.972+0000 7fc1617fa640 1 -- 172.21.15.19:0/172809907 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc170004d20 con 0x55b7c954d2f0 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc1617fa640 1 -- 172.21.15.19:0/172809907 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc17000f040 con 0x55b7c954d2f0 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc186a78740 10 client.18819 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc186a78740 10 client.18819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc186a78740 20 client.18819 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc186a78740 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c9534d80 0x55b7c992e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc186a78740 4 client.18819 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c988bb80 con 0x55b7c9534d80 2024-05-17T09:42:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc17d162640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c9534d80 0x55b7c992e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc1617fa640 10 client.18819 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.973+0000 7fc17d162640 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c9534d80 0x55b7c992e530 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2a0a3ee765f439 server_cookie=24771929fc4e6cd8 in_seq=0 out_seq=0 2024-05-17T09:42:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc1617fa640 1 -- 172.21.15.19:0/172809907 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b7c988bb80 con 0x55b7c9534d80 2024-05-17T09:42:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc1617fa640 10 client.18819 handle_command_reply: tid=0 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 client.18819 shutdown 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 msgr2=0x55b7c9920d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9920d40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc1700098b0 tx=0x7fc170005a90 comp rx=0 tx=0).stop 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 shutdown_connections 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c9534d80 0x55b7c992e530 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c95761b0 0x55b7c99281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9921280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc1617fa640 0 client.18819 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 --2- 172.21.15.19:0/172809907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9920d40 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.974+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 >> 172.21.15.19:0/172809907 conn(0x55b7c956a300 msgr2=0x55b7c9915c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.975+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 shutdown_connections 2024-05-17T09:42:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.975+0000 7fc186a78740 1 -- 172.21.15.19:0/172809907 wait complete. 2024-05-17T09:42:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.975+0000 7fc186a78740 20 client.18819 trim_cache size 0 max 16384 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 Processor -- start 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 -- start start 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 0x55b7c991b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9928170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c99286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781b1580 con 0x55b7c9534d80 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781aee50 con 0x55b7c954d2f0 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc186a78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781acca0 con 0x55b7c95554d0 2024-05-17T09:42:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc17e164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 0x55b7c991b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc17e164640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 0x55b7c991b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52202/0 (socket says 172.21.15.19:52202) 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.990+0000 7fc17e164640 1 -- 172.21.15.19:0/1410426094 learned_addr learned my addr 172.21.15.19:0/1410426094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17d162640 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9928170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17e164640 1 -- 172.21.15.19:0/1410426094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 msgr2=0x55b7c99286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17e164640 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c99286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17e164640 1 -- 172.21.15.19:0/1410426094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 msgr2=0x55b7c9928170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17e164640 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9928170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17e164640 1 -- 172.21.15.19:0/1410426094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c991e6c0 con 0x55b7c9534d80 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17d162640 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9928170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc17e164640 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 0x55b7c991b8f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc170009880 tx=0x7fc170005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71a8605a042eacf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc1637fe640 1 -- 172.21.15.19:0/1410426094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170016070 con 0x55b7c9534d80 2024-05-17T09:42:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc1637fe640 1 -- 172.21.15.19:0/1410426094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17001f030 con 0x55b7c9534d80 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.991+0000 7fc1637fe640 1 -- 172.21.15.19:0/1410426094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170003970 con 0x55b7c9534d80 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 -- 172.21.15.19:0/1410426094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 msgr2=0x55b7c991b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 0x55b7c991b8f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc170009880 tx=0x7fc170005200 comp rx=0 tx=0).stop 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 -- 172.21.15.19:0/1410426094 shutdown_connections 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c99286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c954d2f0 0x55b7c9928170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 --2- 172.21.15.19:0/1410426094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9534d80 0x55b7c991b8f0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 -- 172.21.15.19:0/1410426094 >> 172.21.15.19:0/1410426094 conn(0x55b7c956a300 msgr2=0x55b7c99161a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 -- 172.21.15.19:0/1410426094 shutdown_connections 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 -- 172.21.15.19:0/1410426094 wait complete. 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 -- start start 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.992+0000 7fc186a78740 1 Processor -- start 2024-05-17T09:42:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c9534d80 0x55b7c99273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9927930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9927e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ad310 con 0x55b7c954d2f0 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ae780 con 0x55b7c9534d80 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ad9e0 con 0x55b7c95554d0 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc186a78740 10 client.0 _ll_register_callbacks cb 0x7fc170009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9927930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9927930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52208/0 (socket says 172.21.15.19:52208) 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 -- 172.21.15.19:0/1407719104 learned_addr learned my addr 172.21.15.19:0/1407719104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17e164640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c9534d80 0x55b7c99273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17e164640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9927e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 -- 172.21.15.19:0/1407719104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 msgr2=0x55b7c9927e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9927e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 -- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c9534d80 msgr2=0x55b7c99273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c9534d80 0x55b7c99273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 -- 172.21.15.19:0/1407719104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c9994d00 con 0x55b7c954d2f0 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17e164640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9927e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc17d162640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9927930 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc1680057c0 tx=0x7fc168004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83eed05357dc265 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc1617fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.993+0000 7fc1617fa640 1 -- 172.21.15.19:0/1407719104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168049070 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc1617fa640 1 -- 172.21.15.19:0/1407719104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc168003a00 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17001f030 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc1617fa640 1 -- 172.21.15.19:0/1407719104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc168055050 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 10 client.18825 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b7c9996090 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc1617fa640 1 -- 172.21.15.19:0/1407719104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17001f030 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc1617fa640 1 -- 172.21.15.19:0/1407719104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc1680452c0 con 0x55b7c954d2f0 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 10 client.18825 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 10 client.18825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 20 client.18825 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c95761b0 0x55b7c9998a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 4 client.18825 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:42:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7c9885360 con 0x55b7c95761b0 2024-05-17T09:42:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.994+0000 7fc17e164640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c95761b0 0x55b7c9998a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.995+0000 7fc17e164640 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c95761b0 0x55b7c9998a80 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b5200e53c61e78 server_cookie=6727dfd43911e156 in_seq=0 out_seq=0 2024-05-17T09:42:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:10.995+0000 7fc1617fa640 10 client.18825 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:11.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:10 smithi079 ceph-mon[35502]: pgmap v1110: 129 pgs: 129 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-05-17T09:42:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc1617fa640 1 -- 172.21.15.19:0/1407719104 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b7c9885360 con 0x55b7c95761b0 2024-05-17T09:42:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc1617fa640 10 client.18825 handle_command_reply: tid=0 2024-05-17T09:42:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 client.18825 shutdown 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 msgr2=0x55b7c9927930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9927930 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc1680057c0 tx=0x7fc168004560 comp rx=0 tx=0).stop 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 shutdown_connections 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b7c95761b0 0x55b7c9998a80 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b7c95554d0 0x55b7c9927e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c954d2f0 0x55b7c9927930 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc1617fa640 0 client.18825 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 --2- 172.21.15.19:0/1407719104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7c9534d80 0x55b7c99273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 >> 172.21.15.19:0/1407719104 conn(0x55b7c956a300 msgr2=0x55b7c99161a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 shutdown_connections 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.064+0000 7fc186a78740 1 -- 172.21.15.19:0/1407719104 wait complete. 2024-05-17T09:42:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc186a78740 20 client.18825 trim_cache size 0 max 16384 2024-05-17T09:42:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:42:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:42:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc1640b2920 msgr2=0x7fc1640b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc1640b2920 0x7fc1640b4e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55b7c9918780 tx=0x7fc174091040 comp rx=0 tx=0).stop 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 msgr2=0x7fc178077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 0x7fc178077a80 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc174005fd0 tx=0x7fc17400dde0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 shutdown_connections 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc178109b90 0x7fc178076660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc178104e90 0x7fc178076120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc1640b2920 0x7fc1640b4e30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 --2- 172.21.15.19:0/3451397186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc178103c50 0x7fc178077a80 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 >> 172.21.15.19:0/3451397186 conn(0x7fc1780fee60 msgr2=0x7fc178105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 shutdown_connections 2024-05-17T09:42:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:11.065+0000 7fc17fbc7640 1 -- 172.21.15.19:0/3451397186 wait complete. 2024-05-17T09:42:11.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.50 MB/sec execute 368 sec latency 2417.830 ms 2024-05-17T09:42:12.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:42:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:12.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.49 MB/sec execute 369 sec latency 3417.895 ms 2024-05-17T09:42:12.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:12 smithi019 ceph-mon[31620]: pgmap v1111: 129 pgs: 129 active+clean; 464 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.8 MiB/s wr, 244 op/s 2024-05-17T09:42:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:12 smithi069 ceph-mon[35081]: pgmap v1111: 129 pgs: 129 active+clean; 464 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.8 MiB/s wr, 244 op/s 2024-05-17T09:42:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:12 smithi079 ceph-mon[35502]: pgmap v1111: 129 pgs: 129 active+clean; 464 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.8 MiB/s wr, 244 op/s 2024-05-17T09:42:13.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.47 MB/sec execute 370 sec latency 4417.987 ms 2024-05-17T09:42:14.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 -- 172.21.15.19:0/3412311345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15501034d0 msgr2=0x7f15501038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:14.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 --2- 172.21.15.19:0/3412311345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15501034d0 0x7f15501038e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f15400099c0 tx=0x7f1540009cd0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 -- 172.21.15.19:0/3412311345 shutdown_connections 2024-05-17T09:42:14.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 --2- 172.21.15.19:0/3412311345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1550108750 0x7f15501100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 --2- 172.21.15.19:0/3412311345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15501034d0 0x7f15501038e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 --2- 172.21.15.19:0/3412311345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1550107e00 0x7f1550108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 -- 172.21.15.19:0/3412311345 >> 172.21.15.19:0/3412311345 conn(0x7f15500fef20 msgr2=0x7f1550101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:14.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 -- 172.21.15.19:0/3412311345 shutdown_connections 2024-05-17T09:42:14.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.045+0000 7f1554df8700 1 -- 172.21.15.19:0/3412311345 wait complete. 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.046+0000 7f1554df8700 1 Processor -- start 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.046+0000 7f1554df8700 1 -- start start 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.046+0000 7f1554df8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15501034d0 0x7f155019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.046+0000 7f1554df8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1550107e00 0x7f155019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f1554df8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 0x7f15501a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f1554df8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1550113230 con 0x7f1550108750 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f1554df8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15501130b0 con 0x7f1550107e00 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f1554df8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f15501133b0 con 0x7f15501034d0 2024-05-17T09:42:14.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 0x7f15501a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 0x7f15501a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54636/0 (socket says 172.21.15.19:54636) 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 -- 172.21.15.19:0/2627767526 learned_addr learned my addr 172.21.15.19:0/2627767526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f1545d9b700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1550107e00 0x7f155019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154e59c700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15501034d0 0x7f155019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 -- 172.21.15.19:0/2627767526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15501034d0 msgr2=0x7f155019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15501034d0 0x7f155019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 -- 172.21.15.19:0/2627767526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1550107e00 msgr2=0x7f155019c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1550107e00 0x7f155019c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 -- 172.21.15.19:0/2627767526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155019c980 con 0x7f1550108750 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.047+0000 7f154ed9d700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 0x7f15501a6c10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f154800ada0 tx=0x7f154800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.048+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548017020 con 0x7f1550108750 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.048+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15501a7270 con 0x7f1550108750 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.048+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f154800cd60 con 0x7f1550108750 2024-05-17T09:42:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.048+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548005280 con 0x7f1550108750 2024-05-17T09:42:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.048+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15501a7730 con 0x7f1550108750 2024-05-17T09:42:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.049+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1550104980 con 0x7f1550108750 2024-05-17T09:42:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.050+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f154800c8d0 con 0x7f1550108750 2024-05-17T09:42:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.051+0000 7f1547fff700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f153c0b2880 0x7f153c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:14.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.051+0000 7f154e59c700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f153c0b2880 0x7f153c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:14.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.051+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f15480e22d0 con 0x7f1550108750 2024-05-17T09:42:14.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.052+0000 7f154e59c700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f153c0b2880 0x7f153c0b4d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1540006010 tx=0x7f1540009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.055+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1548021020 con 0x7f1550108750 2024-05-17T09:42:14.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.337+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f155002e5e0 con 0x7f1550108750 2024-05-17T09:42:14.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.338+0000 7f1547fff700 1 -- 172.21.15.19:0/2627767526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f15480a9c70 con 0x7f1550108750 2024-05-17T09:42:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:42:14.337 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:42:14.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f153c0b2880 msgr2=0x7f153c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:14.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f153c0b2880 0x7f153c0b4d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1540006010 tx=0x7f1540009f10 comp rx=0 tx=0).stop 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 msgr2=0x7f15501a6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 0x7f15501a6c10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f154800ada0 tx=0x7f154800c330 comp rx=0 tx=0).stop 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 shutdown_connections 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f153c0b2880 0x7f153c0b4d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15501034d0 0x7f155019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1550108750 0x7f15501a6c10 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 --2- 172.21.15.19:0/2627767526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1550107e00 0x7f155019c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 >> 172.21.15.19:0/2627767526 conn(0x7f15500fef20 msgr2=0x7f1550101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 shutdown_connections 2024-05-17T09:42:14.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:14.340+0000 7f1554df8700 1 -- 172.21.15.19:0/2627767526 wait complete. 2024-05-17T09:42:14.341 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:42:14.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.45 MB/sec execute 371 sec latency 5418.063 ms 2024-05-17T09:42:14.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:14 smithi019 ceph-mon[31620]: pgmap v1112: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-17T09:42:14.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:14 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2627767526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:14 smithi069 ceph-mon[35081]: pgmap v1112: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-17T09:42:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:14 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2627767526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:15.010 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 151dc9b1 --release --await' 2024-05-17T09:42:15.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 151dc9b1 --release --await 2024-05-17T09:42:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:14 smithi079 ceph-mon[35502]: pgmap v1112: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-05-17T09:42:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:14 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2627767526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:15.261 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:15.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.43 MB/sec execute 372 sec latency 6418.140 ms 2024-05-17T09:42:16.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59378 6.42 MB/sec execute 373 sec latency 7418.382 ms 2024-05-17T09:42:16.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:16 smithi019 ceph-mon[31620]: pgmap v1113: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-05-17T09:42:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:16 smithi069 ceph-mon[35081]: pgmap v1113: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-05-17T09:42:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 -- 172.21.15.19:0/3218081116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd410e8e0 msgr2=0x7fbfd4110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 --2- 172.21.15.19:0/3218081116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4110dc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009990 tx=0x7fbfcc009ca0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 -- 172.21.15.19:0/3218081116 shutdown_connections 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 --2- 172.21.15.19:0/3218081116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4110dc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 --2- 172.21.15.19:0/3218081116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4105470 0x7fbfd4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 --2- 172.21.15.19:0/3218081116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfd4105dc0 0x7fbfd410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 -- 172.21.15.19:0/3218081116 >> 172.21.15.19:0/3218081116 conn(0x7fbfd40fef80 msgr2=0x7fbfd41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 -- 172.21.15.19:0/3218081116 shutdown_connections 2024-05-17T09:42:17.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.019+0000 7fbfdc049700 1 -- 172.21.15.19:0/3218081116 wait complete. 2024-05-17T09:42:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 Processor -- start 2024-05-17T09:42:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 -- start start 2024-05-17T09:42:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 0x7fbfd4103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfd4105dc0 0x7fbfd41036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41142f0 con 0x7fbfd4105470 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4114170 con 0x7fbfd410e8e0 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfdc049700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4114470 con 0x7fbfd4105dc0 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfd9de5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 0x7fbfd4103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfd9de5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 0x7fbfd4103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54656/0 (socket says 172.21.15.19:54656) 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfd9de5700 1 -- 172.21.15.19:0/4281484232 learned_addr learned my addr 172.21.15.19:0/4281484232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfd4105dc0 0x7fbfd41036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.020+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd9de5700 1 -- 172.21.15.19:0/4281484232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfd4105dc0 msgr2=0x7fbfd41036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfd4105dc0 0x7fbfd41036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd9de5700 1 -- 172.21.15.19:0/4281484232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd410e8e0 msgr2=0x7fbfd4103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd9de5700 1 -- 172.21.15.19:0/4281484232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd41a8fe0 con 0x7fbfd4105470 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 0x7fbfd4103160 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40060b0 tx=0x7fbfc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd2ffd700 1 -- 172.21.15.19:0/4281484232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc401a070 con 0x7fbfd4105470 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd2ffd700 1 -- 172.21.15.19:0/4281484232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc400ddd0 con 0x7fbfd4105470 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfd2ffd700 1 -- 172.21.15.19:0/4281484232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4014690 con 0x7fbfd4105470 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd41a92d0 con 0x7fbfd4105470 2024-05-17T09:42:17.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.021+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd41a9810 con 0x7fbfd4105470 2024-05-17T09:42:17.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:16 smithi079 ceph-mon[35502]: pgmap v1113: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-05-17T09:42:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.024+0000 7fbfd2ffd700 1 -- 172.21.15.19:0/4281484232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbfc401d070 con 0x7fbfd4105470 2024-05-17T09:42:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.025+0000 7fbfd2ffd700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbfbc0b2880 0x7fbfbc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.025+0000 7fbfeaa5eb80 1 Processor -- start 2024-05-17T09:42:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.025+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbfbc0b2880 0x7fbfbc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.025+0000 7fbfd2ffd700 1 -- 172.21.15.19:0/4281484232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbfc40e1e70 con 0x7fbfd4105470 2024-05-17T09:42:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 -- start start 2024-05-17T09:42:17.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbfbc0b2880 0x7fbfbc0b4d90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4101e40 tx=0x7fbfc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:17.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ef94d0 0x564494000560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494000aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ffa620 0x564493ffaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564493f28d90 con 0x564493e7bf40 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564493f18930 con 0x564493ffa620 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564493f1c970 con 0x564493ef94d0 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494000aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494000aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54666/0 (socket says 172.21.15.19:54666) 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3217662355 learned_addr learned my addr 172.21.15.19:0/3217662355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ef94d0 0x564494000560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ffa620 0x564493ffaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3217662355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ef94d0 msgr2=0x564494000560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ef94d0 0x564494000560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3217662355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ffa620 msgr2=0x564493ffaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ffa620 0x564493ffaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.026+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3217662355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564493e689d0 con 0x564493e7bf40 2024-05-17T09:42:17.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.027+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494000aa0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8010e30 tx=0x7fbfc8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba4d4b916843e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:17.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.027+0000 7fbfbbfff700 1 -- 172.21.15.19:0/3217662355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8011db0 con 0x564493e7bf40 2024-05-17T09:42:17.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.027+0000 7fbfbbfff700 1 -- 172.21.15.19:0/3217662355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc8010e60 con 0x564493e7bf40 2024-05-17T09:42:17.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.027+0000 7fbfbbfff700 1 -- 172.21.15.19:0/3217662355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc801c610 con 0x564493e7bf40 2024-05-17T09:42:17.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3217662355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 msgr2=0x564494000aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494000aa0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8010e30 tx=0x7fbfc8013000 comp rx=0 tx=0).stop 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3217662355 shutdown_connections 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ef94d0 0x564494000560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ffa620 0x564493ffaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3217662355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494000aa0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3217662355 >> 172.21.15.19:0/3217662355 conn(0x564493ebad80 msgr2=0x564493ff5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3217662355 shutdown_connections 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3217662355 wait complete. 2024-05-17T09:42:17.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 -- start start 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.028+0000 7fbfeaa5eb80 1 Processor -- start 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x5644940041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564494004730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494004c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564493e1f0a0 con 0x564493ef94d0 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564493eba8c0 con 0x564493e7bf40 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5644940103d0 con 0x564493ffa620 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfeaa5eb80 10 client.0 _ll_register_callbacks cb 0x564493eb3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564494004730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564494004730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54674/0 (socket says 172.21.15.19:54674) 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 -- 172.21.15.19:0/4161919752 learned_addr learned my addr 172.21.15.19:0/4161919752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x5644940041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494004c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 -- 172.21.15.19:0/4161919752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 msgr2=0x564494004c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494004c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 -- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 msgr2=0x5644940041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x5644940041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.029+0000 7fbfd95e4700 1 -- 172.21.15.19:0/4161919752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564493eb7790 con 0x564493ef94d0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564494004730 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8010e00 tx=0x7fbfc8012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d846370a73ca6c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfb97fa700 1 -- 172.21.15.19:0/4161919752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8036060 con 0x564493ef94d0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfb97fa700 1 -- 172.21.15.19:0/4161919752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbfc8005530 con 0x564493ef94d0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc8010e60 con 0x564493ef94d0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfb97fa700 1 -- 172.21.15.19:0/4161919752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbfc803d050 con 0x564493ef94d0 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfeaa5eb80 10 client.18843 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564494011c80 con 0x564493ef94d0 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfb97fa700 1 -- 172.21.15.19:0/4161919752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc8010e60 con 0x564493ef94d0 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfb97fa700 1 -- 172.21.15.19:0/4161919752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fbfc801e540 con 0x564493ef94d0 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfeaa5eb80 10 client.18843 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.030+0000 7fbfeaa5eb80 10 client.18843 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfeaa5eb80 20 client.18843 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494014200 0x564494016620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfeaa5eb80 4 client.18843 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564493f2ace0 con 0x564494014200 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494014200 0x564494016620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfb97fa700 10 client.18843 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:17.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.031+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494014200 0x564494016620 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfdf2e634b74b3e server_cookie=6c45c07b7daa8d81 in_seq=0 out_seq=0 2024-05-17T09:42:17.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfb97fa700 1 -- 172.21.15.19:0/4161919752 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564493f2ace0 con 0x564494014200 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfb97fa700 10 client.18843 handle_command_reply: tid=0 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 client.18843 shutdown 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 msgr2=0x564494004730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564494004730 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8010e00 tx=0x7fbfc8012ef0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 shutdown_connections 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494004c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494014200 0x564494016620 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x5644940041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/4161919752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564494004730 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfb97fa700 0 client.18843 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 >> 172.21.15.19:0/4161919752 conn(0x564493ebad80 msgr2=0x564493fef280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 shutdown_connections 2024-05-17T09:42:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.032+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/4161919752 wait complete. 2024-05-17T09:42:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.033+0000 7fbfeaa5eb80 20 client.18843 trim_cache size 0 max 16384 2024-05-17T09:42:17.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 Processor -- start 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 -- start start 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x564493fef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564493ffe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564493ff9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41b4db0 con 0x564493ef94d0 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41b2640 con 0x564493e7bf40 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41b0490 con 0x564493ffa620 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfd95e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564493ffe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfd95e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564493ffe8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54682/0 (socket says 172.21.15.19:54682) 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.053+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3163482861 learned_addr learned my addr 172.21.15.19:0/3163482861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564493ff9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x564493fef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3163482861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 msgr2=0x564493ff9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564493ff9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3163482861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 msgr2=0x564493fef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x564493fef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd95e4700 1 -- 172.21.15.19:0/3163482861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564494002760 con 0x564493ef94d0 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x564493fef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564493ffe8a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc8010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cd0a302d7edf4f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfbb7fe700 1 -- 172.21.15.19:0/3163482861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8036040 con 0x564493ef94d0 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfbb7fe700 1 -- 172.21.15.19:0/3163482861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc80050f0 con 0x564493ef94d0 2024-05-17T09:42:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.054+0000 7fbfbb7fe700 1 -- 172.21.15.19:0/3163482861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8011d10 con 0x564493ef94d0 2024-05-17T09:42:17.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3163482861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 msgr2=0x564493ffe8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564493ffe8a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc8010960 comp rx=0 tx=0).stop 2024-05-17T09:42:17.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3163482861 shutdown_connections 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564493ff9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493e7bf40 0x564493fef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/3163482861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493ef94d0 0x564493ffe8a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3163482861 >> 172.21.15.19:0/3163482861 conn(0x564493ebad80 msgr2=0x564493ff9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3163482861 shutdown_connections 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/3163482861 wait complete. 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.055+0000 7fbfeaa5eb80 1 -- start start 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 Processor -- start 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494018390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ef94d0 0x5644940188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494018e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41b0b00 con 0x564493e7bf40 2024-05-17T09:42:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41b1f70 con 0x564493ef94d0 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41b11d0 con 0x564493ffa620 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfeaa5eb80 10 client.0 _ll_register_callbacks cb 0x564493eb3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494018390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494018390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54694/0 (socket says 172.21.15.19:54694) 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 -- 172.21.15.19:0/2909164657 learned_addr learned my addr 172.21.15.19:0/2909164657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ef94d0 0x5644940188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494018e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 -- 172.21.15.19:0/2909164657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 msgr2=0x564494018e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494018e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 -- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ef94d0 msgr2=0x5644940188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.056+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ef94d0 0x5644940188d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfd9de5700 1 -- 172.21.15.19:0/2909164657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56449407ea80 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfda5e6700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494018e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfd9de5700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494018390 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40a8d80 tx=0x7fbfc40a89d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae1b5b1453ac439 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfb97fa700 1 -- 172.21.15.19:0/2909164657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40b1ee0 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfb97fa700 1 -- 172.21.15.19:0/2909164657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbfc40a5310 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc80050f0 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfb97fa700 1 -- 172.21.15.19:0/2909164657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbfc40a92e0 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfeaa5eb80 10 client.18849 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56449407fcd0 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfb97fa700 1 -- 172.21.15.19:0/2909164657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc80050f0 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.057+0000 7fbfb97fa700 1 -- 172.21.15.19:0/2909164657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fbfc40a49f0 con 0x564493e7bf40 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfeaa5eb80 10 client.18849 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfeaa5eb80 10 client.18849 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfeaa5eb80 20 client.18849 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494080390 0x5644940827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfeaa5eb80 4 client.18849 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "151dc9b1", "release": true, "await": true}] 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "151dc9b1", "release": true, "await": true}) -- 0x564493f244c0 con 0x564494080390 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494080390 0x5644940827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfd95e4700 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494080390 0x5644940827b0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ec75a4d490f01b server_cookie=50568118fd12753b in_seq=0 out_seq=0 2024-05-17T09:42:17.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.058+0000 7fbfb97fa700 10 client.18849 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfb97fa700 1 -- 172.21.15.19:0/2909164657 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564493f244c0 con 0x564494080390 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfb97fa700 10 client.18849 handle_command_reply: tid=0 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 client.18849 shutdown 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 msgr2=0x564494018390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494018390 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40a8d80 tx=0x7fbfc40a89d0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 shutdown_connections 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564493ffa620 0x564494018e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564494080390 0x5644940827b0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564493e7bf40 0x564494018390 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 --2- 172.21.15.19:0/2909164657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564493ef94d0 0x5644940188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfb97fa700 0 client.18849 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:17.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 >> 172.21.15.19:0/2909164657 conn(0x564493ebad80 msgr2=0x564493ffe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 shutdown_connections 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 1 -- 172.21.15.19:0/2909164657 wait complete. 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.084+0000 7fbfeaa5eb80 20 client.18849 trim_cache size 0 max 16384 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 80, 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout: "151dc9b1": { 2024-05-17T09:42:17.085 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 80, 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 130.0, 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:17.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:17.087 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbfbc0b2880 msgr2=0x7fbfbc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbfbc0b2880 0x7fbfbc0b4d90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4101e40 tx=0x7fbfc800b410 comp rx=0 tx=0).stop 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 msgr2=0x7fbfd4103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 0x7fbfd4103160 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40060b0 tx=0x7fbfc400d3b0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 shutdown_connections 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd410e8e0 0x7fbfd4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd4105470 0x7fbfd4103160 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbfbc0b2880 0x7fbfbc0b4d90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 --2- 172.21.15.19:0/4281484232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfd4105dc0 0x7fbfd41036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 >> 172.21.15.19:0/4281484232 conn(0x7fbfd40fef80 msgr2=0x7fbfd4100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 shutdown_connections 2024-05-17T09:42:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:17.085+0000 7fbfdc049700 1 -- 172.21.15.19:0/4281484232 wait complete. 2024-05-17T09:42:17.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59472 6.43 MB/sec execute 374 sec latency 7902.390 ms 2024-05-17T09:42:17.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '151dc9b1', total seconds elapsed: 15.9 2024-05-17T09:42:18.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59619 6.42 MB/sec execute 375 sec latency 68.471 ms 2024-05-17T09:42:18.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:18 smithi069 ceph-mon[35081]: pgmap v1114: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-17T09:42:18.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:18 smithi019 ceph-mon[31620]: pgmap v1114: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-17T09:42:19.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:18 smithi079 ceph-mon[35502]: pgmap v1114: 129 pgs: 129 active+clean; 463 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-05-17T09:42:19.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59778 6.43 MB/sec execute 376 sec latency 80.136 ms 2024-05-17T09:42:20.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 59982 6.45 MB/sec execute 377 sec latency 78.411 ms 2024-05-17T09:42:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:20 smithi069 ceph-mon[35081]: pgmap v1115: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-05-17T09:42:20.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:20 smithi019 ceph-mon[31620]: pgmap v1115: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-05-17T09:42:21.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:20 smithi079 ceph-mon[35502]: pgmap v1115: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-05-17T09:42:21.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 60191 6.46 MB/sec execute 378 sec latency 76.154 ms 2024-05-17T09:42:22.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 60328 6.46 MB/sec execute 379 sec latency 64.841 ms 2024-05-17T09:42:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:22 smithi069 ceph-mon[35081]: pgmap v1116: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 880 KiB/s wr, 111 op/s 2024-05-17T09:42:22.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:22 smithi019 ceph-mon[31620]: pgmap v1116: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 880 KiB/s wr, 111 op/s 2024-05-17T09:42:23.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:22 smithi079 ceph-mon[35502]: pgmap v1116: 129 pgs: 129 active+clean; 464 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 880 KiB/s wr, 111 op/s 2024-05-17T09:42:23.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 60434 6.45 MB/sec execute 380 sec latency 71.553 ms 2024-05-17T09:42:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 -- 172.21.15.19:0/2588858060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 msgr2=0x7fc2f4103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2588858060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 0x7fc2f4103840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009900 tx=0x7fc2e802a410 comp rx=0 tx=0).stop 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 -- 172.21.15.19:0/2588858060 shutdown_connections 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2588858060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4108a80 0x7fc2f410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2588858060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2f4103d80 0x7fc2f4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2588858060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 0x7fc2f4103840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 -- 172.21.15.19:0/2588858060 >> 172.21.15.19:0/2588858060 conn(0x7fc2f40fdd30 msgr2=0x7fc2f41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 -- 172.21.15.19:0/2588858060 shutdown_connections 2024-05-17T09:42:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.236+0000 7fc2fad72640 1 -- 172.21.15.19:0/2588858060 wait complete. 2024-05-17T09:42:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.237+0000 7fc2fad72640 1 Processor -- start 2024-05-17T09:42:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.237+0000 7fc2fad72640 1 -- start start 2024-05-17T09:42:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.237+0000 7fc2fad72640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 0x7fc2f4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.237+0000 7fc2fad72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 0x7fc2f4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2fad72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2f4108a80 0x7fc2f41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2fad72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f410e4f0 con 0x7fc2f4103d80 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2fad72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f410e370 con 0x7fc2f4108a80 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2fad72640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f410e670 con 0x7fc2f4102b40 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 0x7fc2f4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 0x7fc2f4199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54948/0 (socket says 172.21.15.19:54948) 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 -- 172.21.15.19:0/2168693763 learned_addr learned my addr 172.21.15.19:0/2168693763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 0x7fc2f4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f930f640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2f4108a80 0x7fc2f41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 -- 172.21.15.19:0/2168693763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 msgr2=0x7fc2f4199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 0x7fc2f4199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 -- 172.21.15.19:0/2168693763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2f4108a80 msgr2=0x7fc2f41a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2f4108a80 0x7fc2f41a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 -- 172.21.15.19:0/2168693763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f41a0c40 con 0x7fc2f4103d80 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.238+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 0x7fc2f4199640 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000eb00 tx=0x7fc2e000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.239+0000 7fc2f1ffb640 1 -- 172.21.15.19:0/2168693763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000bc30 con 0x7fc2f4103d80 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.239+0000 7fc2f1ffb640 1 -- 172.21.15.19:0/2168693763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2e000bdd0 con 0x7fc2f4103d80 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.239+0000 7fc2f1ffb640 1 -- 172.21.15.19:0/2168693763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0004e20 con 0x7fc2f4103d80 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.239+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f406a6b0 con 0x7fc2f4103d80 2024-05-17T09:42:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.239+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f406abb0 con 0x7fc2f4103d80 2024-05-17T09:42:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.242+0000 7fc2f1ffb640 1 -- 172.21.15.19:0/2168693763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2e0005020 con 0x7fc2f4103d80 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.242+0000 7fc301c23740 1 Processor -- start 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 -- start start 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe0f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd222f0 0x55f7fe0f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe05ec30 con 0x55f7fdd4b1b0 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe04e7d0 con 0x55f7fdd2a4d0 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc301c23740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe052810 con 0x55f7fdd222f0 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc2f930f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe0f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc2f930f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe0f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54958/0 (socket says 172.21.15.19:54958) 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc2f930f640 1 -- 172.21.15.19:0/2046068624 learned_addr learned my addr 172.21.15.19:0/2046068624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.243+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f930f640 1 -- 172.21.15.19:0/2046068624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd222f0 msgr2=0x55f7fe0f97b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f930f640 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd222f0 0x55f7fe0f97b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f930f640 1 -- 172.21.15.19:0/2046068624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd2a4d0 msgr2=0x55f7fe0fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f930f640 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f930f640 1 -- 172.21.15.19:0/2046068624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7fdd63210 con 0x55f7fdd4b1b0 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f930f640 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe0f71e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fc0 tx=0x7fc2ec010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b724f2c0d8f1695 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.244+0000 7fc2f1ffb640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2e40b2920 0x7fc2e40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.245+0000 7fc2deffd640 1 -- 172.21.15.19:0/2046068624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec003b70 con 0x55f7fdd4b1b0 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.245+0000 7fc2f1ffb640 1 -- 172.21.15.19:0/2168693763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2e00e3e80 con 0x7fc2f4103d80 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.245+0000 7fc2deffd640 1 -- 172.21.15.19:0/2046068624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2ec003d10 con 0x55f7fdd4b1b0 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.245+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2e40b2920 0x7fc2e40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.245+0000 7fc2deffd640 1 -- 172.21.15.19:0/2046068624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec011970 con 0x55f7fdd4b1b0 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 -- 172.21.15.19:0/2046068624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 msgr2=0x55f7fe0f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2e40b2920 0x7fc2e40b4e30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005760 tx=0x7fc2e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe0f71e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fc0 tx=0x7fc2ec010860 comp rx=0 tx=0).stop 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 -- 172.21.15.19:0/2046068624 shutdown_connections 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd222f0 0x55f7fe0f97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 --2- 172.21.15.19:0/2046068624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe0f71e0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 -- 172.21.15.19:0/2046068624 >> 172.21.15.19:0/2046068624 conn(0x55f7fdd3f300 msgr2=0x55f7fe0e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 -- 172.21.15.19:0/2046068624 shutdown_connections 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 -- 172.21.15.19:0/2046068624 wait complete. 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.246+0000 7fc301c23740 1 -- start start 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 Processor -- start 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe0fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe1059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f7fdd34790 con 0x55f7fdd2a4d0 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe00a890 con 0x55f7fdd222f0 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe0fca60 con 0x55f7fdd4b1b0 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc301c23740 10 client.0 _ll_register_callbacks cb 0x55f7fe00dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe0fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe0fe500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50364/0 (socket says 172.21.15.19:50364) 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f930f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe1059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 -- 172.21.15.19:0/2919413080 learned_addr learned my addr 172.21.15.19:0/2919413080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 -- 172.21.15.19:0/2919413080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd4b1b0 msgr2=0x55f7fe1059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe1059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 -- 172.21.15.19:0/2919413080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd2a4d0 msgr2=0x55f7fe0fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.247+0000 7fc2f8b0e640 1 -- 172.21.15.19:0/2919413080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7fde2f620 con 0x55f7fdd222f0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.248+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe0fe500 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009900 tx=0x7fc2e8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eabaca03ef623411 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.248+0000 7fc2d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.248+0000 7fc2d3fff640 1 -- 172.21.15.19:0/2919413080 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e804c030 con 0x55f7fdd222f0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.248+0000 7fc2d3fff640 1 -- 172.21.15.19:0/2919413080 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2e8004250 con 0x55f7fdd222f0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.248+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ec003d10 con 0x55f7fdd222f0 2024-05-17T09:42:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc2d3fff640 1 -- 172.21.15.19:0/2919413080 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc2e8059050 con 0x55f7fdd222f0 2024-05-17T09:42:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 10 client.28367 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f7fe108940 con 0x55f7fdd222f0 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc2d3fff640 1 -- 172.21.15.19:0/2919413080 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2ec003d10 con 0x55f7fdd222f0 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc2d3fff640 1 -- 172.21.15.19:0/2919413080 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc2e8048440 con 0x55f7fdd222f0 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 10 client.28367 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 10 client.28367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 20 client.28367 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd09d80 0x55f7fe10c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 4 client.28367 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.249+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7fe060b80 con 0x55f7fdd09d80 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.250+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd09d80 0x55f7fe10c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.250+0000 7fc2d3fff640 10 client.28367 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.250+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd09d80 0x55f7fe10c0e0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7473b7e4e08f16e4 server_cookie=9f12bcdb92119a4f in_seq=0 out_seq=0 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc2d3fff640 1 -- 172.21.15.19:0/2919413080 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f7fe060b80 con 0x55f7fdd09d80 2024-05-17T09:42:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc2d3fff640 10 client.28367 handle_command_reply: tid=0 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc301c23740 1 client.28367 shutdown 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 msgr2=0x55f7fe0fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc301c23740 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe0fe500 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009900 tx=0x7fc2e8009260 comp rx=0 tx=0).stop 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 shutdown_connections 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.252+0000 7fc301c23740 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd09d80 0x55f7fe10c0e0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc2d3fff640 0 client.28367 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd4b1b0 0x55f7fe1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd2a4d0 0x55f7fe0fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 1 --2- 172.21.15.19:0/2919413080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe0fe500 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 >> 172.21.15.19:0/2919413080 conn(0x55f7fdd3f300 msgr2=0x55f7fe0e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 shutdown_connections 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 1 -- 172.21.15.19:0/2919413080 wait complete. 2024-05-17T09:42:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.253+0000 7fc301c23740 20 client.28367 trim_cache size 0 max 16384 2024-05-17T09:42:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.273+0000 7fc301c23740 1 Processor -- start 2024-05-17T09:42:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.273+0000 7fc301c23740 1 -- start start 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc301c23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd3f300 0x55f7fe1108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc301c23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd09d80 0x55f7fe105930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc301c23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe110df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f8b0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd3f300 0x55f7fe1108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f8b0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd3f300 0x55f7fe1108b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54978/0 (socket says 172.21.15.19:54978) 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41ad190 con 0x55f7fdd3f300 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd09d80 0x55f7fe105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41aaa60 con 0x55f7fdd222f0 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a88b0 con 0x55f7fdd09d80 2024-05-17T09:42:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f8b0e640 1 -- 172.21.15.19:0/1384175754 learned_addr learned my addr 172.21.15.19:0/1384175754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f930f640 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe110df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f3fff640 1 -- 172.21.15.19:0/1384175754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 msgr2=0x55f7fe110df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe110df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f3fff640 1 -- 172.21.15.19:0/1384175754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd3f300 msgr2=0x55f7fe1108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd3f300 0x55f7fe1108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.274+0000 7fc2f3fff640 1 -- 172.21.15.19:0/1384175754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7fe0fc320 con 0x55f7fdd09d80 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.275+0000 7fc2f930f640 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.275+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd09d80 0x55f7fe105930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00e1940 tx=0x7fc2e00e5750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fef5c8cf78daca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.275+0000 7fc2de7fc640 1 -- 172.21.15.19:0/1384175754 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00ab7f0 con 0x55f7fdd09d80 2024-05-17T09:42:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.275+0000 7fc2de7fc640 1 -- 172.21.15.19:0/1384175754 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2e00e5040 con 0x55f7fdd09d80 2024-05-17T09:42:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.275+0000 7fc2de7fc640 1 -- 172.21.15.19:0/1384175754 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00055b0 con 0x55f7fdd09d80 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.276+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd09d80 msgr2=0x55f7fe105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.276+0000 7fc301c23740 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd09d80 0x55f7fe105930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00e1940 tx=0x7fc2e00e5750 comp rx=0 tx=0).stop 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 shutdown_connections 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd09d80 0x55f7fe105930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 --2- 172.21.15.19:0/1384175754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd3f300 0x55f7fe1108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 >> 172.21.15.19:0/1384175754 conn(0x55f7fe10c450 msgr2=0x55f7fe0f7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 shutdown_connections 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.277+0000 7fc301c23740 1 -- 172.21.15.19:0/1384175754 wait complete. 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.278+0000 7fc301c23740 1 -- start start 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.278+0000 7fc301c23740 1 Processor -- start 2024-05-17T09:42:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.278+0000 7fc301c23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd09d80 0x55f7fe16e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc301c23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe170cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc301c23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd3f300 0x55f7fe173200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc301c23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a8f20 con 0x55f7fdd09d80 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe170cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe170cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50394/0 (socket says 172.21.15.19:50394) 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 -- 172.21.15.19:0/480636841 learned_addr learned my addr 172.21.15.19:0/480636841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f930f640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd3f300 0x55f7fe173200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f8b0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd09d80 0x55f7fe16e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41aa390 con 0x55f7fdd222f0 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41a95f0 con 0x55f7fdd3f300 2024-05-17T09:42:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc301c23740 10 client.0 _ll_register_callbacks cb 0x7fc2e8031700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 -- 172.21.15.19:0/480636841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd3f300 msgr2=0x55f7fe173200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd3f300 0x55f7fe173200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 -- 172.21.15.19:0/480636841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd09d80 msgr2=0x55f7fe16e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd09d80 0x55f7fe16e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f3fff640 1 -- 172.21.15.19:0/480636841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7fe173740 con 0x55f7fdd222f0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.279+0000 7fc2f930f640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd3f300 0x55f7fe173200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc2f3fff640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe170cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00aadf0 tx=0x7fc2e00ab170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892503aa9595a701 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc2d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc2d3fff640 1 -- 172.21.15.19:0/480636841 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00c5060 con 0x55f7fdd222f0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e00e5040 con 0x55f7fdd222f0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc2d3fff640 1 -- 172.21.15.19:0/480636841 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc2e0005310 con 0x55f7fdd222f0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc2d3fff640 1 -- 172.21.15.19:0/480636841 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc2e00b3f50 con 0x55f7fdd222f0 2024-05-17T09:42:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc301c23740 10 client.28373 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.280+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f7fe174840 con 0x55f7fdd222f0 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc2d3fff640 1 -- 172.21.15.19:0/480636841 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2e00e5040 con 0x55f7fdd222f0 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc2d3fff640 1 -- 172.21.15.19:0/480636841 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc2e00e7030 con 0x55f7fdd222f0 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc301c23740 10 client.28373 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc301c23740 10 client.28373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc301c23740 20 client.28373 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc301c23740 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd2a4d0 0x55f7fe177170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc301c23740 4 client.28373 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7fe05a360 con 0x55f7fdd2a4d0 2024-05-17T09:42:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.281+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd2a4d0 0x55f7fe177170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.282+0000 7fc2f8b0e640 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd2a4d0 0x55f7fe177170 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb985c8d916aed33 server_cookie=f9bf14ad78d494f7 in_seq=0 out_seq=0 2024-05-17T09:42:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.282+0000 7fc2d3fff640 10 client.28373 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc2d3fff640 1 -- 172.21.15.19:0/480636841 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f7fe05a360 con 0x55f7fdd2a4d0 2024-05-17T09:42:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc2d3fff640 10 client.28373 handle_command_reply: tid=0 2024-05-17T09:42:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 client.28373 shutdown 2024-05-17T09:42:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 msgr2=0x55f7fe170cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe170cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00aadf0 tx=0x7fc2e00ab170 comp rx=0 tx=0).stop 2024-05-17T09:42:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 shutdown_connections 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f7fdd2a4d0 0x55f7fe177170 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7fdd3f300 0x55f7fe173200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7fdd222f0 0x55f7fe170cb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 --2- 172.21.15.19:0/480636841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f7fdd09d80 0x55f7fe16e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 >> 172.21.15.19:0/480636841 conn(0x55f7fe10c450 msgr2=0x55f7fe0f7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.376+0000 7fc2d3fff640 0 client.28373 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.377+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 shutdown_connections 2024-05-17T09:42:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.377+0000 7fc301c23740 1 -- 172.21.15.19:0/480636841 wait complete. 2024-05-17T09:42:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.377+0000 7fc301c23740 20 client.28373 trim_cache size 0 max 16384 2024-05-17T09:42:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:42:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:42:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:42:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.378+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2e40b2920 msgr2=0x7fc2e40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.378+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2e40b2920 0x7fc2e40b4e30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005760 tx=0x7fc2e8000bc0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.378+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 msgr2=0x7fc2f4199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 0x7fc2f4199640 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000eb00 tx=0x7fc2e000ef00 comp rx=0 tx=0).stop 2024-05-17T09:42:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 shutdown_connections 2024-05-17T09:42:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2f4108a80 0x7fc2f41a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc2e40b2920 0x7fc2e40b4e30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2f4103d80 0x7fc2f4199640 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 --2- 172.21.15.19:0/2168693763 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2f4102b40 0x7fc2f4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 >> 172.21.15.19:0/2168693763 conn(0x7fc2f40fdd30 msgr2=0x7fc2f4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 shutdown_connections 2024-05-17T09:42:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:24.379+0000 7fc2fad72640 1 -- 172.21.15.19:0/2168693763 wait complete. 2024-05-17T09:42:24.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 60571 6.47 MB/sec execute 381 sec latency 116.135 ms 2024-05-17T09:42:24.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:24 smithi019 ceph-mon[31620]: pgmap v1117: 129 pgs: 129 active+clean; 472 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-17T09:42:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:24 smithi069 ceph-mon[35081]: pgmap v1117: 129 pgs: 129 active+clean; 472 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-17T09:42:25.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:24 smithi079 ceph-mon[35502]: pgmap v1117: 129 pgs: 129 active+clean; 472 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-17T09:42:25.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 60759 6.49 MB/sec execute 382 sec latency 76.141 ms 2024-05-17T09:42:26.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 60928 6.50 MB/sec execute 383 sec latency 66.478 ms 2024-05-17T09:42:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:26 smithi069 ceph-mon[35081]: pgmap v1118: 129 pgs: 129 active+clean; 475 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-05-17T09:42:26.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:26 smithi019 ceph-mon[31620]: pgmap v1118: 129 pgs: 129 active+clean; 475 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-05-17T09:42:27.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:26 smithi079 ceph-mon[35502]: pgmap v1118: 129 pgs: 129 active+clean; 475 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-05-17T09:42:27.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61102 6.52 MB/sec execute 384 sec latency 83.285 ms 2024-05-17T09:42:28.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61230 6.52 MB/sec execute 385 sec latency 85.667 ms 2024-05-17T09:42:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:28 smithi069 ceph-mon[35081]: pgmap v1119: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-05-17T09:42:28.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:28 smithi019 ceph-mon[31620]: pgmap v1119: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-05-17T09:42:29.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:28 smithi079 ceph-mon[35502]: pgmap v1119: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-05-17T09:42:29.591 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61366 6.52 MB/sec execute 386 sec latency 84.664 ms 2024-05-17T09:42:30.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61523 6.52 MB/sec execute 387 sec latency 96.875 ms 2024-05-17T09:42:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:30 smithi069 ceph-mon[35081]: pgmap v1120: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-05-17T09:42:30.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:30 smithi019 ceph-mon[31620]: pgmap v1120: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-05-17T09:42:31.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:30 smithi079 ceph-mon[35502]: pgmap v1120: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-05-17T09:42:31.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61673 6.52 MB/sec execute 388 sec latency 129.347 ms 2024-05-17T09:42:32.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61808 6.53 MB/sec execute 389 sec latency 71.026 ms 2024-05-17T09:42:32.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:32 smithi079 ceph-mon[35502]: pgmap v1121: 129 pgs: 129 active+clean; 483 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 216 op/s 2024-05-17T09:42:32.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:32 smithi069 ceph-mon[35081]: pgmap v1121: 129 pgs: 129 active+clean; 483 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 216 op/s 2024-05-17T09:42:32.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:32 smithi019 ceph-mon[31620]: pgmap v1121: 129 pgs: 129 active+clean; 483 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 216 op/s 2024-05-17T09:42:33.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 61963 6.54 MB/sec execute 390 sec latency 89.169 ms 2024-05-17T09:42:34.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62102 6.54 MB/sec execute 391 sec latency 87.018 ms 2024-05-17T09:42:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:34 smithi069 ceph-mon[35081]: pgmap v1122: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.5 MiB/s wr, 229 op/s 2024-05-17T09:42:34.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:34 smithi019 ceph-mon[31620]: pgmap v1122: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.5 MiB/s wr, 229 op/s 2024-05-17T09:42:35.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:34 smithi079 ceph-mon[35502]: pgmap v1122: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.5 MiB/s wr, 229 op/s 2024-05-17T09:42:35.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62244 6.53 MB/sec execute 392 sec latency 96.554 ms 2024-05-17T09:42:36.450 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-17T09:42:36.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:42:36.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62387 6.52 MB/sec execute 393 sec latency 116.538 ms 2024-05-17T09:42:36.900 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:36 smithi069 ceph-mon[35081]: pgmap v1123: 129 pgs: 129 active+clean; 483 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 256 op/s 2024-05-17T09:42:36.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:36 smithi019 ceph-mon[31620]: pgmap v1123: 129 pgs: 129 active+clean; 483 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 256 op/s 2024-05-17T09:42:37.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:36 smithi079 ceph-mon[35502]: pgmap v1123: 129 pgs: 129 active+clean; 483 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 256 op/s 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 -- 172.21.15.19:0/1675204005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1098d0 msgr2=0x7f9b6c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 --2- 172.21.15.19:0/1675204005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1098d0 0x7f9b6c10bde0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009ca0 tx=0x7f9b6402a8c0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 -- 172.21.15.19:0/1675204005 shutdown_connections 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 --2- 172.21.15.19:0/1675204005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1098d0 0x7f9b6c10bde0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 --2- 172.21.15.19:0/1675204005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6c106dc0 0x7f9b6c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 --2- 172.21.15.19:0/1675204005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 -- 172.21.15.19:0/1675204005 >> 172.21.15.19:0/1675204005 conn(0x7f9b6c0fdd30 msgr2=0x7f9b6c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 -- 172.21.15.19:0/1675204005 shutdown_connections 2024-05-17T09:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.565+0000 7f9b720bd640 1 -- 172.21.15.19:0/1675204005 wait complete. 2024-05-17T09:42:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.566+0000 7f9b720bd640 1 Processor -- start 2024-05-17T09:42:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.566+0000 7f9b720bd640 1 -- start start 2024-05-17T09:42:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b720bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b720bd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6c106dc0 0x7f9b6c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b720bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b6c1098d0 0x7f9b6c1a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b720bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c10f640 con 0x7f9b6c1043d0 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b720bd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c10f4c0 con 0x7f9b6c1098d0 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b720bd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c10f7c0 con 0x7f9b6c106dc0 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54248/0 (socket says 172.21.15.19:54248) 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/3594069630 learned_addr learned my addr 172.21.15.19:0/3594069630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b6c1098d0 0x7f9b6c1a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6affd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6c106dc0 0x7f9b6c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/3594069630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6c106dc0 msgr2=0x7f9b6c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6c106dc0 0x7f9b6c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/3594069630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b6c1098d0 msgr2=0x7f9b6c1a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b6c1098d0 0x7f9b6c1a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/3594069630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c1a1c70 con 0x7f9b6c1043d0 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.567+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c19a130 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009f20 tx=0x7f9b5c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.568+0000 7f9b68ff9640 1 -- 172.21.15.19:0/3594069630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c00cd00 con 0x7f9b6c1043d0 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.568+0000 7f9b68ff9640 1 -- 172.21.15.19:0/3594069630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b5c0054b0 con 0x7f9b6c1043d0 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.568+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6c102220 con 0x7f9b6c1043d0 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.568+0000 7f9b68ff9640 1 -- 172.21.15.19:0/3594069630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0047a0 con 0x7f9b6c1043d0 2024-05-17T09:42:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.568+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6c1027c0 con 0x7f9b6c1043d0 2024-05-17T09:42:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 Processor -- start 2024-05-17T09:42:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 -- start start 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b68ff9640 1 -- 172.21.15.19:0/3594069630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9b5c004940 con 0x7f9b6c1043d0 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 0x559af0e3bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e3c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e43030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559af0da5c30 con 0x559af0a921b0 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559af0d957d0 con 0x559af0a714d0 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.571+0000 7f9b78f71740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559af0d99810 con 0x559af0a692f0 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 0x559af0e3bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e3c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 0x559af0e3bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54254/0 (socket says 172.21.15.19:54254) 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6bfff640 1 -- 172.21.15.19:0/1095889385 learned_addr learned my addr 172.21.15.19:0/1095889385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e3c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:59708/0 (socket says 172.21.15.19:59708) 2024-05-17T09:42:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e43030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/1095889385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 msgr2=0x559af0e43030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e43030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/1095889385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 msgr2=0x559af0e3bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 0x559af0e3bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/1095889385 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af0aaa210 con 0x559af0a692f0 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 0x559af0e3bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.572+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e3c130 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0dfc20 tx=0x7f9b5c0e85a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e92d86ecb337c72e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.573+0000 7f9b68ff9640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b540b2920 0x7f9b540b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.573+0000 7f9b51ffb640 1 -- 172.21.15.19:0/1095889385 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0f6070 con 0x559af0a692f0 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.573+0000 7f9b51ffb640 1 -- 172.21.15.19:0/1095889385 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b5c0ad730 con 0x559af0a692f0 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.573+0000 7f9b51ffb640 1 -- 172.21.15.19:0/1095889385 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0ad390 con 0x559af0a692f0 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.573+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b540b2920 0x7f9b540b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.573+0000 7f9b68ff9640 1 -- 172.21.15.19:0/3594069630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9b5c0e2500 con 0x7f9b6c1043d0 2024-05-17T09:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b540b2920 0x7f9b540b4e30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x559af0e3cd80 tx=0x7f9b64031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 -- 172.21.15.19:0/1095889385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 msgr2=0x559af0e3c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e3c130 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0dfc20 tx=0x7f9b5c0e85a0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 -- 172.21.15.19:0/1095889385 shutdown_connections 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e3c130 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 --2- 172.21.15.19:0/1095889385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a921b0 0x559af0e3bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 -- 172.21.15.19:0/1095889385 >> 172.21.15.19:0/1095889385 conn(0x559af0a86300 msgr2=0x559af0e37540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 -- 172.21.15.19:0/1095889385 shutdown_connections 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.574+0000 7f9b78f71740 1 -- 172.21.15.19:0/1095889385 wait complete. 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 -- start start 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 Processor -- start 2024-05-17T09:42:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a692f0 0x559af0e454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559af0a7b790 con 0x559af0a692f0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559af0d51890 con 0x559af0a714d0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559af0e43a80 con 0x559af0a921b0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b78f71740 10 client.0 _ll_register_callbacks cb 0x559af0d54d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.575+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a692f0 0x559af0e454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e45a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50806/0 (socket says 172.21.15.19:50806) 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6affd640 1 -- 172.21.15.19:0/3384919246 learned_addr learned my addr 172.21.15.19:0/3384919246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 -- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 msgr2=0x559af0e45a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e45a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 -- 172.21.15.19:0/3384919246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a692f0 msgr2=0x559af0e454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a692f0 0x559af0e454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 -- 172.21.15.19:0/3384919246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af0b76620 con 0x559af0a921b0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4c990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b64031920 tx=0x7f9b6402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af4295e7bc01096 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.576+0000 7f9b4affd640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b4affd640 1 -- 172.21.15.19:0/3384919246 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64031680 con 0x559af0a921b0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b5c0ad730 con 0x559af0a921b0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b4affd640 1 -- 172.21.15.19:0/3384919246 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9b64004250 con 0x559af0a921b0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b4affd640 1 -- 172.21.15.19:0/3384919246 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b64004450 con 0x559af0a921b0 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b78f71740 10 client.28375 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559af0e4f8c0 con 0x559af0a921b0 2024-05-17T09:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b4affd640 1 -- 172.21.15.19:0/3384919246 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b5c0ad730 con 0x559af0a921b0 2024-05-17T09:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b4affd640 1 -- 172.21.15.19:0/3384919246 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9b64051070 con 0x559af0a921b0 2024-05-17T09:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b78f71740 10 client.28375 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.577+0000 7f9b78f71740 10 client.28375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b78f71740 20 client.28375 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b78f71740 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a50d80 0x559af0e53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b78f71740 4 client.28375 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af0da7b80 con 0x559af0a50d80 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a50d80 0x559af0e53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a50d80 0x559af0e53140 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4535f3431d14e7f5 server_cookie=3d8b7fb0acab3e98 in_seq=0 out_seq=0 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.578+0000 7f9b4affd640 10 client.28375 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b4affd640 1 -- 172.21.15.19:0/3384919246 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559af0da7b80 con 0x559af0a50d80 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b4affd640 10 client.28375 handle_command_reply: tid=0 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 client.28375 shutdown 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 msgr2=0x559af0e4c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4c990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b64031920 tx=0x7f9b6402a8c0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 shutdown_connections 2024-05-17T09:42:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a50d80 0x559af0e53140 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4c990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b4affd640 0 client.28375 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a714d0 0x559af0e45a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 --2- 172.21.15.19:0/3384919246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a692f0 0x559af0e454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 >> 172.21.15.19:0/3384919246 conn(0x559af0a86300 msgr2=0x559af0e3a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 shutdown_connections 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 1 -- 172.21.15.19:0/3384919246 wait complete. 2024-05-17T09:42:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.580+0000 7f9b78f71740 20 client.28375 trim_cache size 0 max 16384 2024-05-17T09:42:37.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62524 6.52 MB/sec execute 394 sec latency 134.920 ms 2024-05-17T09:42:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 Processor -- start 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 -- start start 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e2da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a921b0 0x559af0e4cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1ad070 con 0x559af0a714d0 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1aa940 con 0x559af0a921b0 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b78f71740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1a8790 con 0x559af0a692f0 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54258/0 (socket says 172.21.15.19:54258) 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b6affd640 1 -- 172.21.15.19:0/2981381153 learned_addr learned my addr 172.21.15.19:0/2981381153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a921b0 0x559af0e4cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.603+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e2da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6bfff640 1 -- 172.21.15.19:0/2981381153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 msgr2=0x559af0e2da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e2da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6bfff640 1 -- 172.21.15.19:0/2981381153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 msgr2=0x559af0e4c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6bfff640 1 -- 172.21.15.19:0/2981381153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af0e460f0 con 0x559af0a921b0 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6affd640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a921b0 0x559af0e4cdf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b6402ade0 tx=0x7f9b64004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12136d9b2adea40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b517fa640 1 -- 172.21.15.19:0/2981381153 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640045f0 con 0x559af0a921b0 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b517fa640 1 -- 172.21.15.19:0/2981381153 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b64036050 con 0x559af0a921b0 2024-05-17T09:42:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.604+0000 7f9b517fa640 1 -- 172.21.15.19:0/2981381153 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640309d0 con 0x559af0a921b0 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 -- 172.21.15.19:0/2981381153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a921b0 msgr2=0x559af0e4cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a921b0 0x559af0e4cdf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b6402ade0 tx=0x7f9b64004250 comp rx=0 tx=0).stop 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 -- 172.21.15.19:0/2981381153 shutdown_connections 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a921b0 0x559af0e4cdf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 --2- 172.21.15.19:0/2981381153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a692f0 0x559af0e2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.605+0000 7f9b78f71740 1 -- 172.21.15.19:0/2981381153 >> 172.21.15.19:0/2981381153 conn(0x559af0a86300 msgr2=0x559af0e2d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 -- 172.21.15.19:0/2981381153 shutdown_connections 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 -- 172.21.15.19:0/2981381153 wait complete. 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 -- start start 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 Processor -- start 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 0x559af0e4a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1a8e00 con 0x559af0a714d0 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1aa270 con 0x559af0a692f0 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b78f71740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1a94d0 con 0x559af0a921b0 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.606+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 0x559af0e4a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 0x559af0e4a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50820/0 (socket says 172.21.15.19:50820) 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/831304054 learned_addr learned my addr 172.21.15.19:0/831304054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6affd640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b78f71740 10 client.0 _ll_register_callbacks cb 0x7f9b64031e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6bfff640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/831304054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 msgr2=0x559af0e4ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/831304054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 msgr2=0x559af0e4ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 -- 172.21.15.19:0/831304054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af0e4bff0 con 0x559af0a692f0 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b6b7fe640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 0x559af0e4a5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c000c00 tx=0x7f9b5c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=902fe4672fbb1a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b4affd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:42:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b4affd640 1 -- 172.21.15.19:0/831304054 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c011430 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b4affd640 1 -- 172.21.15.19:0/831304054 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9b5c0aa210 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.607+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b64036050 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b4affd640 1 -- 172.21.15.19:0/831304054 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b5c011620 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 10 client.28391 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559af0e4d2f0 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b4affd640 1 -- 172.21.15.19:0/831304054 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b64036050 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b4affd640 1 -- 172.21.15.19:0/831304054 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9b5c0a9eb0 con 0x559af0a692f0 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 10 client.28391 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 10 client.28391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 20 client.28391 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a261c0 0x559af0ebd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 4 client.28391 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:42:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.608+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559af0da1360 con 0x559af0a261c0 2024-05-17T09:42:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.609+0000 7f9b6affd640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a261c0 0x559af0ebd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.609+0000 7f9b6affd640 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a261c0 0x559af0ebd9f0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e59a381449e348 server_cookie=dbc9c62d48c00397 in_seq=0 out_seq=0 2024-05-17T09:42:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.609+0000 7f9b4affd640 10 client.28391 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b4affd640 1 -- 172.21.15.19:0/831304054 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559af0da1360 con 0x559af0a261c0 2024-05-17T09:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b4affd640 10 client.28391 handle_command_reply: tid=0 2024-05-17T09:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 client.28391 shutdown 2024-05-17T09:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 msgr2=0x559af0e4a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 0x559af0e4a5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c000c00 tx=0x7f9b5c009310 comp rx=0 tx=0).stop 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 shutdown_connections 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559af0a261c0 0x559af0ebd9f0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559af0a921b0 0x559af0e4ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559af0a714d0 0x559af0e4ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 --2- 172.21.15.19:0/831304054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559af0a692f0 0x559af0e4a5c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 >> 172.21.15.19:0/831304054 conn(0x559af0a86300 msgr2=0x559af0e37560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.685+0000 7f9b4affd640 0 client.28391 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.686+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 shutdown_connections 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.686+0000 7f9b78f71740 1 -- 172.21.15.19:0/831304054 wait complete. 2024-05-17T09:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.686+0000 7f9b78f71740 20 client.28391 trim_cache size 0 max 16384 2024-05-17T09:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b540b2920 msgr2=0x7f9b540b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b540b2920 0x7f9b540b4e30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x559af0e3cd80 tx=0x7f9b64031000 comp rx=0 tx=0).stop 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 msgr2=0x7f9b6c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c19a130 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009f20 tx=0x7f9b5c00ee50 comp rx=0 tx=0).stop 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 shutdown_connections 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b6c1098d0 0x7f9b6c1a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6c106dc0 0x7f9b6c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9b540b2920 0x7f9b540b4e30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 --2- 172.21.15.19:0/3594069630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b6c1043d0 0x7f9b6c19a130 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 >> 172.21.15.19:0/3594069630 conn(0x7f9b6c0fdd30 msgr2=0x7f9b6c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 shutdown_connections 2024-05-17T09:42:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:37.687+0000 7f9b720bd640 1 -- 172.21.15.19:0/3594069630 wait complete. 2024-05-17T09:42:38.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62667 6.52 MB/sec execute 395 sec latency 109.765 ms 2024-05-17T09:42:38.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:38 smithi019 ceph-mon[31620]: pgmap v1124: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-05-17T09:42:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 -- 172.21.15.19:0/360162321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008108ef0 msgr2=0x7fa008109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 --2- 172.21.15.19:0/360162321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008108ef0 0x7fa008109300 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0099c0 tx=0x7f9ffc009cd0 comp rx=0 tx=0).stop 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 -- 172.21.15.19:0/360162321 shutdown_connections 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 --2- 172.21.15.19:0/360162321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0081045c0 0x7fa0081049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 --2- 172.21.15.19:0/360162321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa008109840 0x7fa0081111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 --2- 172.21.15.19:0/360162321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008108ef0 0x7fa008109300 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 -- 172.21.15.19:0/360162321 >> 172.21.15.19:0/360162321 conn(0x7fa0080ffe90 msgr2=0x7fa0081022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 -- 172.21.15.19:0/360162321 shutdown_connections 2024-05-17T09:42:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.838+0000 7fa00d10d700 1 -- 172.21.15.19:0/360162321 wait complete. 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 Processor -- start 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 -- start start 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0081045c0 0x7fa008076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 0x7fa008077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa008109840 0x7fa008077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa008114300 con 0x7fa0081045c0 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa008114180 con 0x7fa008109840 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00d10d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa008114480 con 0x7fa008108ef0 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 0x7fa008077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 0x7fa008077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:59734/0 (socket says 172.21.15.19:59734) 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.839+0000 7fa00659c700 1 -- 172.21.15.19:0/3964492455 learned_addr learned my addr 172.21.15.19:0/3964492455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa006d9d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0081045c0 0x7fa008076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00759e700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa008109840 0x7fa008077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00659c700 1 -- 172.21.15.19:0/3964492455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa008109840 msgr2=0x7fa008077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00659c700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa008109840 0x7fa008077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00659c700 1 -- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0081045c0 msgr2=0x7fa008076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00659c700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0081045c0 0x7fa008076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00659c700 1 -- 172.21.15.19:0/3964492455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0081a90f0 con 0x7fa008108ef0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7fa00659c700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 0x7fa008077100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc005f50 tx=0x7f9ffc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc026070 con 0x7fa008108ef0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ffc003c50 con 0x7fa008108ef0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.840+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00f950 con 0x7fa008108ef0 2024-05-17T09:42:38.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.841+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0081a9380 con 0x7fa008108ef0 2024-05-17T09:42:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.841+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0081a9780 con 0x7fa008108ef0 2024-05-17T09:42:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.842+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0080512b0 con 0x7fa008108ef0 2024-05-17T09:42:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.844+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9ffc029070 con 0x7fa008108ef0 2024-05-17T09:42:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.845+0000 7f9feffff700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ff00b2900 0x7f9ff00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.845+0000 7fa006d9d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ff00b2900 0x7f9ff00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.845+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9ffc0e2eb0 con 0x7fa008108ef0 2024-05-17T09:42:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.846+0000 7fa006d9d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ff00b2900 0x7f9ff00b4e10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009c60 tx=0x7f9ff8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:38.848+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ffc112df0 con 0x7fa008108ef0 2024-05-17T09:42:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:38 smithi069 ceph-mon[35081]: pgmap v1124: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-05-17T09:42:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:38 smithi079 ceph-mon[35502]: pgmap v1124: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-05-17T09:42:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.296+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa00802e5e0 con 0x7fa008108ef0 2024-05-17T09:42:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.297+0000 7f9feffff700 1 -- 172.21.15.19:0/3964492455 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f9ffc013070 con 0x7fa008108ef0 2024-05-17T09:42:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:42:39.297 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:42:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ff00b2900 msgr2=0x7f9ff00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ff00b2900 0x7f9ff00b4e10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009c60 tx=0x7f9ff8009260 comp rx=0 tx=0).stop 2024-05-17T09:42:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 msgr2=0x7fa008077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 0x7fa008077100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc005f50 tx=0x7f9ffc004270 comp rx=0 tx=0).stop 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 shutdown_connections 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa008109840 0x7fa008077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0081045c0 0x7fa008076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9ff00b2900 0x7f9ff00b4e10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 --2- 172.21.15.19:0/3964492455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa008108ef0 0x7fa008077100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 >> 172.21.15.19:0/3964492455 conn(0x7fa0080ffe90 msgr2=0x7fa0081022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 shutdown_connections 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:39.300+0000 7fa00d10d700 1 -- 172.21.15.19:0/3964492455 wait complete. 2024-05-17T09:42:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:42:39.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62829 6.52 MB/sec execute 396 sec latency 100.432 ms 2024-05-17T09:42:39.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:39 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3964492455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:39.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:39 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3964492455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:40.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:39 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3964492455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:40.310 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-17T09:42:40.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-17T09:42:40.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 62961 6.53 MB/sec execute 397 sec latency 83.779 ms 2024-05-17T09:42:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:40.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:40 smithi019 ceph-mon[31620]: pgmap v1125: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-05-17T09:42:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:40 smithi069 ceph-mon[35081]: pgmap v1125: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-05-17T09:42:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:40 smithi079 ceph-mon[35502]: pgmap v1125: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-05-17T09:42:41.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63101 6.53 MB/sec execute 398 sec latency 92.171 ms 2024-05-17T09:42:41.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:41 smithi069 ceph-mon[35081]: pgmap v1126: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-05-17T09:42:41.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:41 smithi019 ceph-mon[31620]: pgmap v1126: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-05-17T09:42:42.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:41 smithi079 ceph-mon[35502]: pgmap v1126: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2024-05-17T09:42:42.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63309 6.56 MB/sec execute 399 sec latency 101.610 ms 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 -- 172.21.15.19:0/4161031524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 msgr2=0x7f9548110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 --2- 172.21.15.19:0/4161031524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 0x7f9548110090 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f954000b0d0 tx=0x7f954000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 -- 172.21.15.19:0/4161031524 shutdown_connections 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 --2- 172.21.15.19:0/4161031524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95481034b0 0x7f95481038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 --2- 172.21.15.19:0/4161031524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 0x7f9548110090 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 --2- 172.21.15.19:0/4161031524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9548107de0 0x7f95481081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 -- 172.21.15.19:0/4161031524 >> 172.21.15.19:0/4161031524 conn(0x7f95480fedb0 msgr2=0x7f95481011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 -- 172.21.15.19:0/4161031524 shutdown_connections 2024-05-17T09:42:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.984+0000 7f954f48e700 1 -- 172.21.15.19:0/4161031524 wait complete. 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.985+0000 7f954f48e700 1 Processor -- start 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.985+0000 7f954f48e700 1 -- start start 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954f48e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 0x7f95481999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954f48e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9548107de0 0x7f9548199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954f48e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 0x7f954819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954f48e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9548113260 con 0x7f9548108730 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954f48e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f95481130e0 con 0x7f95481034b0 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954f48e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f95481133e0 con 0x7f9548107de0 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 0x7f95481999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954ca29700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9548107de0 0x7f9548199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 0x7f95481999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50874/0 (socket says 172.21.15.19:50874) 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 -- 172.21.15.19:0/190898782 learned_addr learned my addr 172.21.15.19:0/190898782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954da2b700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 0x7f954819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 -- 172.21.15.19:0/190898782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9548107de0 msgr2=0x7f9548199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9548107de0 0x7f9548199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 -- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 msgr2=0x7f954819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 0x7f954819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.986+0000 7f954d22a700 1 -- 172.21.15.19:0/190898782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95481a90f0 con 0x7f95481034b0 2024-05-17T09:42:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.987+0000 7f954d22a700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 0x7f95481999f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f954400e9a0 tx=0x7f954400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.987+0000 7f953e7fc700 1 -- 172.21.15.19:0/190898782 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954400ca10 con 0x7f95481034b0 2024-05-17T09:42:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.987+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95481a93e0 con 0x7f95481034b0 2024-05-17T09:42:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.987+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95481a9970 con 0x7f95481034b0 2024-05-17T09:42:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.987+0000 7f953e7fc700 1 -- 172.21.15.19:0/190898782 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f954400cbb0 con 0x7f95481034b0 2024-05-17T09:42:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.987+0000 7f953e7fc700 1 -- 172.21.15.19:0/190898782 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544018680 con 0x7f95481034b0 2024-05-17T09:42:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.991+0000 7f953e7fc700 1 -- 172.21.15.19:0/190898782 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9544018820 con 0x7f95481034b0 2024-05-17T09:42:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f955dea3b80 1 Processor -- start 2024-05-17T09:42:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f955dea3b80 1 -- start start 2024-05-17T09:42:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f953e7fc700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f95340b2880 0x7f95340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b467403560 0x55b46750a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f953e7fc700 1 -- 172.21.15.19:0/190898782 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f95440e2740 con 0x7f95481034b0 2024-05-17T09:42:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f954da2b700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f95340b2880 0x7f95340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.992+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4675046b0 0x55b467504ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f955dea3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b467432e20 con 0x55b4675046b0 2024-05-17T09:42:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f955dea3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b4674229c0 con 0x55b467403560 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f955dea3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b467426a00 con 0x55b467385fd0 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:59762/0 (socket says 172.21.15.19:59762) 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 -- 172.21.15.19:0/3753566506 learned_addr learned my addr 172.21.15.19:0/3753566506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954d22a700 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b467403560 0x55b46750a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954da2b700 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4675046b0 0x55b467504ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 -- 172.21.15.19:0/3753566506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b467403560 msgr2=0x55b46750a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b467403560 0x55b46750a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 -- 172.21.15.19:0/3753566506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4675046b0 msgr2=0x55b467504ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4675046b0 0x55b467504ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954ca29700 1 -- 172.21.15.19:0/3753566506 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b467372a60 con 0x55b467385fd0 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.993+0000 7f954d22a700 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b467403560 0x55b46750a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.994+0000 7f954da2b700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f95340b2880 0x7f95340b4d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9540009fd0 tx=0x7f95400093c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.994+0000 7f954ca29700 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750ab30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9538009cb0 tx=0x7f9538011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2588b27dcd075b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.994+0000 7f95337fe700 1 -- 172.21.15.19:0/3753566506 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538011dd0 con 0x55b467385fd0 2024-05-17T09:42:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.994+0000 7f95337fe700 1 -- 172.21.15.19:0/3753566506 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95380044e0 con 0x55b467385fd0 2024-05-17T09:42:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.994+0000 7f95337fe700 1 -- 172.21.15.19:0/3753566506 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538005280 con 0x55b467385fd0 2024-05-17T09:42:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 -- 172.21.15.19:0/3753566506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 msgr2=0x55b46750ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750ab30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9538009cb0 tx=0x7f9538011440 comp rx=0 tx=0).stop 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 -- 172.21.15.19:0/3753566506 shutdown_connections 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4675046b0 0x55b467504ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b467403560 0x55b46750a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 --2- 172.21.15.19:0/3753566506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750ab30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.995+0000 7f955dea3b80 1 -- 172.21.15.19:0/3753566506 >> 172.21.15.19:0/3753566506 conn(0x55b4673c4e10 msgr2=0x55b4674ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.996+0000 7f955dea3b80 1 -- 172.21.15.19:0/3753566506 shutdown_connections 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.996+0000 7f955dea3b80 1 -- 172.21.15.19:0/3753566506 wait complete. 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.996+0000 7f955dea3b80 1 -- start start 2024-05-17T09:42:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.996+0000 7f955dea3b80 1 Processor -- start 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b467329130 con 0x55b467403560 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b4673c4950 con 0x55b4675046b0 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b46751a4c0 con 0x55b467385fd0 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f955dea3b80 10 client.0 _ll_register_callbacks cb 0x55b4673bdab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954da2b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54324/0 (socket says 172.21.15.19:54324) 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 -- 172.21.15.19:0/266704231 learned_addr learned my addr 172.21.15.19:0/266704231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954d22a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 -- 172.21.15.19:0/266704231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 msgr2=0x55b46750e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 -- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 msgr2=0x55b46750ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.997+0000 7f954ca29700 1 -- 172.21.15.19:0/266704231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4673c1820 con 0x55b467403560 2024-05-17T09:42:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f954ca29700 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750e7d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9538000c00 tx=0x7f9538013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c51dca693afd15 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f9530ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f9530ff9700 1 -- 172.21.15.19:0/266704231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538026020 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95380044e0 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f9530ff9700 1 -- 172.21.15.19:0/266704231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9538011de0 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f9530ff9700 1 -- 172.21.15.19:0/266704231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9538013ea0 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f955dea3b80 10 client.18903 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.998+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b46751bdb0 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f9530ff9700 1 -- 172.21.15.19:0/266704231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95380044e0 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f9530ff9700 1 -- 172.21.15.19:0/266704231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9538033020 con 0x55b467403560 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f955dea3b80 10 client.18903 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f955dea3b80 10 client.18903 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f955dea3b80 20 client.18903 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f955dea3b80 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b46751e390 0x55b4675207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f955dea3b80 4 client.18903 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b467434d70 con 0x55b46751e390 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:42.999+0000 7f954d22a700 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b46751e390 0x55b4675207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.000+0000 7f9530ff9700 10 client.18903 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.000+0000 7f954d22a700 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b46751e390 0x55b4675207b0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e4c32f98026673 server_cookie=635f26b7ea52c5b7 in_seq=0 out_seq=0 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f9530ff9700 1 -- 172.21.15.19:0/266704231 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b467434d70 con 0x55b46751e390 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f9530ff9700 10 client.18903 handle_command_reply: tid=0 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f955dea3b80 1 client.18903 shutdown 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 msgr2=0x55b46750e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f955dea3b80 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750e7d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9538000c00 tx=0x7f9538013c80 comp rx=0 tx=0).stop 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 shutdown_connections 2024-05-17T09:42:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f955dea3b80 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.001+0000 7f955dea3b80 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b46751e390 0x55b4675207b0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f955dea3b80 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f955dea3b80 1 --2- 172.21.15.19:0/266704231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750e7d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 >> 172.21.15.19:0/266704231 conn(0x55b4673c4e10 msgr2=0x55b4674f9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f9530ff9700 0 client.18903 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 shutdown_connections 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f955dea3b80 1 -- 172.21.15.19:0/266704231 wait complete. 2024-05-17T09:42:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.002+0000 7f955dea3b80 20 client.18903 trim_cache size 0 max 16384 2024-05-17T09:42:43.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.035+0000 7f955dea3b80 1 Processor -- start 2024-05-17T09:42:43.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 -- start start 2024-05-17T09:42:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b467525e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b467526390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f952c011400 con 0x55b467403560 2024-05-17T09:42:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f952c00ec90 con 0x55b4675046b0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f955dea3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f952c00cae0 con 0x55b467385fd0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954ca29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b467525e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954ca29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b467525e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54334/0 (socket says 172.21.15.19:54334) 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954ca29700 1 -- 172.21.15.19:0/228269605 learned_addr learned my addr 172.21.15.19:0/228269605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954d22a700 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954da2b700 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b467526390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954d22a700 1 -- 172.21.15.19:0/228269605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 msgr2=0x55b467526390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954d22a700 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b467526390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.036+0000 7f954d22a700 1 -- 172.21.15.19:0/228269605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 msgr2=0x55b467525e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.037+0000 7f954d22a700 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b467525e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.037+0000 7f954d22a700 1 -- 172.21.15.19:0/228269605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b467500450 con 0x55b467385fd0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.037+0000 7f954d22a700 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750dbb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9544010c60 tx=0x7f9544091040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2cef87a78fd371a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.037+0000 7f9532ffd700 1 -- 172.21.15.19:0/228269605 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954408f070 con 0x55b467385fd0 2024-05-17T09:42:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.037+0000 7f9532ffd700 1 -- 172.21.15.19:0/228269605 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95440a99b0 con 0x55b467385fd0 2024-05-17T09:42:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.037+0000 7f9532ffd700 1 -- 172.21.15.19:0/228269605 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95440cc0c0 con 0x55b467385fd0 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 -- 172.21.15.19:0/228269605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 msgr2=0x55b46750dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750dbb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9544010c60 tx=0x7f9544091040 comp rx=0 tx=0).stop 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 -- 172.21.15.19:0/228269605 shutdown_connections 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750dbb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b467526390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 --2- 172.21.15.19:0/228269605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b467525e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 -- 172.21.15.19:0/228269605 >> 172.21.15.19:0/228269605 conn(0x55b4673c4e10 msgr2=0x55b467508570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.038+0000 7f955dea3b80 1 -- 172.21.15.19:0/228269605 shutdown_connections 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.039+0000 7f955dea3b80 1 -- 172.21.15.19:0/228269605 wait complete. 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.039+0000 7f955dea3b80 1 -- start start 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.039+0000 7f955dea3b80 1 Processor -- start 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.039+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f955dea3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f955dea3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f952c00d150 con 0x55b467403560 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f955dea3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f952c00e5c0 con 0x55b4675046b0 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f955dea3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f952c00d820 con 0x55b467385fd0 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f955dea3b80 10 client.0 _ll_register_callbacks cb 0x55b4673bdab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:50924/0 (socket says 172.21.15.19:50924) 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 -- 172.21.15.19:0/4031481246 learned_addr learned my addr 172.21.15.19:0/4031481246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954ca29700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954d22a700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 -- 172.21.15.19:0/4031481246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 msgr2=0x55b46750cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 -- 172.21.15.19:0/4031481246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 msgr2=0x55b46750d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.040+0000 7f954da2b700 1 -- 172.21.15.19:0/4031481246 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b467587d20 con 0x55b4675046b0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f954da2b700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750da40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f954000b0d0 tx=0x7f954000b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8da41921ea7b6a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f9530ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f9530ff9700 1 -- 172.21.15.19:0/4031481246 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540035430 con 0x55b4675046b0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f9530ff9700 1 -- 172.21.15.19:0/4031481246 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f954001c410 con 0x55b4675046b0 2024-05-17T09:42:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95440a99b0 con 0x55b4675046b0 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f9530ff9700 1 -- 172.21.15.19:0/4031481246 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f954001c610 con 0x55b4675046b0 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f955dea3b80 10 client.28415 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.041+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b467588e20 con 0x55b4675046b0 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f9530ff9700 1 -- 172.21.15.19:0/4031481246 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f954001cc40 con 0x55b4675046b0 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f9530ff9700 1 -- 172.21.15.19:0/4031481246 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9540011070 con 0x55b4675046b0 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f955dea3b80 10 client.28415 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f955dea3b80 10 client.28415 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f955dea3b80 20 client.28415 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f955dea3b80 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4675895a0 0x55b46758b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f955dea3b80 4 client.28415 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-17T09:42:43.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.042+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7f95440a99b0 con 0x55b4675895a0 2024-05-17T09:42:43.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.043+0000 7f954d22a700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4675895a0 0x55b46758b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:43.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.043+0000 7f9530ff9700 10 client.28415 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:43.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.043+0000 7f954d22a700 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4675895a0 0x55b46758b9c0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdeada249386cdb2 server_cookie=e3b8c0e8de22786d in_seq=0 out_seq=0 2024-05-17T09:42:43.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.436+0000 7f9530ff9700 1 -- 172.21.15.19:0/4031481246 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f95440a99b0 con 0x55b4675895a0 2024-05-17T09:42:43.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.436+0000 7f9530ff9700 10 client.28415 handle_command_reply: tid=0 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.436+0000 7f955dea3b80 1 client.28415 shutdown 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 msgr2=0x55b46750da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750da40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f954000b0d0 tx=0x7f954000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 shutdown_connections 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b467385fd0 0x55b46750cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55b4675895a0 0x55b46758b9c0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4675046b0 0x55b46750da40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f9530ff9700 0 client.28415 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 --2- 172.21.15.19:0/4031481246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b467403560 0x55b46750d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 >> 172.21.15.19:0/4031481246 conn(0x55b4673c4e10 msgr2=0x55b4675036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 shutdown_connections 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.437+0000 7f955dea3b80 1 -- 172.21.15.19:0/4031481246 wait complete. 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.438+0000 7f955dea3b80 20 client.28415 trim_cache size 0 max 16384 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:42:43.438 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 82, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "f7062e47": { 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 82, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 130.0, 2024-05-17T09:42:43.439 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:43.440 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f95340b2880 msgr2=0x7f95340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f95340b2880 0x7f95340b4d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9540009fd0 tx=0x7f95400093c0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 msgr2=0x7f95481999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 0x7f95481999f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f954400e9a0 tx=0x7f954400ecb0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 shutdown_connections 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548108730 0x7f954819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95481034b0 0x7f95481999f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f95340b2880 0x7f95340b4d90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 --2- 172.21.15.19:0/190898782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9548107de0 0x7f9548199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.439+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 >> 172.21.15.19:0/190898782 conn(0x7f95480fedb0 msgr2=0x7f95481075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.440+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 shutdown_connections 2024-05-17T09:42:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:43.440+0000 7f954f48e700 1 -- 172.21.15.19:0/190898782 wait complete. 2024-05-17T09:42:43.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.55 MB/sec execute 400 sec latency 368.526 ms 2024-05-17T09:42:44.167 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f7062e47', quiesce took 7.7 seconds. Will release after: 2.3 2024-05-17T09:42:44.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.54 MB/sec execute 401 sec latency 1262.319 ms 2024-05-17T09:42:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:44 smithi069 ceph-mon[35081]: pgmap v1127: 129 pgs: 129 active+clean; 479 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-05-17T09:42:44.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:44 smithi019 ceph-mon[31620]: pgmap v1127: 129 pgs: 129 active+clean; 479 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-05-17T09:42:45.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:44 smithi079 ceph-mon[35502]: pgmap v1127: 129 pgs: 129 active+clean; 479 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-05-17T09:42:45.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.52 MB/sec execute 402 sec latency 2262.369 ms 2024-05-17T09:42:46.469 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:42:46.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.50 MB/sec execute 403 sec latency 3262.404 ms 2024-05-17T09:42:46.735 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:46.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:46 smithi019 ceph-mon[31620]: pgmap v1128: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 296 op/s 2024-05-17T09:42:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:46 smithi069 ceph-mon[35081]: pgmap v1128: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 296 op/s 2024-05-17T09:42:47.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:46 smithi079 ceph-mon[35502]: pgmap v1128: 129 pgs: 129 active+clean; 482 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 296 op/s 2024-05-17T09:42:47.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.49 MB/sec execute 404 sec latency 4262.463 ms 2024-05-17T09:42:48.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.47 MB/sec execute 405 sec latency 5262.492 ms 2024-05-17T09:42:48.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:48 smithi019 ceph-mon[31620]: pgmap v1129: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-05-17T09:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.849+0000 7fda38445700 1 -- 172.21.15.19:0/171175581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 msgr2=0x7fda301111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.849+0000 7fda38445700 1 --2- 172.21.15.19:0/171175581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda301111a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fda2800b0d0 tx=0x7fda2800b3e0 comp rx=0 tx=0).stop 2024-05-17T09:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.849+0000 7fda38445700 1 -- 172.21.15.19:0/171175581 shutdown_connections 2024-05-17T09:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.849+0000 7fda38445700 1 --2- 172.21.15.19:0/171175581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda301111a0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.849+0000 7fda38445700 1 --2- 172.21.15.19:0/171175581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda301045c0 0x7fda301049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.849+0000 7fda38445700 1 --2- 172.21.15.19:0/171175581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda30108ef0 0x7fda30109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.850+0000 7fda38445700 1 -- 172.21.15.19:0/171175581 >> 172.21.15.19:0/171175581 conn(0x7fda300ffe90 msgr2=0x7fda301022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.850+0000 7fda38445700 1 -- 172.21.15.19:0/171175581 shutdown_connections 2024-05-17T09:42:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.850+0000 7fda38445700 1 -- 172.21.15.19:0/171175581 wait complete. 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.850+0000 7fda38445700 1 Processor -- start 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.850+0000 7fda38445700 1 -- start start 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda38445700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda301045c0 0x7fda301a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda38445700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda30108ef0 0x7fda301a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda38445700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda3019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda38445700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda30114320 con 0x7fda30109840 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda38445700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda301141a0 con 0x7fda30108ef0 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda38445700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fda301144a0 con 0x7fda301045c0 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda3019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda3019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52980/0 (socket says 172.21.15.19:52980) 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 -- 172.21.15.19:0/977003050 learned_addr learned my addr 172.21.15.19:0/977003050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda359e0700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda30108ef0 0x7fda301a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda361e1700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda301045c0 0x7fda301a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 -- 172.21.15.19:0/977003050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda301045c0 msgr2=0x7fda301a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda301045c0 0x7fda301a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 -- 172.21.15.19:0/977003050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda30108ef0 msgr2=0x7fda301a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda30108ef0 0x7fda301a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 -- 172.21.15.19:0/977003050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3019b150 con 0x7fda30109840 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.851+0000 7fda369e2700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda3019aaa0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fda28007d90 tx=0x7fda28007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.852+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2801f030 con 0x7fda30109840 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.852+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda2801bb90 con 0x7fda30109840 2024-05-17T09:42:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.852+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28012c90 con 0x7fda30109840 2024-05-17T09:42:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.852+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3019b3e0 con 0x7fda30109840 2024-05-17T09:42:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.852+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a7710 con 0x7fda30109840 2024-05-17T09:42:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.853+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda300512b0 con 0x7fda30109840 2024-05-17T09:42:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.854+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fda28022030 con 0x7fda30109840 2024-05-17T09:42:48.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.855+0000 7fda237fe700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda1c0b2880 0x7fda1c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:48.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.855+0000 7fda361e1700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda1c0b2880 0x7fda1c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:48.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.855+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fda2800f020 con 0x7fda30109840 2024-05-17T09:42:48.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.856+0000 7fda361e1700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda1c0b2880 0x7fda1c0b4d90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fda2c005950 tx=0x7fda2c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:48.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:48.859+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda28029020 con 0x7fda30109840 2024-05-17T09:42:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:48 smithi069 ceph-mon[35081]: pgmap v1129: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-05-17T09:42:49.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:48 smithi079 ceph-mon[35502]: pgmap v1129: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.1 MiB/s wr, 254 op/s 2024-05-17T09:42:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.124+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda3002e5e0 con 0x7fda30109840 2024-05-17T09:42:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.125+0000 7fda237fe700 1 -- 172.21.15.19:0/977003050 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fda280ab950 con 0x7fda30109840 2024-05-17T09:42:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:42:49.125 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda1c0b2880 msgr2=0x7fda1c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda1c0b2880 0x7fda1c0b4d90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fda2c005950 tx=0x7fda2c005ed0 comp rx=0 tx=0).stop 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 msgr2=0x7fda3019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda3019aaa0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fda28007d90 tx=0x7fda28007e70 comp rx=0 tx=0).stop 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 shutdown_connections 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fda1c0b2880 0x7fda1c0b4d90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda301045c0 0x7fda301a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30109840 0x7fda3019aaa0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.127+0000 7fda38445700 1 --2- 172.21.15.19:0/977003050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda30108ef0 0x7fda301a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.128+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 >> 172.21.15.19:0/977003050 conn(0x7fda300ffe90 msgr2=0x7fda30108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.128+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 shutdown_connections 2024-05-17T09:42:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:49.128+0000 7fda38445700 1 -- 172.21.15.19:0/977003050 wait complete. 2024-05-17T09:42:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:42:49.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.45 MB/sec execute 406 sec latency 6262.520 ms 2024-05-17T09:42:49.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:49 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/977003050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:49.804 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id f7062e47 --release --await' 2024-05-17T09:42:49.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id f7062e47 --release --await 2024-05-17T09:42:49.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:49 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/977003050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:50.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:49 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/977003050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:42:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:42:50.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.44 MB/sec execute 407 sec latency 7262.549 ms 2024-05-17T09:42:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 -- 172.21.15.19:0/3418888644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8103d80 msgr2=0x7fb0b8104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 --2- 172.21.15.19:0/3418888644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8103d80 0x7fb0b8104190 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009900 tx=0x7fb0a002a410 comp rx=0 tx=0).stop 2024-05-17T09:42:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 -- 172.21.15.19:0/3418888644 shutdown_connections 2024-05-17T09:42:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 --2- 172.21.15.19:0/3418888644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0b8108a80 0x7fb0b810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 --2- 172.21.15.19:0/3418888644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8103d80 0x7fb0b8104190 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 --2- 172.21.15.19:0/3418888644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8102b40 0x7fb0b8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 -- 172.21.15.19:0/3418888644 >> 172.21.15.19:0/3418888644 conn(0x7fb0b80fdd30 msgr2=0x7fb0b81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 -- 172.21.15.19:0/3418888644 shutdown_connections 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 -- 172.21.15.19:0/3418888644 wait complete. 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 Processor -- start 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.816+0000 7fb0bf205640 1 -- start start 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 0x7fb0b806b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0b8103d80 0x7fb0b819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8108a80 0x7fb0b81a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810e5c0 con 0x7fb0b8102b40 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810e440 con 0x7fb0b8108a80 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810e740 con 0x7fb0b8103d80 2024-05-17T09:42:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 0x7fb0b806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 0x7fb0b806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52992/0 (socket says 172.21.15.19:52992) 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 -- 172.21.15.19:0/2907086586 learned_addr learned my addr 172.21.15.19:0/2907086586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8108a80 0x7fb0b81a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0b8103d80 0x7fb0b819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 -- 172.21.15.19:0/2907086586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0b8103d80 msgr2=0x7fb0b819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0b8103d80 0x7fb0b819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 -- 172.21.15.19:0/2907086586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8108a80 msgr2=0x7fb0b81a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8108a80 0x7fb0b81a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 -- 172.21.15.19:0/2907086586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b81a1d50 con 0x7fb0b8102b40 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8108a80 0x7fb0b81a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bcfa1640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 0x7fb0b806b120 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d740 tx=0x7fb0b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0b5ffb640 1 -- 172.21.15.19:0/2907086586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0b8102b40 2024-05-17T09:42:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0b5ffb640 1 -- 172.21.15.19:0/2907086586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0b000de50 con 0x7fb0b8102b40 2024-05-17T09:42:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0b5ffb640 1 -- 172.21.15.19:0/2907086586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0003bb0 con 0x7fb0b8102b40 2024-05-17T09:42:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.817+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b81a1fe0 con 0x7fb0b8102b40 2024-05-17T09:42:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.818+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a2570 con 0x7fb0b8102b40 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.820+0000 7fb0c6194740 1 Processor -- start 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.820+0000 7fb0c6194740 1 -- start start 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.820+0000 7fb0b5ffb640 1 -- 172.21.15.19:0/2907086586 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb0b0003d50 con 0x7fb0b8102b40 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349de63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349de6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349ded820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0c6194740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565349d58c30 con 0x565349a451b0 2024-05-17T09:42:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0c6194740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565349d487d0 con 0x565349a1c2f0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0c6194740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565349d4c810 con 0x565349a244d0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349de63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349de63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52996/0 (socket says 172.21.15.19:52996) 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3571648408 learned_addr learned my addr 172.21.15.19:0/3571648408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349ded820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bcfa1640 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349de6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3571648408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 msgr2=0x565349ded820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349ded820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3571648408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 msgr2=0x565349de6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349de6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3571648408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565349a5d210 con 0x565349a451b0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349de63a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c450 tx=0x7fb0a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e769087fa78454 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0b5ffb640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0a40b2920 0x7fb0a40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0aeffd640 1 -- 172.21.15.19:0/3571648408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8007430 con 0x565349a451b0 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0bcfa1640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0a40b2920 0x7fb0a40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.821+0000 7fb0aeffd640 1 -- 172.21.15.19:0/3571648408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a8007a60 con 0x565349a451b0 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0aeffd640 1 -- 172.21.15.19:0/3571648408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800a920 con 0x565349a451b0 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0b5ffb640 1 -- 172.21.15.19:0/2907086586 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0b00e1fd0 con 0x7fb0b8102b40 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 -- 172.21.15.19:0/3571648408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 msgr2=0x565349de63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0bcfa1640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0a40b2920 0x7fb0a40b4e30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x565349de7780 tx=0x7fb0b00e7040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349de63a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c450 tx=0x7fb0a800c910 comp rx=0 tx=0).stop 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 -- 172.21.15.19:0/3571648408 shutdown_connections 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349ded820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349de6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3571648408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349de63a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 -- 172.21.15.19:0/3571648408 >> 172.21.15.19:0/3571648408 conn(0x565349a39300 msgr2=0x565349de1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 -- 172.21.15.19:0/3571648408 shutdown_connections 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.822+0000 7fb0c6194740 1 -- 172.21.15.19:0/3571648408 wait complete. 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 -- start start 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 Processor -- start 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349defc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349df01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349df7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565349a2e790 con 0x565349a451b0 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565349d04890 con 0x565349a1c2f0 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565349dee270 con 0x565349a244d0 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0c6194740 10 client.0 _ll_register_callbacks cb 0x565349d07d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349df7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349df7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52998/0 (socket says 172.21.15.19:52998) 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3453930191 learned_addr learned my addr 172.21.15.19:0/3453930191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.823+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349defc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349df01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3453930191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 msgr2=0x565349df01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349df01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 msgr2=0x565349defc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349defc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3453930191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565349b29620 con 0x565349a451b0 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349df7120 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c420 tx=0x7fb0a8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b48af7354cb91ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb097fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb097fff640 1 -- 172.21.15.19:0/3453930191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800a8b0 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb097fff640 1 -- 172.21.15.19:0/3453930191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0a8012040 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8007a60 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb097fff640 1 -- 172.21.15.19:0/3453930191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0a800aa50 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0c6194740 10 client.18927 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565349dfa050 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.824+0000 7fb097fff640 1 -- 172.21.15.19:0/3453930191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a8007a60 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb097fff640 1 -- 172.21.15.19:0/3453930191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb0a801e030 con 0x565349a451b0 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0c6194740 10 client.18927 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0c6194740 10 client.18927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0c6194740 20 client.18927 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a03d80 0x565349dfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0c6194740 4 client.18927 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565349d5ab80 con 0x565349a03d80 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a03d80 0x565349dfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb097fff640 10 client.18927 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.825+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a03d80 0x565349dfd590 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e7eb1b845b5a45 server_cookie=3f6ec816066a1e5f in_seq=0 out_seq=0 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb097fff640 1 -- 172.21.15.19:0/3453930191 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565349d5ab80 con 0x565349a03d80 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb097fff640 10 client.18927 handle_command_reply: tid=0 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 client.18927 shutdown 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 msgr2=0x565349df7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349df7120 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c420 tx=0x7fb0a8004b50 comp rx=0 tx=0).stop 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 shutdown_connections 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a03d80 0x565349dfd590 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a451b0 0x565349df7120 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a244d0 0x565349df01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3453930191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a1c2f0 0x565349defc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb097fff640 0 client.18927 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 >> 172.21.15.19:0/3453930191 conn(0x565349a39300 msgr2=0x565349de0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.826+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 shutdown_connections 2024-05-17T09:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.827+0000 7fb0c6194740 1 -- 172.21.15.19:0/3453930191 wait complete. 2024-05-17T09:42:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.827+0000 7fb0c6194740 20 client.18927 trim_cache size 0 max 16384 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 Processor -- start 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 -- start start 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a03d80 0x565349de1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 0x565349df7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81af470 con 0x565349a244d0 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81acd40 con 0x565349a03d80 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0c6194740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81aab90 con 0x565349a1c2f0 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 0x565349df7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 0x565349df7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53008/0 (socket says 172.21.15.19:53008) 2024-05-17T09:42:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/1069552327 learned_addr learned my addr 172.21.15.19:0/1069552327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.842+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a03d80 0x565349de1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/1069552327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 msgr2=0x565349df7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/1069552327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a03d80 msgr2=0x565349de1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a03d80 0x565349de1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/1069552327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565349ddd540 con 0x565349a244d0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 0x565349df7580 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007160 tx=0x7fb0a8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe98a36b342af3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0ae7fc640 1 -- 172.21.15.19:0/1069552327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800a8b0 con 0x565349a244d0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0ae7fc640 1 -- 172.21.15.19:0/1069552327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a800ce10 con 0x565349a244d0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.843+0000 7fb0ae7fc640 1 -- 172.21.15.19:0/1069552327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800a400 con 0x565349a244d0 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- 172.21.15.19:0/1069552327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 msgr2=0x565349df7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 0x565349df7580 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007160 tx=0x7fb0a8009960 comp rx=0 tx=0).stop 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- 172.21.15.19:0/1069552327 shutdown_connections 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a244d0 0x565349df7580 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- 172.21.15.19:0/1069552327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a03d80 0x565349de1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- 172.21.15.19:0/1069552327 >> 172.21.15.19:0/1069552327 conn(0x565349a39300 msgr2=0x565349de16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- 172.21.15.19:0/1069552327 shutdown_connections 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- 172.21.15.19:0/1069552327 wait complete. 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- start start 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 Processor -- start 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 0x565349df7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a244d0 0x565349df4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ab200 con 0x565349a03d80 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ac670 con 0x565349a244d0 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.844+0000 7fb0c6194740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ab8d0 con 0x565349a1c2f0 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0c6194740 10 client.0 _ll_register_callbacks cb 0x7fb0a8007880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 0x565349df7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 0x565349df7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53010/0 (socket says 172.21.15.19:53010) 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3966047195 learned_addr learned my addr 172.21.15.19:0/3966047195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3966047195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 msgr2=0x565349df4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a244d0 msgr2=0x565349df4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a244d0 0x565349df4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 -- 172.21.15.19:0/3966047195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565349e63c80 con 0x565349a03d80 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0bd7a2640 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 0x565349df7880 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8004a80 tx=0x7fb0a800a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b83107405508897 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb097fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb097fff640 1 -- 172.21.15.19:0/3966047195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800a8b0 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb097fff640 1 -- 172.21.15.19:0/3966047195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0a801b070 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a800ce10 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb097fff640 1 -- 172.21.15.19:0/3966047195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0a80094f0 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0c6194740 10 client.18933 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.845+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565349e64e60 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb097fff640 1 -- 172.21.15.19:0/3966047195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0a800ce10 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb097fff640 1 -- 172.21.15.19:0/3966047195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb0a801e040 con 0x565349a03d80 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0c6194740 10 client.18933 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0c6194740 10 client.18933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0c6194740 20 client.18933 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a451b0 0x565349e677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0c6194740 4 client.18933 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565349d54360 con 0x565349a451b0 2024-05-17T09:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a451b0 0x565349e677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb0b7fff640 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a451b0 0x565349e677f0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea1e9f3b586e433 server_cookie=7c120204143eaa50 in_seq=0 out_seq=0 2024-05-17T09:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.846+0000 7fb097fff640 10 client.18933 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.897+0000 7fb097fff640 1 -- 172.21.15.19:0/3966047195 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565349d54360 con 0x565349a451b0 2024-05-17T09:42:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.897+0000 7fb097fff640 10 client.18933 handle_command_reply: tid=0 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.897+0000 7fb0c6194740 1 client.18933 shutdown 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 msgr2=0x565349df7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 0x565349df7880 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8004a80 tx=0x7fb0a800a630 comp rx=0 tx=0).stop 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 shutdown_connections 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565349a451b0 0x565349e677f0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565349a244d0 0x565349df4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565349a1c2f0 0x565349df4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb097fff640 0 client.18933 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 --2- 172.21.15.19:0/3966047195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565349a03d80 0x565349df7880 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 >> 172.21.15.19:0/3966047195 conn(0x565349a39300 msgr2=0x565349de4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 shutdown_connections 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 1 -- 172.21.15.19:0/3966047195 wait complete. 2024-05-17T09:42:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0c6194740 20 client.18933 trim_cache size 0 max 16384 2024-05-17T09:42:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:42:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:42:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0a40b2920 msgr2=0x7fb0a40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0bf205640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0a40b2920 0x7fb0a40b4e30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x565349de7780 tx=0x7fb0b00e7040 comp rx=0 tx=0).stop 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 msgr2=0x7fb0b806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.898+0000 7fb0bf205640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 0x7fb0b806b120 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d740 tx=0x7fb0b000dc00 comp rx=0 tx=0).stop 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 shutdown_connections 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0b8108a80 0x7fb0b81a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0b8103d80 0x7fb0b819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0a40b2920 0x7fb0a40b4e30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 --2- 172.21.15.19:0/2907086586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b8102b40 0x7fb0b806b120 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 >> 172.21.15.19:0/2907086586 conn(0x7fb0b80fdd30 msgr2=0x7fb0b8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 shutdown_connections 2024-05-17T09:42:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:42:50.899+0000 7fb0bf205640 1 -- 172.21.15.19:0/2907086586 wait complete. 2024-05-17T09:42:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:50 smithi069 ceph-mon[35081]: pgmap v1130: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-05-17T09:42:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:50 smithi019 ceph-mon[31620]: pgmap v1130: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-05-17T09:42:51.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:50 smithi079 ceph-mon[35502]: pgmap v1130: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-05-17T09:42:51.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63406 6.42 MB/sec execute 408 sec latency 8262.582 ms 2024-05-17T09:42:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.877+0000 7f9c8b028700 1 -- 172.21.15.19:0/112263380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c84107e00 msgr2=0x7f9c84108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.877+0000 7f9c8b028700 1 --2- 172.21.15.19:0/112263380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c84107e00 0x7f9c84108210 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9c780099c0 tx=0x7f9c78009cd0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.877+0000 7f9c8b028700 1 -- 172.21.15.19:0/112263380 shutdown_connections 2024-05-17T09:42:51.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.877+0000 7f9c8b028700 1 --2- 172.21.15.19:0/112263380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c84108750 0x7f9c841100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.877+0000 7f9c8b028700 1 --2- 172.21.15.19:0/112263380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c841034d0 0x7f9c841038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.878+0000 7f9c8b028700 1 --2- 172.21.15.19:0/112263380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c84107e00 0x7f9c84108210 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.878+0000 7f9c8b028700 1 -- 172.21.15.19:0/112263380 >> 172.21.15.19:0/112263380 conn(0x7f9c840fef40 msgr2=0x7f9c84101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.878+0000 7f9c8b028700 1 -- 172.21.15.19:0/112263380 shutdown_connections 2024-05-17T09:42:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.878+0000 7f9c8b028700 1 -- 172.21.15.19:0/112263380 wait complete. 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.878+0000 7f9c8b028700 1 Processor -- start 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.878+0000 7f9c8b028700 1 -- start start 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c8b028700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 0x7f9c8419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c8b028700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c84107e00 0x7f9c8419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c8b028700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c84108750 0x7f9c84199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c8b028700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84113210 con 0x7f9c841034d0 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c8b028700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84113090 con 0x7f9c84108750 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c8b028700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84113390 con 0x7f9c84107e00 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 0x7f9c8419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 0x7f9c8419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53026/0 (socket says 172.21.15.19:53026) 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 -- 172.21.15.19:0/1930971104 learned_addr learned my addr 172.21.15.19:0/1930971104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c83fff700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c84107e00 0x7f9c8419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c895c5700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c84108750 0x7f9c84199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 -- 172.21.15.19:0/1930971104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c84107e00 msgr2=0x7f9c8419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c84107e00 0x7f9c8419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 -- 172.21.15.19:0/1930971104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c84108750 msgr2=0x7f9c84199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c84108750 0x7f9c84199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.879+0000 7f9c88dc4700 1 -- 172.21.15.19:0/1930971104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8419a160 con 0x7f9c841034d0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.880+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 0x7f9c8419f890 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400ada0 tx=0x7f9c7400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.880+0000 7f9c81ffb700 1 -- 172.21.15.19:0/1930971104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74017020 con 0x7f9c841034d0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.880+0000 7f9c81ffb700 1 -- 172.21.15.19:0/1930971104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c7400ce10 con 0x7f9c841034d0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.880+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8419a450 con 0x7f9c841034d0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.880+0000 7f9c81ffb700 1 -- 172.21.15.19:0/1930971104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74005310 con 0x7f9c841034d0 2024-05-17T09:42:51.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.880+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a6720 con 0x7f9c841034d0 2024-05-17T09:42:51.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.883+0000 7f9c81ffb700 1 -- 172.21.15.19:0/1930971104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c740054b0 con 0x7f9c841034d0 2024-05-17T09:42:51.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.883+0000 7f9c99a3db80 1 Processor -- start 2024-05-17T09:42:51.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.883+0000 7f9c81ffb700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c6c0b2880 0x7f9c6c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.883+0000 7f9c99a3db80 1 -- start start 2024-05-17T09:42:51.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c895c5700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c6c0b2880 0x7f9c6c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c81ffb700 1 -- 172.21.15.19:0/1930971104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9c740e1e50 con 0x7f9c841034d0 2024-05-17T09:42:51.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea208ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea208b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea2085020 0x557ea20854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ea1fb2e70 con 0x557ea1f05f40 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557ea1fa2a10 con 0x557ea1ff1530 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557ea1fa6a50 con 0x557ea2085020 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea208b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea208b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53032/0 (socket says 172.21.15.19:53032) 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 -- 172.21.15.19:0/1216646907 learned_addr learned my addr 172.21.15.19:0/1216646907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c895c5700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c6c0b2880 0x7f9c6c0b4d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ca50 tx=0x7f9c7c005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea208ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c895c5700 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea2085020 0x557ea20854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 -- 172.21.15.19:0/1216646907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea2085020 msgr2=0x557ea20854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea2085020 0x557ea20854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 -- 172.21.15.19:0/1216646907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 msgr2=0x557ea208ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea208ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.884+0000 7f9c83fff700 1 -- 172.21.15.19:0/1216646907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea1f4c0b0 con 0x557ea1f05f40 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.885+0000 7f9c83fff700 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea208b430 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f9c84076b30 tx=0x7f9c78005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165633bf11660ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.885+0000 7f9c6affd700 1 -- 172.21.15.19:0/1216646907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78027070 con 0x557ea1f05f40 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.885+0000 7f9c6affd700 1 -- 172.21.15.19:0/1216646907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c780055a0 con 0x557ea1f05f40 2024-05-17T09:42:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.885+0000 7f9c6affd700 1 -- 172.21.15.19:0/1216646907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003bf0 con 0x557ea1f05f40 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 -- 172.21.15.19:0/1216646907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 msgr2=0x557ea208b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea208b430 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f9c84076b30 tx=0x7f9c78005100 comp rx=0 tx=0).stop 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 -- 172.21.15.19:0/1216646907 shutdown_connections 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea2085020 0x557ea20854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea208ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/1216646907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea208b430 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 -- 172.21.15.19:0/1216646907 >> 172.21.15.19:0/1216646907 conn(0x557ea1f662c0 msgr2=0x557ea207fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 -- 172.21.15.19:0/1216646907 shutdown_connections 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 -- 172.21.15.19:0/1216646907 wait complete. 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 -- start start 2024-05-17T09:42:51.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.886+0000 7f9c99a3db80 1 Processor -- start 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea208eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 0x557ea208f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea208f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ea1ea90a0 con 0x557ea1ff1530 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557ea1f0aea0 con 0x557ea2085020 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557ea2091b20 con 0x557ea1f05f40 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c99a3db80 10 client.0 _ll_register_callbacks cb 0x557ea1f3da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 0x557ea208f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 0x557ea208f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53042/0 (socket says 172.21.15.19:53042) 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 -- 172.21.15.19:0/2621600777 learned_addr learned my addr 172.21.15.19:0/2621600777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea208eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c895c5700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea208f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 -- 172.21.15.19:0/2621600777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 msgr2=0x557ea208eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea208eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 -- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 msgr2=0x557ea208f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea208f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c83fff700 1 -- 172.21.15.19:0/2621600777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea1ef29d0 con 0x557ea1ff1530 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.887+0000 7f9c895c5700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea208f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c83fff700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 0x557ea208f000 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005ef0 tx=0x7f9c78003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca8e25b14a9202 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c63fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c63fff700 1 -- 172.21.15.19:0/2621600777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78027040 con 0x557ea1ff1530 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c63fff700 1 -- 172.21.15.19:0/2621600777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9c78003dc0 con 0x557ea1ff1530 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c780055a0 con 0x557ea1ff1530 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c63fff700 1 -- 172.21.15.19:0/2621600777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9c7803a050 con 0x557ea1ff1530 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c99a3db80 10 client.18945 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:51.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ea1f5c860 con 0x557ea1ff1530 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c63fff700 1 -- 172.21.15.19:0/2621600777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c780055a0 con 0x557ea1ff1530 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c63fff700 1 -- 172.21.15.19:0/2621600777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9c78022610 con 0x557ea1ff1530 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c99a3db80 10 client.18945 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.888+0000 7f9c99a3db80 10 client.18945 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c99a3db80 20 client.18945 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea209e790 0x557ea20a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c99a3db80 4 client.18945 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea2050fd0 con 0x557ea209e790 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea209e790 0x557ea20a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c63fff700 10 client.18945 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.889+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea209e790 0x557ea20a0bb0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb7fd17b1b2812 server_cookie=7560f6bb0c4ac3b3 in_seq=0 out_seq=0 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c63fff700 1 -- 172.21.15.19:0/2621600777 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ea2050fd0 con 0x557ea209e790 2024-05-17T09:42:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c63fff700 10 client.18945 handle_command_reply: tid=0 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 client.18945 shutdown 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 msgr2=0x557ea208f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 0x557ea208f000 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005ef0 tx=0x7f9c78003aa0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 shutdown_connections 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea208eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea209e790 0x557ea20a0bb0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea208f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/2621600777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1ff1530 0x557ea208f000 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 >> 172.21.15.19:0/2621600777 conn(0x557ea1f662c0 msgr2=0x557ea2079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c63fff700 0 client.18945 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 shutdown_connections 2024-05-17T09:42:51.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 1 -- 172.21.15.19:0/2621600777 wait complete. 2024-05-17T09:42:51.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.890+0000 7f9c99a3db80 20 client.18945 trim_cache size 0 max 16384 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.910+0000 7f9c99a3db80 1 Processor -- start 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 -- start start 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea2090f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea2084a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 0x557ea209ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b2c40 con 0x557ea2085020 2024-05-17T09:42:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b04d0 con 0x557ea1ff1530 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841ae320 con 0x557ea1f05f40 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 0x557ea209ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 0x557ea209ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53058/0 (socket says 172.21.15.19:53058) 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 -- 172.21.15.19:0/559685371 learned_addr learned my addr 172.21.15.19:0/559685371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea2090f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c83fff700 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea2084a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 -- 172.21.15.19:0/559685371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 msgr2=0x557ea2090f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea2090f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 -- 172.21.15.19:0/559685371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 msgr2=0x557ea2084a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea2084a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 -- 172.21.15.19:0/559685371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea20807c0 con 0x557ea2085020 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.911+0000 7f9c895c5700 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 0x557ea209ac00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ed20 tx=0x7f9c7c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab0810d090f51c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c6a7fc700 1 -- 172.21.15.19:0/559685371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c010d80 con 0x557ea2085020 2024-05-17T09:42:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c6a7fc700 1 -- 172.21.15.19:0/559685371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c7c022070 con 0x557ea2085020 2024-05-17T09:42:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c6a7fc700 1 -- 172.21.15.19:0/559685371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c01e6c0 con 0x557ea2085020 2024-05-17T09:42:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 -- 172.21.15.19:0/559685371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 msgr2=0x557ea209ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 0x557ea209ac00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ed20 tx=0x7f9c7c011000 comp rx=0 tx=0).stop 2024-05-17T09:42:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 -- 172.21.15.19:0/559685371 shutdown_connections 2024-05-17T09:42:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1f05f40 0x557ea2090f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea2085020 0x557ea209ac00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/559685371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea1ff1530 0x557ea2084a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.912+0000 7f9c99a3db80 1 -- 172.21.15.19:0/559685371 >> 172.21.15.19:0/559685371 conn(0x557ea1f662c0 msgr2=0x557ea207a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 -- 172.21.15.19:0/559685371 shutdown_connections 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 -- 172.21.15.19:0/559685371 wait complete. 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 -- start start 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 Processor -- start 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea20a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1ff1530 0x557ea20a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea20a3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841ae990 con 0x557ea1f05f40 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841afe00 con 0x557ea2085020 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841af060 con 0x557ea1ff1530 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.913+0000 7f9c99a3db80 10 client.0 _ll_register_callbacks cb 0x557ea1f3da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:42:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea20a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea20a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53062/0 (socket says 172.21.15.19:53062) 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 -- 172.21.15.19:0/3373892374 learned_addr learned my addr 172.21.15.19:0/3373892374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c895c5700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea20a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c83fff700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1ff1530 0x557ea20a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 -- 172.21.15.19:0/3373892374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1ff1530 msgr2=0x557ea20a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1ff1530 0x557ea20a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 -- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 msgr2=0x557ea20a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea20a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 -- 172.21.15.19:0/3373892374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea208f8b0 con 0x557ea1f05f40 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c88dc4700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea20a27e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400a0a0 tx=0x7f9c740a8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9874541518a974fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c63fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c63fff700 1 -- 172.21.15.19:0/3373892374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c740b1f20 con 0x557ea1f05f40 2024-05-17T09:42:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c63fff700 1 -- 172.21.15.19:0/3373892374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9c740ad120 con 0x557ea1f05f40 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7c022070 con 0x557ea1f05f40 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c63fff700 1 -- 172.21.15.19:0/3373892374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9c740a93b0 con 0x557ea1f05f40 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c99a3db80 10 client.18951 fetch_fsmap learned FSMap version 39 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.914+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ea210a350 con 0x557ea1f05f40 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c63fff700 1 -- 172.21.15.19:0/3373892374 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c7c022070 con 0x557ea1f05f40 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c63fff700 1 -- 172.21.15.19:0/3373892374 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9c74012070 con 0x557ea1f05f40 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c99a3db80 10 client.18951 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c99a3db80 10 client.18951 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c99a3db80 20 client.18951 populate_metadata read hostname 'smithi019' 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea210a9b0 0x557ea210cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c99a3db80 4 client.18951 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f7062e47", "release": true, "await": true}] 2024-05-17T09:42:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f7062e47", "release": true, "await": true}) -- 0x557ea1fb4dc0 con 0x557ea210a9b0 2024-05-17T09:42:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.915+0000 7f9c83fff700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea210a9b0 0x557ea210cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:42:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.916+0000 7f9c83fff700 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea210a9b0 0x557ea210cdd0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af028e43658c6ab8 server_cookie=af2020cd10152315 in_seq=0 out_seq=0 2024-05-17T09:42:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.916+0000 7f9c63fff700 10 client.18951 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c63fff700 1 -- 172.21.15.19:0/3373892374 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557ea1fb4dc0 con 0x557ea210a9b0 2024-05-17T09:42:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c63fff700 10 client.18951 handle_command_reply: tid=0 2024-05-17T09:42:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 client.18951 shutdown 2024-05-17T09:42:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 msgr2=0x557ea20a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea20a27e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400a0a0 tx=0x7f9c740a8cb0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 shutdown_connections 2024-05-17T09:42:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557ea210a9b0 0x557ea210cdd0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ea2085020 0x557ea20a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ea1f05f40 0x557ea20a27e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 --2- 172.21.15.19:0/3373892374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557ea1ff1530 0x557ea20a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 >> 172.21.15.19:0/3373892374 conn(0x557ea1f662c0 msgr2=0x557ea208d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.950+0000 7f9c63fff700 0 client.18951 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.951+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 shutdown_connections 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.951+0000 7f9c99a3db80 1 -- 172.21.15.19:0/3373892374 wait complete. 2024-05-17T09:42:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.951+0000 7f9c99a3db80 20 client.18951 trim_cache size 0 max 16384 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 84, 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "f7062e47": { 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 84, 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:42:51.952 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:51.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:42:51.953 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:42:51.953 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 130.0, 2024-05-17T09:42:51.953 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:42:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:42:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:42:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:42:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:42:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:42:51.954 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:51.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:51.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:51.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:51.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:42:51.955 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:42:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c6c0b2880 msgr2=0x7f9c6c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c6c0b2880 0x7f9c6c0b4d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ca50 tx=0x7f9c7c005d70 comp rx=0 tx=0).stop 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 msgr2=0x7f9c8419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 0x7f9c8419f890 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400ada0 tx=0x7f9c7400c330 comp rx=0 tx=0).stop 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 shutdown_connections 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c84108750 0x7f9c84199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841034d0 0x7f9c8419f890 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9c6c0b2880 0x7f9c6c0b4d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 --2- 172.21.15.19:0/1930971104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c84107e00 0x7f9c8419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.952+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 >> 172.21.15.19:0/1930971104 conn(0x7f9c840fef40 msgr2=0x7f9c841096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.953+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 shutdown_connections 2024-05-17T09:42:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:42:51.953+0000 7f9c8b028700 1 -- 172.21.15.19:0/1930971104 wait complete. 2024-05-17T09:42:52.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63518 6.42 MB/sec execute 409 sec latency 8627.063 ms 2024-05-17T09:42:52.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:52 smithi019 ceph-mon[31620]: pgmap v1131: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-05-17T09:42:52.876 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f7062e47', total seconds elapsed: 16.4 2024-05-17T09:42:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:52 smithi069 ceph-mon[35081]: pgmap v1131: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-05-17T09:42:53.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:52 smithi079 ceph-mon[35502]: pgmap v1131: 129 pgs: 129 active+clean; 483 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-05-17T09:42:53.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63687 6.42 MB/sec execute 410 sec latency 72.682 ms 2024-05-17T09:42:54.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63813 6.42 MB/sec execute 411 sec latency 82.084 ms 2024-05-17T09:42:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:54 smithi069 ceph-mon[35081]: pgmap v1132: 129 pgs: 129 active+clean; 484 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-17T09:42:54.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:54 smithi019 ceph-mon[31620]: pgmap v1132: 129 pgs: 129 active+clean; 484 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-17T09:42:55.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:54 smithi079 ceph-mon[35502]: pgmap v1132: 129 pgs: 129 active+clean; 484 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-05-17T09:42:55.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 63941 6.43 MB/sec execute 412 sec latency 77.401 ms 2024-05-17T09:42:56.592 INFO:tasks.workunit.client.0.smithi019.stdout: 10 64120 6.44 MB/sec execute 413 sec latency 78.036 ms 2024-05-17T09:42:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:56 smithi069 ceph-mon[35081]: pgmap v1133: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-05-17T09:42:56.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:56 smithi019 ceph-mon[31620]: pgmap v1133: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-05-17T09:42:57.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:56 smithi079 ceph-mon[35502]: pgmap v1133: 129 pgs: 129 active+clean; 481 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-05-17T09:42:57.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 64299 6.46 MB/sec execute 414 sec latency 85.994 ms 2024-05-17T09:42:58.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 64471 6.46 MB/sec execute 415 sec latency 124.306 ms 2024-05-17T09:42:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:42:58 smithi069 ceph-mon[35081]: pgmap v1134: 129 pgs: 129 active+clean; 487 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-17T09:42:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:42:58 smithi019 ceph-mon[31620]: pgmap v1134: 129 pgs: 129 active+clean; 487 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-17T09:42:59.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:42:58 smithi079 ceph-mon[35502]: pgmap v1134: 129 pgs: 129 active+clean; 487 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-05-17T09:42:59.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 64590 6.45 MB/sec execute 416 sec latency 94.501 ms 2024-05-17T09:43:00.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 64715 6.45 MB/sec execute 417 sec latency 103.209 ms 2024-05-17T09:43:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:00 smithi069 ceph-mon[35081]: pgmap v1135: 129 pgs: 129 active+clean; 489 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2024-05-17T09:43:00.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:00 smithi019 ceph-mon[31620]: pgmap v1135: 129 pgs: 129 active+clean; 489 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2024-05-17T09:43:01.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:00 smithi079 ceph-mon[35502]: pgmap v1135: 129 pgs: 129 active+clean; 489 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2024-05-17T09:43:01.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 64889 6.47 MB/sec execute 418 sec latency 100.802 ms 2024-05-17T09:43:02.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65060 6.48 MB/sec execute 419 sec latency 102.882 ms 2024-05-17T09:43:02.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:02 smithi079 ceph-mon[35502]: pgmap v1136: 129 pgs: 129 active+clean; 491 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.3 MiB/s wr, 136 op/s 2024-05-17T09:43:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:02 smithi069 ceph-mon[35081]: pgmap v1136: 129 pgs: 129 active+clean; 491 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.3 MiB/s wr, 136 op/s 2024-05-17T09:43:02.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:02 smithi019 ceph-mon[31620]: pgmap v1136: 129 pgs: 129 active+clean; 491 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.3 MiB/s wr, 136 op/s 2024-05-17T09:43:03.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65194 6.48 MB/sec execute 420 sec latency 109.866 ms 2024-05-17T09:43:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 -- 172.21.15.19:0/3174071661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 msgr2=0x7efe6c1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 --2- 172.21.15.19:0/3174071661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c1041a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe54009900 tx=0x7efe5402a410 comp rx=0 tx=0).stop 2024-05-17T09:43:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 -- 172.21.15.19:0/3174071661 shutdown_connections 2024-05-17T09:43:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 --2- 172.21.15.19:0/3174071661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 0x7efe6c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 --2- 172.21.15.19:0/3174071661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c1041a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 --2- 172.21.15.19:0/3174071661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 -- 172.21.15.19:0/3174071661 >> 172.21.15.19:0/3174071661 conn(0x7efe6c0fdd40 msgr2=0x7efe6c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 -- 172.21.15.19:0/3174071661 shutdown_connections 2024-05-17T09:43:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.048+0000 7efe720f2640 1 -- 172.21.15.19:0/3174071661 wait complete. 2024-05-17T09:43:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.049+0000 7efe720f2640 1 Processor -- start 2024-05-17T09:43:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.049+0000 7efe720f2640 1 -- start start 2024-05-17T09:43:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe720f2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c06bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe720f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe720f2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 0x7efe6c1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe720f2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10e550 con 0x7efe6c102b50 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe720f2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10e3d0 con 0x7efe6c103d90 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe720f2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10e6d0 con 0x7efe6c108a90 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c06bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c06bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51566/0 (socket says 172.21.15.19:51566) 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 -- 172.21.15.19:0/1698212955 learned_addr learned my addr 172.21.15.19:0/1698212955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6bfff640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 0x7efe6c1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6affd640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 -- 172.21.15.19:0/1698212955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 msgr2=0x7efe6c1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 0x7efe6c1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 -- 172.21.15.19:0/1698212955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 msgr2=0x7efe6c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6b7fe640 1 -- 172.21.15.19:0/1698212955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1a1cc0 con 0x7efe6c102b50 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6affd640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.050+0000 7efe6bfff640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 0x7efe6c1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.051+0000 7efe6b7fe640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c06bd90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e9f0 tx=0x7efe5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.051+0000 7efe68ff9640 1 -- 172.21.15.19:0/1698212955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00ccc0 con 0x7efe6c102b50 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.051+0000 7efe68ff9640 1 -- 172.21.15.19:0/1698212955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5c0044b0 con 0x7efe6c102b50 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.051+0000 7efe68ff9640 1 -- 172.21.15.19:0/1698212955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c010430 con 0x7efe6c102b50 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.051+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1a1fb0 con 0x7efe6c102b50 2024-05-17T09:43:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.052+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1a2410 con 0x7efe6c102b50 2024-05-17T09:43:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 Processor -- start 2024-05-17T09:43:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 -- start start 2024-05-17T09:43:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe68ff9640 1 -- 172.21.15.19:0/1698212955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efe5c003f90 con 0x7efe6c102b50 2024-05-17T09:43:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 0x560a60f657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f65d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a60ed3c30 con 0x560a60bc01b0 2024-05-17T09:43:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.055+0000 7efe78fa6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a60ec37d0 con 0x560a60b9f4d0 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe78fa6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560a60ec7810 con 0x560a60b972f0 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f65d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 0x560a60f657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 0x560a60f657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51578/0 (socket says 172.21.15.19:51578) 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f65d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:52336/0 (socket says 172.21.15.19:52336) 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 -- 172.21.15.19:0/4256551205 learned_addr learned my addr 172.21.15.19:0/4256551205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 -- 172.21.15.19:0/4256551205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 msgr2=0x560a60f65d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f65d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 -- 172.21.15.19:0/4256551205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 msgr2=0x560a60f6cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f6cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 -- 172.21.15.19:0/4256551205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a60bd8210 con 0x560a60bc01b0 2024-05-17T09:43:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6b7fe640 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f65d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.056+0000 7efe6bfff640 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 0x560a60f657b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7efe640117e0 tx=0x7efe64011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a076eee53ea2eefb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.057+0000 7efe68ff9640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe580b29f0 0x7efe580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.057+0000 7efe61ffb640 1 -- 172.21.15.19:0/4256551205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64011310 con 0x560a60bc01b0 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.057+0000 7efe6b7fe640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe580b29f0 0x7efe580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.057+0000 7efe61ffb640 1 -- 172.21.15.19:0/4256551205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe64003e00 con 0x560a60bc01b0 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.057+0000 7efe61ffb640 1 -- 172.21.15.19:0/4256551205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64004410 con 0x560a60bc01b0 2024-05-17T09:43:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.057+0000 7efe68ff9640 1 -- 172.21.15.19:0/1698212955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efe5c0e27e0 con 0x7efe6c102b50 2024-05-17T09:43:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 -- 172.21.15.19:0/4256551205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 msgr2=0x560a60f657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 0x560a60f657b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7efe640117e0 tx=0x7efe64011ca0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe6b7fe640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe580b29f0 0x7efe580b4f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x560a60f66b90 tx=0x7efe5c0cc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 -- 172.21.15.19:0/4256551205 shutdown_connections 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f6cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f65d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 --2- 172.21.15.19:0/4256551205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bc01b0 0x560a60f657b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 -- 172.21.15.19:0/4256551205 >> 172.21.15.19:0/4256551205 conn(0x560a60bb4300 msgr2=0x560a60f61140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 -- 172.21.15.19:0/4256551205 shutdown_connections 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.058+0000 7efe78fa6740 1 -- 172.21.15.19:0/4256551205 wait complete. 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 -- start start 2024-05-17T09:43:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 Processor -- start 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 0x560a60f6f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 0x560a60f6f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 0x560a60f6f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51582/0 (socket says 172.21.15.19:51582) 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe6bfff640 1 -- 172.21.15.19:0/3540855556 learned_addr learned my addr 172.21.15.19:0/3540855556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60bc01b0 0x560a60f765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a60ba9790 con 0x560a60b972f0 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a60e7f890 con 0x560a60bc01b0 2024-05-17T09:43:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560a60f6d680 con 0x560a60b9f4d0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe6affd640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.059+0000 7efe78fa6740 10 client.0 _ll_register_callbacks cb 0x560a60e82d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60bc01b0 0x560a60f765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 -- 172.21.15.19:0/3540855556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 msgr2=0x560a60f6f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f6f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 -- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60bc01b0 msgr2=0x560a60f765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60bc01b0 0x560a60f765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a60ca4620 con 0x560a60b972f0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 0x560a60f6f150 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7efe64006010 tx=0x7efe64005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc34600e7450f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe4affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe4affd640 1 -- 172.21.15.19:0/3540855556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6400fd30 con 0x560a60b972f0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.060+0000 7efe4affd640 1 -- 172.21.15.19:0/3540855556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efe64004a40 con 0x560a60b972f0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe64003e00 con 0x560a60b972f0 2024-05-17T09:43:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe4affd640 1 -- 172.21.15.19:0/3540855556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe64034050 con 0x560a60b972f0 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 10 client.18963 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a60f79500 con 0x560a60b972f0 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe4affd640 1 -- 172.21.15.19:0/3540855556 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe64003e00 con 0x560a60b972f0 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe4affd640 1 -- 172.21.15.19:0/3540855556 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efe64028030 con 0x560a60b972f0 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 10 client.18963 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 10 client.18963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 20 client.18963 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60b7ed80 0x560a60f7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 4 client.18963 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.061+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a60ed5b80 con 0x560a60b7ed80 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.062+0000 7efe6affd640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60b7ed80 0x560a60f7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.062+0000 7efe6affd640 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60b7ed80 0x560a60f7c9b0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8a8a15a4b691e2 server_cookie=8f1427ebac436ae1 in_seq=0 out_seq=0 2024-05-17T09:43:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.062+0000 7efe4affd640 10 client.18963 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe4affd640 1 -- 172.21.15.19:0/3540855556 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560a60ed5b80 con 0x560a60b7ed80 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe4affd640 10 client.18963 handle_command_reply: tid=0 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 client.18963 shutdown 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 msgr2=0x560a60f6f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 0x560a60f6f150 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7efe64006010 tx=0x7efe64005ab0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 shutdown_connections 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60b7ed80 0x560a60f7c9b0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60bc01b0 0x560a60f765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f6f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe4affd640 0 client.18963 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3540855556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60b972f0 0x560a60f6f150 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 >> 172.21.15.19:0/3540855556 conn(0x560a60bb4300 msgr2=0x560a60f64050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 shutdown_connections 2024-05-17T09:43:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 1 -- 172.21.15.19:0/3540855556 wait complete. 2024-05-17T09:43:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.063+0000 7efe78fa6740 20 client.18963 trim_cache size 0 max 16384 2024-05-17T09:43:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.096+0000 7efe78fa6740 1 Processor -- start 2024-05-17T09:43:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.098+0000 7efe78fa6740 1 -- start start 2024-05-17T09:43:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.099+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60f69d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.099+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b972f0 0x560a60f64650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.099+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f64b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.099+0000 7efe78fa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1ad0e0 con 0x560a60bb4300 2024-05-17T09:43:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.099+0000 7efe78fa6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1aa9b0 con 0x560a60b972f0 2024-05-17T09:43:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.099+0000 7efe78fa6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1a8800 con 0x560a60b9f4d0 2024-05-17T09:43:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.100+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60f69d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.100+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60f69d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51594/0 (socket says 172.21.15.19:51594) 2024-05-17T09:43:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.100+0000 7efe6bfff640 1 -- 172.21.15.19:0/3562630300 learned_addr learned my addr 172.21.15.19:0/3562630300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.100+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f64b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.101+0000 7efe6bfff640 1 -- 172.21.15.19:0/3562630300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b972f0 msgr2=0x560a60f64650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:43:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.101+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b972f0 0x560a60f64650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.101+0000 7efe6bfff640 1 -- 172.21.15.19:0/3562630300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 msgr2=0x560a60f69d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.101+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60f69d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.101+0000 7efe6bfff640 1 -- 172.21.15.19:0/3562630300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a60f77630 con 0x560a60b9f4d0 2024-05-17T09:43:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.101+0000 7efe6bfff640 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f64b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efe64005d70 tx=0x7efe64000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1bb71b1cb60bd8c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.102+0000 7efe617fa640 1 -- 172.21.15.19:0/3562630300 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640055c0 con 0x560a60b9f4d0 2024-05-17T09:43:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.102+0000 7efe617fa640 1 -- 172.21.15.19:0/3562630300 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe64005760 con 0x560a60b9f4d0 2024-05-17T09:43:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.102+0000 7efe617fa640 1 -- 172.21.15.19:0/3562630300 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6401a7a0 con 0x560a60b9f4d0 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.103+0000 7efe78fa6740 1 -- 172.21.15.19:0/3562630300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 msgr2=0x560a60f64b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.103+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f64b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efe64005d70 tx=0x7efe64000f40 comp rx=0 tx=0).stop 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 -- 172.21.15.19:0/3562630300 shutdown_connections 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b9f4d0 0x560a60f64b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b972f0 0x560a60f64650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 --2- 172.21.15.19:0/3562630300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60f69d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 -- 172.21.15.19:0/3562630300 >> 172.21.15.19:0/3562630300 conn(0x560a60f58000 msgr2=0x560a60f63f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 -- 172.21.15.19:0/3562630300 shutdown_connections 2024-05-17T09:43:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.104+0000 7efe78fa6740 1 -- 172.21.15.19:0/3562630300 wait complete. 2024-05-17T09:43:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 -- start start 2024-05-17T09:43:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 Processor -- start 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60fe7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1a8e70 con 0x560a60bb4300 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1aa2e0 con 0x560a60b9f4d0 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1a9540 con 0x560a60b972f0 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.105+0000 7efe78fa6740 10 client.0 _ll_register_callbacks cb 0x7efe640116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f74ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f74ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:51492/0 (socket says 172.21.15.19:51492) 2024-05-17T09:43:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 -- 172.21.15.19:0/819542778 learned_addr learned my addr 172.21.15.19:0/819542778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6bfff640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6bfff640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60fe7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 -- 172.21.15.19:0/819542778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 msgr2=0x560a60f72560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f72560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 -- 172.21.15.19:0/819542778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 msgr2=0x560a60fe7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60fe7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 -- 172.21.15.19:0/819542778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a60fe7da0 con 0x560a60b9f4d0 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe6affd640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f74ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe5402aa00 tx=0x7efe54003750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b34a3644ce11a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.106+0000 7efe4affd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:43:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.107+0000 7efe4affd640 1 -- 172.21.15.19:0/819542778 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5404c020 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.107+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe64005760 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.107+0000 7efe4affd640 1 -- 172.21.15.19:0/819542778 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efe54003d10 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.107+0000 7efe4affd640 1 -- 172.21.15.19:0/819542778 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe54057050 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.107+0000 7efe78fa6740 10 client.28463 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.107+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a60fe8f90 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe4affd640 1 -- 172.21.15.19:0/819542778 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe64005760 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe4affd640 1 -- 172.21.15.19:0/819542778 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efe540316f0 con 0x560a60b9f4d0 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe78fa6740 10 client.28463 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe78fa6740 10 client.28463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe78fa6740 20 client.28463 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe78fa6740 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60bc01b0 0x560a60feb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe78fa6740 4 client.28463 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:43:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a60ecf360 con 0x560a60bc01b0 2024-05-17T09:43:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.108+0000 7efe6bfff640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60bc01b0 0x560a60feb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.109+0000 7efe6bfff640 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60bc01b0 0x560a60feb9b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85327033503904f server_cookie=eadfdb8a72111898 in_seq=0 out_seq=0 2024-05-17T09:43:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.109+0000 7efe4affd640 10 client.28463 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.192+0000 7efe4affd640 1 -- 172.21.15.19:0/819542778 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560a60ecf360 con 0x560a60bc01b0 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.192+0000 7efe4affd640 10 client.28463 handle_command_reply: tid=0 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 client.28463 shutdown 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 msgr2=0x560a60f74ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f74ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe5402aa00 tx=0x7efe54003750 comp rx=0 tx=0).stop 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 shutdown_connections 2024-05-17T09:43:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560a60bc01b0 0x560a60feb9b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a60bb4300 0x560a60fe7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a60b9f4d0 0x560a60f74ab0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 --2- 172.21.15.19:0/819542778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560a60b972f0 0x560a60f72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe4affd640 0 client.28463 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 >> 172.21.15.19:0/819542778 conn(0x560a60f58000 msgr2=0x560a60f64970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 shutdown_connections 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 1 -- 172.21.15.19:0/819542778 wait complete. 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.193+0000 7efe78fa6740 20 client.28463 trim_cache size 0 max 16384 2024-05-17T09:43:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:43:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:43:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe580b29f0 msgr2=0x7efe580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe580b29f0 0x7efe580b4f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x560a60f66b90 tx=0x7efe5c0cc040 comp rx=0 tx=0).stop 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 msgr2=0x7efe6c06bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c06bd90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e9f0 tx=0x7efe5c00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 shutdown_connections 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe6c108a90 0x7efe6c1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe6c103d90 0x7efe6c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efe580b29f0 0x7efe580b4f00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 --2- 172.21.15.19:0/1698212955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe6c102b50 0x7efe6c06bd90 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 >> 172.21.15.19:0/1698212955 conn(0x7efe6c0fdd40 msgr2=0x7efe6c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 shutdown_connections 2024-05-17T09:43:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:04.194+0000 7efe720f2640 1 -- 172.21.15.19:0/1698212955 wait complete. 2024-05-17T09:43:04.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65309 6.47 MB/sec execute 421 sec latency 124.067 ms 2024-05-17T09:43:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:04 smithi069 ceph-mon[35081]: pgmap v1137: 129 pgs: 129 active+clean; 492 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-05-17T09:43:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:04 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:43:04.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:04 smithi019 ceph-mon[31620]: pgmap v1137: 129 pgs: 129 active+clean; 492 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-05-17T09:43:04.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:04 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:43:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:04 smithi079 ceph-mon[35502]: pgmap v1137: 129 pgs: 129 active+clean; 492 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-05-17T09:43:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:04 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:43:05.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65460 6.48 MB/sec execute 422 sec latency 96.004 ms 2024-05-17T09:43:06.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65615 6.48 MB/sec execute 423 sec latency 90.217 ms 2024-05-17T09:43:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:06 smithi069 ceph-mon[35081]: pgmap v1138: 129 pgs: 129 active+clean; 490 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 3.5 MiB/s wr, 196 op/s 2024-05-17T09:43:06.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:06 smithi019 ceph-mon[31620]: pgmap v1138: 129 pgs: 129 active+clean; 490 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 3.5 MiB/s wr, 196 op/s 2024-05-17T09:43:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:06 smithi079 ceph-mon[35502]: pgmap v1138: 129 pgs: 129 active+clean; 490 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 3.5 MiB/s wr, 196 op/s 2024-05-17T09:43:07.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65778 6.47 MB/sec execute 424 sec latency 111.489 ms 2024-05-17T09:43:07.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:43:07.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:43:07.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:43:07.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:43:07.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:43:07.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:43:08.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:43:08.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:43:08.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:43:08.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 65915 6.47 MB/sec execute 425 sec latency 109.801 ms 2024-05-17T09:43:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:08 smithi069 ceph-mon[35081]: pgmap v1139: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-05-17T09:43:08.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:08 smithi019 ceph-mon[31620]: pgmap v1139: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-05-17T09:43:09.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:08 smithi079 ceph-mon[35502]: pgmap v1139: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-05-17T09:43:09.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 66050 6.46 MB/sec execute 426 sec latency 79.713 ms 2024-05-17T09:43:10.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 66210 6.48 MB/sec execute 427 sec latency 83.862 ms 2024-05-17T09:43:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:10 smithi069 ceph-mon[35081]: pgmap v1140: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-05-17T09:43:10.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:10 smithi019 ceph-mon[31620]: pgmap v1140: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-05-17T09:43:11.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:10 smithi079 ceph-mon[35502]: pgmap v1140: 129 pgs: 129 active+clean; 491 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-05-17T09:43:11.449 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:43:11.449 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.5 2024-05-17T09:43:11.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:43:11.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 66388 6.49 MB/sec execute 428 sec latency 76.663 ms 2024-05-17T09:43:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:43:12.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 66576 6.50 MB/sec execute 429 sec latency 96.822 ms 2024-05-17T09:43:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:12 smithi069 ceph-mon[35081]: pgmap v1141: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-05-17T09:43:12.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:12 smithi019 ceph-mon[31620]: pgmap v1141: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-05-17T09:43:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:12 smithi079 ceph-mon[35502]: pgmap v1141: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-05-17T09:43:13.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 66756 6.52 MB/sec execute 430 sec latency 99.295 ms 2024-05-17T09:43:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 -- 172.21.15.19:0/939332636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 msgr2=0x7f1c04105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 --2- 172.21.15.19:0/939332636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c04105920 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009990 tx=0x7f1bf4009ca0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 -- 172.21.15.19:0/939332636 shutdown_connections 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 --2- 172.21.15.19:0/939332636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c0410e8f0 0x7f1c04110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 --2- 172.21.15.19:0/939332636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c04105920 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 --2- 172.21.15.19:0/939332636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c0410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 -- 172.21.15.19:0/939332636 >> 172.21.15.19:0/939332636 conn(0x7f1c040fedb0 msgr2=0x7f1c041011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 -- 172.21.15.19:0/939332636 shutdown_connections 2024-05-17T09:43:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.249+0000 7f1c0982b700 1 -- 172.21.15.19:0/939332636 wait complete. 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 Processor -- start 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 -- start start 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c041a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c041a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c0410e8f0 0x7f1c0419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041142d0 con 0x7f1c04105510 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04114150 con 0x7f1c04105e60 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c0982b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04114450 con 0x7f1c0410e8f0 2024-05-17T09:43:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.250+0000 7f1c027fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c041a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c041a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c041a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:60552/0 (socket says 172.21.15.19:60552) 2024-05-17T09:43:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 -- 172.21.15.19:0/1206626457 learned_addr learned my addr 172.21.15.19:0/1206626457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c02ffd700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c041a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53576/0 (socket says 172.21.15.19:53576) 2024-05-17T09:43:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c037fe700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c0410e8f0 0x7f1c0419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 -- 172.21.15.19:0/1206626457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c0410e8f0 msgr2=0x7f1c0419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c0410e8f0 0x7f1c0419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 -- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 msgr2=0x7f1c041a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c041a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 -- 172.21.15.19:0/1206626457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0419b260 con 0x7f1c04105e60 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.251+0000 7f1c027fc700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c041a0ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800e9a0 tx=0x7f1bf800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.252+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800ca10 con 0x7f1c04105e60 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.252+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bf800cbb0 con 0x7f1c04105e60 2024-05-17T09:43:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.252+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8018680 con 0x7f1c04105e60 2024-05-17T09:43:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.252+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0419b550 con 0x7f1c04105e60 2024-05-17T09:43:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.252+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0419b980 con 0x7f1c04105e60 2024-05-17T09:43:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.253+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0406be20 con 0x7f1c04105e60 2024-05-17T09:43:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.255+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1bf8010430 con 0x7f1c04105e60 2024-05-17T09:43:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.256+0000 7f1c08829700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1bec0b2a50 0x7f1bec0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.257+0000 7f1c02ffd700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1bec0b2a50 0x7f1bec0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.257+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1bf80e26b0 con 0x7f1c04105e60 2024-05-17T09:43:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.257+0000 7f1c02ffd700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1bec0b2a50 0x7f1bec0b4f60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400b2e0 tx=0x7f1bf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.264+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bf80a9fc0 con 0x7f1c04105e60 2024-05-17T09:43:14.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 66933 6.52 MB/sec execute 431 sec latency 85.859 ms 2024-05-17T09:43:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.727+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c0402e5e0 con 0x7f1c04105e60 2024-05-17T09:43:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.728+0000 7f1c08829700 1 -- 172.21.15.19:0/1206626457 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f1bf80a98c0 con 0x7f1c04105e60 2024-05-17T09:43:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:43:14.727 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1bec0b2a50 msgr2=0x7f1bec0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1bec0b2a50 0x7f1bec0b4f60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400b2e0 tx=0x7f1bf4005fb0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 msgr2=0x7f1c041a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c041a0ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800e9a0 tx=0x7f1bf800ecb0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 shutdown_connections 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f1bec0b2a50 0x7f1bec0b4f60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c0410e8f0 0x7f1c0419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c04105510 0x7f1c041a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 --2- 172.21.15.19:0/1206626457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c04105e60 0x7f1c041a0ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 >> 172.21.15.19:0/1206626457 conn(0x7f1c040fedb0 msgr2=0x7f1c040ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:14.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.732+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 shutdown_connections 2024-05-17T09:43:14.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:14.733+0000 7f1c0982b700 1 -- 172.21.15.19:0/1206626457 wait complete. 2024-05-17T09:43:14.732 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:43:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:14 smithi069 ceph-mon[35081]: pgmap v1142: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-17T09:43:14.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:14 smithi019 ceph-mon[31620]: pgmap v1142: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-17T09:43:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:14 smithi079 ceph-mon[35502]: pgmap v1142: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-17T09:43:15.497 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.5 --await-for 45.0' 2024-05-17T09:43:15.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.5 --await-for 45.0 2024-05-17T09:43:15.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67083 6.53 MB/sec execute 432 sec latency 82.149 ms 2024-05-17T09:43:15.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:15 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1206626457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:15.880 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:43:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:15 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1206626457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:16.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:15 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1206626457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:16.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67190 6.52 MB/sec execute 433 sec latency 87.120 ms 2024-05-17T09:43:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:16 smithi069 ceph-mon[35081]: pgmap v1143: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-05-17T09:43:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:16 smithi019 ceph-mon[31620]: pgmap v1143: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-05-17T09:43:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:16 smithi079 ceph-mon[35502]: pgmap v1143: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-05-17T09:43:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 -- 172.21.15.19:0/1953990466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514102b40 msgr2=0x7f2514103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 --2- 172.21.15.19:0/1953990466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514102b40 0x7f2514103840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2510009900 tx=0x7f251002a410 comp rx=0 tx=0).stop 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 -- 172.21.15.19:0/1953990466 shutdown_connections 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 --2- 172.21.15.19:0/1953990466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514108a80 0x7f251410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 --2- 172.21.15.19:0/1953990466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103d80 0x7f2514104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 --2- 172.21.15.19:0/1953990466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514102b40 0x7f2514103840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 -- 172.21.15.19:0/1953990466 >> 172.21.15.19:0/1953990466 conn(0x7f25140fdd30 msgr2=0x7f25141001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 -- 172.21.15.19:0/1953990466 shutdown_connections 2024-05-17T09:43:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.364+0000 7f251c661640 1 -- 172.21.15.19:0/1953990466 wait complete. 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 Processor -- start 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 -- start start 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514102b40 0x7f251419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514103d80 0x7f251419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514108a80 0x7f25141a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f251410f610 con 0x7f2514102b40 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f251410f490 con 0x7f2514103d80 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251c661640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f251410f790 con 0x7f2514108a80 2024-05-17T09:43:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514102b40 0x7f251419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.365+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514103d80 0x7f251419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514102b40 0x7f251419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53592/0 (socket says 172.21.15.19:53592) 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514103d80 0x7f251419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:60576/0 (socket says 172.21.15.19:60576) 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251a3fd640 1 -- 172.21.15.19:0/2923019786 learned_addr learned my addr 172.21.15.19:0/2923019786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514108a80 0x7f25141a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 -- 172.21.15.19:0/2923019786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514103d80 msgr2=0x7f251419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514103d80 0x7f251419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 -- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514102b40 msgr2=0x7f251419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514102b40 0x7f251419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 -- 172.21.15.19:0/2923019786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141a1d10 con 0x7f2514108a80 2024-05-17T09:43:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251abfe640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514108a80 0x7f25141a16a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f250c00ec50 tx=0x7f250c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f250b7fe640 1 -- 172.21.15.19:0/2923019786 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c003b70 con 0x7f2514108a80 2024-05-17T09:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f250b7fe640 1 -- 172.21.15.19:0/2923019786 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f250c003d10 con 0x7f2514108a80 2024-05-17T09:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25141a1fa0 con 0x7f2514108a80 2024-05-17T09:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.366+0000 7f250b7fe640 1 -- 172.21.15.19:0/2923019786 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0107a0 con 0x7f2514108a80 2024-05-17T09:43:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.367+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a2400 con 0x7f2514108a80 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.370+0000 7f25235fb740 1 Processor -- start 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.370+0000 7f250b7fe640 1 -- 172.21.15.19:0/2923019786 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f250c003fc0 con 0x7f2514108a80 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.370+0000 7f25235fb740 1 -- start start 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f25235fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687841b0 0x557c68b297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f25235fb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6875b2f0 0x557c68b29d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f25235fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b30c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f25235fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c68a97c30 con 0x557c687841b0 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f25235fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c68a877d0 con 0x557c687634d0 2024-05-17T09:43:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f25235fb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557c68a8b810 con 0x557c6875b2f0 2024-05-17T09:43:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687841b0 0x557c68b297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687841b0 0x557c68b297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53608/0 (socket says 172.21.15.19:53608) 2024-05-17T09:43:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f2519bfc640 1 -- 172.21.15.19:0/1241837191 learned_addr learned my addr 172.21.15.19:0/1241837191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f251a3fd640 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6875b2f0 0x557c68b29d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f2519bfc640 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b30c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f251a3fd640 1 -- 172.21.15.19:0/1241837191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 msgr2=0x557c68b30c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f251a3fd640 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b30c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f251a3fd640 1 -- 172.21.15.19:0/1241837191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687841b0 msgr2=0x557c68b297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f251a3fd640 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687841b0 0x557c68b297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f251a3fd640 1 -- 172.21.15.19:0/1241837191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6879c210 con 0x557c6875b2f0 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.371+0000 7f2519bfc640 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b30c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.372+0000 7f251a3fd640 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6875b2f0 0x557c68b29d30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25100057c0 tx=0x7f2510031500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156096531ad84492 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.372+0000 7f250b7fe640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25000b29a0 0x7f25000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.372+0000 7f2508bfe640 1 -- 172.21.15.19:0/1241837191 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510035030 con 0x557c6875b2f0 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.372+0000 7f2508bfe640 1 -- 172.21.15.19:0/1241837191 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2510040070 con 0x557c6875b2f0 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.372+0000 7f2519bfc640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25000b29a0 0x7f25000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.372+0000 7f250b7fe640 1 -- 172.21.15.19:0/2923019786 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f250c0e2ac0 con 0x7f2514108a80 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f2508bfe640 1 -- 172.21.15.19:0/1241837191 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510003850 con 0x557c6875b2f0 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 -- 172.21.15.19:0/1241837191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6875b2f0 msgr2=0x557c68b29d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f2519bfc640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25000b29a0 0x7f25000b4eb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2504009fd0 tx=0x7f2504015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6875b2f0 0x557c68b29d30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25100057c0 tx=0x7f2510031500 comp rx=0 tx=0).stop 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 -- 172.21.15.19:0/1241837191 shutdown_connections 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b30c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6875b2f0 0x557c68b29d30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 --2- 172.21.15.19:0/1241837191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687841b0 0x557c68b297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 -- 172.21.15.19:0/1241837191 >> 172.21.15.19:0/1241837191 conn(0x557c68778300 msgr2=0x557c68b20d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 -- 172.21.15.19:0/1241837191 shutdown_connections 2024-05-17T09:43:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.373+0000 7f25235fb740 1 -- 172.21.15.19:0/1241837191 wait complete. 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 -- start start 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 Processor -- start 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 0x557c68b330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687634d0 0x557c68b335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b3a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c6876d790 con 0x557c687634d0 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c68a43890 con 0x557c6875b2f0 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557c68b316c0 con 0x557c687841b0 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f25235fb740 10 client.0 _ll_register_callbacks cb 0x557c68a46d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 0x557c68b330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687634d0 0x557c68b335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 0x557c68b330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:60588/0 (socket says 172.21.15.19:60588) 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f251a3fd640 1 -- 172.21.15.19:0/227669797 learned_addr learned my addr 172.21.15.19:0/227669797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.374+0000 7f251abfe640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b3a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f251a3fd640 1 -- 172.21.15.19:0/227669797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 msgr2=0x557c68b3a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f251a3fd640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b3a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f251a3fd640 1 -- 172.21.15.19:0/227669797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687634d0 msgr2=0x557c68b335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f251a3fd640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687634d0 0x557c68b335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f251a3fd640 1 -- 172.21.15.19:0/227669797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c68868620 con 0x557c6875b2f0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f2519bfc640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687634d0 0x557c68b335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f251a3fd640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 0x557c68b330a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f251002a450 tx=0x7f25100318a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01bd22a0498ef90 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f24fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f24fdffb640 1 -- 172.21.15.19:0/227669797 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510035070 con 0x557c6875b2f0 2024-05-17T09:43:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f24fdffb640 1 -- 172.21.15.19:0/227669797 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f251003c440 con 0x557c6875b2f0 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.375+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2510040070 con 0x557c6875b2f0 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f24fdffb640 1 -- 172.21.15.19:0/227669797 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f251003c640 con 0x557c6875b2f0 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 10 client.28481 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557c68b3d400 con 0x557c6875b2f0 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f24fdffb640 1 -- 172.21.15.19:0/227669797 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2510040070 con 0x557c6875b2f0 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f24fdffb640 1 -- 172.21.15.19:0/227669797 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f25100445b0 con 0x557c6875b2f0 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 10 client.28481 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 10 client.28481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 20 client.28481 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c68742d80 0x557c68b407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 4 client.28481 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c68a99b80 con 0x557c68742d80 2024-05-17T09:43:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.376+0000 7f2519bfc640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c68742d80 0x557c68b407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.377+0000 7f2519bfc640 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c68742d80 0x557c68b407a0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d01e374d9f204d2 server_cookie=4f53962fb4051032 in_seq=0 out_seq=0 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.377+0000 7f24fdffb640 10 client.28481 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f24fdffb640 1 -- 172.21.15.19:0/227669797 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557c68a99b80 con 0x557c68742d80 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f24fdffb640 10 client.28481 handle_command_reply: tid=0 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 client.28481 shutdown 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 msgr2=0x557c68b330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 0x557c68b330a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f251002a450 tx=0x7f25100318a0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 shutdown_connections 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c68742d80 0x557c68b407a0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b3a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c687634d0 0x557c68b335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f24fdffb640 0 client.28481 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 --2- 172.21.15.19:0/227669797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c6875b2f0 0x557c68b330a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 >> 172.21.15.19:0/227669797 conn(0x557c68778300 msgr2=0x557c68b23c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 shutdown_connections 2024-05-17T09:43:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 1 -- 172.21.15.19:0/227669797 wait complete. 2024-05-17T09:43:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.378+0000 7f25235fb740 20 client.28481 trim_cache size 0 max 16384 2024-05-17T09:43:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 Processor -- start 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 -- start start 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b24240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b24780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25141acf90 con 0x557c6875b2f0 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25141aa860 con 0x557c687634d0 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f25235fb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25141a86b0 con 0x557c687841b0 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b24240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53630/0 (socket says 172.21.15.19:53630) 2024-05-17T09:43:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.409+0000 7f251a3fd640 1 -- 172.21.15.19:0/3351486448 learned_addr learned my addr 172.21.15.19:0/3351486448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f251abfe640 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b24780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f2519bfc640 1 -- 172.21.15.19:0/3351486448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 msgr2=0x557c68b24780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f2519bfc640 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b24780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f2519bfc640 1 -- 172.21.15.19:0/3351486448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 msgr2=0x557c68b336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f2519bfc640 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f2519bfc640 1 -- 172.21.15.19:0/3351486448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c68b32140 con 0x557c687634d0 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f251a3fd640 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f2519bfc640 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b24240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x557c68b40f10 tx=0x7f2504004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2546fc1c19c5ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f24fffff640 1 -- 172.21.15.19:0/3351486448 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250400fe60 con 0x557c687634d0 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f24fffff640 1 -- 172.21.15.19:0/3351486448 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25040054b0 con 0x557c687634d0 2024-05-17T09:43:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.410+0000 7f24fffff640 1 -- 172.21.15.19:0/3351486448 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504004c00 con 0x557c687634d0 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 -- 172.21.15.19:0/3351486448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 msgr2=0x557c68b24240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b24240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x557c68b40f10 tx=0x7f2504004730 comp rx=0 tx=0).stop 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 -- 172.21.15.19:0/3351486448 shutdown_connections 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687841b0 0x557c68b24780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687634d0 0x557c68b24240 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 --2- 172.21.15.19:0/3351486448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 -- 172.21.15.19:0/3351486448 >> 172.21.15.19:0/3351486448 conn(0x557c68778300 msgr2=0x557c68b23b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 -- 172.21.15.19:0/3351486448 shutdown_connections 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.412+0000 7f25235fb740 1 -- 172.21.15.19:0/3351486448 wait complete. 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.413+0000 7f25235fb740 1 -- start start 2024-05-17T09:43:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.413+0000 7f25235fb740 1 Processor -- start 2024-05-17T09:43:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.413+0000 7f25235fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b37da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f25235fb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 0x557c68b382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f25235fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687841b0 0x557c68b39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 0x557c68b382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b37da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f2519bfc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 0x557c68b382e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46684/0 (socket says 172.21.15.19:46684) 2024-05-17T09:43:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f251a3fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b37da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53642/0 (socket says 172.21.15.19:53642) 2024-05-17T09:43:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f2519bfc640 1 -- 172.21.15.19:0/609117250 learned_addr learned my addr 172.21.15.19:0/609117250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f251abfe640 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687841b0 0x557c68b39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25141a8d20 con 0x557c6875b2f0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25141aa190 con 0x557c687841b0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.414+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25141a93f0 con 0x557c687634d0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f25235fb740 10 client.0 _ll_register_callbacks cb 0x7f251002ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f2519bfc640 1 -- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687841b0 msgr2=0x557c68b39210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f2519bfc640 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687841b0 0x557c68b39210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f2519bfc640 1 -- 172.21.15.19:0/609117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 msgr2=0x557c68b37da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f2519bfc640 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b37da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f2519bfc640 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c68b397d0 con 0x557c687634d0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.415+0000 7f2519bfc640 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 0x557c68b382e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f250400fc30 tx=0x7f250400fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee3a974ed8742c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f24fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:43:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f24fdffb640 1 -- 172.21.15.19:0/609117250 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250402e660 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f24fdffb640 1 -- 172.21.15.19:0/609117250 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f250402e800 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25040054b0 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f24fdffb640 1 -- 172.21.15.19:0/609117250 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f250402ea00 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f25235fb740 10 client.28474 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.416+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557c68b3ab00 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f24fdffb640 1 -- 172.21.15.19:0/609117250 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25040054b0 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f24fdffb640 1 -- 172.21.15.19:0/609117250 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f2504038410 con 0x557c687634d0 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f25235fb740 10 client.28474 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f25235fb740 10 client.28474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f25235fb740 20 client.28474 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f25235fb740 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c687181c0 0x557c68bab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f25235fb740 4 client.28474 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:43:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c68a93360 con 0x557c687181c0 2024-05-17T09:43:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.417+0000 7f251a3fd640 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c687181c0 0x557c68bab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.418+0000 7f24fdffb640 10 client.28474 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.418+0000 7f251a3fd640 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c687181c0 0x557c68bab150 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbfa2995d9d54ee server_cookie=5016973835eab6ef in_seq=0 out_seq=0 2024-05-17T09:43:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.523+0000 7f24fdffb640 1 -- 172.21.15.19:0/609117250 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557c68a93360 con 0x557c687181c0 2024-05-17T09:43:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.523+0000 7f24fdffb640 10 client.28474 handle_command_reply: tid=0 2024-05-17T09:43:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.523+0000 7f25235fb740 1 client.28474 shutdown 2024-05-17T09:43:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 msgr2=0x557c68b382e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 0x557c68b382e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f250400fc30 tx=0x7f250400fe60 comp rx=0 tx=0).stop 2024-05-17T09:43:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 shutdown_connections 2024-05-17T09:43:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x557c687181c0 0x557c68bab150 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c687841b0 0x557c68b39210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c687634d0 0x557c68b382e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 --2- 172.21.15.19:0/609117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c6875b2f0 0x557c68b37da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 >> 172.21.15.19:0/609117250 conn(0x557c68778300 msgr2=0x557c68b333e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f24fdffb640 0 client.28474 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.524+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 shutdown_connections 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.525+0000 7f25235fb740 1 -- 172.21.15.19:0/609117250 wait complete. 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.525+0000 7f25235fb740 20 client.28474 trim_cache size 0 max 16384 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:43:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25000b29a0 msgr2=0x7f25000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25000b29a0 0x7f25000b4eb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2504009fd0 tx=0x7f2504015040 comp rx=0 tx=0).stop 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514108a80 msgr2=0x7f25141a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514108a80 0x7f25141a16a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f250c00ec50 tx=0x7f250c00c6a0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 shutdown_connections 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2514108a80 0x7f25141a16a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2514103d80 0x7f251419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25000b29a0 0x7f25000b4eb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 --2- 172.21.15.19:0/2923019786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514102b40 0x7f251419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 >> 172.21.15.19:0/2923019786 conn(0x7f25140fdd30 msgr2=0x7f2514104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.526+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 shutdown_connections 2024-05-17T09:43:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:17.527+0000 7f251c661640 1 -- 172.21.15.19:0/2923019786 wait complete. 2024-05-17T09:43:17.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67324 6.53 MB/sec execute 434 sec latency 155.503 ms 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 -- 172.21.15.19:0/1490196500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65641045e0 msgr2=0x7f65641049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 --2- 172.21.15.19:0/1490196500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65641045e0 0x7f65641049f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f65600098f0 tx=0x7f6560009c00 comp rx=0 tx=0).stop 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 -- 172.21.15.19:0/1490196500 shutdown_connections 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 --2- 172.21.15.19:0/1490196500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f65641111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 --2- 172.21.15.19:0/1490196500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65641045e0 0x7f65641049f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 --2- 172.21.15.19:0/1490196500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6564108f10 0x7f6564109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 -- 172.21.15.19:0/1490196500 >> 172.21.15.19:0/1490196500 conn(0x7f6564100030 msgr2=0x7f6564102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.332+0000 7f656c459700 1 -- 172.21.15.19:0/1490196500 shutdown_connections 2024-05-17T09:43:18.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.333+0000 7f656c459700 1 -- 172.21.15.19:0/1490196500 wait complete. 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.333+0000 7f656c459700 1 Processor -- start 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.333+0000 7f656c459700 1 -- start start 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656c459700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65641045e0 0x7f6564075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656c459700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564108f10 0x7f6564075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656c459700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f6564076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656c459700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6564114390 con 0x7f6564109860 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656c459700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6564114210 con 0x7f6564108f10 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656c459700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6564114510 con 0x7f65641045e0 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f65699f4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564108f10 0x7f6564075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a9f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f6564076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a9f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f6564076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53660/0 (socket says 172.21.15.19:53660) 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f65699f4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564108f10 0x7f6564075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:60628/0 (socket says 172.21.15.19:60628) 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a9f6700 1 -- 172.21.15.19:0/3781301050 learned_addr learned my addr 172.21.15.19:0/3781301050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65641045e0 0x7f6564075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a1f5700 1 -- 172.21.15.19:0/3781301050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564108f10 msgr2=0x7f6564075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564108f10 0x7f6564075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a1f5700 1 -- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 msgr2=0x7f6564076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.334+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f6564076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f656a1f5700 1 -- 172.21.15.19:0/3781301050 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65641a91c0 con 0x7f65641045e0 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f656a9f6700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f6564076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65641045e0 0x7f6564075ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6560009fd0 tx=0x7f656000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f655b7fe700 1 -- 172.21.15.19:0/3781301050 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656000b560 con 0x7f65641045e0 2024-05-17T09:43:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656419ab80 con 0x7f65641045e0 2024-05-17T09:43:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f655b7fe700 1 -- 172.21.15.19:0/3781301050 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f656000bb90 con 0x7f65641045e0 2024-05-17T09:43:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f655b7fe700 1 -- 172.21.15.19:0/3781301050 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65600038e0 con 0x7f65641045e0 2024-05-17T09:43:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.335+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656419b0b0 con 0x7f65641045e0 2024-05-17T09:43:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.339+0000 7f655b7fe700 1 -- 172.21.15.19:0/3781301050 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6560003a80 con 0x7f65641045e0 2024-05-17T09:43:18.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.340+0000 7f657ae6eb80 1 Processor -- start 2024-05-17T09:43:18.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.340+0000 7f657ae6eb80 1 -- start start 2024-05-17T09:43:18.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f655b7fe700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65500b2900 0x7f65500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7c795c0 0x5602b7d12f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d0d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602b7c3af00 con 0x5602b7b8dfd0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5602b7c2aaa0 con 0x5602b7d0d0b0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5602b7c2eae0 con 0x5602b7c795c0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f656a9f6700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65500b2900 0x7f65500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f655b7fe700 1 -- 172.21.15.19:0/3781301050 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f65600ebfe0 con 0x7f65641045e0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f656a1f5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7c795c0 0x5602b7d12f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f656a1f5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7c795c0 0x5602b7d12f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46716/0 (socket says 172.21.15.19:46716) 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f656a1f5700 1 -- 172.21.15.19:0/1057192450 learned_addr learned my addr 172.21.15.19:0/1057192450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f65699f4700 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.341+0000 7f656a9f6700 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d0d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 -- 172.21.15.19:0/1057192450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7c795c0 msgr2=0x5602b7d12f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7c795c0 0x5602b7d12f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 -- 172.21.15.19:0/1057192450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 msgr2=0x5602b7d134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 -- 172.21.15.19:0/1057192450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b7bd4140 con 0x5602b7d0d0b0 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f65699f4700 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d0d580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c090 tx=0x7f655c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5538d78cbb832a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f656a9f6700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65500b2900 0x7f65500b4e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f655c009fd0 tx=0x7f655c00cc00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f6558d47700 1 -- 172.21.15.19:0/1057192450 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c009320 con 0x5602b7d0d0b0 2024-05-17T09:43:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.342+0000 7f6558d47700 1 -- 172.21.15.19:0/1057192450 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c0075c0 con 0x5602b7d0d0b0 2024-05-17T09:43:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f6558d47700 1 -- 172.21.15.19:0/1057192450 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c025620 con 0x5602b7d0d0b0 2024-05-17T09:43:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f657ae6eb80 1 -- 172.21.15.19:0/1057192450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 msgr2=0x5602b7d0d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d0d580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c090 tx=0x7f655c01d000 comp rx=0 tx=0).stop 2024-05-17T09:43:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f657ae6eb80 1 -- 172.21.15.19:0/1057192450 shutdown_connections 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7c795c0 0x5602b7d12f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d0d580 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.343+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/1057192450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.344+0000 7f657ae6eb80 1 -- 172.21.15.19:0/1057192450 >> 172.21.15.19:0/1057192450 conn(0x5602b7bee350 msgr2=0x5602b7d07d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.344+0000 7f657ae6eb80 1 -- 172.21.15.19:0/1057192450 shutdown_connections 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.344+0000 7f657ae6eb80 1 -- 172.21.15.19:0/1057192450 wait complete. 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.344+0000 7f657ae6eb80 1 -- start start 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.344+0000 7f657ae6eb80 1 Processor -- start 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7b8dfd0 0x5602b7d16b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7c795c0 0x5602b7d17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f65699f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7c795c0 0x5602b7d17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602b7b31130 con 0x5602b7c795c0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f65699f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7c795c0 0x5602b7d17090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53680/0 (socket says 172.21.15.19:53680) 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f65699f4700 1 -- 172.21.15.19:0/3157502598 learned_addr learned my addr 172.21.15.19:0/3157502598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5602b7b92f30 con 0x5602b7d0d0b0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5602b7d19bb0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7b8dfd0 0x5602b7d16b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f657ae6eb80 10 client.0 _ll_register_callbacks cb 0x5602b7bc5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a9f6700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a1f5700 1 -- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 msgr2=0x5602b7d175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a1f5700 1 -- 172.21.15.19:0/3157502598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7c795c0 msgr2=0x5602b7d17090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7c795c0 0x5602b7d17090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a1f5700 1 -- 172.21.15.19:0/3157502598 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b7b7aa60 con 0x5602b7b8dfd0 2024-05-17T09:43:18.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.345+0000 7f656a9f6700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f656a1f5700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7b8dfd0 0x5602b7d16b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f65600b2fe0 tx=0x7f65600b2a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b576419277bfb32c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f654dffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f654dffb700 1 -- 172.21.15.19:0/3157502598 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65600cc070 con 0x5602b7b8dfd0 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f654dffb700 1 -- 172.21.15.19:0/3157502598 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f65600af570 con 0x5602b7b8dfd0 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c0075c0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f654dffb700 1 -- 172.21.15.19:0/3157502598 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f65600b39e0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f657ae6eb80 10 client.28486 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5602b7be48f0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f654dffb700 1 -- 172.21.15.19:0/3157502598 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c0075c0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.346+0000 7f654dffb700 1 -- 172.21.15.19:0/3157502598 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f65600b3540 con 0x5602b7b8dfd0 2024-05-17T09:43:18.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f657ae6eb80 10 client.28486 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:18.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f657ae6eb80 10 client.28486 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:43:18.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f657ae6eb80 20 client.28486 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7d28dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f657ae6eb80 4 client.28486 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602b7cd9060 con 0x5602b7d26910 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f65699f4700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7d28dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f654dffb700 10 client.28486 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.347+0000 7f65699f4700 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7d28dc0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f5be2adb5b6c21 server_cookie=67bb06053025ff17 in_seq=0 out_seq=0 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f654dffb700 1 -- 172.21.15.19:0/3157502598 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5602b7cd9060 con 0x5602b7d26910 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f654dffb700 10 client.28486 handle_command_reply: tid=0 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 client.28486 shutdown 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7b8dfd0 msgr2=0x5602b7d16b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7b8dfd0 0x5602b7d16b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f65600b2fe0 tx=0x7f65600b2a50 comp rx=0 tx=0).stop 2024-05-17T09:43:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 shutdown_connections 2024-05-17T09:43:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7b8dfd0 0x5602b7d16b50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7d28dc0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f654dffb700 0 client.28486 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7d0d0b0 0x5602b7d175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/3157502598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7c795c0 0x5602b7d17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.350+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 >> 172.21.15.19:0/3157502598 conn(0x5602b7bee350 msgr2=0x5602b7d0c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:18.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.351+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 shutdown_connections 2024-05-17T09:43:18.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.351+0000 7f657ae6eb80 1 -- 172.21.15.19:0/3157502598 wait complete. 2024-05-17T09:43:18.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.351+0000 7f657ae6eb80 20 client.28486 trim_cache size 0 max 16384 2024-05-17T09:43:18.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.391+0000 7f657ae6eb80 1 Processor -- start 2024-05-17T09:43:18.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.391+0000 7f657ae6eb80 1 -- start start 2024-05-17T09:43:18.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d02680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b0990 con 0x5602b7b8dfd0 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65641ae220 con 0x5602b7c795c0 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f657ae6eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65641ac070 con 0x5602b7d0d0b0 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f656a1f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d02680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f656a1f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d02680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53682/0 (socket says 172.21.15.19:53682) 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f656a9f6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f656a1f5700 1 -- 172.21.15.19:0/772691569 learned_addr learned my addr 172.21.15.19:0/772691569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:18.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.392+0000 7f65699f4700 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f656a9f6700 1 -- 172.21.15.19:0/772691569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 msgr2=0x5602b7d145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f656a9f6700 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f656a9f6700 1 -- 172.21.15.19:0/772691569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 msgr2=0x5602b7d02680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f656a9f6700 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d02680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f656a9f6700 1 -- 172.21.15.19:0/772691569 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b7d15d80 con 0x5602b7d0d0b0 2024-05-17T09:43:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f656a9f6700 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d22b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f655c027040 tx=0x7f655c0121d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fac1662b73590f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f654ffff700 1 -- 172.21.15.19:0/772691569 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c033020 con 0x5602b7d0d0b0 2024-05-17T09:43:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f654ffff700 1 -- 172.21.15.19:0/772691569 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c009320 con 0x5602b7d0d0b0 2024-05-17T09:43:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.393+0000 7f654ffff700 1 -- 172.21.15.19:0/772691569 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c003cf0 con 0x5602b7d0d0b0 2024-05-17T09:43:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 -- 172.21.15.19:0/772691569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 msgr2=0x5602b7d22b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d22b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f655c027040 tx=0x7f655c0121d0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 -- 172.21.15.19:0/772691569 shutdown_connections 2024-05-17T09:43:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d22b40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d02680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/772691569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.394+0000 7f657ae6eb80 1 -- 172.21.15.19:0/772691569 >> 172.21.15.19:0/772691569 conn(0x5602b7bee350 msgr2=0x5602b7d10ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.395+0000 7f657ae6eb80 1 -- 172.21.15.19:0/772691569 shutdown_connections 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.395+0000 7f657ae6eb80 1 -- 172.21.15.19:0/772691569 wait complete. 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.395+0000 7f657ae6eb80 1 -- start start 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.395+0000 7f657ae6eb80 1 Processor -- start 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d1de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d1de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d1de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53686/0 (socket says 172.21.15.19:53686) 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65641ac6e0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65641adb50 con 0x5602b7c795c0 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 -- 172.21.15.19:0/2948830572 learned_addr learned my addr 172.21.15.19:0/2948830572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65641acdb0 con 0x5602b7d0d0b0 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f657ae6eb80 10 client.0 _ll_register_callbacks cb 0x5602b7bc5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a9f6700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f65699f4700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 -- 172.21.15.19:0/2948830572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 msgr2=0x5602b7d1e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d1e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 -- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 msgr2=0x5602b7d1e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d1e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a1f5700 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b7d1f060 con 0x5602b7b8dfd0 2024-05-17T09:43:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.396+0000 7f656a9f6700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d1e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f656a1f5700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d1de40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f65600e9780 tx=0x7f6560005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01d817021c70fed server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f654dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f654dffb700 1 -- 172.21.15.19:0/2948830572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65600cc070 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f654dffb700 1 -- 172.21.15.19:0/2948830572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f65600bbc20 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c009320 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f654dffb700 1 -- 172.21.15.19:0/2948830572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f65600d2080 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f657ae6eb80 10 client.19008 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.397+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5602b7da2be0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f654dffb700 1 -- 172.21.15.19:0/2948830572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655c009320 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f654dffb700 1 -- 172.21.15.19:0/2948830572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6560073ba0 con 0x5602b7b8dfd0 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f657ae6eb80 10 client.19008 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f657ae6eb80 10 client.19008 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:43:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f657ae6eb80 20 client.19008 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:18.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7da5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:18.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f657ae6eb80 4 client.19008 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.5, "await_for": 45.0}] 2024-05-17T09:43:18.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.5, "await_for": 45.0}) -- 0x5602b7c3ce50 con 0x5602b7d26910 2024-05-17T09:43:18.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.398+0000 7f65699f4700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7da5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:18.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.399+0000 7f654dffb700 10 client.19008 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:18.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.399+0000 7f65699f4700 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7da5130 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5a8fc3dfc1146 server_cookie=4352c45550658b39 in_seq=0 out_seq=0 2024-05-17T09:43:18.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67449 6.54 MB/sec execute 435 sec latency 213.286 ms 2024-05-17T09:43:18.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:18 smithi019 ceph-mon[31620]: pgmap v1144: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f654dffb700 1 -- 172.21.15.19:0/2948830572 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5602b7c3ce50 con 0x5602b7d26910 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f654dffb700 10 client.19008 handle_command_reply: tid=0 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 client.19008 shutdown 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 msgr2=0x5602b7d1de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d1de40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f65600e9780 tx=0x7f6560005ee0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 shutdown_connections 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602b7d0d0b0 0x5602b7d1e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5602b7d26910 0x5602b7da5130 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602b7b8dfd0 0x5602b7d1de40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 --2- 172.21.15.19:0/2948830572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5602b7c795c0 0x5602b7d1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 >> 172.21.15.19:0/2948830572 conn(0x5602b7bee350 msgr2=0x5602b7d0c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f654dffb700 0 client.19008 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 shutdown_connections 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 1 -- 172.21.15.19:0/2948830572 wait complete. 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.808+0000 7f657ae6eb80 20 client.19008 trim_cache size 0 max 16384 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 86, 2024-05-17T09:43:18.808 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "bc19407a": { 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 86, 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.5, 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:43:18.809 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:18.810 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65500b2900 msgr2=0x7f65500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65500b2900 0x7f65500b4e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f655c009fd0 tx=0x7f655c00cc00 comp rx=0 tx=0).stop 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65641045e0 msgr2=0x7f6564075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65641045e0 0x7f6564075ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6560009fd0 tx=0x7f656000fa00 comp rx=0 tx=0).stop 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 shutdown_connections 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f65500b2900 0x7f65500b4e10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65641045e0 0x7f6564075ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564109860 0x7f6564076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 --2- 172.21.15.19:0/3781301050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6564108f10 0x7f6564075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 >> 172.21.15.19:0/3781301050 conn(0x7f6564100030 msgr2=0x7f6564102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:18.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 shutdown_connections 2024-05-17T09:43:18.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:18.809+0000 7f656c459700 1 -- 172.21.15.19:0/3781301050 wait complete. 2024-05-17T09:43:18.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:18 smithi069 ceph-mon[35081]: pgmap v1144: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-05-17T09:43:19.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:18 smithi079 ceph-mon[35502]: pgmap v1144: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-05-17T09:43:19.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.52 MB/sec execute 436 sec latency 1202.267 ms 2024-05-17T09:43:19.631 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bc19407a', quiesce took 8.2 seconds. Will release after: 14.3 2024-05-17T09:43:20.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.51 MB/sec execute 437 sec latency 2202.332 ms 2024-05-17T09:43:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:20 smithi069 ceph-mon[35081]: pgmap v1145: 129 pgs: 129 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-05-17T09:43:20.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:20 smithi019 ceph-mon[31620]: pgmap v1145: 129 pgs: 129 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-05-17T09:43:21.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:20 smithi079 ceph-mon[35502]: pgmap v1145: 129 pgs: 129 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-05-17T09:43:21.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.50 MB/sec execute 438 sec latency 3202.394 ms 2024-05-17T09:43:22.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.48 MB/sec execute 439 sec latency 4202.434 ms 2024-05-17T09:43:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:22 smithi069 ceph-mon[35081]: pgmap v1146: 129 pgs: 129 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 246 op/s 2024-05-17T09:43:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:22 smithi019 ceph-mon[31620]: pgmap v1146: 129 pgs: 129 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 246 op/s 2024-05-17T09:43:23.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:22 smithi079 ceph-mon[35502]: pgmap v1146: 129 pgs: 129 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 246 op/s 2024-05-17T09:43:23.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.47 MB/sec execute 440 sec latency 5202.474 ms 2024-05-17T09:43:24.593 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.45 MB/sec execute 441 sec latency 6202.513 ms 2024-05-17T09:43:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:24 smithi069 ceph-mon[35081]: pgmap v1147: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-05-17T09:43:24.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:24 smithi019 ceph-mon[31620]: pgmap v1147: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-05-17T09:43:25.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:24 smithi079 ceph-mon[35502]: pgmap v1147: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-05-17T09:43:25.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.44 MB/sec execute 442 sec latency 7202.576 ms 2024-05-17T09:43:25.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:25 smithi069 ceph-mon[35081]: pgmap v1148: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.7 MiB/s wr, 198 op/s 2024-05-17T09:43:25.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:25 smithi019 ceph-mon[31620]: pgmap v1148: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.7 MiB/s wr, 198 op/s 2024-05-17T09:43:26.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:25 smithi079 ceph-mon[35502]: pgmap v1148: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.7 MiB/s wr, 198 op/s 2024-05-17T09:43:26.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.42 MB/sec execute 443 sec latency 8202.647 ms 2024-05-17T09:43:27.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.41 MB/sec execute 444 sec latency 9202.769 ms 2024-05-17T09:43:28.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.39 MB/sec execute 445 sec latency 10202.855 ms 2024-05-17T09:43:28.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:28 smithi079 ceph-mon[35502]: pgmap v1149: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-17T09:43:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:28 smithi069 ceph-mon[35081]: pgmap v1149: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-17T09:43:28.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:28 smithi019 ceph-mon[31620]: pgmap v1149: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-17T09:43:29.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.38 MB/sec execute 446 sec latency 11202.944 ms 2024-05-17T09:43:30.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.36 MB/sec execute 447 sec latency 12203.016 ms 2024-05-17T09:43:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 -- 172.21.15.19:0/2183818407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c1022c0 msgr2=0x7ff31c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 --2- 172.21.15.19:0/2183818407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c1022c0 0x7ff31c102fc0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff31800aa10 tx=0x7ff31802b720 comp rx=0 tx=0).stop 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 -- 172.21.15.19:0/2183818407 shutdown_connections 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 --2- 172.21.15.19:0/2183818407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff31c108200 0x7ff31c10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 --2- 172.21.15.19:0/2183818407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff31c103500 0x7ff31c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 --2- 172.21.15.19:0/2183818407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c1022c0 0x7ff31c102fc0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.656+0000 7ff323ad9640 1 -- 172.21.15.19:0/2183818407 >> 172.21.15.19:0/2183818407 conn(0x7ff31c0fd4d0 msgr2=0x7ff31c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.657+0000 7ff323ad9640 1 -- 172.21.15.19:0/2183818407 shutdown_connections 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.657+0000 7ff323ad9640 1 -- 172.21.15.19:0/2183818407 wait complete. 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.657+0000 7ff323ad9640 1 Processor -- start 2024-05-17T09:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.657+0000 7ff323ad9640 1 -- start start 2024-05-17T09:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff323ad9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff31c1022c0 0x7ff31c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff323ad9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff31c103500 0x7ff31c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff323ad9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 0x7ff31c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff323ad9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10d400 con 0x7ff31c108200 2024-05-17T09:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff323ad9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10d280 con 0x7ff31c103500 2024-05-17T09:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff323ad9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10d580 con 0x7ff31c1022c0 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 0x7ff31c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 0x7ff31c1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57102/0 (socket says 172.21.15.19:57102) 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 -- 172.21.15.19:0/3131470738 learned_addr learned my addr 172.21.15.19:0/3131470738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff321875640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff31c1022c0 0x7ff31c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff321074640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff31c103500 0x7ff31c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 -- 172.21.15.19:0/3131470738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff31c1022c0 msgr2=0x7ff31c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff31c1022c0 0x7ff31c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 -- 172.21.15.19:0/3131470738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff31c103500 msgr2=0x7ff31c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff31c103500 0x7ff31c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 -- 172.21.15.19:0/3131470738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c1a1d10 con 0x7ff31c108200 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.658+0000 7ff322076640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 0x7ff31c1a16a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff31400eb10 tx=0x7ff31400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.659+0000 7ff312ffd640 1 -- 172.21.15.19:0/3131470738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314004b20 con 0x7ff31c108200 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.659+0000 7ff312ffd640 1 -- 172.21.15.19:0/3131470738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff314004cc0 con 0x7ff31c108200 2024-05-17T09:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.659+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1a1fa0 con 0x7ff31c108200 2024-05-17T09:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.659+0000 7ff312ffd640 1 -- 172.21.15.19:0/3131470738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314010690 con 0x7ff31c108200 2024-05-17T09:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.659+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1a24b0 con 0x7ff31c108200 2024-05-17T09:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.662+0000 7ff32a98e740 1 Processor -- start 2024-05-17T09:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.662+0000 7ff312ffd640 1 -- 172.21.15.19:0/3131470738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff314005020 con 0x7ff31c108200 2024-05-17T09:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.662+0000 7ff32a98e740 1 -- start start 2024-05-17T09:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dca97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dca9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcb0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff32a98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56231dc17c30 con 0x56231d8db2f0 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff32a98e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56231dc077d0 con 0x56231d8e34d0 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff32a98e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56231dc0b810 con 0x56231d9041b0 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dca9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dca9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57104/0 (socket says 172.21.15.19:57104) 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 -- 172.21.15.19:0/4127950098 learned_addr learned my addr 172.21.15.19:0/4127950098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321074640 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcb0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321074640 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dca97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff312ffd640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3080b2920 0x7ff3080b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 -- 172.21.15.19:0/4127950098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 msgr2=0x56231dca97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dca97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 -- 172.21.15.19:0/4127950098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 msgr2=0x56231dcb0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcb0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff321875640 1 -- 172.21.15.19:0/4127950098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231d91c210 con 0x56231d8db2f0 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff322076640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3080b2920 0x7ff3080b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.663+0000 7ff312ffd640 1 -- 172.21.15.19:0/3131470738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff3140e2a50 con 0x7ff31c108200 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.664+0000 7ff321875640 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dca9d30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff3180057c0 tx=0x7ff318005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76205f47b61f243a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.664+0000 7ff307fff640 1 -- 172.21.15.19:0/4127950098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318036040 con 0x56231d8db2f0 2024-05-17T09:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.664+0000 7ff322076640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3080b2920 0x7ff3080b4e30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff3140a6f30 tx=0x7ff31406f2b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.664+0000 7ff307fff640 1 -- 172.21.15.19:0/4127950098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff318004030 con 0x56231d8db2f0 2024-05-17T09:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.664+0000 7ff307fff640 1 -- 172.21.15.19:0/4127950098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3180039a0 con 0x56231d8db2f0 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 -- 172.21.15.19:0/4127950098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 msgr2=0x56231dca9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dca9d30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff3180057c0 tx=0x7ff318005fd0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 -- 172.21.15.19:0/4127950098 shutdown_connections 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcb0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dca9d30 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 --2- 172.21.15.19:0/4127950098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dca97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 -- 172.21.15.19:0/4127950098 >> 172.21.15.19:0/4127950098 conn(0x56231d8f8300 msgr2=0x56231dca0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 -- 172.21.15.19:0/4127950098 shutdown_connections 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 -- 172.21.15.19:0/4127950098 wait complete. 2024-05-17T09:43:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 -- start start 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.665+0000 7ff32a98e740 1 Processor -- start 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d8e34d0 0x56231dcb3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 0x56231dcba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56231d8ed790 con 0x56231d9041b0 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56231dbc3890 con 0x56231d8db2f0 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56231dcb1620 con 0x56231d8e34d0 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff32a98e740 10 client.0 _ll_register_callbacks cb 0x56231dbc6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 0x56231dcba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 0x56231dcba4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57114/0 (socket says 172.21.15.19:57114) 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 -- 172.21.15.19:0/484975854 learned_addr learned my addr 172.21.15.19:0/484975854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321875640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d8e34d0 0x56231dcb3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 -- 172.21.15.19:0/484975854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d8e34d0 msgr2=0x56231dcb3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d8e34d0 0x56231dcb3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 -- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 msgr2=0x56231dcb3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.666+0000 7ff321074640 1 -- 172.21.15.19:0/484975854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231d9e8620 con 0x56231d9041b0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff321074640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 0x56231dcba4d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff30c012c80 tx=0x7ff30c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84cfa0c8a386d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff3057fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff3057fa640 1 -- 172.21.15.19:0/484975854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c010e70 con 0x56231d9041b0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff3057fa640 1 -- 172.21.15.19:0/484975854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff30c004050 con 0x56231d9041b0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff318004030 con 0x56231d9041b0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff3057fa640 1 -- 172.21.15.19:0/484975854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff30c02a050 con 0x56231d9041b0 2024-05-17T09:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff32a98e740 10 client.19020 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56231dcbd4d0 con 0x56231d9041b0 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff3057fa640 1 -- 172.21.15.19:0/484975854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff318004030 con 0x56231d9041b0 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.667+0000 7ff3057fa640 1 -- 172.21.15.19:0/484975854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ff30c0047a0 con 0x56231d9041b0 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff32a98e740 10 client.19020 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff32a98e740 10 client.19020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff32a98e740 20 client.19020 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff32a98e740 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8c2d80 0x56231dcc0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff32a98e740 4 client.19020 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231dc19b80 con 0x56231d8c2d80 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff321875640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8c2d80 0x56231dcc0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff321875640 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8c2d80 0x56231dcc0ca0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60c4b71aa2c9101 server_cookie=6ee703acc5aa5103 in_seq=0 out_seq=0 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.668+0000 7ff3057fa640 10 client.19020 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff3057fa640 1 -- 172.21.15.19:0/484975854 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56231dc19b80 con 0x56231d8c2d80 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff3057fa640 10 client.19020 handle_command_reply: tid=0 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 client.19020 shutdown 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 msgr2=0x56231dcba4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 0x56231dcba4d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff30c012c80 tx=0x7ff30c004730 comp rx=0 tx=0).stop 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 shutdown_connections 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8c2d80 0x56231dcc0ca0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d9041b0 0x56231dcba4d0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d8e34d0 0x56231dcb3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 --2- 172.21.15.19:0/484975854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff3057fa640 0 client.19020 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 >> 172.21.15.19:0/484975854 conn(0x56231d8f8300 msgr2=0x56231dca3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.669+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 shutdown_connections 2024-05-17T09:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.670+0000 7ff32a98e740 1 -- 172.21.15.19:0/484975854 wait complete. 2024-05-17T09:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.670+0000 7ff32a98e740 20 client.19020 trim_cache size 0 max 16384 2024-05-17T09:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 Processor -- start 2024-05-17T09:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 -- start start 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dcacbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1ad0e0 con 0x56231d8db2f0 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1aa9b0 con 0x56231d8e34d0 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff32a98e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1a8800 con 0x56231d9041b0 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff321074640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dcacbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff321074640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dcacbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57128/0 (socket says 172.21.15.19:57128) 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.685+0000 7ff321074640 1 -- 172.21.15.19:0/3426551129 learned_addr learned my addr 172.21.15.19:0/3426551129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321875640 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321074640 1 -- 172.21.15.19:0/3426551129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 msgr2=0x56231dcba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321074640 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321074640 1 -- 172.21.15.19:0/3426551129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 msgr2=0x56231dcba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321074640 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcba460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321074640 1 -- 172.21.15.19:0/3426551129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231dcb4170 con 0x56231d8db2f0 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321875640 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff321074640 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dcacbd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff30c001eb0 tx=0x7ff30c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905ff969586174a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff3077fe640 1 -- 172.21.15.19:0/3426551129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c033070 con 0x56231d8db2f0 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff3077fe640 1 -- 172.21.15.19:0/3426551129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff30c0120c0 con 0x56231d8db2f0 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.686+0000 7ff3077fe640 1 -- 172.21.15.19:0/3426551129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0054e0 con 0x56231d8db2f0 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 -- 172.21.15.19:0/3426551129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 msgr2=0x56231dcacbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dcacbd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff30c001eb0 tx=0x7ff30c004640 comp rx=0 tx=0).stop 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 -- 172.21.15.19:0/3426551129 shutdown_connections 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8e34d0 0x56231dcba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 --2- 172.21.15.19:0/3426551129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8db2f0 0x56231dcacbd0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 -- 172.21.15.19:0/3426551129 >> 172.21.15.19:0/3426551129 conn(0x56231d8f8300 msgr2=0x56231dca0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 -- 172.21.15.19:0/3426551129 shutdown_connections 2024-05-17T09:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 -- 172.21.15.19:0/3426551129 wait complete. 2024-05-17T09:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 -- start start 2024-05-17T09:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.687+0000 7ff32a98e740 1 Processor -- start 2024-05-17T09:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 0x56231dcb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcb9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1a8e70 con 0x56231d8e34d0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1aa2e0 con 0x56231d8db2f0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1a9540 con 0x56231d9041b0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff32a98e740 10 client.0 _ll_register_callbacks cb 0x7ff30c0128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 0x56231dcb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 0x56231dcb8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57138/0 (socket says 172.21.15.19:57138) 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 -- 172.21.15.19:0/1147059761 learned_addr learned my addr 172.21.15.19:0/1147059761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321074640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcb9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 -- 172.21.15.19:0/1147059761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 msgr2=0x56231dcb9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcb9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321074640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 -- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 msgr2=0x56231dcb7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321875640 1 -- 172.21.15.19:0/1147059761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231dcb9830 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.688+0000 7ff321074640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff321875640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 0x56231dcb8390 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff318050fd0 tx=0x7ff318005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7503d878cae43e40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff3057fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff3057fa640 1 -- 172.21.15.19:0/1147059761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318036040 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff3057fa640 1 -- 172.21.15.19:0/1147059761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff318030610 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30c0120c0 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff3057fa640 1 -- 172.21.15.19:0/1147059761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff318060050 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 10 client.19026 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56231dcbab50 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff3057fa640 1 -- 172.21.15.19:0/1147059761 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff30c0120c0 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff3057fa640 1 -- 172.21.15.19:0/1147059761 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ff31800b040 con 0x56231d8e34d0 2024-05-17T09:43:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 10 client.19026 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 10 client.19026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 20 client.19026 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8981c0 0x56231dd2b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 4 client.19026 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.689+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56231dc13360 con 0x56231d8981c0 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.690+0000 7ff321074640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8981c0 0x56231dd2b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.690+0000 7ff321074640 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8981c0 0x56231dd2b550 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98c68f55c4fb92a server_cookie=be8cf08bb3d7fd1e in_seq=0 out_seq=0 2024-05-17T09:43:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.690+0000 7ff3057fa640 10 client.19026 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.733+0000 7ff3057fa640 1 -- 172.21.15.19:0/1147059761 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56231dc13360 con 0x56231d8981c0 2024-05-17T09:43:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.733+0000 7ff3057fa640 10 client.19026 handle_command_reply: tid=0 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.733+0000 7ff32a98e740 1 client.19026 shutdown 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 msgr2=0x56231dcb8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 0x56231dcb8390 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff318050fd0 tx=0x7ff318005890 comp rx=0 tx=0).stop 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 shutdown_connections 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56231d8981c0 0x56231dd2b550 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56231d9041b0 0x56231dcb9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56231d8e34d0 0x56231dcb8390 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff3057fa640 0 client.19026 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 --2- 172.21.15.19:0/1147059761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56231d8db2f0 0x56231dcb7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 >> 172.21.15.19:0/1147059761 conn(0x56231d8f8300 msgr2=0x56231dca0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 shutdown_connections 2024-05-17T09:43:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 1 -- 172.21.15.19:0/1147059761 wait complete. 2024-05-17T09:43:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.734+0000 7ff32a98e740 20 client.19026 trim_cache size 0 max 16384 2024-05-17T09:43:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:43:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:43:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3080b2920 msgr2=0x7ff3080b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3080b2920 0x7ff3080b4e30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff3140a6f30 tx=0x7ff31406f2b0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 msgr2=0x7ff31c1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 0x7ff31c1a16a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff31400eb10 tx=0x7ff31400c2d0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 shutdown_connections 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff3080b2920 0x7ff3080b4e30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c108200 0x7ff31c1a16a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff31c103500 0x7ff31c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 --2- 172.21.15.19:0/3131470738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff31c1022c0 0x7ff31c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 >> 172.21.15.19:0/3131470738 conn(0x7ff31c0fd4d0 msgr2=0x7ff31c1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 shutdown_connections 2024-05-17T09:43:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:30.735+0000 7ff323ad9640 1 -- 172.21.15.19:0/3131470738 wait complete. 2024-05-17T09:43:30.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:30 smithi079 ceph-mon[35502]: pgmap v1150: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:43:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:30 smithi069 ceph-mon[35081]: pgmap v1150: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:43:30.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:30 smithi019 ceph-mon[31620]: pgmap v1150: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-17T09:43:31.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.35 MB/sec execute 448 sec latency 13203.131 ms 2024-05-17T09:43:32.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.34 MB/sec execute 449 sec latency 14203.223 ms 2024-05-17T09:43:32.731 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:32 smithi069 ceph-mon[35081]: pgmap v1151: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 489 KiB/s wr, 49 op/s 2024-05-17T09:43:32.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:32 smithi079 ceph-mon[35502]: pgmap v1151: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 489 KiB/s wr, 49 op/s 2024-05-17T09:43:32.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:32 smithi019 ceph-mon[31620]: pgmap v1151: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 489 KiB/s wr, 49 op/s 2024-05-17T09:43:33.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.32 MB/sec execute 450 sec latency 15203.349 ms 2024-05-17T09:43:33.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:43:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:43:34.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.31 MB/sec execute 451 sec latency 16203.409 ms 2024-05-17T09:43:34.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:34 smithi019 ceph-mon[31620]: pgmap v1152: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 297 KiB/s wr, 44 op/s 2024-05-17T09:43:34.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:34 smithi079 ceph-mon[35502]: pgmap v1152: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 297 KiB/s wr, 44 op/s 2024-05-17T09:43:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:34 smithi069 ceph-mon[35081]: pgmap v1152: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 297 KiB/s wr, 44 op/s 2024-05-17T09:43:35.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.29 MB/sec execute 452 sec latency 17203.508 ms 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 -- 172.21.15.19:0/2396565236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 msgr2=0x7f5ec41049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 --2- 172.21.15.19:0/2396565236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41049d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40099c0 tx=0x7f5eb4009cd0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 -- 172.21.15.19:0/2396565236 shutdown_connections 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 --2- 172.21.15.19:0/2396565236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ec4109840 0x7f5ec41111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 --2- 172.21.15.19:0/2396565236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41049d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 --2- 172.21.15.19:0/2396565236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 0x7f5ec4109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 -- 172.21.15.19:0/2396565236 >> 172.21.15.19:0/2396565236 conn(0x7f5ec40ffe90 msgr2=0x7f5ec41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 -- 172.21.15.19:0/2396565236 shutdown_connections 2024-05-17T09:43:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.193+0000 7f5eca487700 1 -- 172.21.15.19:0/2396565236 wait complete. 2024-05-17T09:43:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.194+0000 7f5eca487700 1 Processor -- start 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.194+0000 7f5eca487700 1 -- start start 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5eca487700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5eca487700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 0x7f5ec41a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5eca487700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ec4109840 0x7f5ec419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5eca487700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4114320 con 0x7f5ec41045c0 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5eca487700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41141a0 con 0x7f5ec4108ef0 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5eca487700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41144a0 con 0x7f5ec4109840 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44112/0 (socket says 172.21.15.19:44112) 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 -- 172.21.15.19:0/1569212116 learned_addr learned my addr 172.21.15.19:0/1569212116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec8a24700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ec4109840 0x7f5ec419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec37fe700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 0x7f5ec41a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 -- 172.21.15.19:0/1569212116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ec4109840 msgr2=0x7f5ec419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ec4109840 0x7f5ec419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 -- 172.21.15.19:0/1569212116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 msgr2=0x7f5ec41a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 0x7f5ec41a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 -- 172.21.15.19:0/1569212116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec419b270 con 0x7f5ec41045c0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec37fe700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 0x7f5ec41a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.195+0000 7f5ec3fff700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41a09b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4006010 tx=0x7f5eb4004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.196+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4026070 con 0x7f5ec41045c0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.196+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5eb4003c50 con 0x7f5ec41045c0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.196+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec419b500 con 0x7f5ec41045c0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.196+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb400fb10 con 0x7f5ec41045c0 2024-05-17T09:43:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.196+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec41a7760 con 0x7f5ec41045c0 2024-05-17T09:43:36.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.197+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec4105a70 con 0x7f5ec41045c0 2024-05-17T09:43:36.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.199+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5eb4029070 con 0x7f5ec41045c0 2024-05-17T09:43:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.199+0000 7f5ec17fa700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5eb00b2900 0x7f5eb00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.199+0000 7f5ec37fe700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5eb00b2900 0x7f5eb00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.199+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5eb40e32e0 con 0x7f5ec41045c0 2024-05-17T09:43:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.200+0000 7f5ec37fe700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5eb00b2900 0x7f5eb00b4e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5ec419c270 tx=0x7f5eac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:36.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.203+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5eb4013070 con 0x7f5ec41045c0 2024-05-17T09:43:36.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.468+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec419bfe0 con 0x7f5ec41045c0 2024-05-17T09:43:36.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.469+0000 7f5ec17fa700 1 -- 172.21.15.19:0/1569212116 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f5eb40aace0 con 0x7f5ec41045c0 2024-05-17T09:43:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:43:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.471+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5eb00b2900 msgr2=0x7f5eb00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.471+0000 7f5eca487700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5eb00b2900 0x7f5eb00b4e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5ec419c270 tx=0x7f5eac009260 comp rx=0 tx=0).stop 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.471+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 msgr2=0x7f5ec41a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.471+0000 7f5eca487700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41a09b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4006010 tx=0x7f5eb4004320 comp rx=0 tx=0).stop 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 shutdown_connections 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5eb00b2900 0x7f5eb00b4e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ec4109840 0x7f5ec419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41045c0 0x7f5ec41a09b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 --2- 172.21.15.19:0/1569212116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec4108ef0 0x7f5ec41a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 >> 172.21.15.19:0/1569212116 conn(0x7f5ec40ffe90 msgr2=0x7f5ec41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 shutdown_connections 2024-05-17T09:43:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:36.472+0000 7f5eca487700 1 -- 172.21.15.19:0/1569212116 wait complete. 2024-05-17T09:43:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:43:36.594 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.28 MB/sec execute 453 sec latency 18203.564 ms 2024-05-17T09:43:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:36 smithi079 ceph-mon[35502]: pgmap v1153: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-05-17T09:43:36.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:36 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1569212116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:36.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:36 smithi019 ceph-mon[31620]: pgmap v1153: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-05-17T09:43:36.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:36 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1569212116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:36 smithi069 ceph-mon[35081]: pgmap v1153: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-05-17T09:43:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:36 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1569212116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:37.138 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id bc19407a --release --await' 2024-05-17T09:43:37.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id bc19407a --release --await 2024-05-17T09:43:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:43:37.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.27 MB/sec execute 454 sec latency 19203.647 ms 2024-05-17T09:43:38.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67457 6.25 MB/sec execute 455 sec latency 20203.771 ms 2024-05-17T09:43:38.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:38 smithi079 ceph-mon[35502]: pgmap v1154: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-05-17T09:43:38.831 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:38 smithi019 ceph-mon[31620]: pgmap v1154: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-05-17T09:43:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:38 smithi069 ceph-mon[35081]: pgmap v1154: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-05-17T09:43:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 -- 172.21.15.19:0/1030744196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 msgr2=0x7f3bbc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/1030744196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc108200 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0099c0 tx=0x7f3bac009cd0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 -- 172.21.15.19:0/1030744196 shutdown_connections 2024-05-17T09:43:39.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/1030744196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc108740 0x7f3bbc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/1030744196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bbc1034c0 0x7f3bbc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/1030744196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc108200 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 -- 172.21.15.19:0/1030744196 >> 172.21.15.19:0/1030744196 conn(0x7f3bbc0fef10 msgr2=0x7f3bbc101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 -- 172.21.15.19:0/1030744196 shutdown_connections 2024-05-17T09:43:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.223+0000 7f3bc3a35700 1 -- 172.21.15.19:0/1030744196 wait complete. 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 Processor -- start 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 -- start start 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc1034c0 0x7f3bbc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bbc108740 0x7f3bbc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc113200 con 0x7f3bbc107df0 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc113080 con 0x7f3bbc108740 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc3a35700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc113380 con 0x7f3bbc1034c0 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc0fd0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc0fd0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44132/0 (socket says 172.21.15.19:44132) 2024-05-17T09:43:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/3834565159 learned_addr learned my addr 172.21.15.19:0/3834565159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.224+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bbc108740 0x7f3bbc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc1034c0 0x7f3bbc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/3834565159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc1034c0 msgr2=0x7f3bbc19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc1034c0 0x7f3bbc19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/3834565159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bbc108740 msgr2=0x7f3bbc19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bbc108740 0x7f3bbc19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/3834565159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc19fc50 con 0x7f3bbc107df0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc1034c0 0x7f3bbc19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc19b320 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0045e0 tx=0x7f3bac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bb27fc700 1 -- 172.21.15.19:0/3834565159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac026070 con 0x7f3bbc107df0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc19fee0 con 0x7f3bbc107df0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bb27fc700 1 -- 172.21.15.19:0/3834565159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bac00f460 con 0x7f3bbc107df0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bb27fc700 1 -- 172.21.15.19:0/3834565159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac009e60 con 0x7f3bbc107df0 2024-05-17T09:43:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.225+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc1a7860 con 0x7f3bbc107df0 2024-05-17T09:43:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.229+0000 7f3bb27fc700 1 -- 172.21.15.19:0/3834565159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3bac029070 con 0x7f3bbc107df0 2024-05-17T09:43:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bb27fc700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3ba80b2880 0x7f3ba80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 Processor -- start 2024-05-17T09:43:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3ba80b2880 0x7f3ba80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 -- start start 2024-05-17T09:43:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bb27fc700 1 -- 172.21.15.19:0/3834565159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3bac0e2ed0 con 0x7f3bbc107df0 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef0064d0 0x561aef10d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aeef88f40 0x561aef10daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 0x561aef107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561aef035d90 con 0x561aef107620 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.230+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561aef025930 con 0x561aeef88f40 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561aef029970 con 0x561aef0064d0 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 0x561aef107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 0x561aef107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44138/0 (socket says 172.21.15.19:44138) 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 -- 172.21.15.19:0/1469157240 learned_addr learned my addr 172.21.15.19:0/1469157240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3ba80b2880 0x7f3ba80b4d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8005950 tx=0x7f3bb80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aeef88f40 0x561aef10daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef0064d0 0x561aef10d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 -- 172.21.15.19:0/1469157240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef0064d0 msgr2=0x561aef10d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef0064d0 0x561aef10d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 -- 172.21.15.19:0/1469157240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aeef88f40 msgr2=0x561aef10daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aeef88f40 0x561aef10daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 -- 172.21.15.19:0/1469157240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aeef759d0 con 0x561aef107620 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aeef88f40 0x561aef10daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 0x561aef107a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4011680 tx=0x7f3bb4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f094e8e1c05482 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3ba77fe700 1 -- 172.21.15.19:0/1469157240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4019030 con 0x561aef107620 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3ba77fe700 1 -- 172.21.15.19:0/1469157240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb40092e0 con 0x561aef107620 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.231+0000 7f3ba77fe700 1 -- 172.21.15.19:0/1469157240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4007500 con 0x561aef107620 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 -- 172.21.15.19:0/1469157240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 msgr2=0x561aef107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 0x561aef107a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4011680 tx=0x7f3bb4009920 comp rx=0 tx=0).stop 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 -- 172.21.15.19:0/1469157240 shutdown_connections 2024-05-17T09:43:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef0064d0 0x561aef10d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef107620 0x561aef107a30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/1469157240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aeef88f40 0x561aef10daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 -- 172.21.15.19:0/1469157240 >> 172.21.15.19:0/1469157240 conn(0x561aeefc7d80 msgr2=0x561aef102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 -- 172.21.15.19:0/1469157240 shutdown_connections 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.232+0000 7f3bd244ab80 1 -- 172.21.15.19:0/1469157240 wait complete. 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 -- start start 2024-05-17T09:43:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 Processor -- start 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef1111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef111720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef111c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561aeef2c0a0 con 0x561aeef88f40 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561aeefc78c0 con 0x561aef0064d0 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561aef11d3d0 con 0x561aef107620 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bd244ab80 10 client.0 _ll_register_callbacks cb 0x561aeefc0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bc17d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef1111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bc17d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef1111e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44152/0 (socket says 172.21.15.19:44152) 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.233+0000 7f3bc17d1700 1 -- 172.21.15.19:0/2377030854 learned_addr learned my addr 172.21.15.19:0/2377030854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef111720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc17d1700 1 -- 172.21.15.19:0/2377030854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 msgr2=0x561aef111c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef111c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc17d1700 1 -- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 msgr2=0x561aef111720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef111720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc17d1700 1 -- 172.21.15.19:0/2377030854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aeefc4790 con 0x561aeef88f40 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef1111e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8004080 tx=0x7f3bb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86edfca849175682 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3ba4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/2377030854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8024020 con 0x561aeef88f40 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/2377030854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3bb8003d10 con 0x561aeef88f40 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb40092e0 con 0x561aeef88f40 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/2377030854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3bb802f050 con 0x561aeef88f40 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bd244ab80 10 client.19044 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.234+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561aef11ecd0 con 0x561aeef88f40 2024-05-17T09:43:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/2377030854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb40092e0 con 0x561aeef88f40 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/2377030854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3bb8011440 con 0x561aeef88f40 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bd244ab80 10 client.19044 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bd244ab80 10 client.19044 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bd244ab80 20 client.19044 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef123780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bd244ab80 4 client.19044 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aef037ce0 con 0x561aef121360 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.235+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.236+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef123780 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04f7b3b430c25ce server_cookie=3e03cd4bdf27fee1 in_seq=0 out_seq=0 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.236+0000 7f3ba4ff9700 10 client.19044 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.236+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/2377030854 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561aef037ce0 con 0x561aef121360 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.236+0000 7f3ba4ff9700 10 client.19044 handle_command_reply: tid=0 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.236+0000 7f3bd244ab80 1 client.19044 shutdown 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 msgr2=0x561aef1111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef1111e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8004080 tx=0x7f3bb8004750 comp rx=0 tx=0).stop 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 shutdown_connections 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef111c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef123780 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef1111e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3ba4ff9700 0 client.19044 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/2377030854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef111720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 >> 172.21.15.19:0/2377030854 conn(0x561aeefc7d80 msgr2=0x561aef106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 shutdown_connections 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 1 -- 172.21.15.19:0/2377030854 wait complete. 2024-05-17T09:43:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.237+0000 7f3bd244ab80 20 client.19044 trim_cache size 0 max 16384 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.257+0000 7f3bd244ab80 1 Processor -- start 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.257+0000 7f3bd244ab80 1 -- start start 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aeef88f40 0x561aef113270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 0x561aef10be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef107620 0x561aef10c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1b08c0 con 0x561aef0064d0 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1ae150 con 0x561aef107620 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1abfa0 con 0x561aeef88f40 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 0x561aef10be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 0x561aef10be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44158/0 (socket says 172.21.15.19:44158) 2024-05-17T09:43:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/565936850 learned_addr learned my addr 172.21.15.19:0/565936850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aeef88f40 0x561aef113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef107620 0x561aef10c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/565936850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aeef88f40 msgr2=0x561aef113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aeef88f40 0x561aef113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/565936850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef107620 msgr2=0x561aef10c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef107620 0x561aef10c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 -- 172.21.15.19:0/565936850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aef103b10 con 0x561aef0064d0 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.258+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 0x561aef10be80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3bac003760 tx=0x7f3bac0a9ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cff0d84e5a556d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3ba6ffd700 1 -- 172.21.15.19:0/565936850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0ae520 con 0x561aef0064d0 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3ba6ffd700 1 -- 172.21.15.19:0/565936850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bac0b2f40 con 0x561aef0064d0 2024-05-17T09:43:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3ba6ffd700 1 -- 172.21.15.19:0/565936850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac057260 con 0x561aef0064d0 2024-05-17T09:43:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3bd244ab80 1 -- 172.21.15.19:0/565936850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 msgr2=0x561aef10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 0x561aef10be80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3bac003760 tx=0x7f3bac0a9ac0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3bd244ab80 1 -- 172.21.15.19:0/565936850 shutdown_connections 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aeef88f40 0x561aef113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.259+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef107620 0x561aef10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.260+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/565936850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aef0064d0 0x561aef10be80 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.260+0000 7f3bd244ab80 1 -- 172.21.15.19:0/565936850 >> 172.21.15.19:0/565936850 conn(0x561aeefc7d80 msgr2=0x561aef106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.260+0000 7f3bd244ab80 1 -- 172.21.15.19:0/565936850 shutdown_connections 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.260+0000 7f3bd244ab80 1 -- 172.21.15.19:0/565936850 wait complete. 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.260+0000 7f3bd244ab80 1 -- start start 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.260+0000 7f3bd244ab80 1 Processor -- start 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef116300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef116840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef116d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1ac610 con 0x561aeef88f40 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1ada80 con 0x561aef0064d0 2024-05-17T09:43:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1acce0 con 0x561aef107620 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 10 client.0 _ll_register_callbacks cb 0x561aeefc0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44172/0 (socket says 172.21.15.19:44172) 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 -- 172.21.15.19:0/709182786 learned_addr learned my addr 172.21.15.19:0/709182786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc1fd2700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef116d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 -- 172.21.15.19:0/709182786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 msgr2=0x561aef116d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef116d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 -- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 msgr2=0x561aef116840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef116840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 -- 172.21.15.19:0/709182786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aef18be40 con 0x561aeef88f40 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bc17d1700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef116300 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b5d0 tx=0x7f3bb80116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94fc64b438a6d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3ba4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/709182786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8024050 con 0x561aeef88f40 2024-05-17T09:43:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/709182786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3bb8003b60 con 0x561aeef88f40 2024-05-17T09:43:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.261+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bac0b2f40 con 0x561aeef88f40 2024-05-17T09:43:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/709182786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3bb8003d60 con 0x561aeef88f40 2024-05-17T09:43:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 10 client.19050 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561aef18cfe0 con 0x561aeef88f40 2024-05-17T09:43:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/709182786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bac0b2f40 con 0x561aeef88f40 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/709182786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3bb803b410 con 0x561aeef88f40 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 10 client.19050 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 10 client.19050 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 20 client.19050 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef18f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 4 client.19050 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bc19407a", "release": true, "await": true}] 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bc19407a", "release": true, "await": true}) -- 0x561aef0314c0 con 0x561aef121360 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.262+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef18f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.263+0000 7f3bc0fd0700 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef18f690 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a3db9a52be9271 server_cookie=56776785987e9c61 in_seq=0 out_seq=0 2024-05-17T09:43:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.263+0000 7f3ba4ff9700 10 client.19050 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.301+0000 7f3ba4ff9700 1 -- 172.21.15.19:0/709182786 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561aef0314c0 con 0x561aef121360 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.301+0000 7f3ba4ff9700 10 client.19050 handle_command_reply: tid=0 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 client.19050 shutdown 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 msgr2=0x561aef116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef116300 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b5d0 tx=0x7f3bb80116f0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 shutdown_connections 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561aef107620 0x561aef116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x561aef121360 0x561aef18f690 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3ba4ff9700 0 client.19050 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561aeef88f40 0x561aef116300 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 --2- 172.21.15.19:0/709182786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561aef0064d0 0x561aef116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 >> 172.21.15.19:0/709182786 conn(0x561aeefc7d80 msgr2=0x561aef11c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:39.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 shutdown_connections 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 1 -- 172.21.15.19:0/709182786 wait complete. 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.302+0000 7f3bd244ab80 20 client.19050 trim_cache size 0 max 16384 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 88, 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "bc19407a": { 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 88, 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:43:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.5, 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:39.305 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:43:39.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3ba80b2880 msgr2=0x7f3ba80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3ba80b2880 0x7f3ba80b4d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8005950 tx=0x7f3bb80058e0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 msgr2=0x7f3bbc19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:39.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc19b320 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0045e0 tx=0x7f3bac004610 comp rx=0 tx=0).stop 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 shutdown_connections 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3ba80b2880 0x7f3ba80b4d90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bbc1034c0 0x7f3bbc19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bbc108740 0x7f3bbc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 --2- 172.21.15.19:0/3834565159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc107df0 0x7f3bbc19b320 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 >> 172.21.15.19:0/3834565159 conn(0x7f3bbc0fef10 msgr2=0x7f3bbc06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.303+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 shutdown_connections 2024-05-17T09:43:39.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:39.304+0000 7f3bc3a35700 1 -- 172.21.15.19:0/3834565159 wait complete. 2024-05-17T09:43:39.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67514 6.24 MB/sec execute 456 sec latency 20889.636 ms 2024-05-17T09:43:40.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67667 6.25 MB/sec execute 457 sec latency 65.473 ms 2024-05-17T09:43:40.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:40 smithi079 ceph-mon[35502]: pgmap v1155: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-17T09:43:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:40 smithi069 ceph-mon[35081]: pgmap v1155: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-17T09:43:40.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:40 smithi019 ceph-mon[31620]: pgmap v1155: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-17T09:43:41.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67807 6.25 MB/sec execute 458 sec latency 142.344 ms 2024-05-17T09:43:42.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 67949 6.25 MB/sec execute 459 sec latency 133.679 ms 2024-05-17T09:43:42.776 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:42 smithi079 ceph-mon[35502]: pgmap v1156: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 180 KiB/s wr, 21 op/s 2024-05-17T09:43:42.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:42 smithi069 ceph-mon[35081]: pgmap v1156: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 180 KiB/s wr, 21 op/s 2024-05-17T09:43:42.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:42 smithi019 ceph-mon[31620]: pgmap v1156: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 180 KiB/s wr, 21 op/s 2024-05-17T09:43:43.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68080 6.25 MB/sec execute 460 sec latency 143.684 ms 2024-05-17T09:43:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 -- 172.21.15.19:0/3782164146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c58109a10 msgr2=0x7f5c5810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 --2- 172.21.15.19:0/3782164146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c58109a10 0x7f5c5810be30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009ca0 tx=0x7f5c5002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 -- 172.21.15.19:0/3782164146 shutdown_connections 2024-05-17T09:43:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 --2- 172.21.15.19:0/3782164146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c58109a10 0x7f5c5810be30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 --2- 172.21.15.19:0/3782164146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c581070b0 0x7f5c581094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 --2- 172.21.15.19:0/3782164146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58104750 0x7f5c58106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 -- 172.21.15.19:0/3782164146 >> 172.21.15.19:0/3782164146 conn(0x7f5c580fdd50 msgr2=0x7f5c58100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 -- 172.21.15.19:0/3782164146 shutdown_connections 2024-05-17T09:43:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.888+0000 7f5c5d284640 1 -- 172.21.15.19:0/3782164146 wait complete. 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.889+0000 7f5c5d284640 1 Processor -- start 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.889+0000 7f5c5d284640 1 -- start start 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.889+0000 7f5c5d284640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 0x7f5c5819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.889+0000 7f5c5d284640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c581070b0 0x7f5c5819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c5d284640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 0x7f5c581a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c5d284640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5810f670 con 0x7f5c58104750 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c5d284640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5810f4f0 con 0x7f5c58109a10 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c5d284640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5810f7f0 con 0x7f5c581070b0 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 0x7f5c5819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 0x7f5c581a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 0x7f5c581a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45036/0 (socket says 172.21.15.19:45036) 2024-05-17T09:43:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 0x7f5c5819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54294/0 (socket says 172.21.15.19:54294) 2024-05-17T09:43:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 -- 172.21.15.19:0/1763458062 learned_addr learned my addr 172.21.15.19:0/1763458062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c567fc640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c581070b0 0x7f5c5819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 -- 172.21.15.19:0/1763458062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c581070b0 msgr2=0x7f5c5819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c581070b0 0x7f5c5819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 -- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 msgr2=0x7f5c5819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 0x7f5c5819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c577fe640 1 -- 172.21.15.19:0/1763458062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581a1cd0 con 0x7f5c58109a10 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.890+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 0x7f5c5819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.891+0000 7f5c577fe640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 0x7f5c581a1660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c50031920 tx=0x7f5c50009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.891+0000 7f5c3ffff640 1 -- 172.21.15.19:0/1763458062 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50003cb0 con 0x7f5c58109a10 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.891+0000 7f5c3ffff640 1 -- 172.21.15.19:0/1763458062 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c50003e50 con 0x7f5c58109a10 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.891+0000 7f5c3ffff640 1 -- 172.21.15.19:0/1763458062 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50030b80 con 0x7f5c58109a10 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.891+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c58102120 con 0x7f5c58109a10 2024-05-17T09:43:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.891+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581025b0 con 0x7f5c58109a10 2024-05-17T09:43:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.895+0000 7f5c3ffff640 1 -- 172.21.15.19:0/1763458062 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5c5004e440 con 0x7f5c58109a10 2024-05-17T09:43:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.896+0000 7f5c64206740 1 Processor -- start 2024-05-17T09:43:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.896+0000 7f5c64206740 1 -- start start 2024-05-17T09:43:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c64206740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9b243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c64206740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b24920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c64206740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b2b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c64206740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560ee9a96c30 con 0x560ee97624d0 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c64206740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ee9a867d0 con 0x560ee97831b0 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c64206740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560ee9a8a810 con 0x560ee975a2f0 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b24920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9b243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9b243a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45040/0 (socket says 172.21.15.19:45040) 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c567fc640 1 -- 172.21.15.19:0/309737852 learned_addr learned my addr 172.21.15.19:0/309737852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c567fc640 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b2b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c56ffd640 1 -- 172.21.15.19:0/309737852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 msgr2=0x560ee9b243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9b243a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.897+0000 7f5c56ffd640 1 -- 172.21.15.19:0/309737852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 msgr2=0x560ee9b2b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c3ffff640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5c400b28a0 0x7f5c400b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b2b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c56ffd640 1 -- 172.21.15.19:0/309737852 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee979b210 con 0x560ee975a2f0 2024-05-17T09:43:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c567fc640 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b2b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c567fc640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5c400b28a0 0x7f5c400b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b24920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c48013770 tx=0x7f5c48013c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86187f454c372c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.898+0000 7f5c3ffff640 1 -- 172.21.15.19:0/1763458062 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5c501038f0 con 0x7f5c58109a10 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.899+0000 7f5c3d7fa640 1 -- 172.21.15.19:0/309737852 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004110 con 0x560ee975a2f0 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.899+0000 7f5c567fc640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5c400b28a0 0x7f5c400b4db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x560ee9b25570 tx=0x7f5c4c004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.899+0000 7f5c3d7fa640 1 -- 172.21.15.19:0/309737852 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c4800edc0 con 0x560ee975a2f0 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.899+0000 7f5c3d7fa640 1 -- 172.21.15.19:0/309737852 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48003b80 con 0x560ee975a2f0 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 -- 172.21.15.19:0/309737852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 msgr2=0x560ee9b24920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b24920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c48013770 tx=0x7f5c48013c30 comp rx=0 tx=0).stop 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 -- 172.21.15.19:0/309737852 shutdown_connections 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b2b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b24920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 --2- 172.21.15.19:0/309737852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9b243a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 -- 172.21.15.19:0/309737852 >> 172.21.15.19:0/309737852 conn(0x560ee9777300 msgr2=0x560ee9b1fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 -- 172.21.15.19:0/309737852 shutdown_connections 2024-05-17T09:43:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.900+0000 7f5c64206740 1 -- 172.21.15.19:0/309737852 wait complete. 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 -- start start 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 Processor -- start 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b2dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b2e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560ee976c790 con 0x560ee97831b0 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ee9a42890 con 0x560ee97624d0 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560ee9b2c270 con 0x560ee975a2f0 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.901+0000 7f5c64206740 10 client.0 _ll_register_callbacks cb 0x560ee9a45d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b2e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b2e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45042/0 (socket says 172.21.15.19:45042) 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 -- 172.21.15.19:0/4129236500 learned_addr learned my addr 172.21.15.19:0/4129236500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c577fe640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b2dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 -- 172.21.15.19:0/4129236500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 msgr2=0x560ee9b2dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b2dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 -- 172.21.15.19:0/4129236500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 msgr2=0x560ee9b35180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b35180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 -- 172.21.15.19:0/4129236500 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee9867620 con 0x560ee97624d0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b2dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c567fc640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b2e290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c004650 tx=0x7f5c4c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89cc9cb5895885db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.902+0000 7f5c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c3aff5640 1 -- 172.21.15.19:0/4129236500 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c011e10 con 0x560ee97624d0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c3aff5640 1 -- 172.21.15.19:0/4129236500 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5c4c020420 con 0x560ee97624d0 2024-05-17T09:43:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4800edc0 con 0x560ee97624d0 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c3aff5640 1 -- 172.21.15.19:0/4129236500 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5c4c020620 con 0x560ee97624d0 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c64206740 10 client.28553 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ee9b38150 con 0x560ee97624d0 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c3aff5640 1 -- 172.21.15.19:0/4129236500 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c4800edc0 con 0x560ee97624d0 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.903+0000 7f5c3aff5640 1 -- 172.21.15.19:0/4129236500 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5c4c02c700 con 0x560ee97624d0 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c64206740 10 client.28553 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c64206740 10 client.28553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c64206740 20 client.28553 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c64206740 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee9741d80 0x560ee9b3b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c64206740 4 client.28553 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee9a98b80 con 0x560ee9741d80 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee9741d80 0x560ee9b3b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c3aff5640 10 client.28553 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.904+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee9741d80 0x560ee9b3b9d0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82130f355a7dd03 server_cookie=17f7ecf9cbeec881 in_seq=0 out_seq=0 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.905+0000 7f5c3aff5640 1 -- 172.21.15.19:0/4129236500 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560ee9a98b80 con 0x560ee9741d80 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.905+0000 7f5c3aff5640 10 client.28553 handle_command_reply: tid=0 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.905+0000 7f5c64206740 1 client.28553 shutdown 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 msgr2=0x560ee9b2e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b2e290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c004650 tx=0x7f5c4c0046c0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 shutdown_connections 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee9741d80 0x560ee9b3b9d0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b35180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b2e290 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 --2- 172.21.15.19:0/4129236500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b2dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 >> 172.21.15.19:0/4129236500 conn(0x560ee9777300 msgr2=0x560ee9b22c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c3aff5640 0 client.28553 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 shutdown_connections 2024-05-17T09:43:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 1 -- 172.21.15.19:0/4129236500 wait complete. 2024-05-17T09:43:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.906+0000 7f5c64206740 20 client.28553 trim_cache size 0 max 16384 2024-05-17T09:43:43.928 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bc19407a', total seconds elapsed: 32.5 2024-05-17T09:43:43.928 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.5 2024-05-17T09:43:43.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 Processor -- start 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 -- start start 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b28810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581acff0 con 0x560ee97831b0 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581aa8c0 con 0x560ee97624d0 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c64206740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581a8710 con 0x560ee975a2f0 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54328/0 (socket says 172.21.15.19:54328) 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c577fe640 1 -- 172.21.15.19:0/3726260704 learned_addr learned my addr 172.21.15.19:0/3726260704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c567fc640 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b28810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.931+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c567fc640 1 -- 172.21.15.19:0/3726260704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 msgr2=0x560ee9b282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c567fc640 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b282d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c567fc640 1 -- 172.21.15.19:0/3726260704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 msgr2=0x560ee9b350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c567fc640 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c567fc640 1 -- 172.21.15.19:0/3726260704 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee9741fb0 con 0x560ee97624d0 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c577fe640 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c567fc640 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b28810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009a30 tx=0x7f5c4c00f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8f0ca652d6516f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.932+0000 7f5c3cff9640 1 -- 172.21.15.19:0/3726260704 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c022c90 con 0x560ee97624d0 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.933+0000 7f5c3cff9640 1 -- 172.21.15.19:0/3726260704 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c4c02c400 con 0x560ee97624d0 2024-05-17T09:43:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.933+0000 7f5c3cff9640 1 -- 172.21.15.19:0/3726260704 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c020b40 con 0x560ee97624d0 2024-05-17T09:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 -- 172.21.15.19:0/3726260704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 msgr2=0x560ee9b28810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b28810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009a30 tx=0x7f5c4c00f840 comp rx=0 tx=0).stop 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 -- 172.21.15.19:0/3726260704 shutdown_connections 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97831b0 0x560ee9b350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97624d0 0x560ee9b28810 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 --2- 172.21.15.19:0/3726260704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 -- 172.21.15.19:0/3726260704 >> 172.21.15.19:0/3726260704 conn(0x560ee9777300 msgr2=0x560ee9b231a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 -- 172.21.15.19:0/3726260704 shutdown_connections 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 -- 172.21.15.19:0/3726260704 wait complete. 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.934+0000 7f5c64206740 1 -- start start 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 Processor -- start 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b34a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b34f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9ba2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581a8d80 con 0x560ee97624d0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581aa1f0 con 0x560ee97831b0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581a9450 con 0x560ee975a2f0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c64206740 10 client.0 _ll_register_callbacks cb 0x7f5c4c00fc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b34a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9ba2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b34a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58518/0 (socket says 172.21.15.19:58518) 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9ba2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45054/0 (socket says 172.21.15.19:45054) 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b34f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.935+0000 7f5c56ffd640 1 -- 172.21.15.19:0/1158297030 learned_addr learned my addr 172.21.15.19:0/1158297030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c577fe640 1 -- 172.21.15.19:0/1158297030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 msgr2=0x560ee9b34a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c577fe640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b34a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c577fe640 1 -- 172.21.15.19:0/1158297030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 msgr2=0x560ee9b34f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c577fe640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b34f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c577fe640 1 -- 172.21.15.19:0/1158297030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee9ba2650 con 0x560ee97831b0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b34a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c567fc640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b34f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c577fe640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9ba2090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5c500c7ab0 tx=0x7f5c500cb540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d421eb08adc764 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c3aff5640 1 -- 172.21.15.19:0/1158297030 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500c7cc0 con 0x560ee97831b0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c3aff5640 1 -- 172.21.15.19:0/1158297030 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5c500cb990 con 0x560ee97831b0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4c02c400 con 0x560ee97831b0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c3aff5640 1 -- 172.21.15.19:0/1158297030 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5c500ea050 con 0x560ee97831b0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c64206740 10 client.28559 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.936+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ee9ba3930 con 0x560ee97831b0 2024-05-17T09:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c3aff5640 1 -- 172.21.15.19:0/1158297030 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c4c02c400 con 0x560ee97831b0 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c3aff5640 1 -- 172.21.15.19:0/1158297030 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5c50004160 con 0x560ee97831b0 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c64206740 10 client.28559 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c64206740 10 client.28559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c64206740 20 client.28559 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c64206740 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee97171c0 0x560ee9ba6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c64206740 4 client.28559 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ee9a92360 con 0x560ee97171c0 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.937+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee97171c0 0x560ee9ba6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.938+0000 7f5c3aff5640 10 client.28559 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:43.938+0000 7f5c56ffd640 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee97171c0 0x560ee9ba6320 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d85c4689c40b0b server_cookie=d8e04e4978ea71b in_seq=0 out_seq=0 2024-05-17T09:43:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.018+0000 7f5c3aff5640 1 -- 172.21.15.19:0/1158297030 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560ee9a92360 con 0x560ee97171c0 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c3aff5640 10 client.28559 handle_command_reply: tid=0 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 client.28559 shutdown 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 msgr2=0x560ee9ba2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9ba2090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5c500c7ab0 tx=0x7f5c500cb540 comp rx=0 tx=0).stop 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 shutdown_connections 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560ee97171c0 0x560ee9ba6320 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ee97831b0 0x560ee9ba2090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560ee97624d0 0x560ee9b34f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 --2- 172.21.15.19:0/1158297030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ee975a2f0 0x560ee9b34a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 >> 172.21.15.19:0/1158297030 conn(0x560ee9777300 msgr2=0x560ee9b281f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c3aff5640 0 client.28559 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 shutdown_connections 2024-05-17T09:43:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 1 -- 172.21.15.19:0/1158297030 wait complete. 2024-05-17T09:43:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.019+0000 7f5c64206740 20 client.28559 trim_cache size 0 max 16384 2024-05-17T09:43:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:43:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:43:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5c400b28a0 msgr2=0x7f5c400b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5c400b28a0 0x7f5c400b4db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x560ee9b25570 tx=0x7f5c4c004730 comp rx=0 tx=0).stop 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 msgr2=0x7f5c581a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 0x7f5c581a1660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c50031920 tx=0x7f5c50009880 comp rx=0 tx=0).stop 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 shutdown_connections 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c58109a10 0x7f5c581a1660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c581070b0 0x7f5c5819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5c400b28a0 0x7f5c400b4db0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 --2- 172.21.15.19:0/1763458062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c58104750 0x7f5c5819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 >> 172.21.15.19:0/1763458062 conn(0x7f5c580fdd50 msgr2=0x7f5c58105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.020+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 shutdown_connections 2024-05-17T09:43:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:44.021+0000 7f5c5d284640 1 -- 172.21.15.19:0/1763458062 wait complete. 2024-05-17T09:43:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:43:44.595 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68204 6.25 MB/sec execute 461 sec latency 118.906 ms 2024-05-17T09:43:44.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:44 smithi079 ceph-mon[35502]: pgmap v1157: 129 pgs: 129 active+clean; 501 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 442 KiB/s wr, 43 op/s 2024-05-17T09:43:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:44 smithi069 ceph-mon[35081]: pgmap v1157: 129 pgs: 129 active+clean; 501 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 442 KiB/s wr, 43 op/s 2024-05-17T09:43:44.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:44 smithi019 ceph-mon[31620]: pgmap v1157: 129 pgs: 129 active+clean; 501 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 442 KiB/s wr, 43 op/s 2024-05-17T09:43:45.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68348 6.25 MB/sec execute 462 sec latency 99.863 ms 2024-05-17T09:43:46.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68501 6.26 MB/sec execute 463 sec latency 130.217 ms 2024-05-17T09:43:46.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:46 smithi019 ceph-mon[31620]: pgmap v1158: 129 pgs: 129 active+clean; 502 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1005 KiB/s wr, 85 op/s 2024-05-17T09:43:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:46 smithi079 ceph-mon[35502]: pgmap v1158: 129 pgs: 129 active+clean; 502 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1005 KiB/s wr, 85 op/s 2024-05-17T09:43:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:46 smithi069 ceph-mon[35081]: pgmap v1158: 129 pgs: 129 active+clean; 502 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1005 KiB/s wr, 85 op/s 2024-05-17T09:43:47.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 -- 172.21.15.19:0/4234589825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90010c5b0 msgr2=0x7fc90010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 --2- 172.21.15.19:0/4234589825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90010c5b0 0x7fc90010c9c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009990 tx=0x7fc8f0009ca0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 -- 172.21.15.19:0/4234589825 shutdown_connections 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 --2- 172.21.15.19:0/4234589825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc900106ec0 0x7fc900113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 --2- 172.21.15.19:0/4234589825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90010c5b0 0x7fc90010c9c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 --2- 172.21.15.19:0/4234589825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc900106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 -- 172.21.15.19:0/4234589825 >> 172.21.15.19:0/4234589825 conn(0x7fc900102070 msgr2=0x7fc9001044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 -- 172.21.15.19:0/4234589825 shutdown_connections 2024-05-17T09:43:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.184+0000 7fc904b93700 1 -- 172.21.15.19:0/4234589825 wait complete. 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 Processor -- start 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 -- start start 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc90019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900106ec0 0x7fc90019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc90010c5b0 0x7fc9001a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001165c0 con 0x7fc900106ec0 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc900116440 con 0x7fc90010c5b0 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc904b93700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc900116740 con 0x7fc900106570 2024-05-17T09:43:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc8fe59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc90019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc8fe59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc90019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58530/0 (socket says 172.21.15.19:58530) 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc8fe59c700 1 -- 172.21.15.19:0/2181135810 learned_addr learned my addr 172.21.15.19:0/2181135810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc8fdd9b700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900106ec0 0x7fc90019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.185+0000 7fc8fed9d700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc90010c5b0 0x7fc9001a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fe59c700 1 -- 172.21.15.19:0/2181135810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc90010c5b0 msgr2=0x7fc9001a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fe59c700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc90010c5b0 0x7fc9001a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fe59c700 1 -- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900106ec0 msgr2=0x7fc90019d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fe59c700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900106ec0 0x7fc90019d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fe59c700 1 -- 172.21.15.19:0/2181135810 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9001a1e60 con 0x7fc900106570 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fdd9b700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900106ec0 0x7fc90019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8fe59c700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc90019cf90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00047b0 tx=0x7fc8f0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0026070 con 0x7fc900106570 2024-05-17T09:43:47.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001a20f0 con 0x7fc900106570 2024-05-17T09:43:47.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8f0021410 con 0x7fc900106570 2024-05-17T09:43:47.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0017650 con 0x7fc900106570 2024-05-17T09:43:47.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.186+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a9a70 con 0x7fc900106570 2024-05-17T09:43:47.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.187+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc900107c90 con 0x7fc900106570 2024-05-17T09:43:47.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.189+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc8f0029070 con 0x7fc900106570 2024-05-17T09:43:47.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.190+0000 7fc8ef7fe700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc8e80b2900 0x7fc8e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:47.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.191+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc8f00e3680 con 0x7fc900106570 2024-05-17T09:43:47.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.191+0000 7fc8fdd9b700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc8e80b2900 0x7fc8e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:47.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.192+0000 7fc8fdd9b700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc8e80b2900 0x7fc8e80b4e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc900103f90 tx=0x7fc8f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:47.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.194+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8f0013070 con 0x7fc900106570 2024-05-17T09:43:47.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68641 6.26 MB/sec execute 464 sec latency 100.168 ms 2024-05-17T09:43:47.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.658+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc90019e2f0 con 0x7fc900106570 2024-05-17T09:43:47.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.659+0000 7fc8ef7fe700 1 -- 172.21.15.19:0/2181135810 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fc8f00ab080 con 0x7fc900106570 2024-05-17T09:43:47.659 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:43:47.659 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:43:47.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc8e80b2900 msgr2=0x7fc8e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:47.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc8e80b2900 0x7fc8e80b4e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc900103f90 tx=0x7fc8f4009260 comp rx=0 tx=0).stop 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 msgr2=0x7fc90019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc90019cf90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00047b0 tx=0x7fc8f0005e80 comp rx=0 tx=0).stop 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 shutdown_connections 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc8e80b2900 0x7fc8e80b4e10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc900106570 0x7fc90019cf90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc90010c5b0 0x7fc9001a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 --2- 172.21.15.19:0/2181135810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc900106ec0 0x7fc90019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 >> 172.21.15.19:0/2181135810 conn(0x7fc900102070 msgr2=0x7fc9001048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 shutdown_connections 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:47.663+0000 7fc904b93700 1 -- 172.21.15.19:0/2181135810 wait complete. 2024-05-17T09:43:47.663 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:43:48.381 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.5 --await-for 45.0' 2024-05-17T09:43:48.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.5 --await-for 45.0 2024-05-17T09:43:48.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68775 6.25 MB/sec execute 465 sec latency 114.102 ms 2024-05-17T09:43:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:48 smithi019 ceph-mon[31620]: pgmap v1159: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.8 MiB/s wr, 131 op/s 2024-05-17T09:43:48.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:48 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2181135810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:48 smithi079 ceph-mon[35502]: pgmap v1159: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.8 MiB/s wr, 131 op/s 2024-05-17T09:43:48.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:48 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2181135810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:43:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:48 smithi069 ceph-mon[35081]: pgmap v1159: 129 pgs: 129 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.8 MiB/s wr, 131 op/s 2024-05-17T09:43:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:48 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2181135810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:43:49.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 68928 6.26 MB/sec execute 466 sec latency 119.223 ms 2024-05-17T09:43:50.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:50 smithi019 ceph-mon[31620]: pgmap v1160: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.4 MiB/s wr, 193 op/s 2024-05-17T09:43:50.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69070 6.26 MB/sec execute 467 sec latency 91.055 ms 2024-05-17T09:43:50.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:50 smithi079 ceph-mon[35502]: pgmap v1160: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.4 MiB/s wr, 193 op/s 2024-05-17T09:43:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:50 smithi069 ceph-mon[35081]: pgmap v1160: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.4 MiB/s wr, 193 op/s 2024-05-17T09:43:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 -- 172.21.15.19:0/3377394674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48107de0 msgr2=0x7f6a481081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3377394674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48107de0 0x7f6a481081f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6a380099c0 tx=0x7f6a38009cd0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 -- 172.21.15.19:0/3377394674 shutdown_connections 2024-05-17T09:43:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3377394674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a48108730 0x7f6a48110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3377394674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a481034b0 0x7f6a481038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3377394674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48107de0 0x7f6a481081f0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 -- 172.21.15.19:0/3377394674 >> 172.21.15.19:0/3377394674 conn(0x7f6a480fedb0 msgr2=0x7f6a481011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 -- 172.21.15.19:0/3377394674 shutdown_connections 2024-05-17T09:43:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.016+0000 7f6a4f410700 1 -- 172.21.15.19:0/3377394674 wait complete. 2024-05-17T09:43:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.017+0000 7f6a4f410700 1 Processor -- start 2024-05-17T09:43:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.017+0000 7f6a4f410700 1 -- start start 2024-05-17T09:43:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4f410700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 0x7f6a48075470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4f410700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48107de0 0x7f6a4806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4f410700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108730 0x7f6a4806ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4f410700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48114380 con 0x7f6a48108730 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4c9ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48107de0 0x7f6a4806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 0x7f6a48075470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 0x7f6a48075470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58548/0 (socket says 172.21.15.19:58548) 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4c9ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48107de0 0x7f6a4806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:45096/0 (socket says 172.21.15.19:45096) 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/3207222556 learned_addr learned my addr 172.21.15.19:0/3207222556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48114200 con 0x7f6a48107de0 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48114500 con 0x7f6a481034b0 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.018+0000 7f6a4d9ad700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108730 0x7f6a4806ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.019+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/3207222556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48107de0 msgr2=0x7f6a4806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.019+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48107de0 0x7f6a4806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.019+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108730 msgr2=0x7f6a4806ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.019+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108730 0x7f6a4806ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.019+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/3207222556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a48072fa0 con 0x7f6a481034b0 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.019+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 0x7f6a48075470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400e9a0 tx=0x7f6a4400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.020+0000 7f6a3e7fc700 1 -- 172.21.15.19:0/3207222556 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4400ca10 con 0x7f6a481034b0 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.020+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a48073290 con 0x7f6a481034b0 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.020+0000 7f6a3e7fc700 1 -- 172.21.15.19:0/3207222556 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a4400cbb0 con 0x7f6a481034b0 2024-05-17T09:43:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.020+0000 7f6a3e7fc700 1 -- 172.21.15.19:0/3207222556 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44018680 con 0x7f6a481034b0 2024-05-17T09:43:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.020+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a48073690 con 0x7f6a481034b0 2024-05-17T09:43:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.024+0000 7f6a3e7fc700 1 -- 172.21.15.19:0/3207222556 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a44010430 con 0x7f6a481034b0 2024-05-17T09:43:51.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.025+0000 7f6a3e7fc700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a340b2900 0x7f6a340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.025+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a340b2900 0x7f6a340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.025+0000 7f6a3e7fc700 1 -- 172.21.15.19:0/3207222556 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a440e2710 con 0x7f6a481034b0 2024-05-17T09:43:51.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.025+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a340b2900 0x7f6a340b4e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000c00 tx=0x7f6a3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:51.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.025+0000 7f6a5de25b80 1 Processor -- start 2024-05-17T09:43:51.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 -- start start 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 0x55f85ba79b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85ba736b0 0x55f85ba73ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f85b9a1e20 con 0x55f85ba736b0 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f85b9919c0 con 0x55f85b972560 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a4c9ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 0x55f85ba79b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a4c9ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 0x55f85ba79b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58560/0 (socket says 172.21.15.19:58560) 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/1426099546 learned_addr learned my addr 172.21.15.19:0/1426099546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a4d9ad700 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85ba736b0 0x55f85ba73ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a5de25b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f85b995a00 con 0x55f85b8f4fd0 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.026+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1426099546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 msgr2=0x55f85ba79b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 0x55f85ba79b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1426099546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85ba736b0 msgr2=0x55f85ba73ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85ba736b0 0x55f85ba73ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1426099546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85b8e1a60 con 0x55f85b972560 2024-05-17T09:43:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 0x55f85ba79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba795f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a440a6b90 tx=0x7f6a440b2140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da034383b0818046 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a337fe700 1 -- 172.21.15.19:0/1426099546 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440aa020 con 0x55f85b972560 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a337fe700 1 -- 172.21.15.19:0/1426099546 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a440a9920 con 0x55f85b972560 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.027+0000 7f6a337fe700 1 -- 172.21.15.19:0/1426099546 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440cc0d0 con 0x55f85b972560 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.028+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1426099546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 msgr2=0x55f85ba795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.028+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba795f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a440a6b90 tx=0x7f6a440b2140 comp rx=0 tx=0).stop 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1426099546 shutdown_connections 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85ba736b0 0x55f85ba73ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba795f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1426099546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b8f4fd0 0x55f85ba79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1426099546 >> 172.21.15.19:0/1426099546 conn(0x55f85b933e10 msgr2=0x55f85ba6e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1426099546 shutdown_connections 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1426099546 wait complete. 2024-05-17T09:43:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- start start 2024-05-17T09:43:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 Processor -- start 2024-05-17T09:43:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85ba736b0 0x55f85ba7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f85b898130 con 0x55f85b8f4fd0 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f85b933950 con 0x55f85b972560 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.029+0000 7f6a5de25b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f85ba744d0 con 0x55f85ba736b0 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a5de25b80 10 client.0 _ll_register_callbacks cb 0x55f85b92cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54376/0 (socket says 172.21.15.19:54376) 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1152617033 learned_addr learned my addr 172.21.15.19:0/1152617033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1152617033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85ba736b0 msgr2=0x55f85ba7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85ba736b0 0x55f85ba7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 msgr2=0x55f85ba7d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba7d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1152617033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85b930820 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7d300 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6a440dfe00 tx=0x7f6a440b28a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b01d094bc43ee05 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1152617033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440cabd0 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1152617033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a440a5050 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.030+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a440a9920 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1152617033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6a440cad70 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 10 client.19086 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f85ba93420 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1152617033 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a440a9920 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1152617033 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6a440a8c10 con 0x55f85b8f4fd0 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 10 client.19086 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 10 client.19086 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 20 client.19086 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85ba958f0 0x55f85ba97d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 4 client.19086 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.031+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85b9a3d70 con 0x55f85ba958f0 2024-05-17T09:43:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.032+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85ba958f0 0x55f85ba97d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.032+0000 7f6a30ff9700 10 client.19086 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.032+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85ba958f0 0x55f85ba97d10 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9f5afe71a34753 server_cookie=529cf91f34dc3b4c in_seq=0 out_seq=0 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1152617033 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f85b9a3d70 con 0x55f85ba958f0 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a30ff9700 10 client.19086 handle_command_reply: tid=0 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a5de25b80 1 client.19086 shutdown 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 msgr2=0x55f85ba7d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7d300 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6a440dfe00 tx=0x7f6a440b28a0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 shutdown_connections 2024-05-17T09:43:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85ba736b0 0x55f85ba7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.033+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85ba958f0 0x55f85ba97d10 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7d300 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1152617033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85b972560 0x55f85ba7d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 >> 172.21.15.19:0/1152617033 conn(0x55f85b933e10 msgr2=0x55f85ba68310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a30ff9700 0 client.19086 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 shutdown_connections 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1152617033 wait complete. 2024-05-17T09:43:51.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.034+0000 7f6a5de25b80 20 client.19086 trim_cache size 0 max 16384 2024-05-17T09:43:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.063+0000 7f6a5de25b80 1 Processor -- start 2024-05-17T09:43:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.063+0000 7f6a5de25b80 1 -- start start 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba7cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54384/0 (socket says 172.21.15.19:54384) 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/2263053301 learned_addr learned my addr 172.21.15.19:0/2263053301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85ba72db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba7cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b9280 con 0x55f85b8f4fd0 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a4d9ad700 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85ba72db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b6b10 con 0x55f85ba736b0 2024-05-17T09:43:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.064+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b4980 con 0x55f85b972560 2024-05-17T09:43:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/2263053301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 msgr2=0x55f85ba72db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85ba72db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/2263053301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 msgr2=0x55f85ba7dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/2263053301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85ba6eaf0 con 0x55f85b972560 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba7cc20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55f85ba790b0 tx=0x7f6a38009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dda55615acdf2f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.065+0000 7f6a32ffd700 1 -- 172.21.15.19:0/2263053301 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800b790 con 0x55f85b972560 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.066+0000 7f6a32ffd700 1 -- 172.21.15.19:0/2263053301 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a3800bdc0 con 0x55f85b972560 2024-05-17T09:43:51.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.066+0000 7f6a32ffd700 1 -- 172.21.15.19:0/2263053301 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800fbf0 con 0x55f85b972560 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 msgr2=0x55f85ba7cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba7cc20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55f85ba790b0 tx=0x7f6a38009cd0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 shutdown_connections 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85ba72db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba7dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/2263053301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba7cc20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 >> 172.21.15.19:0/2263053301 conn(0x55f85b933e10 msgr2=0x55f85ba726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 shutdown_connections 2024-05-17T09:43:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.067+0000 7f6a5de25b80 1 -- 172.21.15.19:0/2263053301 wait complete. 2024-05-17T09:43:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.068+0000 7f6a5de25b80 1 -- start start 2024-05-17T09:43:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.068+0000 7f6a5de25b80 1 Processor -- start 2024-05-17T09:43:51.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.068+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba98a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.068+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba98f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a5de25b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85baff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba98a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a4d1ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba98a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54390/0 (socket says 172.21.15.19:54390) 2024-05-17T09:43:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a4c9ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba98f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a4c9ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba98f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58578/0 (socket says 172.21.15.19:58578) 2024-05-17T09:43:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a4d1ac700 1 -- 172.21.15.19:0/1485969655 learned_addr learned my addr 172.21.15.19:0/1485969655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b4fd0 con 0x55f85b8f4fd0 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a4d9ad700 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85baff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b6440 con 0x55f85ba736b0 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b56a0 con 0x55f85b972560 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.069+0000 7f6a5de25b80 10 client.0 _ll_register_callbacks cb 0x55f85b92cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 msgr2=0x55f85baff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85baff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/1485969655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 msgr2=0x55f85ba98a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba98a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a4c9ab700 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85ba99830 con 0x55f85b972560 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a4c9ab700 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba98f80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55f85b99e110 tx=0x7f6a3800b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1d84780f55cb21 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1485969655 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800bba0 con 0x55f85b972560 2024-05-17T09:43:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a3800bdc0 con 0x55f85b972560 2024-05-17T09:43:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.070+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1485969655 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6a3800f460 con 0x55f85b972560 2024-05-17T09:43:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1485969655 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6a3804e050 con 0x55f85b972560 2024-05-17T09:43:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 10 client.28567 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f85bb00680 con 0x55f85b972560 2024-05-17T09:43:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1485969655 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a3800faf0 con 0x55f85b972560 2024-05-17T09:43:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1485969655 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6a38045780 con 0x55f85b972560 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 10 client.28567 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 10 client.28567 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 20 client.28567 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85bb00c40 0x55f85bb03060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 4 client.28567 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.5, "await_for": 45.0}] 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.071+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.5, "await_for": 45.0}) -- 0x7f6a3800bdc0 con 0x55f85bb00c40 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.072+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85bb00c40 0x55f85bb03060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.072+0000 7f6a30ff9700 10 client.28567 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.072+0000 7f6a4d1ac700 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85bb00c40 0x55f85bb03060 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ce444d4694c377 server_cookie=f3749fc60d7c4d18 in_seq=0 out_seq=0 2024-05-17T09:43:51.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.462+0000 7f6a30ff9700 1 -- 172.21.15.19:0/1485969655 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6a3800bdc0 con 0x55f85bb00c40 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.462+0000 7f6a30ff9700 10 client.28567 handle_command_reply: tid=0 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.462+0000 7f6a5de25b80 1 client.28567 shutdown 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 msgr2=0x55f85ba98f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba98f80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55f85b99e110 tx=0x7f6a3800b6a0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 shutdown_connections 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55f85bb00c40 0x55f85bb03060 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a30ff9700 0 client.28567 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f85ba736b0 0x55f85baff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f85b8f4fd0 0x55f85ba98a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 --2- 172.21.15.19:0/1485969655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f85b972560 0x55f85ba98f80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 >> 172.21.15.19:0/1485969655 conn(0x55f85b933e10 msgr2=0x55f85ba72e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 shutdown_connections 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 1 -- 172.21.15.19:0/1485969655 wait complete. 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.463+0000 7f6a5de25b80 20 client.28567 trim_cache size 0 max 16384 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 90, 2024-05-17T09:43:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "c21059e4": { 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 90, 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.5, 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:43:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:43:51.466 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a340b2900 msgr2=0x7f6a340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a340b2900 0x7f6a340b4e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000c00 tx=0x7f6a3801a040 comp rx=0 tx=0).stop 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 msgr2=0x7f6a48075470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 0x7f6a48075470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400e9a0 tx=0x7f6a4400ecb0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 shutdown_connections 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6a340b2900 0x7f6a340b4e10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a481034b0 0x7f6a48075470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108730 0x7f6a4806ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 --2- 172.21.15.19:0/3207222556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48107de0 0x7f6a4806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 >> 172.21.15.19:0/3207222556 conn(0x7f6a480fedb0 msgr2=0x7f6a481011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 shutdown_connections 2024-05-17T09:43:51.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:43:51.464+0000 7f6a4f410700 1 -- 172.21.15.19:0/3207222556 wait complete. 2024-05-17T09:43:51.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.25 MB/sec execute 468 sec latency 398.393 ms 2024-05-17T09:43:52.180 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c21059e4', quiesce took 8.3 seconds. Will release after: 14.2 2024-05-17T09:43:52.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.24 MB/sec execute 469 sec latency 1169.725 ms 2024-05-17T09:43:52.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:52 smithi079 ceph-mon[35502]: pgmap v1161: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-05-17T09:43:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:52 smithi069 ceph-mon[35081]: pgmap v1161: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-05-17T09:43:52.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:52 smithi019 ceph-mon[31620]: pgmap v1161: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-05-17T09:43:53.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.22 MB/sec execute 470 sec latency 2169.838 ms 2024-05-17T09:43:54.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.21 MB/sec execute 471 sec latency 3169.915 ms 2024-05-17T09:43:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:54 smithi079 ceph-mon[35502]: pgmap v1162: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.1 MiB/s wr, 256 op/s 2024-05-17T09:43:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:54 smithi069 ceph-mon[35081]: pgmap v1162: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.1 MiB/s wr, 256 op/s 2024-05-17T09:43:54.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:54 smithi019 ceph-mon[31620]: pgmap v1162: 129 pgs: 129 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.1 MiB/s wr, 256 op/s 2024-05-17T09:43:55.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.20 MB/sec execute 472 sec latency 4170.006 ms 2024-05-17T09:43:56.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.18 MB/sec execute 473 sec latency 5170.133 ms 2024-05-17T09:43:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:56 smithi079 ceph-mon[35502]: pgmap v1163: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-05-17T09:43:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:56 smithi069 ceph-mon[35081]: pgmap v1163: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-05-17T09:43:56.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:56 smithi019 ceph-mon[31620]: pgmap v1163: 129 pgs: 129 active+clean; 498 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-05-17T09:43:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 -- 172.21.15.19:0/2068018940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 msgr2=0x7f2b7810be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 --2- 172.21.15.19:0/2068018940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7810be20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ca0 tx=0x7f2b7002a8c0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 -- 172.21.15.19:0/2068018940 shutdown_connections 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 --2- 172.21.15.19:0/2068018940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7810be20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 --2- 172.21.15.19:0/2068018940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b781018c0 0x7f2b78101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 --2- 172.21.15.19:0/2068018940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b78105210 0x7f2b78105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 -- 172.21.15.19:0/2068018940 >> 172.21.15.19:0/2068018940 conn(0x7f2b780fd500 msgr2=0x7f2b780ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 -- 172.21.15.19:0/2068018940 shutdown_connections 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.150+0000 7f2b7f740640 1 -- 172.21.15.19:0/2068018940 wait complete. 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 Processor -- start 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 -- start start 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b781018c0 0x7f2b7819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78105210 0x7f2b781a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810f6a0 con 0x7f2b781023c0 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810f520 con 0x7f2b781018c0 2024-05-17T09:43:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7f740640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810f820 con 0x7f2b78105210 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59486/0 (socket says 172.21.15.19:59486) 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 -- 172.21.15.19:0/279282945 learned_addr learned my addr 172.21.15.19:0/279282945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b781018c0 0x7f2b7819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78105210 0x7f2b781a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 -- 172.21.15.19:0/279282945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78105210 msgr2=0x7f2b781a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78105210 0x7f2b781a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 -- 172.21.15.19:0/279282945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b781018c0 msgr2=0x7f2b7819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b781018c0 0x7f2b7819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7ccdb640 1 -- 172.21.15.19:0/279282945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b781a1cc0 con 0x7f2b781023c0 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b781018c0 0x7f2b7819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.151+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78105210 0x7f2b781a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.152+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7819a6c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800d770 tx=0x7f2b6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.152+0000 7f2b6e7fc640 1 -- 172.21.15.19:0/279282945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68017070 con 0x7f2b781023c0 2024-05-17T09:43:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.152+0000 7f2b6e7fc640 1 -- 172.21.15.19:0/279282945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b680054b0 con 0x7f2b781023c0 2024-05-17T09:43:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.152+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b781a1fb0 con 0x7f2b781023c0 2024-05-17T09:43:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.152+0000 7f2b6e7fc640 1 -- 172.21.15.19:0/279282945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004c60 con 0x7f2b781023c0 2024-05-17T09:43:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.152+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b781a2410 con 0x7f2b781023c0 2024-05-17T09:43:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.154+0000 7f2b865f0740 1 Processor -- start 2024-05-17T09:43:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 -- start start 2024-05-17T09:43:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b6e7fc640 1 -- 172.21.15.19:0/279282945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2b68004e00 con 0x7f2b781023c0 2024-05-17T09:43:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd427b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd427bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd4282c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abd41e9c30 con 0x55abd3ed61b0 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55abd41d97d0 con 0x55abd3eb54d0 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b865f0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55abd41dd810 con 0x55abd3ead2f0 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd427b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd427b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59498/0 (socket says 172.21.15.19:59498) 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/4256501424 learned_addr learned my addr 172.21.15.19:0/4256501424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd427bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd4282c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/4256501424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 msgr2=0x55abd427bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd427bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/4256501424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 msgr2=0x55abd4282c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd4282c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/4256501424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd3eee210 con 0x55abd3ed61b0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.155+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd4282c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd427b7b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009910 tx=0x7f2b70009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86a6d4924b3b2a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b6e7fc640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2b640b28a0 0x7f2b640b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b637fe640 1 -- 172.21.15.19:0/4256501424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70031a80 con 0x55abd3ed61b0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2b640b28a0 0x7f2b640b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b637fe640 1 -- 172.21.15.19:0/4256501424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70036040 con 0x55abd3ed61b0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b637fe640 1 -- 172.21.15.19:0/4256501424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70004390 con 0x55abd3ed61b0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.156+0000 7f2b6e7fc640 1 -- 172.21.15.19:0/279282945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2b680e20d0 con 0x7f2b781023c0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 -- 172.21.15.19:0/4256501424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 msgr2=0x55abd427b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2b640b28a0 0x7f2b640b4db0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55abd427cb90 tx=0x7f2b74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd427b7b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009910 tx=0x7f2b70009940 comp rx=0 tx=0).stop 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 -- 172.21.15.19:0/4256501424 shutdown_connections 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd4282c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd427bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 --2- 172.21.15.19:0/4256501424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd427b7b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 -- 172.21.15.19:0/4256501424 >> 172.21.15.19:0/4256501424 conn(0x55abd3eca300 msgr2=0x55abd4272d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 -- 172.21.15.19:0/4256501424 shutdown_connections 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 -- 172.21.15.19:0/4256501424 wait complete. 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 -- start start 2024-05-17T09:43:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.157+0000 7f2b865f0740 1 Processor -- start 2024-05-17T09:43:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ead2f0 0x55abd42850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd42855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abd3ebf790 con 0x55abd3ed61b0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55abd4195890 con 0x55abd3ead2f0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55abd4283620 con 0x55abd3eb54d0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 10 client.0 _ll_register_callbacks cb 0x55abd4198d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59504/0 (socket says 172.21.15.19:59504) 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1751171087 learned_addr learned my addr 172.21.15.19:0/1751171087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ead2f0 0x55abd42850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd42855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1751171087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 msgr2=0x55abd42855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd42855e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ead2f0 msgr2=0x55abd42850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ead2f0 0x55abd42850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1751171087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd3fba620 con 0x55abd3ed61b0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c540 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2b70006010 tx=0x7f2b70031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf4f87699f6d1e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1751171087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7004a040 con 0x55abd3ed61b0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1751171087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2b70003970 con 0x55abd3ed61b0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.158+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b70036040 con 0x55abd3ed61b0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1751171087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2b7005c050 con 0x55abd3ed61b0 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 10 client.19104 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abd428f470 con 0x55abd3ed61b0 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1751171087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70036040 con 0x55abd3ed61b0 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1751171087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f2b7004d030 con 0x55abd3ed61b0 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 10 client.19104 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 10 client.19104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 20 client.19104 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e94d80 0x55abd42927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 4 client.19104 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd41ebb80 con 0x55abd3e94d80 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.159+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e94d80 0x55abd42927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.160+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e94d80 0x55abd42927d0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88af736fcacbc47a server_cookie=3ba57ed5edbeec57 in_seq=0 out_seq=0 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.160+0000 7f2b60ff9640 10 client.19104 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1751171087 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55abd41ebb80 con 0x55abd3e94d80 2024-05-17T09:43:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b60ff9640 10 client.19104 handle_command_reply: tid=0 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 client.19104 shutdown 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 msgr2=0x55abd428c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c540 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2b70006010 tx=0x7f2b70031740 comp rx=0 tx=0).stop 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 shutdown_connections 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e94d80 0x55abd42927d0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c540 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd42855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b60ff9640 0 client.19104 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1751171087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ead2f0 0x55abd42850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 >> 172.21.15.19:0/1751171087 conn(0x55abd3eca300 msgr2=0x55abd4275c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 shutdown_connections 2024-05-17T09:43:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 1 -- 172.21.15.19:0/1751171087 wait complete. 2024-05-17T09:43:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.161+0000 7f2b865f0740 20 client.19104 trim_cache size 0 max 16384 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.176+0000 7f2b865f0740 1 Processor -- start 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.176+0000 7f2b865f0740 1 -- start start 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd42762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd428c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b865f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781ad180 con 0x55abd3ed61b0 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b865f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781aaa50 con 0x55abd3eb54d0 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b865f0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a88a0 con 0x55abd3ead2f0 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59510/0 (socket says 172.21.15.19:59510) 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1932844643 learned_addr learned my addr 172.21.15.19:0/1932844643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd428c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd42762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1932844643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 msgr2=0x55abd42762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd42762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1932844643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 msgr2=0x55abd428c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd428c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 -- 172.21.15.19:0/1932844643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd4282fe0 con 0x55abd3ed61b0 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c9a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002ade0 tx=0x7f2b700317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589da341481d16ee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b62ffd640 1 -- 172.21.15.19:0/1932844643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70031a80 con 0x55abd3ed61b0 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b62ffd640 1 -- 172.21.15.19:0/1932844643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70036040 con 0x55abd3ed61b0 2024-05-17T09:43:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.177+0000 7f2b62ffd640 1 -- 172.21.15.19:0/1932844643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70030cb0 con 0x55abd3ed61b0 2024-05-17T09:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 -- 172.21.15.19:0/1932844643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 msgr2=0x55abd428c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c9a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002ade0 tx=0x7f2b700317f0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 -- 172.21.15.19:0/1932844643 shutdown_connections 2024-05-17T09:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ed61b0 0x55abd428c9a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3eb54d0 0x55abd428c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1932844643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3ead2f0 0x55abd42762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 -- 172.21.15.19:0/1932844643 >> 172.21.15.19:0/1932844643 conn(0x55abd3eca300 msgr2=0x55abd4275bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 -- 172.21.15.19:0/1932844643 shutdown_connections 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 -- 172.21.15.19:0/1932844643 wait complete. 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.178+0000 7f2b865f0740 1 -- start start 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 Processor -- start 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 0x55abd428a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd428a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ed61b0 0x55abd428b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a8f10 con 0x55abd3ead2f0 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781aa380 con 0x55abd3ed61b0 2024-05-17T09:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a95e0 con 0x55abd3eb54d0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b865f0740 10 client.0 _ll_register_callbacks cb 0x7f2b70031550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 0x55abd428a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 0x55abd428a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59518/0 (socket says 172.21.15.19:59518) 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 -- 172.21.15.19:0/1917443362 learned_addr learned my addr 172.21.15.19:0/1917443362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd428a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ed61b0 0x55abd428b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 -- 172.21.15.19:0/1917443362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 msgr2=0x55abd428a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd428a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 -- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ed61b0 msgr2=0x55abd428b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ed61b0 0x55abd428b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7d4dc640 1 -- 172.21.15.19:0/1917443362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd428bc20 con 0x55abd3ead2f0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.179+0000 7f2b7dcdd640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ed61b0 0x55abd428b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b7d4dc640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 0x55abd428a1f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x55abd4273ac0 tx=0x7f2b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1801d2f25f796be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:43:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1917443362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740035f0 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b70036040 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1917443362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2b74003790 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1917443362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2b74037050 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 10 client.19110 fetch_fsmap learned FSMap version 39 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abd428cf80 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1917443362 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b70036040 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1917443362 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f2b74029600 con 0x55abd3ead2f0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 10 client.19110 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 10 client.19110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 20 client.19110 populate_metadata read hostname 'smithi019' 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e6a1c0 0x55abd42fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 4 client.19110 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.180+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abd41e5360 con 0x55abd3e6a1c0 2024-05-17T09:43:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.181+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e6a1c0 0x55abd42fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:43:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.181+0000 7f2b60ff9640 10 client.19110 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.181+0000 7f2b7ccdb640 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e6a1c0 0x55abd42fd5f0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200ce7fbde71655f server_cookie=b943ae81afdfc2ce in_seq=0 out_seq=0 2024-05-17T09:43:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.236+0000 7f2b60ff9640 1 -- 172.21.15.19:0/1917443362 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55abd41e5360 con 0x55abd3e6a1c0 2024-05-17T09:43:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.236+0000 7f2b60ff9640 10 client.19110 handle_command_reply: tid=0 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.236+0000 7f2b865f0740 1 client.19110 shutdown 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 msgr2=0x55abd428a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 0x55abd428a1f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x55abd4273ac0 tx=0x7f2b740046c0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 shutdown_connections 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55abd3e6a1c0 0x55abd42fd5f0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55abd3ed61b0 0x55abd428b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55abd3eb54d0 0x55abd428a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 --2- 172.21.15.19:0/1917443362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abd3ead2f0 0x55abd428a1f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b60ff9640 0 client.19110 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 >> 172.21.15.19:0/1917443362 conn(0x55abd3eca300 msgr2=0x55abd4271330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 shutdown_connections 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 1 -- 172.21.15.19:0/1917443362 wait complete. 2024-05-17T09:43:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b865f0740 20 client.19110 trim_cache size 0 max 16384 2024-05-17T09:43:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:43:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:43:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2b640b28a0 msgr2=0x7f2b640b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2b640b28a0 0x7f2b640b4db0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55abd427cb90 tx=0x7f2b74005f90 comp rx=0 tx=0).stop 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 msgr2=0x7f2b7819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7819a6c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800d770 tx=0x7f2b6800dc30 comp rx=0 tx=0).stop 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 shutdown_connections 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78105210 0x7f2b781a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f2b640b28a0 0x7f2b640b4db0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b781023c0 0x7f2b7819a6c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 --2- 172.21.15.19:0/279282945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b781018c0 0x7f2b7819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.237+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 >> 172.21.15.19:0/279282945 conn(0x7f2b780fd500 msgr2=0x7f2b78107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:43:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.238+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 shutdown_connections 2024-05-17T09:43:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:43:57.238+0000 7f2b7f740640 1 -- 172.21.15.19:0/279282945 wait complete. 2024-05-17T09:43:57.596 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.17 MB/sec execute 474 sec latency 6170.259 ms 2024-05-17T09:43:58.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.16 MB/sec execute 475 sec latency 7170.366 ms 2024-05-17T09:43:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:43:58 smithi079 ceph-mon[35502]: pgmap v1164: 129 pgs: 129 active+clean; 497 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-05-17T09:43:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:43:58 smithi069 ceph-mon[35081]: pgmap v1164: 129 pgs: 129 active+clean; 497 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-05-17T09:43:58.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:43:58 smithi019 ceph-mon[31620]: pgmap v1164: 129 pgs: 129 active+clean; 497 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-05-17T09:43:59.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.14 MB/sec execute 476 sec latency 8170.476 ms 2024-05-17T09:44:00.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.13 MB/sec execute 477 sec latency 9170.603 ms 2024-05-17T09:44:00.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:00 smithi079 ceph-mon[35502]: pgmap v1165: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-17T09:44:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:00 smithi069 ceph-mon[35081]: pgmap v1165: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-17T09:44:00.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:00 smithi019 ceph-mon[31620]: pgmap v1165: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-17T09:44:01.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.12 MB/sec execute 478 sec latency 10170.699 ms 2024-05-17T09:44:02.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.11 MB/sec execute 479 sec latency 11170.814 ms 2024-05-17T09:44:02.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:02 smithi079 ceph-mon[35502]: pgmap v1166: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 977 KiB/s wr, 114 op/s 2024-05-17T09:44:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:02 smithi069 ceph-mon[35081]: pgmap v1166: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 977 KiB/s wr, 114 op/s 2024-05-17T09:44:02.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:02 smithi019 ceph-mon[31620]: pgmap v1166: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 977 KiB/s wr, 114 op/s 2024-05-17T09:44:03.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.09 MB/sec execute 480 sec latency 12170.921 ms 2024-05-17T09:44:04.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.08 MB/sec execute 481 sec latency 13171.011 ms 2024-05-17T09:44:04.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:04 smithi079 ceph-mon[35502]: pgmap v1167: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 587 KiB/s wr, 100 op/s 2024-05-17T09:44:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:04 smithi069 ceph-mon[35081]: pgmap v1167: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 587 KiB/s wr, 100 op/s 2024-05-17T09:44:04.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:04 smithi019 ceph-mon[31620]: pgmap v1167: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 587 KiB/s wr, 100 op/s 2024-05-17T09:44:05.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.07 MB/sec execute 482 sec latency 14171.136 ms 2024-05-17T09:44:06.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:44:06.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.06 MB/sec execute 483 sec latency 15171.207 ms 2024-05-17T09:44:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:44:06.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:06 smithi019 ceph-mon[31620]: pgmap v1168: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 148 KiB/s wr, 48 op/s 2024-05-17T09:44:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:06 smithi069 ceph-mon[35081]: pgmap v1168: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 148 KiB/s wr, 48 op/s 2024-05-17T09:44:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:06 smithi079 ceph-mon[35502]: pgmap v1168: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 148 KiB/s wr, 48 op/s 2024-05-17T09:44:07.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.04 MB/sec execute 484 sec latency 16171.239 ms 2024-05-17T09:44:07.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:07 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:44:07.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:07 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:44:08.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:07 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:44:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 -- 172.21.15.19:0/2655311394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 msgr2=0x7fc4281038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 --2- 172.21.15.19:0/2655311394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc4281038c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc4240099c0 tx=0x7fc424009cd0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 -- 172.21.15.19:0/2655311394 shutdown_connections 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 --2- 172.21.15.19:0/2655311394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc428108730 0x7fc428110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 --2- 172.21.15.19:0/2655311394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc4281038c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 --2- 172.21.15.19:0/2655311394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc428107de0 0x7fc4281081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 -- 172.21.15.19:0/2655311394 >> 172.21.15.19:0/2655311394 conn(0x7fc4280fede0 msgr2=0x7fc428101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 -- 172.21.15.19:0/2655311394 shutdown_connections 2024-05-17T09:44:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.583+0000 7fc430447700 1 -- 172.21.15.19:0/2655311394 wait complete. 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 Processor -- start 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 -- start start 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc42819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc428107de0 0x7fc42819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc428108730 0x7fc42819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc428113280 con 0x7fc4281034b0 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc428113100 con 0x7fc428107de0 2024-05-17T09:44:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc430447700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc428113400 con 0x7fc428108730 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc42e1e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc42819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc42e1e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc42819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42668/0 (socket says 172.21.15.19:42668) 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.584+0000 7fc42e1e3700 1 -- 172.21.15.19:0/696319335 learned_addr learned my addr 172.21.15.19:0/696319335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42d9e2700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc428107de0 0x7fc42819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e9e4700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc428108730 0x7fc42819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e1e3700 1 -- 172.21.15.19:0/696319335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc428108730 msgr2=0x7fc42819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e1e3700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc428108730 0x7fc42819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e1e3700 1 -- 172.21.15.19:0/696319335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc428107de0 msgr2=0x7fc42819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e1e3700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc428107de0 0x7fc42819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e1e3700 1 -- 172.21.15.19:0/696319335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4281a6fa0 con 0x7fc4281034b0 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc42e1e3700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc42819bc80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc424005910 tx=0x7fc4240045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424026070 con 0x7fc4281034b0 2024-05-17T09:44:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4281a7230 con 0x7fc4281034b0 2024-05-17T09:44:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4281a76e0 con 0x7fc4281034b0 2024-05-17T09:44:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42400f460 con 0x7fc4281034b0 2024-05-17T09:44:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.585+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424009e60 con 0x7fc4281034b0 2024-05-17T09:44:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.586+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c005330 con 0x7fc4281034b0 2024-05-17T09:44:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.588+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc424029070 con 0x7fc4281034b0 2024-05-17T09:44:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.589+0000 7fc41f7fe700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4140b2980 0x7fc4140b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.590+0000 7fc42d9e2700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4140b2980 0x7fc4140b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.590+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc4240e2e50 con 0x7fc4281034b0 2024-05-17T09:44:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.590+0000 7fc42d9e2700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4140b2980 0x7fc4140b4e90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc418009680 tx=0x7fc418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.594+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc42405edf0 con 0x7fc4281034b0 2024-05-17T09:44:08.597 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.03 MB/sec execute 485 sec latency 17171.274 ms 2024-05-17T09:44:08.724 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:08 smithi019 ceph-mon[31620]: pgmap v1169: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 58 KiB/s wr, 39 op/s 2024-05-17T09:44:08.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:08 smithi079 ceph-mon[35502]: pgmap v1169: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 58 KiB/s wr, 39 op/s 2024-05-17T09:44:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.859+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc40c005620 con 0x7fc4281034b0 2024-05-17T09:44:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.860+0000 7fc41f7fe700 1 -- 172.21.15.19:0/696319335 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7fc424013070 con 0x7fc4281034b0 2024-05-17T09:44:08.860 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:44:08.860 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:44:08.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.862+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4140b2980 msgr2=0x7fc4140b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:08.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.862+0000 7fc430447700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4140b2980 0x7fc4140b4e90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc418009680 tx=0x7fc418009260 comp rx=0 tx=0).stop 2024-05-17T09:44:08.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.862+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 msgr2=0x7fc42819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:08.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.862+0000 7fc430447700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc42819bc80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc424005910 tx=0x7fc4240045e0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 shutdown_connections 2024-05-17T09:44:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc4140b2980 0x7fc4140b4e90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc428108730 0x7fc42819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4281034b0 0x7fc42819bc80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 --2- 172.21.15.19:0/696319335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc428107de0 0x7fc42819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:08.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 >> 172.21.15.19:0/696319335 conn(0x7fc4280fede0 msgr2=0x7fc428101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:08.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 shutdown_connections 2024-05-17T09:44:08.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:08.863+0000 7fc430447700 1 -- 172.21.15.19:0/696319335 wait complete. 2024-05-17T09:44:08.864 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:44:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:08 smithi069 ceph-mon[35081]: pgmap v1169: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 58 KiB/s wr, 39 op/s 2024-05-17T09:44:09.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.02 MB/sec execute 486 sec latency 18171.368 ms 2024-05-17T09:44:09.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:09 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/696319335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:44:09.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:09 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/696319335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:44:09.870 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:09 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/696319335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 -- 172.21.15.19:0/3031021480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8103d80 msgr2=0x7f59d8104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 --2- 172.21.15.19:0/3031021480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8103d80 0x7f59d8104190 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009900 tx=0x7f59c802a410 comp rx=0 tx=0).stop 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 -- 172.21.15.19:0/3031021480 shutdown_connections 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 --2- 172.21.15.19:0/3031021480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8108a80 0x7f59d810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 --2- 172.21.15.19:0/3031021480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8103d80 0x7f59d8104190 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 --2- 172.21.15.19:0/3031021480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8102b40 0x7f59d8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 -- 172.21.15.19:0/3031021480 >> 172.21.15.19:0/3031021480 conn(0x7f59d80fdd50 msgr2=0x7f59d81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 -- 172.21.15.19:0/3031021480 shutdown_connections 2024-05-17T09:44:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.353+0000 7f59e0714640 1 -- 172.21.15.19:0/3031021480 wait complete. 2024-05-17T09:44:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 Processor -- start 2024-05-17T09:44:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 -- start start 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 0x7f59d819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 0x7f59d819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8108a80 0x7f59d81a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59d810e4f0 con 0x7f59d8102b40 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59d810e370 con 0x7f59d8108a80 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59e0714640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59d810e670 con 0x7f59d8103d80 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59ddcaf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 0x7f59d819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 0x7f59d819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59ddcaf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 0x7f59d819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58646/0 (socket says 172.21.15.19:58646) 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59ddcaf640 1 -- 172.21.15.19:0/4049684526 learned_addr learned my addr 172.21.15.19:0/4049684526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59de4b0640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 0x7f59d819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42686/0 (socket says 172.21.15.19:42686) 2024-05-17T09:44:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59decb1640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8108a80 0x7f59d81a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59ddcaf640 1 -- 172.21.15.19:0/4049684526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8108a80 msgr2=0x7f59d81a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8108a80 0x7f59d81a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.354+0000 7f59ddcaf640 1 -- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 msgr2=0x7f59d819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 0x7f59d819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59ddcaf640 1 -- 172.21.15.19:0/4049684526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d81a1ce0 con 0x7f59d8103d80 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59de4b0640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 0x7f59d819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 0x7f59d819a750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009230 tx=0x7f59c8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59cf7fe640 1 -- 172.21.15.19:0/4049684526 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c803f030 con 0x7f59d8103d80 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59cf7fe640 1 -- 172.21.15.19:0/4049684526 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59c80043d0 con 0x7f59d8103d80 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d81a1f70 con 0x7f59d8103d80 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59cf7fe640 1 -- 172.21.15.19:0/4049684526 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8035070 con 0x7f59d8103d80 2024-05-17T09:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.355+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d81a2380 con 0x7f59d8103d80 2024-05-17T09:44:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 Processor -- start 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 -- start start 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c42920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c49820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561923bb4c30 con 0x5619238782f0 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561923ba47d0 con 0x5619238804d0 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59e75c6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561923ba8810 con 0x5619238a11b0 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c42920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59decb1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59decb1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58662/0 (socket says 172.21.15.19:58662) 2024-05-17T09:44:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59decb1640 1 -- 172.21.15.19:0/1839105498 learned_addr learned my addr 172.21.15.19:0/1839105498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59cf7fe640 1 -- 172.21.15.19:0/4049684526 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f59c8042030 con 0x7f59d8103d80 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59de4b0640 1 -- 172.21.15.19:0/1839105498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 msgr2=0x561923c423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59de4b0640 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59de4b0640 1 -- 172.21.15.19:0/1839105498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 msgr2=0x561923c49820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59de4b0640 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c49820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59de4b0640 1 -- 172.21.15.19:0/1839105498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619238b9210 con 0x5619238782f0 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.358+0000 7f59decb1640 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59de4b0640 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c42920 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f59d8102410 tx=0x7f59d40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393025a72b3056e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59ccff9640 1 -- 172.21.15.19:0/1839105498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4004310 con 0x5619238782f0 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59ccff9640 1 -- 172.21.15.19:0/1839105498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d40044b0 con 0x5619238782f0 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59ccff9640 1 -- 172.21.15.19:0/1839105498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40035f0 con 0x5619238782f0 2024-05-17T09:44:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59cf7fe640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f59c40b28a0 0x7f59c40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59decb1640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f59c40b28a0 0x7f59c40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.359+0000 7f59cf7fe640 1 -- 172.21.15.19:0/4049684526 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f59c81029d0 con 0x7f59d8103d80 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 -- 172.21.15.19:0/1839105498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 msgr2=0x561923c42920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c42920 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f59d8102410 tx=0x7f59d40118d0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 -- 172.21.15.19:0/1839105498 shutdown_connections 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c49820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c42920 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1839105498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 -- 172.21.15.19:0/1839105498 >> 172.21.15.19:0/1839105498 conn(0x561923895300 msgr2=0x561923c3dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59decb1640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f59c40b28a0 0x7f59c40b4db0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x561923c43570 tx=0x7f59d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 -- 172.21.15.19:0/1839105498 shutdown_connections 2024-05-17T09:44:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 -- 172.21.15.19:0/1839105498 wait complete. 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 -- start start 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.360+0000 7f59e75c6740 1 Processor -- start 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c53220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56192388a790 con 0x5619238782f0 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561923b60890 con 0x5619238804d0 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561923c4a270 con 0x5619238a11b0 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59e75c6740 10 client.0 _ll_register_callbacks cb 0x561923b63d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c4bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42714/0 (socket says 172.21.15.19:42714) 2024-05-17T09:44:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59de4b0640 1 -- 172.21.15.19:0/1693231861 learned_addr learned my addr 172.21.15.19:0/1693231861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59decb1640 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c53220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 -- 172.21.15.19:0/1693231861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 msgr2=0x561923c53220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c53220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 -- 172.21.15.19:0/1693231861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 msgr2=0x561923c4bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c4bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 -- 172.21.15.19:0/1693231861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561923985620 con 0x5619238804d0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c4c2c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59c8100120 tx=0x7f59c80c9460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc03c170e537f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.361+0000 7f59c1ffb640 1 -- 172.21.15.19:0/1693231861 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8049040 con 0x5619238804d0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d40044b0 con 0x5619238804d0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59c1ffb640 1 -- 172.21.15.19:0/1693231861 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f59c80c9f50 con 0x5619238804d0 2024-05-17T09:44:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59c1ffb640 1 -- 172.21.15.19:0/1693231861 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59c80c9cf0 con 0x5619238804d0 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 10 client.28616 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561923c56150 con 0x5619238804d0 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59c1ffb640 1 -- 172.21.15.19:0/1693231861 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d40044b0 con 0x5619238804d0 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59c1ffb640 1 -- 172.21.15.19:0/1693231861 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f59c80c5660 con 0x5619238804d0 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 10 client.28616 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 10 client.28616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 20 client.28616 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192385fd80 0x561923c598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 4 client.28616 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561923bb6b80 con 0x56192385fd80 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.362+0000 7f59de4b0640 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192385fd80 0x561923c598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.363+0000 7f59c1ffb640 10 client.28616 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.363+0000 7f59de4b0640 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192385fd80 0x561923c598f0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2780e1a65ef1c0 server_cookie=916793678860f31c in_seq=0 out_seq=0 2024-05-17T09:44:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59c1ffb640 1 -- 172.21.15.19:0/1693231861 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561923bb6b80 con 0x56192385fd80 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59c1ffb640 10 client.28616 handle_command_reply: tid=0 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 client.28616 shutdown 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 msgr2=0x561923c4c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c4c2c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59c8100120 tx=0x7f59c80c9460 comp rx=0 tx=0).stop 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 shutdown_connections 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x56192385fd80 0x561923c598f0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238a11b0 0x561923c53220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c4c2c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 --2- 172.21.15.19:0/1693231861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c4bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59c1ffb640 0 client.28616 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 >> 172.21.15.19:0/1693231861 conn(0x561923895300 msgr2=0x561923c40c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 shutdown_connections 2024-05-17T09:44:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 1 -- 172.21.15.19:0/1693231861 wait complete. 2024-05-17T09:44:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.364+0000 7f59e75c6740 20 client.28616 trim_cache size 0 max 16384 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.379+0000 7f59e75c6740 1 Processor -- start 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.379+0000 7f59e75c6740 1 -- start start 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 0x561923c5f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238782f0 0x561923c53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238804d0 0x561923c5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59e75c6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59d81af1d0 con 0x561923895300 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59e75c6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59d81acaa0 con 0x5619238782f0 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59e75c6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59d81aa8f0 con 0x5619238804d0 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 0x561923c5f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 0x561923c5f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42724/0 (socket says 172.21.15.19:42724) 2024-05-17T09:44:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 -- 172.21.15.19:0/747686781 learned_addr learned my addr 172.21.15.19:0/747686781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59decb1640 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238804d0 0x561923c5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238782f0 0x561923c53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 -- 172.21.15.19:0/747686781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238804d0 msgr2=0x561923c5f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238804d0 0x561923c5f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 -- 172.21.15.19:0/747686781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238782f0 msgr2=0x561923c53140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238782f0 0x561923c53140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 -- 172.21.15.19:0/747686781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561923c49710 con 0x561923895300 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238782f0 0x561923c53140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.380+0000 7f59de4b0640 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 0x561923c5f0c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f59d40058c0 tx=0x7f59d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922253f1cee37b65 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59c3fff640 1 -- 172.21.15.19:0/747686781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4033070 con 0x561923895300 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59c3fff640 1 -- 172.21.15.19:0/747686781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d4003a80 con 0x561923895300 2024-05-17T09:44:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59c3fff640 1 -- 172.21.15.19:0/747686781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d402e800 con 0x561923895300 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 -- 172.21.15.19:0/747686781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 msgr2=0x561923c5f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 0x561923c5f0c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f59d40058c0 tx=0x7f59d4005f90 comp rx=0 tx=0).stop 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 -- 172.21.15.19:0/747686781 shutdown_connections 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5619238804d0 0x561923c5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238782f0 0x561923c53140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 --2- 172.21.15.19:0/747686781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561923895300 0x561923c5f0c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 -- 172.21.15.19:0/747686781 >> 172.21.15.19:0/747686781 conn(0x561923c4abe0 msgr2=0x561923c45940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 -- 172.21.15.19:0/747686781 shutdown_connections 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.381+0000 7f59e75c6740 1 -- 172.21.15.19:0/747686781 wait complete. 2024-05-17T09:44:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 -- start start 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 Processor -- start 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c52e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561923895300 0x561923cc0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59d81aaf60 con 0x5619238782f0 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59d81ac3d0 con 0x5619238804d0 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59d81ab630 con 0x561923895300 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59e75c6740 10 client.0 _ll_register_callbacks cb 0x7f59c80c9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59de4b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42728/0 (socket says 172.21.15.19:42728) 2024-05-17T09:44:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59de4b0640 1 -- 172.21.15.19:0/2653437370 learned_addr learned my addr 172.21.15.19:0/2653437370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c52e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.382+0000 7f59decb1640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561923895300 0x561923cc0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59de4b0640 1 -- 172.21.15.19:0/2653437370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561923895300 msgr2=0x561923cc0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59de4b0640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561923895300 0x561923cc0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59de4b0640 1 -- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 msgr2=0x561923c52e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59de4b0640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c52e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59de4b0640 1 -- 172.21.15.19:0/2653437370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561923cc0c10 con 0x5619238782f0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59de4b0640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c528e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f59d4011910 tx=0x7f59d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d2fc500e1a8d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59c1ffb640 1 -- 172.21.15.19:0/2653437370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d403b070 con 0x5619238782f0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59c1ffb640 1 -- 172.21.15.19:0/2653437370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f59d4003c30 con 0x5619238782f0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.383+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d4003a80 con 0x5619238782f0 2024-05-17T09:44:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59c1ffb640 1 -- 172.21.15.19:0/2653437370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59d4047050 con 0x5619238782f0 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 10 client.19134 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561923cc1ea0 con 0x5619238782f0 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59c1ffb640 1 -- 172.21.15.19:0/2653437370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d4003a80 con 0x5619238782f0 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59c1ffb640 1 -- 172.21.15.19:0/2653437370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f59d4040440 con 0x5619238782f0 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 10 client.19134 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 10 client.19134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 20 client.19134 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619238a11b0 0x561923cc46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 4 client.19134 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561923bb0360 con 0x5619238a11b0 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.384+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619238a11b0 0x561923cc46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.385+0000 7f59c1ffb640 10 client.19134 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.385+0000 7f59ddcaf640 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619238a11b0 0x561923cc46c0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a24ff8cfdaf49a server_cookie=47332eb045b0de81 in_seq=0 out_seq=0 2024-05-17T09:44:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59c1ffb640 1 -- 172.21.15.19:0/2653437370 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561923bb0360 con 0x5619238a11b0 2024-05-17T09:44:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59c1ffb640 10 client.19134 handle_command_reply: tid=0 2024-05-17T09:44:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 client.19134 shutdown 2024-05-17T09:44:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 msgr2=0x561923c528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c528e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f59d4011910 tx=0x7f59d4009b50 comp rx=0 tx=0).stop 2024-05-17T09:44:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 shutdown_connections 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5619238a11b0 0x561923cc46c0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561923895300 0x561923cc0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619238804d0 0x561923c52e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59c1ffb640 0 client.19134 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 --2- 172.21.15.19:0/2653437370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619238782f0 0x561923c528e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 >> 172.21.15.19:0/2653437370 conn(0x561923c4abe0 msgr2=0x561923c46310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 shutdown_connections 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 1 -- 172.21.15.19:0/2653437370 wait complete. 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.418+0000 7f59e75c6740 20 client.19134 trim_cache size 0 max 16384 2024-05-17T09:44:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:44:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:44:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:44:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f59c40b28a0 msgr2=0x7f59c40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f59c40b28a0 0x7f59c40b4db0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x561923c43570 tx=0x7f59d0009500 comp rx=0 tx=0).stop 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 msgr2=0x7f59d819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 0x7f59d819a750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009230 tx=0x7f59c8005e80 comp rx=0 tx=0).stop 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 shutdown_connections 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8108a80 0x7f59d81a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f59d8103d80 0x7f59d819a750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f59c40b28a0 0x7f59c40b4db0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 --2- 172.21.15.19:0/4049684526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59d8102b40 0x7f59d819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 >> 172.21.15.19:0/4049684526 conn(0x7f59d80fdd50 msgr2=0x7f59d8109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 shutdown_connections 2024-05-17T09:44:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:10.419+0000 7f59e0714640 1 -- 172.21.15.19:0/4049684526 wait complete. 2024-05-17T09:44:10.429 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id c21059e4 --release --await' 2024-05-17T09:44:10.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id c21059e4 --release --await 2024-05-17T09:44:10.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 6.01 MB/sec execute 487 sec latency 19171.418 ms 2024-05-17T09:44:10.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:10 smithi019 ceph-mon[31620]: pgmap v1170: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-17T09:44:10.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:10 smithi069 ceph-mon[35081]: pgmap v1170: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-17T09:44:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:44:10.970 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:10 smithi079 ceph-mon[35502]: pgmap v1170: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-17T09:44:11.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 5.99 MB/sec execute 488 sec latency 20171.474 ms 2024-05-17T09:44:12.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69172 5.98 MB/sec execute 489 sec latency 21171.751 ms 2024-05-17T09:44:12.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:12 smithi019 ceph-mon[31620]: pgmap v1171: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-05-17T09:44:12.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:12 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:12 smithi069 ceph-mon[35081]: pgmap v1171: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-05-17T09:44:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:12 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:12 smithi079 ceph-mon[35502]: pgmap v1171: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-05-17T09:44:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:12 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 -- 172.21.15.19:0/732661712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73001034b0 msgr2=0x7f73001038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 --2- 172.21.15.19:0/732661712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73001034b0 0x7f73001038c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f72f00099c0 tx=0x7f72f0009cd0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 -- 172.21.15.19:0/732661712 shutdown_connections 2024-05-17T09:44:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 --2- 172.21.15.19:0/732661712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7300108730 0x7f7300110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 --2- 172.21.15.19:0/732661712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73001034b0 0x7f73001038c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 --2- 172.21.15.19:0/732661712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300107de0 0x7f73001081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 -- 172.21.15.19:0/732661712 >> 172.21.15.19:0/732661712 conn(0x7f73000fefa0 msgr2=0x7f73001013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.308+0000 7f7307104700 1 -- 172.21.15.19:0/732661712 shutdown_connections 2024-05-17T09:44:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.309+0000 7f7307104700 1 -- 172.21.15.19:0/732661712 wait complete. 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.309+0000 7f7307104700 1 Processor -- start 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.309+0000 7f7307104700 1 -- start start 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7307104700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 0x7f730019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7307104700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300107de0 0x7f730019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7307104700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7300108730 0x7f730019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7307104700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7300113210 con 0x7f7300108730 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7307104700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7300113090 con 0x7f7300107de0 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7307104700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7300113390 con 0x7f73001034b0 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 0x7f730019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 0x7f730019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58704/0 (socket says 172.21.15.19:58704) 2024-05-17T09:44:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f73056a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7300108730 0x7f730019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 -- 172.21.15.19:0/906437196 learned_addr learned my addr 172.21.15.19:0/906437196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f72fffff700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300107de0 0x7f730019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 -- 172.21.15.19:0/906437196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300107de0 msgr2=0x7f730019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300107de0 0x7f730019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 -- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7300108730 msgr2=0x7f730019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7300108730 0x7f730019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 -- 172.21.15.19:0/906437196 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730019fc20 con 0x7f73001034b0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f73056a1700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7300108730 0x7f730019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.310+0000 7f7304ea0700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 0x7f730019ad50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f72f0006010 tx=0x7f72f0005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.311+0000 7f72fdffb700 1 -- 172.21.15.19:0/906437196 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0026070 con 0x7f73001034b0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.311+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730019feb0 con 0x7f73001034b0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.311+0000 7f72fdffb700 1 -- 172.21.15.19:0/906437196 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72f0005680 con 0x7f73001034b0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.311+0000 7f72fdffb700 1 -- 172.21.15.19:0/906437196 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f000faa0 con 0x7f73001034b0 2024-05-17T09:44:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.311+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73001a7830 con 0x7f73001034b0 2024-05-17T09:44:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.314+0000 7f72fdffb700 1 -- 172.21.15.19:0/906437196 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f72f0029070 con 0x7f73001034b0 2024-05-17T09:44:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.314+0000 7f7315b19b80 1 Processor -- start 2024-05-17T09:44:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.314+0000 7f7315b19b80 1 -- start start 2024-05-17T09:44:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.314+0000 7f72fdffb700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72e80b2880 0x7f72e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f73056a1700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72e80b2880 0x7f72e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f72fdffb700 1 -- 172.21.15.19:0/906437196 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f72f00e2d60 con 0x7f73001034b0 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 0x55cbbbb6e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbbb68620 0x55cbbbb68a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7315b19b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbbba96d90 con 0x55cbbba674d0 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7315b19b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cbbba86930 con 0x55cbbb9e9f40 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7315b19b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cbbba8a970 con 0x55cbbbb68620 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 0x55cbbbb6e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 0x55cbbbb6e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42762/0 (socket says 172.21.15.19:42762) 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 -- 172.21.15.19:0/815899517 learned_addr learned my addr 172.21.15.19:0/815899517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f72fffff700 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f73056a1700 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbbb68620 0x55cbbbb68a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 -- 172.21.15.19:0/815899517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbbb68620 msgr2=0x55cbbbb68a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbbb68620 0x55cbbbb68a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 -- 172.21.15.19:0/815899517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 msgr2=0x55cbbbb6eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f7304ea0700 1 -- 172.21.15.19:0/815899517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbb9d69d0 con 0x55cbbba674d0 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.315+0000 7f73056a1700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72e80b2880 0x7f72e80b4d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f72f8003e80 tx=0x7f72f8009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.316+0000 7f7304ea0700 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 0x55cbbbb6e560 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f72f00a60e0 tx=0x7f72f00a98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9610bd41f240b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.316+0000 7f72e6ffd700 1 -- 172.21.15.19:0/815899517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f000f460 con 0x55cbbba674d0 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.316+0000 7f72e6ffd700 1 -- 172.21.15.19:0/815899517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72f00a9b70 con 0x55cbbba674d0 2024-05-17T09:44:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.316+0000 7f72e6ffd700 1 -- 172.21.15.19:0/815899517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00b29a0 con 0x55cbbba674d0 2024-05-17T09:44:13.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 -- 172.21.15.19:0/815899517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 msgr2=0x55cbbbb6e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 0x55cbbbb6e560 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f72f00a60e0 tx=0x7f72f00a98c0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 -- 172.21.15.19:0/815899517 shutdown_connections 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbbb68620 0x55cbbbb68a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbba674d0 0x55cbbbb6e560 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 --2- 172.21.15.19:0/815899517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 -- 172.21.15.19:0/815899517 >> 172.21.15.19:0/815899517 conn(0x55cbbba28d80 msgr2=0x55cbbbb63520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 -- 172.21.15.19:0/815899517 shutdown_connections 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 -- 172.21.15.19:0/815899517 wait complete. 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 -- start start 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.317+0000 7f7315b19b80 1 Processor -- start 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbbb98d0a0 con 0x55cbbb9e9f40 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cbbba288c0 con 0x55cbbbb68620 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cbbbb69260 con 0x55cbbba674d0 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7315b19b80 10 client.0 _ll_register_callbacks cb 0x55cbbba21a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb77cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42778/0 (socket says 172.21.15.19:42778) 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 -- 172.21.15.19:0/1954953901 learned_addr learned my addr 172.21.15.19:0/1954953901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f72fffff700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f73056a1700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 -- 172.21.15.19:0/1954953901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 msgr2=0x55cbbbb781f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb781f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 -- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 msgr2=0x55cbbbb720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 -- 172.21.15.19:0/1954953901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbba25790 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f73056a1700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f7304ea0700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb77cb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f72f00a6090 tx=0x7f72f000f550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1179dd786b4aa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.318+0000 7f72dffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f72dffff700 1 -- 172.21.15.19:0/1954953901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f006e060 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f72dffff700 1 -- 172.21.15.19:0/1954953901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f72f00a6650 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f00a9b70 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f72dffff700 1 -- 172.21.15.19:0/1954953901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f72f00c3030 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 10 client.19146 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cbbbb7f8d0 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f72dffff700 1 -- 172.21.15.19:0/1954953901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72f00a9b70 con 0x55cbbb9e9f40 2024-05-17T09:44:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f72dffff700 1 -- 172.21.15.19:0/1954953901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f72f00a5bb0 con 0x55cbbb9e9f40 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 10 client.19146 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 10 client.19146 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 20 client.19146 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbb81bd0 0x55cbbbb83ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 4 client.19146 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.319+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbbba98ce0 con 0x55cbbbb81bd0 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.320+0000 7f72fffff700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbb81bd0 0x55cbbbb83ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.320+0000 7f72fffff700 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbb81bd0 0x55cbbbb83ff0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64657e50f524ef1 server_cookie=54ccbb3746fce220 in_seq=0 out_seq=0 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.320+0000 7f72dffff700 10 client.19146 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f72dffff700 1 -- 172.21.15.19:0/1954953901 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cbbba98ce0 con 0x55cbbbb81bd0 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f72dffff700 10 client.19146 handle_command_reply: tid=0 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 client.19146 shutdown 2024-05-17T09:44:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 msgr2=0x55cbbbb77cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb77cb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f72f00a6090 tx=0x7f72f000f550 comp rx=0 tx=0).stop 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 shutdown_connections 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbb81bd0 0x55cbbbb83ff0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb77cb0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f72dffff700 0 client.19146 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1954953901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb781f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 >> 172.21.15.19:0/1954953901 conn(0x55cbbba28d80 msgr2=0x55cbbbb5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 shutdown_connections 2024-05-17T09:44:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 1 -- 172.21.15.19:0/1954953901 wait complete. 2024-05-17T09:44:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.321+0000 7f7315b19b80 20 client.19146 trim_cache size 0 max 16384 2024-05-17T09:44:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.341+0000 7f7315b19b80 1 Processor -- start 2024-05-17T09:44:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.341+0000 7f7315b19b80 1 -- start start 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb89770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbbb68620 0x55cbbbb89cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7315b19b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73001b2aa0 con 0x55cbbbb68620 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7315b19b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73001b0330 con 0x55cbbb9e9f40 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7315b19b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73001ae180 con 0x55cbbba674d0 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f73056a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbbb68620 0x55cbbbb89cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f72fffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb89770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f73056a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbbb68620 0x55cbbbb89cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42792/0 (socket says 172.21.15.19:42792) 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f72fffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb89770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:58740/0 (socket says 172.21.15.19:58740) 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f73056a1700 1 -- 172.21.15.19:0/1996909720 learned_addr learned my addr 172.21.15.19:0/1996909720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 -- 172.21.15.19:0/1996909720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 msgr2=0x55cbbbb89770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb89770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 -- 172.21.15.19:0/1996909720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbbb68620 msgr2=0x55cbbbb89cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbbb68620 0x55cbbbb89cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 -- 172.21.15.19:0/1996909720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbbb64d40 con 0x55cbbb9e9f40 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f7304ea0700 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6c580 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72f00a9ff0 tx=0x7f72f00b2d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca91a0e0936ec2ab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.342+0000 7f72e67fc700 1 -- 172.21.15.19:0/1996909720 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f006e070 con 0x55cbbb9e9f40 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f72e67fc700 1 -- 172.21.15.19:0/1996909720 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72f00aa760 con 0x55cbbb9e9f40 2024-05-17T09:44:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f72e67fc700 1 -- 172.21.15.19:0/1996909720 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00a5c30 con 0x55cbbb9e9f40 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 -- 172.21.15.19:0/1996909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 msgr2=0x55cbbbb6c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6c580 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72f00a9ff0 tx=0x7f72f00b2d60 comp rx=0 tx=0).stop 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 -- 172.21.15.19:0/1996909720 shutdown_connections 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbbb68620 0x55cbbbb89cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb6c580 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 --2- 172.21.15.19:0/1996909720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb89770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 -- 172.21.15.19:0/1996909720 >> 172.21.15.19:0/1996909720 conn(0x55cbbba28d80 msgr2=0x55cbbbb6be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 -- 172.21.15.19:0/1996909720 shutdown_connections 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.343+0000 7f7315b19b80 1 -- 172.21.15.19:0/1996909720 wait complete. 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 -- start start 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 Processor -- start 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb7bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb7c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb7c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73001ae7f0 con 0x55cbbb9e9f40 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73001afc60 con 0x55cbbbb68620 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73001aeec0 con 0x55cbbba674d0 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7315b19b80 10 client.0 _ll_register_callbacks cb 0x55cbbba21a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb7bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.344+0000 7f7304ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb7bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42794/0 (socket says 172.21.15.19:42794) 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 -- 172.21.15.19:0/2202177377 learned_addr learned my addr 172.21.15.19:0/2202177377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:13.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f72fffff700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb7c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f73056a1700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb7c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 -- 172.21.15.19:0/2202177377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 msgr2=0x55cbbbb7c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb7c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 -- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 msgr2=0x55cbbbb7c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb7c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 -- 172.21.15.19:0/2202177377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbbbeb780 con 0x55cbbb9e9f40 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f72fffff700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f73056a1700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb7c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7304ea0700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb7bea0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f72f00058b0 tx=0x7f72f00a5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9084d689c03101ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f72dffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f72dffff700 1 -- 172.21.15.19:0/2202177377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f006e070 con 0x55cbbb9e9f40 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f72dffff700 1 -- 172.21.15.19:0/2202177377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f72f00a6650 con 0x55cbbb9e9f40 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f00aa760 con 0x55cbbb9e9f40 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f72dffff700 1 -- 172.21.15.19:0/2202177377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f72f00b2e30 con 0x55cbbb9e9f40 2024-05-17T09:44:13.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7315b19b80 10 client.19152 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.345+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cbbbbeb780 con 0x55cbbb9e9f40 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f72dffff700 1 -- 172.21.15.19:0/2202177377 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72f00aa760 con 0x55cbbb9e9f40 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f72dffff700 1 -- 172.21.15.19:0/2202177377 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f72f00a8f40 con 0x55cbbb9e9f40 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f7315b19b80 10 client.19152 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f7315b19b80 10 client.19152 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f7315b19b80 20 client.19152 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f7315b19b80 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbbecc50 0x55cbbbbef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f7315b19b80 4 client.19152 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c21059e4", "release": true, "await": true}] 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c21059e4", "release": true, "await": true}) -- 0x55cbbba924c0 con 0x55cbbbbecc50 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f72fffff700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbbecc50 0x55cbbbbef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f72dffff700 10 client.19152 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.346+0000 7f72fffff700 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbbecc50 0x55cbbbbef070 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57aaf2f951c14925 server_cookie=9583115ac9c6fa36 in_seq=0 out_seq=0 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f72dffff700 1 -- 172.21.15.19:0/2202177377 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cbbba924c0 con 0x55cbbbbecc50 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f72dffff700 10 client.19152 handle_command_reply: tid=0 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 client.19152 shutdown 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 msgr2=0x55cbbbb7bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb7bea0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f72f00058b0 tx=0x7f72f00a5c00 comp rx=0 tx=0).stop 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 shutdown_connections 2024-05-17T09:44:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55cbbbbecc50 0x55cbbbbef070 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cbbbb68620 0x55cbbbb7c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbbb9e9f40 0x55cbbbb7bea0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f72dffff700 0 client.19152 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 --2- 172.21.15.19:0/2202177377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cbbba674d0 0x55cbbbb7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.385+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 >> 172.21.15.19:0/2202177377 conn(0x55cbbba28d80 msgr2=0x55cbbbb5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.386+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 shutdown_connections 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.386+0000 7f7315b19b80 1 -- 172.21.15.19:0/2202177377 wait complete. 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.386+0000 7f7315b19b80 20 client.19152 trim_cache size 0 max 16384 2024-05-17T09:44:13.386 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 92, 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "c21059e4": { 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 92, 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:44:13.387 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.5, 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:44:13.388 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72e80b2880 msgr2=0x7f72e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72e80b2880 0x7f72e80b4d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f72f8003e80 tx=0x7f72f8009920 comp rx=0 tx=0).stop 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 msgr2=0x7f730019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 0x7f730019ad50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f72f0006010 tx=0x7f72f0005280 comp rx=0 tx=0).stop 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 shutdown_connections 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f72e80b2880 0x7f72e80b4d90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73001034b0 0x7f730019ad50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7300108730 0x7f730019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 --2- 172.21.15.19:0/906437196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300107de0 0x7f730019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 >> 172.21.15.19:0/906437196 conn(0x7f73000fefa0 msgr2=0x7f7300075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.387+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 shutdown_connections 2024-05-17T09:44:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:44:13.388+0000 7f7307104700 1 -- 172.21.15.19:0/906437196 wait complete. 2024-05-17T09:44:13.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69211 5.97 MB/sec execute 490 sec latency 21947.218 ms 2024-05-17T09:44:14.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69362 5.97 MB/sec execute 491 sec latency 79.303 ms 2024-05-17T09:44:14.680 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c21059e4', total seconds elapsed: 30.8 2024-05-17T09:44:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: pgmap v1172: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:44:14.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: pgmap v1172: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:14.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:44:14.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:44:14.942 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: pgmap v1172: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:44:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:44:15.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:44:15.022 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:44:15.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69531 5.98 MB/sec execute 492 sec latency 81.417 ms 2024-05-17T09:44:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:15 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:44:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:15 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:44:15.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:15 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:44:15.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:15 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:44:16.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:15 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-17T09:44:16.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:15 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-17T09:44:16.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69691 5.98 MB/sec execute 493 sec latency 78.480 ms 2024-05-17T09:44:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:16 smithi069 ceph-mon[35081]: pgmap v1173: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-17T09:44:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:16 smithi019 ceph-mon[31620]: pgmap v1173: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-17T09:44:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:16 smithi079 ceph-mon[35502]: pgmap v1173: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-17T09:44:17.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 69834 5.99 MB/sec execute 494 sec latency 70.807 ms 2024-05-17T09:44:18.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70026 6.01 MB/sec execute 495 sec latency 68.272 ms 2024-05-17T09:44:18.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:18 smithi069 ceph-mon[35081]: pgmap v1174: 129 pgs: 129 active+clean; 496 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 622 KiB/s wr, 59 op/s 2024-05-17T09:44:18.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:18 smithi019 ceph-mon[31620]: pgmap v1174: 129 pgs: 129 active+clean; 496 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 622 KiB/s wr, 59 op/s 2024-05-17T09:44:19.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:18 smithi079 ceph-mon[35502]: pgmap v1174: 129 pgs: 129 active+clean; 496 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 622 KiB/s wr, 59 op/s 2024-05-17T09:44:19.598 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70238 6.03 MB/sec execute 496 sec latency 77.122 ms 2024-05-17T09:44:20.599 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70433 6.04 MB/sec execute 497 sec latency 105.758 ms 2024-05-17T09:44:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:20 smithi069 ceph-mon[35081]: pgmap v1175: 129 pgs: 129 active+clean; 500 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-05-17T09:44:20.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:20 smithi019 ceph-mon[31620]: pgmap v1175: 129 pgs: 129 active+clean; 500 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-05-17T09:44:21.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:20 smithi079 ceph-mon[35502]: pgmap v1175: 129 pgs: 129 active+clean; 500 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-05-17T09:44:21.599 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70549 6.03 MB/sec execute 498 sec latency 75.578 ms 2024-05-17T09:44:22.599 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70665 6.03 MB/sec execute 499 sec latency 85.325 ms 2024-05-17T09:44:22.909 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:22 smithi069 ceph-mon[35081]: pgmap v1176: 129 pgs: 129 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-05-17T09:44:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:22 smithi019 ceph-mon[31620]: pgmap v1176: 129 pgs: 129 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-05-17T09:44:22.995 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:22 smithi079 ceph-mon[35502]: pgmap v1176: 129 pgs: 129 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-05-17T09:44:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 -- 172.21.15.19:0/1787954435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c103d60 msgr2=0x7fc52c104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 --2- 172.21.15.19:0/1787954435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c103d60 0x7fc52c104170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc518009860 tx=0x7fc51802a340 comp rx=0 tx=0).stop 2024-05-17T09:44:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 -- 172.21.15.19:0/1787954435 shutdown_connections 2024-05-17T09:44:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 --2- 172.21.15.19:0/1787954435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c108a60 0x7fc52c10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 --2- 172.21.15.19:0/1787954435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c103d60 0x7fc52c104170 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 --2- 172.21.15.19:0/1787954435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 -- 172.21.15.19:0/1787954435 >> 172.21.15.19:0/1787954435 conn(0x7fc52c0fdd50 msgr2=0x7fc52c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 -- 172.21.15.19:0/1787954435 shutdown_connections 2024-05-17T09:44:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.576+0000 7fc532ea8640 1 -- 172.21.15.19:0/1787954435 wait complete. 2024-05-17T09:44:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.577+0000 7fc532ea8640 1 Processor -- start 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.577+0000 7fc532ea8640 1 -- start start 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc532ea8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc532ea8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c103d60 0x7fc52c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc532ea8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c108a60 0x7fc52c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc532ea8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c10e4d0 con 0x7fc52c108a60 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc532ea8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c10e350 con 0x7fc52c103d60 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc532ea8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c10e650 con 0x7fc52c102b20 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c103d60 0x7fc52c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39230/0 (socket says 172.21.15.19:39230) 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 -- 172.21.15.19:0/847357672 learned_addr learned my addr 172.21.15.19:0/847357672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c103d60 0x7fc52c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33724/0 (socket says 172.21.15.19:33724) 2024-05-17T09:44:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc531445640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c108a60 0x7fc52c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 -- 172.21.15.19:0/847357672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c103d60 msgr2=0x7fc52c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c103d60 0x7fc52c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 -- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c108a60 msgr2=0x7fc52c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c108a60 0x7fc52c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.578+0000 7fc530c44640 1 -- 172.21.15.19:0/847357672 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c1a1d10 con 0x7fc52c102b20 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.579+0000 7fc530c44640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c19a1d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc520009c90 tx=0x7fc52000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.579+0000 7fc529ffb640 1 -- 172.21.15.19:0/847357672 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52001e070 con 0x7fc52c102b20 2024-05-17T09:44:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.579+0000 7fc529ffb640 1 -- 172.21.15.19:0/847357672 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5200198a0 con 0x7fc52c102b20 2024-05-17T09:44:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.579+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c0fe1c0 con 0x7fc52c102b20 2024-05-17T09:44:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.579+0000 7fc529ffb640 1 -- 172.21.15.19:0/847357672 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520018730 con 0x7fc52c102b20 2024-05-17T09:44:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.579+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c0fe730 con 0x7fc52c102b20 2024-05-17T09:44:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.582+0000 7fc529ffb640 1 -- 172.21.15.19:0/847357672 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc520021070 con 0x7fc52c102b20 2024-05-17T09:44:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 Processor -- start 2024-05-17T09:44:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 -- start start 2024-05-17T09:44:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdf44d0 0x55afed1bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55afed128c30 con 0x55afecdf44d0 2024-05-17T09:44:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afed1187d0 con 0x55afecdec2f0 2024-05-17T09:44:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc539e37740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55afed11c810 con 0x55afece151b0 2024-05-17T09:44:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc531445640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc530c44640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc531445640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39234/0 (socket says 172.21.15.19:39234) 2024-05-17T09:44:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdf44d0 0x55afed1bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.583+0000 7fc531445640 1 -- 172.21.15.19:0/319522880 learned_addr learned my addr 172.21.15.19:0/319522880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc529ffb640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc51c0b29a0 0x7fc51c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc52bfff640 1 -- 172.21.15.19:0/319522880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 msgr2=0x55afed1b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc52bfff640 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc52bfff640 1 -- 172.21.15.19:0/319522880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 msgr2=0x55afed1b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc52bfff640 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc52bfff640 1 -- 172.21.15.19:0/319522880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afece2d210 con 0x55afecdf44d0 2024-05-17T09:44:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc531445640 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc529ffb640 1 -- 172.21.15.19:0/847357672 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5200e5d40 con 0x7fc52c102b20 2024-05-17T09:44:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc531445640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc51c0b29a0 0x7fc51c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.584+0000 7fc52bfff640 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdf44d0 0x55afed1bd820 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fc51802a860 tx=0x7fc518005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaba692ebdb8f5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.585+0000 7fc516ffd640 1 -- 172.21.15.19:0/319522880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518037040 con 0x55afecdf44d0 2024-05-17T09:44:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.585+0000 7fc531445640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc51c0b29a0 0x7fc51c0b4eb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55afed1b7570 tx=0x7fc524004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.585+0000 7fc516ffd640 1 -- 172.21.15.19:0/319522880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5180306b0 con 0x55afecdf44d0 2024-05-17T09:44:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.585+0000 7fc516ffd640 1 -- 172.21.15.19:0/319522880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180035f0 con 0x55afecdf44d0 2024-05-17T09:44:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 -- 172.21.15.19:0/319522880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdf44d0 msgr2=0x55afed1bd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdf44d0 0x55afed1bd820 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fc51802a860 tx=0x7fc518005e40 comp rx=0 tx=0).stop 2024-05-17T09:44:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 -- 172.21.15.19:0/319522880 shutdown_connections 2024-05-17T09:44:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdf44d0 0x55afed1bd820 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 --2- 172.21.15.19:0/319522880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 -- 172.21.15.19:0/319522880 >> 172.21.15.19:0/319522880 conn(0x55afece09300 msgr2=0x55afed1b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 -- 172.21.15.19:0/319522880 shutdown_connections 2024-05-17T09:44:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 -- 172.21.15.19:0/319522880 wait complete. 2024-05-17T09:44:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.586+0000 7fc539e37740 1 -- start start 2024-05-17T09:44:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 Processor -- start 2024-05-17T09:44:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 0x55afed1c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55afecdfe790 con 0x55afecdec2f0 2024-05-17T09:44:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afed0d4890 con 0x55afecdd3d80 2024-05-17T09:44:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55afed1be260 con 0x55afece151b0 2024-05-17T09:44:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.587+0000 7fc539e37740 10 client.0 _ll_register_callbacks cb 0x55afed0d7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.589+0000 7fc531445640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 0x55afed1c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.589+0000 7fc531445640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 0x55afed1c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33744/0 (socket says 172.21.15.19:33744) 2024-05-17T09:44:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.589+0000 7fc531445640 1 -- 172.21.15.19:0/289978413 learned_addr learned my addr 172.21.15.19:0/289978413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.590+0000 7fc531445640 1 -- 172.21.15.19:0/289978413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 msgr2=0x55afed1c01e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:44:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.590+0000 7fc531445640 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1c01e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.590+0000 7fc531445640 1 -- 172.21.15.19:0/289978413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 msgr2=0x55afed1bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.590+0000 7fc531445640 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.590+0000 7fc531445640 1 -- 172.21.15.19:0/289978413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afecef9620 con 0x55afecdd3d80 2024-05-17T09:44:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.591+0000 7fc531445640 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 0x55afed1c7140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc52401f6c0 tx=0x7fc524004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c225c343746a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.591+0000 7fc50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:44:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.591+0000 7fc50bfff640 1 -- 172.21.15.19:0/289978413 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400de30 con 0x55afecdd3d80 2024-05-17T09:44:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.591+0000 7fc50bfff640 1 -- 172.21.15.19:0/289978413 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc524003970 con 0x55afecdd3d80 2024-05-17T09:44:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.591+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5180306b0 con 0x55afecdd3d80 2024-05-17T09:44:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.591+0000 7fc50bfff640 1 -- 172.21.15.19:0/289978413 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc524033050 con 0x55afecdd3d80 2024-05-17T09:44:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.592+0000 7fc539e37740 10 client.28652 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.592+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55afed1ca190 con 0x55afecdd3d80 2024-05-17T09:44:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc50bfff640 1 -- 172.21.15.19:0/289978413 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5180306b0 con 0x55afecdd3d80 2024-05-17T09:44:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc50bfff640 1 -- 172.21.15.19:0/289978413 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc52400b040 con 0x55afecdd3d80 2024-05-17T09:44:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc539e37740 10 client.28652 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc539e37740 10 client.28652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc539e37740 20 client.28652 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc539e37740 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afecda91c0 0x55afed1cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.593+0000 7fc539e37740 4 client.28652 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:44:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.594+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afed12ab80 con 0x55afecda91c0 2024-05-17T09:44:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.595+0000 7fc530c44640 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afecda91c0 0x55afed1cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.597+0000 7fc530c44640 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afecda91c0 0x55afed1cd9e0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed24a368cc39bcbd server_cookie=d74b8f63e63b0c3c in_seq=0 out_seq=0 2024-05-17T09:44:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.598+0000 7fc50bfff640 10 client.28652 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.600+0000 7fc50bfff640 1 -- 172.21.15.19:0/289978413 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55afed12ab80 con 0x55afecda91c0 2024-05-17T09:44:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.600+0000 7fc50bfff640 10 client.28652 handle_command_reply: tid=0 2024-05-17T09:44:23.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70856 6.06 MB/sec execute 500 sec latency 91.374 ms 2024-05-17T09:44:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.602+0000 7fc539e37740 1 client.28652 shutdown 2024-05-17T09:44:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.605+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 msgr2=0x55afed1c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.605+0000 7fc539e37740 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 0x55afed1c7140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc52401f6c0 tx=0x7fc524004a40 comp rx=0 tx=0).stop 2024-05-17T09:44:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.608+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 shutdown_connections 2024-05-17T09:44:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.608+0000 7fc539e37740 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afecda91c0 0x55afed1cd9e0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.608+0000 7fc539e37740 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdd3d80 0x55afed1c7140 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.608+0000 7fc539e37740 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afece151b0 0x55afed1c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.608+0000 7fc539e37740 1 --2- 172.21.15.19:0/289978413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.608+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 >> 172.21.15.19:0/289978413 conn(0x55afece09300 msgr2=0x55afed1b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.609+0000 7fc50bfff640 0 client.28652 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.610+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 shutdown_connections 2024-05-17T09:44:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.610+0000 7fc539e37740 1 -- 172.21.15.19:0/289978413 wait complete. 2024-05-17T09:44:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.610+0000 7fc539e37740 20 client.28652 trim_cache size 0 max 16384 2024-05-17T09:44:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.643+0000 7fc539e37740 1 Processor -- start 2024-05-17T09:44:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.643+0000 7fc539e37740 1 -- start start 2024-05-17T09:44:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.643+0000 7fc539e37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.643+0000 7fc539e37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.643+0000 7fc539e37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afece151b0 0x55afed1c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc539e37740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1ad090 con 0x55afece151b0 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc530c44640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:33756/0 (socket says 172.21.15.19:33756) 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc52bfff640 1 -- 172.21.15.19:0/3598114620 learned_addr learned my addr 172.21.15.19:0/3598114620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc531445640 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afece151b0 0x55afed1c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1aa960 con 0x55afecdec2f0 2024-05-17T09:44:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a87b0 con 0x55afecdd3d80 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc530c44640 1 -- 172.21.15.19:0/3598114620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 msgr2=0x55afed1c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc530c44640 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc530c44640 1 -- 172.21.15.19:0/3598114620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afece151b0 msgr2=0x55afed1c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc530c44640 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afece151b0 0x55afed1c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.644+0000 7fc530c44640 1 -- 172.21.15.19:0/3598114620 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afecda93f0 con 0x55afecdd3d80 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.645+0000 7fc530c44640 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1b1d20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5200acf90 tx=0x7fc520019c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d484d1116563ce0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.645+0000 7fc5167fc640 1 -- 172.21.15.19:0/3598114620 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200ac770 con 0x55afecdd3d80 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.645+0000 7fc5167fc640 1 -- 172.21.15.19:0/3598114620 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5200acb10 con 0x55afecdd3d80 2024-05-17T09:44:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.645+0000 7fc5167fc640 1 -- 172.21.15.19:0/3598114620 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200a9290 con 0x55afecdd3d80 2024-05-17T09:44:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.646+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 msgr2=0x55afed1b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1b1d20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5200acf90 tx=0x7fc520019c50 comp rx=0 tx=0).stop 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 shutdown_connections 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afece151b0 0x55afed1c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afecdec2f0 0x55afed1c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 --2- 172.21.15.19:0/3598114620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1b1d20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 >> 172.21.15.19:0/3598114620 conn(0x55afece09300 msgr2=0x55afed1b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 shutdown_connections 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 -- 172.21.15.19:0/3598114620 wait complete. 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.647+0000 7fc539e37740 1 -- start start 2024-05-17T09:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.648+0000 7fc539e37740 1 Processor -- start 2024-05-17T09:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.648+0000 7fc539e37740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.648+0000 7fc539e37740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.648+0000 7fc530c44640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.648+0000 7fc530c44640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:39248/0 (socket says 172.21.15.19:39248) 2024-05-17T09:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.648+0000 7fc530c44640 1 -- 172.21.15.19:0/3114972910 learned_addr learned my addr 172.21.15.19:0/3114972910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afece151b0 0x55afed1c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a8e20 con 0x55afecdec2f0 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1aa290 con 0x55afece151b0 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1a94f0 con 0x55afecdd3d80 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc52bfff640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc539e37740 10 client.0 _ll_register_callbacks cb 0x7fc52400dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc531445640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afece151b0 0x55afed1c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc530c44640 1 -- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afece151b0 msgr2=0x55afed1c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc530c44640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afece151b0 0x55afed1c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc530c44640 1 -- 172.21.15.19:0/3114972910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 msgr2=0x55afed1c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc530c44640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc530c44640 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afed1c62b0 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc52bfff640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.649+0000 7fc531445640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afece151b0 0x55afed1c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc530c44640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1c4880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc52000bb90 tx=0x7fc520005a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af87417ef39facf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc50bfff640 1 -- 172.21.15.19:0/3114972910 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200ad000 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc50bfff640 1 -- 172.21.15.19:0/3114972910 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc5200a9290 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5200acb10 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc50bfff640 1 -- 172.21.15.19:0/3114972910 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc520019a50 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc539e37740 10 client.28636 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55afed1c7540 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc50bfff640 1 -- 172.21.15.19:0/3114972910 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5200acb10 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc50bfff640 1 -- 172.21.15.19:0/3114972910 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc5200a8a00 con 0x55afecdd3d80 2024-05-17T09:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc539e37740 10 client.28636 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.650+0000 7fc539e37740 10 client.28636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc539e37740 20 client.28636 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afed2363b0 0x55afed2387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc539e37740 4 client.28636 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afed124360 con 0x55afed2363b0 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc52bfff640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afed2363b0 0x55afed2387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc52bfff640 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afed2363b0 0x55afed2387d0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be947d67133f2f9 server_cookie=8a23b84e35a589c9 in_seq=0 out_seq=0 2024-05-17T09:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.651+0000 7fc50bfff640 10 client.28636 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.760+0000 7fc50bfff640 1 -- 172.21.15.19:0/3114972910 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55afed124360 con 0x55afed2363b0 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.760+0000 7fc50bfff640 10 client.28636 handle_command_reply: tid=0 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.760+0000 7fc539e37740 1 client.28636 shutdown 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 msgr2=0x55afed1c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1c4880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc52000bb90 tx=0x7fc520005a70 comp rx=0 tx=0).stop 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 shutdown_connections 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55afed2363b0 0x55afed2387d0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afece151b0 0x55afed1c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc50bfff640 0 client.28636 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afecdec2f0 0x55afed1c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 --2- 172.21.15.19:0/3114972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55afecdd3d80 0x55afed1c4880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 >> 172.21.15.19:0/3114972910 conn(0x55afece09300 msgr2=0x55afed1b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 shutdown_connections 2024-05-17T09:44:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 1 -- 172.21.15.19:0/3114972910 wait complete. 2024-05-17T09:44:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.761+0000 7fc539e37740 20 client.28636 trim_cache size 0 max 16384 2024-05-17T09:44:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:44:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:44:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc51c0b29a0 msgr2=0x7fc51c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc51c0b29a0 0x7fc51c0b4eb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55afed1b7570 tx=0x7fc524004ab0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 msgr2=0x7fc52c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c19a1d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc520009c90 tx=0x7fc52000b690 comp rx=0 tx=0).stop 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 shutdown_connections 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc51c0b29a0 0x7fc51c0b4eb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c108a60 0x7fc52c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc52c103d60 0x7fc52c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 --2- 172.21.15.19:0/847357672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc52c102b20 0x7fc52c19a1d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 >> 172.21.15.19:0/847357672 conn(0x7fc52c0fdd50 msgr2=0x7fc52c109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 shutdown_connections 2024-05-17T09:44:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:23.762+0000 7fc532ea8640 1 -- 172.21.15.19:0/847357672 wait complete. 2024-05-17T09:44:24.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 70989 6.06 MB/sec execute 501 sec latency 109.857 ms 2024-05-17T09:44:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:24 smithi069 ceph-mon[35081]: pgmap v1177: 129 pgs: 129 active+clean; 498 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-05-17T09:44:24.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:24 smithi019 ceph-mon[31620]: pgmap v1177: 129 pgs: 129 active+clean; 498 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-05-17T09:44:25.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:24 smithi079 ceph-mon[35502]: pgmap v1177: 129 pgs: 129 active+clean; 498 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-05-17T09:44:25.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71114 6.06 MB/sec execute 502 sec latency 117.063 ms 2024-05-17T09:44:26.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71256 6.06 MB/sec execute 503 sec latency 65.762 ms 2024-05-17T09:44:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:26 smithi069 ceph-mon[35081]: pgmap v1178: 129 pgs: 129 active+clean; 499 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-05-17T09:44:26.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:26 smithi019 ceph-mon[31620]: pgmap v1178: 129 pgs: 129 active+clean; 499 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-05-17T09:44:27.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:26 smithi079 ceph-mon[35502]: pgmap v1178: 129 pgs: 129 active+clean; 499 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-05-17T09:44:27.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71418 6.07 MB/sec execute 504 sec latency 83.211 ms 2024-05-17T09:44:28.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71562 6.07 MB/sec execute 505 sec latency 84.328 ms 2024-05-17T09:44:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:28 smithi069 ceph-mon[35081]: pgmap v1179: 129 pgs: 129 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.5 MiB/s wr, 238 op/s 2024-05-17T09:44:28.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:28 smithi019 ceph-mon[31620]: pgmap v1179: 129 pgs: 129 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.5 MiB/s wr, 238 op/s 2024-05-17T09:44:29.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:28 smithi079 ceph-mon[35502]: pgmap v1179: 129 pgs: 129 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.5 MiB/s wr, 238 op/s 2024-05-17T09:44:29.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71693 6.07 MB/sec execute 506 sec latency 77.038 ms 2024-05-17T09:44:30.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71824 6.06 MB/sec execute 507 sec latency 109.072 ms 2024-05-17T09:44:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:30 smithi069 ceph-mon[35081]: pgmap v1180: 129 pgs: 129 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 4.5 MiB/s wr, 236 op/s 2024-05-17T09:44:30.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:30 smithi019 ceph-mon[31620]: pgmap v1180: 129 pgs: 129 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 4.5 MiB/s wr, 236 op/s 2024-05-17T09:44:31.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:30 smithi079 ceph-mon[35502]: pgmap v1180: 129 pgs: 129 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 4.5 MiB/s wr, 236 op/s 2024-05-17T09:44:31.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 71978 6.07 MB/sec execute 508 sec latency 93.142 ms 2024-05-17T09:44:32.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 72140 6.07 MB/sec execute 509 sec latency 77.728 ms 2024-05-17T09:44:32.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:32 smithi079 ceph-mon[35502]: pgmap v1181: 129 pgs: 129 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 257 op/s 2024-05-17T09:44:32.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:32 smithi069 ceph-mon[35081]: pgmap v1181: 129 pgs: 129 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 257 op/s 2024-05-17T09:44:32.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:32 smithi019 ceph-mon[31620]: pgmap v1181: 129 pgs: 129 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 257 op/s 2024-05-17T09:44:33.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 72295 6.08 MB/sec execute 510 sec latency 87.154 ms 2024-05-17T09:44:34.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 72436 6.08 MB/sec execute 511 sec latency 116.070 ms 2024-05-17T09:44:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:34 smithi069 ceph-mon[35081]: pgmap v1182: 129 pgs: 129 active+clean; 501 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-05-17T09:44:34.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:34 smithi019 ceph-mon[31620]: pgmap v1182: 129 pgs: 129 active+clean; 501 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-05-17T09:44:35.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:34 smithi079 ceph-mon[35502]: pgmap v1182: 129 pgs: 129 active+clean; 501 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-05-17T09:44:35.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 72614 6.08 MB/sec execute 512 sec latency 80.243 ms 2024-05-17T09:44:36.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 72776 6.08 MB/sec execute 513 sec latency 78.730 ms 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 -- 172.21.15.19:0/2449473248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98102b60 msgr2=0x7fdc98103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 --2- 172.21.15.19:0/2449473248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98102b60 0x7fdc98103860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009860 tx=0x7fdc9402a340 comp rx=0 tx=0).stop 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 -- 172.21.15.19:0/2449473248 shutdown_connections 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 --2- 172.21.15.19:0/2449473248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 0x7fdc9810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 --2- 172.21.15.19:0/2449473248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98103da0 0x7fdc981041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 --2- 172.21.15.19:0/2449473248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98102b60 0x7fdc98103860 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 -- 172.21.15.19:0/2449473248 >> 172.21.15.19:0/2449473248 conn(0x7fdc980fdd50 msgr2=0x7fdc98100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 -- 172.21.15.19:0/2449473248 shutdown_connections 2024-05-17T09:44:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.925+0000 7fdca053f640 1 -- 172.21.15.19:0/2449473248 wait complete. 2024-05-17T09:44:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.926+0000 7fdca053f640 1 Processor -- start 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.926+0000 7fdca053f640 1 -- start start 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdca053f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 0x7fdc9819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdca053f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98103da0 0x7fdc9819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdca053f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 0x7fdc981a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdca053f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9810e510 con 0x7fdc98102b60 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdca053f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9810e390 con 0x7fdc98108aa0 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdca053f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9810e690 con 0x7fdc98103da0 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 0x7fdc9819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 0x7fdc9819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60976/0 (socket says 172.21.15.19:60976) 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 -- 172.21.15.19:0/3136324051 learned_addr learned my addr 172.21.15.19:0/3136324051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 0x7fdc981a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9dada640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98103da0 0x7fdc9819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 -- 172.21.15.19:0/3136324051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98103da0 msgr2=0x7fdc9819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98103da0 0x7fdc9819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 -- 172.21.15.19:0/3136324051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 msgr2=0x7fdc981a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 0x7fdc981a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 -- 172.21.15.19:0/3136324051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc981a1d50 con 0x7fdc98102b60 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 0x7fdc981a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.927+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 0x7fdc9819a210 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdc9402a930 tx=0x7fdc9402ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.928+0000 7fdc8f7fe640 1 -- 172.21.15.19:0/3136324051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94040030 con 0x7fdc98102b60 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.928+0000 7fdc8f7fe640 1 -- 172.21.15.19:0/3136324051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc940305f0 con 0x7fdc98102b60 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.928+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc981a1fe0 con 0x7fdc98102b60 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.928+0000 7fdc8f7fe640 1 -- 172.21.15.19:0/3136324051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94037040 con 0x7fdc98102b60 2024-05-17T09:44:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.928+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981a2390 con 0x7fdc98102b60 2024-05-17T09:44:36.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:36 smithi069 ceph-mon[35081]: pgmap v1183: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 244 op/s 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 Processor -- start 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 -- start start 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be44550920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be44557820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be444c2c30 con 0x55be441af1b0 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55be444b27d0 con 0x55be4418e4d0 2024-05-17T09:44:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdca73f6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55be444b6810 con 0x55be441862f0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdc9eadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdc9eadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60986/0 (socket says 172.21.15.19:60986) 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdc9eadc640 1 -- 172.21.15.19:0/2441070818 learned_addr learned my addr 172.21.15.19:0/2441070818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.931+0000 7fdc9dada640 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be44557820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc9eadc640 1 -- 172.21.15.19:0/2441070818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 msgr2=0x55be44550920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be44550920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc9eadc640 1 -- 172.21.15.19:0/2441070818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 msgr2=0x55be44557820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be44557820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc9eadc640 1 -- 172.21.15.19:0/2441070818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be441c7210 con 0x55be441af1b0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445503a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011830 tx=0x7fdc90011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231555088eae953e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc8cbfe640 1 -- 172.21.15.19:0/2441070818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90011250 con 0x55be441af1b0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc8cbfe640 1 -- 172.21.15.19:0/2441070818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc90003e00 con 0x55be441af1b0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc8cbfe640 1 -- 172.21.15.19:0/2441070818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90004450 con 0x55be441af1b0 2024-05-17T09:44:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.932+0000 7fdc8f7fe640 1 -- 172.21.15.19:0/3136324051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdc94043070 con 0x7fdc98102b60 2024-05-17T09:44:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.933+0000 7fdca73f6740 1 -- 172.21.15.19:0/2441070818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 msgr2=0x55be445503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.933+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445503a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011830 tx=0x7fdc90011cf0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.933+0000 7fdca73f6740 1 -- 172.21.15.19:0/2441070818 shutdown_connections 2024-05-17T09:44:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.933+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be44557820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.933+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be44550920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.933+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2441070818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445503a0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdca73f6740 1 -- 172.21.15.19:0/2441070818 >> 172.21.15.19:0/2441070818 conn(0x55be441a3300 msgr2=0x55be4454bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdca73f6740 1 -- 172.21.15.19:0/2441070818 shutdown_connections 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdca73f6740 1 -- 172.21.15.19:0/2441070818 wait complete. 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdc8f7fe640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc840b27b0 0x7fdc840b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdc9dada640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc840b27b0 0x7fdc840b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdc8f7fe640 1 -- 172.21.15.19:0/3136324051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdc94102330 con 0x7fdc98102b60 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdca73f6740 1 -- start start 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.934+0000 7fdca73f6740 1 Processor -- start 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441862f0 0x55be4455e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9dada640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc840b27b0 0x7fdc840b4cc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55be4454cac0 tx=0x7fdc88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be4418e4d0 0x55be4455e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441af1b0 0x55be44565680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be44198790 con 0x55be4418e4d0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55be4446e890 con 0x55be441862f0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55be445582c0 con 0x55be441af1b0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdca73f6740 10 client.0 _ll_register_callbacks cb 0x55be44471d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441862f0 0x55be4455e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9dada640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be4418e4d0 0x55be4455e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441af1b0 0x55be44565680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441862f0 0x55be4455e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:46334/0 (socket says 172.21.15.19:46334) 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9e2db640 1 -- 172.21.15.19:0/863613524 learned_addr learned my addr 172.21.15.19:0/863613524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 -- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441862f0 msgr2=0x55be4455e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441862f0 0x55be4455e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 -- 172.21.15.19:0/863613524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be4418e4d0 msgr2=0x55be4455e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be4418e4d0 0x55be4455e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 -- 172.21.15.19:0/863613524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be44293620 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441af1b0 0x55be44565680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011830 tx=0x7fdc9000fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2fc11f61a11ce13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.935+0000 7fdc81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdc81ffb640 1 -- 172.21.15.19:0/863613524 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90004e50 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdc81ffb640 1 -- 172.21.15.19:0/863613524 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdc90011050 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc90003e00 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdc81ffb640 1 -- 172.21.15.19:0/863613524 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdc9002f050 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdca73f6740 10 client.28645 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55be445685b0 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.936+0000 7fdc81ffb640 1 -- 172.21.15.19:0/863613524 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc90003e00 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdc81ffb640 1 -- 172.21.15.19:0/863613524 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdc90018020 con 0x55be441af1b0 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdca73f6740 10 client.28645 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdca73f6740 10 client.28645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdca73f6740 20 client.28645 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdca73f6740 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be4416dd80 0x55be4456bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdca73f6740 4 client.28645 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:44:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be444c4b80 con 0x55be4416dd80 2024-05-17T09:44:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.937+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be4416dd80 0x55be4456bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.938+0000 7fdc81ffb640 10 client.28645 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.938+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be4416dd80 0x55be4456bad0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8998e5c703c99935 server_cookie=92c182ea71f0395d in_seq=0 out_seq=0 2024-05-17T09:44:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdc81ffb640 1 -- 172.21.15.19:0/863613524 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55be444c4b80 con 0x55be4416dd80 2024-05-17T09:44:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdc81ffb640 10 client.28645 handle_command_reply: tid=0 2024-05-17T09:44:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 client.28645 shutdown 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441af1b0 msgr2=0x55be44565680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441af1b0 0x55be44565680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011830 tx=0x7fdc9000fc40 comp rx=0 tx=0).stop 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 shutdown_connections 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be4416dd80 0x55be4456bad0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441af1b0 0x55be44565680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be4418e4d0 0x55be4455e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdc81ffb640 0 client.28645 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 --2- 172.21.15.19:0/863613524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441862f0 0x55be4455e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 >> 172.21.15.19:0/863613524 conn(0x55be441a3300 msgr2=0x55be4454ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 shutdown_connections 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 1 -- 172.21.15.19:0/863613524 wait complete. 2024-05-17T09:44:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.940+0000 7fdca73f6740 20 client.28645 trim_cache size 0 max 16384 2024-05-17T09:44:36.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:36 smithi019 ceph-mon[31620]: pgmap v1183: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 244 op/s 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.962+0000 7fdca73f6740 1 Processor -- start 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.962+0000 7fdca73f6740 1 -- start start 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.962+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be4455ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.962+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be4454aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9eadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9dada640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be4454aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdca73f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981ad150 con 0x55be441af1b0 2024-05-17T09:44:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdca73f6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981aaa20 con 0x55be4418e4d0 2024-05-17T09:44:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be4455ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9dada640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be4454aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:46340/0 (socket says 172.21.15.19:46340) 2024-05-17T09:44:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdca73f6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981a8890 con 0x55be441862f0 2024-05-17T09:44:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9dada640 1 -- 172.21.15.19:0/1642332809 learned_addr learned my addr 172.21.15.19:0/1642332809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 -- 172.21.15.19:0/1642332809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 msgr2=0x55be4454aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be4454aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 -- 172.21.15.19:0/1642332809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 msgr2=0x55be445655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 -- 172.21.15.19:0/1642332809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be4454c8c0 con 0x55be441862f0 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.963+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be4455ce70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009830 tx=0x7fdc94005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8558e11d071b599 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.964+0000 7fdc83fff640 1 -- 172.21.15.19:0/1642332809 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc940307a0 con 0x55be441862f0 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.964+0000 7fdc83fff640 1 -- 172.21.15.19:0/1642332809 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc940c9140 con 0x55be441862f0 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.964+0000 7fdc83fff640 1 -- 172.21.15.19:0/1642332809 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc940d2400 con 0x55be441862f0 2024-05-17T09:44:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 -- 172.21.15.19:0/1642332809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 msgr2=0x55be4455ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be4455ce70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009830 tx=0x7fdc94005f90 comp rx=0 tx=0).stop 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 -- 172.21.15.19:0/1642332809 shutdown_connections 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441af1b0 0x55be445655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be4418e4d0 0x55be4454aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 --2- 172.21.15.19:0/1642332809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be441862f0 0x55be4455ce70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 -- 172.21.15.19:0/1642332809 >> 172.21.15.19:0/1642332809 conn(0x55be441a3300 msgr2=0x55be4454a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 -- 172.21.15.19:0/1642332809 shutdown_connections 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 -- 172.21.15.19:0/1642332809 wait complete. 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 -- start start 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.965+0000 7fdca73f6740 1 Processor -- start 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 0x55be44562fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be4418e4d0 0x55be44563520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441af1b0 0x55be44564450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981a8ee0 con 0x55be441862f0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981aa350 con 0x55be441af1b0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981a95b0 con 0x55be4418e4d0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdca73f6740 10 client.0 _ll_register_callbacks cb 0x7fdc900116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 0x55be44562fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 0x55be44562fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32778/0 (socket says 172.21.15.19:32778) 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 -- 172.21.15.19:0/2113683163 learned_addr learned my addr 172.21.15.19:0/2113683163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9dada640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be4418e4d0 0x55be44563520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9eadc640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441af1b0 0x55be44564450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 -- 172.21.15.19:0/2113683163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be4418e4d0 msgr2=0x55be44563520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be4418e4d0 0x55be44563520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 -- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441af1b0 msgr2=0x55be44564450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441af1b0 0x55be44564450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 -- 172.21.15.19:0/2113683163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be44564a10 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.966+0000 7fdc9e2db640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 0x55be44562fe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fdc940328a0 tx=0x7fdc940c9630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bee4d29846865d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdc81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdc81ffb640 1 -- 172.21.15.19:0/2113683163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94102160 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc940c9140 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdc81ffb640 1 -- 172.21.15.19:0/2113683163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fdc940b1110 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdc81ffb640 1 -- 172.21.15.19:0/2113683163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdc940c9d30 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdca73f6740 10 client.19185 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55be44565cf0 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdc81ffb640 1 -- 172.21.15.19:0/2113683163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc940c9140 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdc81ffb640 1 -- 172.21.15.19:0/2113683163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdc9404d060 con 0x55be441862f0 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.967+0000 7fdca73f6740 10 client.19185 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdca73f6740 10 client.19185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdca73f6740 20 client.19185 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be441431c0 0x55be445d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdca73f6740 4 client.19185 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:44:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be444be360 con 0x55be441431c0 2024-05-17T09:44:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdc9dada640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be441431c0 0x55be445d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdc81ffb640 10 client.19185 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:36.968+0000 7fdc9dada640 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be441431c0 0x55be445d6370 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98088e8a536c7b07 server_cookie=b1059cc20ad958d9 in_seq=0 out_seq=0 2024-05-17T09:44:37.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:36 smithi079 ceph-mon[35502]: pgmap v1183: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 244 op/s 2024-05-17T09:44:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdc81ffb640 1 -- 172.21.15.19:0/2113683163 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be444be360 con 0x55be441431c0 2024-05-17T09:44:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdc81ffb640 10 client.19185 handle_command_reply: tid=0 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdca73f6740 1 client.19185 shutdown 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 msgr2=0x55be44562fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 0x55be44562fe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fdc940328a0 tx=0x7fdc940c9630 comp rx=0 tx=0).stop 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 shutdown_connections 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.071+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55be441431c0 0x55be445d6370 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdc81ffb640 0 client.19185 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55be441af1b0 0x55be44564450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55be4418e4d0 0x55be44563520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 1 --2- 172.21.15.19:0/2113683163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be441862f0 0x55be44562fe0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 >> 172.21.15.19:0/2113683163 conn(0x55be441a3300 msgr2=0x55be4454a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 shutdown_connections 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 1 -- 172.21.15.19:0/2113683163 wait complete. 2024-05-17T09:44:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.072+0000 7fdca73f6740 20 client.19185 trim_cache size 0 max 16384 2024-05-17T09:44:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:44:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:44:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc840b27b0 msgr2=0x7fdc840b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc840b27b0 0x7fdc840b4cc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55be4454cac0 tx=0x7fdc88009260 comp rx=0 tx=0).stop 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 msgr2=0x7fdc9819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 0x7fdc9819a210 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdc9402a930 tx=0x7fdc9402ad70 comp rx=0 tx=0).stop 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 shutdown_connections 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc98108aa0 0x7fdc981a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc98103da0 0x7fdc9819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fdc840b27b0 0x7fdc840b4cc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 --2- 172.21.15.19:0/3136324051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc98102b60 0x7fdc9819a210 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 >> 172.21.15.19:0/3136324051 conn(0x7fdc980fdd50 msgr2=0x7fdc9806b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.073+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 shutdown_connections 2024-05-17T09:44:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:37.074+0000 7fdca053f640 1 -- 172.21.15.19:0/3136324051 wait complete. 2024-05-17T09:44:37.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 72920 6.08 MB/sec execute 514 sec latency 129.781 ms 2024-05-17T09:44:38.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 73095 6.09 MB/sec execute 515 sec latency 110.596 ms 2024-05-17T09:44:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:38 smithi069 ceph-mon[35081]: pgmap v1184: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 299 op/s 2024-05-17T09:44:38.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:38 smithi019 ceph-mon[31620]: pgmap v1184: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 299 op/s 2024-05-17T09:44:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:38 smithi079 ceph-mon[35502]: pgmap v1184: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 299 op/s 2024-05-17T09:44:39.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 73237 6.09 MB/sec execute 516 sec latency 77.055 ms 2024-05-17T09:44:40.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 73430 6.10 MB/sec execute 517 sec latency 64.948 ms 2024-05-17T09:44:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:40 smithi069 ceph-mon[35081]: pgmap v1185: 129 pgs: 129 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-05-17T09:44:40.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:40 smithi019 ceph-mon[31620]: pgmap v1185: 129 pgs: 129 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-05-17T09:44:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:40 smithi079 ceph-mon[35502]: pgmap v1185: 129 pgs: 129 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-05-17T09:44:41.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 73619 6.12 MB/sec execute 518 sec latency 76.239 ms 2024-05-17T09:44:42.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 73823 6.13 MB/sec execute 519 sec latency 70.009 ms 2024-05-17T09:44:42.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:42 smithi069 ceph-mon[35081]: pgmap v1186: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-05-17T09:44:42.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:42 smithi019 ceph-mon[31620]: pgmap v1186: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-05-17T09:44:43.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:42 smithi079 ceph-mon[35502]: pgmap v1186: 129 pgs: 129 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-05-17T09:44:43.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 73980 6.14 MB/sec execute 520 sec latency 104.151 ms 2024-05-17T09:44:44.601 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74107 6.13 MB/sec execute 521 sec latency 112.936 ms 2024-05-17T09:44:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:44 smithi069 ceph-mon[35081]: pgmap v1187: 129 pgs: 129 active+clean; 501 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.6 MiB/s wr, 287 op/s 2024-05-17T09:44:44.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:44 smithi019 ceph-mon[31620]: pgmap v1187: 129 pgs: 129 active+clean; 501 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.6 MiB/s wr, 287 op/s 2024-05-17T09:44:45.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:44 smithi079 ceph-mon[35502]: pgmap v1187: 129 pgs: 129 active+clean; 501 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 3.6 MiB/s wr, 287 op/s 2024-05-17T09:44:45.600 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74261 6.14 MB/sec execute 522 sec latency 78.308 ms 2024-05-17T09:44:46.602 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74411 6.15 MB/sec execute 523 sec latency 76.230 ms 2024-05-17T09:44:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:46 smithi069 ceph-mon[35081]: pgmap v1188: 129 pgs: 129 active+clean; 505 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-05-17T09:44:46.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:46 smithi019 ceph-mon[31620]: pgmap v1188: 129 pgs: 129 active+clean; 505 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-05-17T09:44:47.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:46 smithi079 ceph-mon[35502]: pgmap v1188: 129 pgs: 129 active+clean; 505 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 3.7 MiB/s wr, 273 op/s 2024-05-17T09:44:47.602 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74574 6.16 MB/sec execute 524 sec latency 103.571 ms 2024-05-17T09:44:48.602 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74707 6.17 MB/sec execute 525 sec latency 119.257 ms 2024-05-17T09:44:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:48 smithi069 ceph-mon[35081]: pgmap v1189: 129 pgs: 129 active+clean; 512 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-05-17T09:44:48.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:48 smithi019 ceph-mon[31620]: pgmap v1189: 129 pgs: 129 active+clean; 512 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-05-17T09:44:49.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:48 smithi079 ceph-mon[35502]: pgmap v1189: 129 pgs: 129 active+clean; 512 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-05-17T09:44:49.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74869 6.17 MB/sec execute 526 sec latency 69.914 ms 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 -- 172.21.15.19:0/3696112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c103a80 msgr2=0x7f9f3c104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 --2- 172.21.15.19:0/3696112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c103a80 0x7f9f3c104780 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009900 tx=0x7f9f2c02a400 comp rx=0 tx=0).stop 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 -- 172.21.15.19:0/3696112 shutdown_connections 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 --2- 172.21.15.19:0/3696112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 --2- 172.21.15.19:0/3696112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f3c104cc0 0x7f9f3c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 --2- 172.21.15.19:0/3696112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c103a80 0x7f9f3c104780 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 -- 172.21.15.19:0/3696112 >> 172.21.15.19:0/3696112 conn(0x7f9f3c0fec70 msgr2=0x7f9f3c101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 -- 172.21.15.19:0/3696112 shutdown_connections 2024-05-17T09:44:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.235+0000 7f9f4109e640 1 -- 172.21.15.19:0/3696112 wait complete. 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 Processor -- start 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 -- start start 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f3c103a80 0x7f9f3c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c104cc0 0x7f9f3c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0764d0 con 0x7f9f3c104cc0 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c076350 con 0x7f9f3c1099c0 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.236+0000 7f9f4109e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c076650 con 0x7f9f3c103a80 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c1a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:39824/0 (socket says 172.21.15.19:39824) 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 -- 172.21.15.19:0/831974439 learned_addr learned my addr 172.21.15.19:0/831974439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c104cc0 0x7f9f3c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 -- 172.21.15.19:0/831974439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f3c103a80 msgr2=0x7f9f3c19c2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f3c103a80 0x7f9f3c19c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 -- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c104cc0 msgr2=0x7f9f3c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c104cc0 0x7f9f3c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 -- 172.21.15.19:0/831974439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c1a3e20 con 0x7f9f3c1099c0 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c104cc0 0x7f9f3c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.237+0000 7f9f4089d640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c1a37b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400eb10 tx=0x7f9f3400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.238+0000 7f9f397fa640 1 -- 172.21.15.19:0/831974439 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34004b20 con 0x7f9f3c1099c0 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.238+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c1a4110 con 0x7f9f3c1099c0 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.238+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1a4660 con 0x7f9f3c1099c0 2024-05-17T09:44:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.238+0000 7f9f397fa640 1 -- 172.21.15.19:0/831974439 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f34004cc0 con 0x7f9f3c1099c0 2024-05-17T09:44:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.238+0000 7f9f397fa640 1 -- 172.21.15.19:0/831974439 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34010690 con 0x7f9f3c1099c0 2024-05-17T09:44:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.241+0000 7f9f491bb740 1 Processor -- start 2024-05-17T09:44:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.241+0000 7f9f397fa640 1 -- 172.21.15.19:0/831974439 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9f34010830 con 0x7f9f3c1099c0 2024-05-17T09:44:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.241+0000 7f9f491bb740 1 -- start start 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.241+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x560666160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x5606661679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f491bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5606660d2c30 con 0x560665dbf1b0 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f491bb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5606660c27d0 con 0x560665d962f0 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f491bb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5606660c6810 con 0x560665d9e4d0 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x560666160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x560666160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55556/0 (socket says 172.21.15.19:55556) 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:39838/0 (socket says 172.21.15.19:39838) 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3b7fe640 1 -- 172.21.15.19:0/4111856982 learned_addr learned my addr 172.21.15.19:0/4111856982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x5606661679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 -- 172.21.15.19:0/4111856982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 msgr2=0x5606661679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x5606661679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 -- 172.21.15.19:0/4111856982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 msgr2=0x560666160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x560666160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3bfff640 1 -- 172.21.15.19:0/4111856982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560665dd7210 con 0x560665d962f0 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x560666160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.242+0000 7f9f397fa640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9f240b2920 0x7f9f240b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.243+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666160ac0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c02a920 tx=0x7f9f2c031500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90d42d8270d9ba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.243+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9f240b2920 0x7f9f240b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.243+0000 7f9f397fa640 1 -- 172.21.15.19:0/831974439 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9f340e2ab0 con 0x7f9f3c1099c0 2024-05-17T09:44:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.243+0000 7f9f227fc640 1 -- 172.21.15.19:0/4111856982 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c03f070 con 0x560665d962f0 2024-05-17T09:44:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.243+0000 7f9f227fc640 1 -- 172.21.15.19:0/4111856982 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c004040 con 0x560665d962f0 2024-05-17T09:44:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.243+0000 7f9f227fc640 1 -- 172.21.15.19:0/4111856982 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c035030 con 0x560665d962f0 2024-05-17T09:44:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 -- 172.21.15.19:0/4111856982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 msgr2=0x560666160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9f240b2920 0x7f9f240b4e30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x560666161710 tx=0x7f9f30009d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666160ac0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c02a920 tx=0x7f9f2c031500 comp rx=0 tx=0).stop 2024-05-17T09:44:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 -- 172.21.15.19:0/4111856982 shutdown_connections 2024-05-17T09:44:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x5606661679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666160ac0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 --2- 172.21.15.19:0/4111856982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x560666160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 -- 172.21.15.19:0/4111856982 >> 172.21.15.19:0/4111856982 conn(0x560665db3300 msgr2=0x56066615bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 -- 172.21.15.19:0/4111856982 shutdown_connections 2024-05-17T09:44:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.244+0000 7f9f491bb740 1 -- 172.21.15.19:0/4111856982 wait complete. 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 -- start start 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 Processor -- start 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665d962f0 0x5606661725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666172af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665dbf1b0 0x560666179a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560665da8790 con 0x560665d962f0 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56066607e890 con 0x560665d9e4d0 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5606661683b0 con 0x560665dbf1b0 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665d962f0 0x5606661725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f491bb740 10 client.0 _ll_register_callbacks cb 0x560666081d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666172af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665d962f0 0x5606661725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55566/0 (socket says 172.21.15.19:55566) 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f4089d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665dbf1b0 0x560666179a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f3bfff640 1 -- 172.21.15.19:0/1865186504 learned_addr learned my addr 172.21.15.19:0/1865186504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.245+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666172af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:39842/0 (socket says 172.21.15.19:39842) 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f4089d640 1 -- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 msgr2=0x560666172af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f4089d640 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666172af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f4089d640 1 -- 172.21.15.19:0/1865186504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665d962f0 msgr2=0x5606661725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f4089d640 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665d962f0 0x5606661725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f4089d640 1 -- 172.21.15.19:0/1865186504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560665ea3620 con 0x560665dbf1b0 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f4089d640 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665dbf1b0 0x560666179a50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005860 tx=0x7f9f340a9550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f677ad627c5554ec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/1865186504 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f340c0030 con 0x560665dbf1b0 2024-05-17T09:44:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/1865186504 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9f340a6310 con 0x560665dbf1b0 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c004040 con 0x560665dbf1b0 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/1865186504 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9f340ae0d0 con 0x560665dbf1b0 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f491bb740 10 client.28660 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56066617ca50 con 0x560665dbf1b0 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.246+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/1865186504 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f340b2b80 con 0x560665dbf1b0 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/1865186504 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9f340a5690 con 0x560665dbf1b0 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f491bb740 10 client.28660 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f491bb740 10 client.28660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f491bb740 20 client.28660 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d7dd80 0x5606661801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f491bb740 4 client.28660 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9f2c004040 con 0x560665d7dd80 2024-05-17T09:44:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d7dd80 0x5606661801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.247+0000 7f9f1b7fe640 10 client.28660 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.248+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d7dd80 0x5606661801e0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db853d9ba30e14c2 server_cookie=ea632675fd5b42ab in_seq=0 out_seq=0 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/1865186504 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f9f2c004040 con 0x560665d7dd80 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f1b7fe640 10 client.28660 handle_command_reply: tid=0 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 client.28660 shutdown 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665dbf1b0 msgr2=0x560666179a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665dbf1b0 0x560666179a50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005860 tx=0x7f9f340a9550 comp rx=0 tx=0).stop 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 shutdown_connections 2024-05-17T09:44:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d7dd80 0x5606661801e0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665dbf1b0 0x560666179a50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666172af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1865186504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665d962f0 0x5606661725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 >> 172.21.15.19:0/1865186504 conn(0x560665db3300 msgr2=0x56066615ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f1b7fe640 0 client.28660 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 shutdown_connections 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.249+0000 7f9f491bb740 1 -- 172.21.15.19:0/1865186504 wait complete. 2024-05-17T09:44:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.250+0000 7f9f491bb740 20 client.28660 trim_cache size 0 max 16384 2024-05-17T09:44:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 Processor -- start 2024-05-17T09:44:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 -- start start 2024-05-17T09:44:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d962f0 0x560666164940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666179970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x56066615f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28012500 con 0x560665dbf1b0 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2800fdd0 con 0x560665d9e4d0 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.272+0000 7f9f491bb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2800dc00 con 0x560665d962f0 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f4089d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x56066615f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666179970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d962f0 0x560666164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666179970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:39856/0 (socket says 172.21.15.19:39856) 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f4089d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x56066615f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55572/0 (socket says 172.21.15.19:55572) 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3b7fe640 1 -- 172.21.15.19:0/1139714718 learned_addr learned my addr 172.21.15.19:0/1139714718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 -- 172.21.15.19:0/1139714718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 msgr2=0x560666179970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666179970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 -- 172.21.15.19:0/1139714718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 msgr2=0x56066615f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x56066615f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 -- 172.21.15.19:0/1139714718 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560666180590 con 0x560665d962f0 2024-05-17T09:44:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f4089d640 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x56066615f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:44:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.273+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d962f0 0x560666164940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c006010 tx=0x7f9f2c005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896921a6049e047a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.274+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1139714718 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c054070 con 0x560665d962f0 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.274+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1139714718 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c031810 con 0x560665d962f0 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.274+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1139714718 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c03f040 con 0x560665d962f0 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 -- 172.21.15.19:0/1139714718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d962f0 msgr2=0x560666164940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d962f0 0x560666164940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c006010 tx=0x7f9f2c005890 comp rx=0 tx=0).stop 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 -- 172.21.15.19:0/1139714718 shutdown_connections 2024-05-17T09:44:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x56066615f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d9e4d0 0x560666179970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 --2- 172.21.15.19:0/1139714718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d962f0 0x560666164940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 -- 172.21.15.19:0/1139714718 >> 172.21.15.19:0/1139714718 conn(0x560665db3300 msgr2=0x5606661604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 -- 172.21.15.19:0/1139714718 shutdown_connections 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.275+0000 7f9f491bb740 1 -- 172.21.15.19:0/1139714718 wait complete. 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.276+0000 7f9f491bb740 1 -- start start 2024-05-17T09:44:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.276+0000 7f9f491bb740 1 Processor -- start 2024-05-17T09:44:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.276+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666178bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.276+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x560666179130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.276+0000 7f9f491bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x5606661e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666178bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x560666179130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666178bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:39862/0 (socket says 172.21.15.19:39862) 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x560666179130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:49968/0 (socket says 172.21.15.19:49968) 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f491bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2800e290 con 0x560665dbf1b0 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f491bb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2800f700 con 0x560665d962f0 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f491bb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2800e960 con 0x560665d9e4d0 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 -- 172.21.15.19:0/367605188 learned_addr learned my addr 172.21.15.19:0/367605188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:44:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f491bb740 10 client.0 _ll_register_callbacks cb 0x7f9f340a9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f4089d640 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x5606661e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 -- 172.21.15.19:0/367605188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 msgr2=0x560666179130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x560666179130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 -- 172.21.15.19:0/367605188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 msgr2=0x5606661e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x5606661e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 -- 172.21.15.19:0/367605188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606661e6eb0 con 0x560665d962f0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f3bfff640 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666178bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005f70 tx=0x7f9f2c009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3665f143e42c53b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.277+0000 7f9f1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/367605188 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c03f050 con 0x560665d962f0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/367605188 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9f2c04f5c0 con 0x560665d962f0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c031810 con 0x560665d962f0 2024-05-17T09:44:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/367605188 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c066050 con 0x560665d962f0 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f491bb740 10 client.28694 fetch_fsmap learned FSMap version 39 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5606661e8100 con 0x560665d962f0 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/367605188 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c05d070 con 0x560665d962f0 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.278+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/367605188 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9f2c057070 con 0x560665d962f0 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f491bb740 10 client.28694 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f491bb740 10 client.28694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f491bb740 20 client.28694 populate_metadata read hostname 'smithi019' 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f491bb740 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d531c0 0x5606661ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f491bb740 4 client.28694 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9f2c031810 con 0x560665d531c0 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d531c0 0x5606661ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f1b7fe640 10 client.28694 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.279+0000 7f9f3b7fe640 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d531c0 0x5606661ea5c0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9047de5ffb93c74a server_cookie=8f49cbccaea0ed22 in_seq=0 out_seq=0 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.389+0000 7f9f1b7fe640 1 -- 172.21.15.19:0/367605188 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f9f2c031810 con 0x560665d531c0 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f1b7fe640 10 client.28694 handle_command_reply: tid=0 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 client.28694 shutdown 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 msgr2=0x560666178bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666178bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005f70 tx=0x7f9f2c009300 comp rx=0 tx=0).stop 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 shutdown_connections 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x560665d531c0 0x5606661ea5c0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f1b7fe640 0 client.28694 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560665dbf1b0 0x5606661e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560665d9e4d0 0x560666179130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 --2- 172.21.15.19:0/367605188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560665d962f0 0x560666178bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 >> 172.21.15.19:0/367605188 conn(0x560665db3300 msgr2=0x560666163930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 shutdown_connections 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 1 -- 172.21.15.19:0/367605188 wait complete. 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.390+0000 7f9f491bb740 20 client.28694 trim_cache size 0 max 16384 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:44:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.391+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9f240b2920 msgr2=0x7f9f240b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.391+0000 7f9f4109e640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9f240b2920 0x7f9f240b4e30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x560666161710 tx=0x7f9f30009d20 comp rx=0 tx=0).stop 2024-05-17T09:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 msgr2=0x7f9f3c1a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c1a37b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400eb10 tx=0x7f9f3400c2d0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 shutdown_connections 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f3c1099c0 0x7f9f3c1a37b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f9f240b2920 0x7f9f240b4e30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c104cc0 0x7f9f3c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 --2- 172.21.15.19:0/831974439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f3c103a80 0x7f9f3c19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 >> 172.21.15.19:0/831974439 conn(0x7f9f3c0fec70 msgr2=0x7f9f3c101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 shutdown_connections 2024-05-17T09:44:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:44:50.392+0000 7f9f4109e640 1 -- 172.21.15.19:0/831974439 wait complete. 2024-05-17T09:44:50.602 INFO:tasks.workunit.client.0.smithi019.stdout: 10 74991 6.16 MB/sec execute 527 sec latency 137.330 ms 2024-05-17T09:44:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:50 smithi069 ceph-mon[35081]: pgmap v1190: 129 pgs: 129 active+clean; 514 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-05-17T09:44:50.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:50 smithi019 ceph-mon[31620]: pgmap v1190: 129 pgs: 129 active+clean; 514 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-05-17T09:44:51.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:50 smithi079 ceph-mon[35502]: pgmap v1190: 129 pgs: 129 active+clean; 514 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.2 MiB/s wr, 248 op/s 2024-05-17T09:44:51.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 75137 6.16 MB/sec execute 528 sec latency 90.768 ms 2024-05-17T09:44:52.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 75272 6.16 MB/sec execute 529 sec latency 87.715 ms 2024-05-17T09:44:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:52 smithi069 ceph-mon[35081]: pgmap v1191: 129 pgs: 129 active+clean; 518 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 247 op/s 2024-05-17T09:44:52.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:52 smithi019 ceph-mon[31620]: pgmap v1191: 129 pgs: 129 active+clean; 518 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 247 op/s 2024-05-17T09:44:53.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:52 smithi079 ceph-mon[35502]: pgmap v1191: 129 pgs: 129 active+clean; 518 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 247 op/s 2024-05-17T09:44:53.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 75420 6.16 MB/sec execute 530 sec latency 119.413 ms 2024-05-17T09:44:54.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 75581 6.18 MB/sec execute 531 sec latency 82.666 ms 2024-05-17T09:44:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:54 smithi069 ceph-mon[35081]: pgmap v1192: 129 pgs: 129 active+clean; 517 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.4 MiB/s wr, 243 op/s 2024-05-17T09:44:54.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:54 smithi019 ceph-mon[31620]: pgmap v1192: 129 pgs: 129 active+clean; 517 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.4 MiB/s wr, 243 op/s 2024-05-17T09:44:55.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:54 smithi079 ceph-mon[35502]: pgmap v1192: 129 pgs: 129 active+clean; 517 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.4 MiB/s wr, 243 op/s 2024-05-17T09:44:55.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 75738 6.19 MB/sec execute 532 sec latency 77.485 ms 2024-05-17T09:44:56.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 75894 6.19 MB/sec execute 533 sec latency 81.045 ms 2024-05-17T09:44:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:56 smithi069 ceph-mon[35081]: pgmap v1193: 129 pgs: 129 active+clean; 515 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-17T09:44:56.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:56 smithi019 ceph-mon[31620]: pgmap v1193: 129 pgs: 129 active+clean; 515 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-17T09:44:57.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:56 smithi079 ceph-mon[35502]: pgmap v1193: 129 pgs: 129 active+clean; 515 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-05-17T09:44:57.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76056 6.19 MB/sec execute 534 sec latency 105.784 ms 2024-05-17T09:44:58.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76193 6.18 MB/sec execute 535 sec latency 82.888 ms 2024-05-17T09:44:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:44:58 smithi069 ceph-mon[35081]: pgmap v1194: 129 pgs: 129 active+clean; 511 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 291 op/s 2024-05-17T09:44:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:44:58 smithi019 ceph-mon[31620]: pgmap v1194: 129 pgs: 129 active+clean; 511 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 291 op/s 2024-05-17T09:44:59.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:44:58 smithi079 ceph-mon[35502]: pgmap v1194: 129 pgs: 129 active+clean; 511 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 291 op/s 2024-05-17T09:44:59.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76338 6.17 MB/sec execute 536 sec latency 77.886 ms 2024-05-17T09:45:00.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76492 6.17 MB/sec execute 537 sec latency 99.815 ms 2024-05-17T09:45:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:00 smithi069 ceph-mon[35081]: pgmap v1195: 129 pgs: 129 active+clean; 512 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-05-17T09:45:00.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:00 smithi019 ceph-mon[31620]: pgmap v1195: 129 pgs: 129 active+clean; 512 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-05-17T09:45:01.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:00 smithi079 ceph-mon[35502]: pgmap v1195: 129 pgs: 129 active+clean; 512 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-05-17T09:45:01.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76634 6.17 MB/sec execute 538 sec latency 80.953 ms 2024-05-17T09:45:02.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76787 6.18 MB/sec execute 539 sec latency 70.764 ms 2024-05-17T09:45:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:02 smithi069 ceph-mon[35081]: pgmap v1196: 129 pgs: 129 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-05-17T09:45:02.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:02 smithi019 ceph-mon[31620]: pgmap v1196: 129 pgs: 129 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-05-17T09:45:03.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:02 smithi079 ceph-mon[35502]: pgmap v1196: 129 pgs: 129 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 -- 172.21.15.19:0/890090992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 msgr2=0x7f853410aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 --2- 172.21.15.19:0/890090992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f853410aea0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f852c009ca0 tx=0x7f852c02a8c0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 -- 172.21.15.19:0/890090992 shutdown_connections 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 --2- 172.21.15.19:0/890090992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f853410aea0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 --2- 172.21.15.19:0/890090992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534103d80 0x7f8534104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 --2- 172.21.15.19:0/890090992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534102b40 0x7f8534103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 -- 172.21.15.19:0/890090992 >> 172.21.15.19:0/890090992 conn(0x7f85340fdd30 msgr2=0x7f85341001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 -- 172.21.15.19:0/890090992 shutdown_connections 2024-05-17T09:45:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.549+0000 7f853997d640 1 -- 172.21.15.19:0/890090992 wait complete. 2024-05-17T09:45:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 Processor -- start 2024-05-17T09:45:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 -- start start 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534102b40 0x7f853406b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534103d80 0x7f853419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f85341a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f853410e570 con 0x7f8534108a80 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f853410e3f0 con 0x7f8534103d80 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.550+0000 7f853997d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f853410e6f0 con 0x7f8534102b40 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f85337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f85341a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f85337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f85341a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55232/0 (socket says 172.21.15.19:55232) 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534102b40 0x7f853406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f85337fe640 1 -- 172.21.15.19:0/460811839 learned_addr learned my addr 172.21.15.19:0/460811839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f85327fc640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534103d80 0x7f853419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 -- 172.21.15.19:0/460811839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534103d80 msgr2=0x7f853419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534103d80 0x7f853419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 -- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 msgr2=0x7f85341a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f85341a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 -- 172.21.15.19:0/460811839 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85341a1c80 con 0x7f8534102b40 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f85337fe640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f85341a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f85327fc640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534103d80 0x7f853419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f8532ffd640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534102b40 0x7f853406b120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8534102410 tx=0x7f852400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.551+0000 7f853897b640 1 -- 172.21.15.19:0/460811839 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400cc60 con 0x7f8534102b40 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.552+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8534198ce0 con 0x7f8534102b40 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.552+0000 7f853897b640 1 -- 172.21.15.19:0/460811839 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85240044b0 con 0x7f8534102b40 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.552+0000 7f853897b640 1 -- 172.21.15.19:0/460811839 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85240035f0 con 0x7f8534102b40 2024-05-17T09:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.552+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8534199280 con 0x7f8534102b40 2024-05-17T09:45:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.555+0000 7f853897b640 1 -- 172.21.15.19:0/460811839 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8524022020 con 0x7f8534102b40 2024-05-17T09:45:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.555+0000 7f8540834740 1 Processor -- start 2024-05-17T09:45:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.555+0000 7f8540834740 1 -- start start 2024-05-17T09:45:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8540834740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 0x5632b8b113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8540834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b11920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8540834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8540834740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5632b8a83c30 con 0x5632b87701b0 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8540834740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5632b8a737d0 con 0x5632b87472f0 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8540834740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5632b8a77810 con 0x5632b874f4d0 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b11920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 0x5632b8b113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 0x5632b8b113a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55240/0 (socket says 172.21.15.19:55240) 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 -- 172.21.15.19:0/1849440660 learned_addr learned my addr 172.21.15.19:0/1849440660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85327fc640 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 -- 172.21.15.19:0/1849440660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 msgr2=0x5632b8b18820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b18820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f853897b640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f851c0b28a0 0x7f851c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 -- 172.21.15.19:0/1849440660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 msgr2=0x5632b8b11920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b11920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.556+0000 7f85337fe640 1 -- 172.21.15.19:0/1849440660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b8788210 con 0x5632b87701b0 2024-05-17T09:45:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f8532ffd640 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b11920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f85337fe640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f851c0b28a0 0x7f851c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f853897b640 1 -- 172.21.15.19:0/460811839 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f85240e65d0 con 0x7f8534102b40 2024-05-17T09:45:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f85337fe640 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 0x5632b8b113a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f852c02aeb0 tx=0x7f852c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05efb42360abce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f8519ffb640 1 -- 172.21.15.19:0/1849440660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c003970 con 0x5632b87701b0 2024-05-17T09:45:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f8519ffb640 1 -- 172.21.15.19:0/1849440660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f852c036030 con 0x5632b87701b0 2024-05-17T09:45:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.557+0000 7f8519ffb640 1 -- 172.21.15.19:0/1849440660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c030820 con 0x5632b87701b0 2024-05-17T09:45:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.560+0000 7f85337fe640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f851c0b28a0 0x7f851c0b4db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f852c04a000 tx=0x7f852c030b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.560+0000 7f8540834740 1 -- 172.21.15.19:0/1849440660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 msgr2=0x5632b8b113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.560+0000 7f8540834740 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 0x5632b8b113a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f852c02aeb0 tx=0x7f852c009a20 comp rx=0 tx=0).stop 2024-05-17T09:45:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 -- 172.21.15.19:0/1849440660 shutdown_connections 2024-05-17T09:45:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b18820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b11920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 --2- 172.21.15.19:0/1849440660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87701b0 0x5632b8b113a0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 -- 172.21.15.19:0/1849440660 >> 172.21.15.19:0/1849440660 conn(0x5632b8764300 msgr2=0x5632b8b0cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 -- 172.21.15.19:0/1849440660 shutdown_connections 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.561+0000 7f8540834740 1 -- 172.21.15.19:0/1849440660 wait complete. 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.562+0000 7f8540834740 1 -- start start 2024-05-17T09:45:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.562+0000 7f8540834740 1 Processor -- start 2024-05-17T09:45:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b87472f0 0x5632b8b1abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b874f4d0 0x5632b8b1b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 0x5632b8b22070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5632b8759790 con 0x5632b874f4d0 2024-05-17T09:45:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5632b8a2f890 con 0x5632b87701b0 2024-05-17T09:45:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5632b8b190f0 con 0x5632b87472f0 2024-05-17T09:45:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 0x5632b8b22070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b87472f0 0x5632b8b1abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85327fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b874f4d0 0x5632b8b1b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 0x5632b8b22070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36582/0 (socket says 172.21.15.19:36582) 2024-05-17T09:45:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 -- 172.21.15.19:0/432547003 learned_addr learned my addr 172.21.15.19:0/432547003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f8540834740 10 client.0 _ll_register_callbacks cb 0x5632b8a32d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 -- 172.21.15.19:0/432547003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b87472f0 msgr2=0x5632b8b1abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b87472f0 0x5632b8b1abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 -- 172.21.15.19:0/432547003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b874f4d0 msgr2=0x5632b8b1b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b874f4d0 0x5632b8b1b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.563+0000 7f85337fe640 1 -- 172.21.15.19:0/432547003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b8854620 con 0x5632b87701b0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f8532ffd640 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b87472f0 0x5632b8b1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f85337fe640 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 0x5632b8b22070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f852c0319e0 tx=0x7f852c041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982dd933f368ccac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f8512ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f8512ffd640 1 -- 172.21.15.19:0/432547003 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c061060 con 0x5632b87701b0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f8512ffd640 1 -- 172.21.15.19:0/432547003 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f852c04c050 con 0x5632b87701b0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852c036030 con 0x5632b87701b0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.564+0000 7f8512ffd640 1 -- 172.21.15.19:0/432547003 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f852c031e20 con 0x5632b87701b0 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 10 client.28706 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5632b8b24d50 con 0x5632b87701b0 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8512ffd640 1 -- 172.21.15.19:0/432547003 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f852c036030 con 0x5632b87701b0 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8512ffd640 1 -- 172.21.15.19:0/432547003 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f852c052910 con 0x5632b87701b0 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 10 client.28706 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 10 client.28706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 20 client.28706 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b872ed80 0x5632b8b28140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 4 client.28706 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:45:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.565+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632b8a85b80 con 0x5632b872ed80 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.566+0000 7f8532ffd640 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b872ed80 0x5632b8b28140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.566+0000 7f8512ffd640 10 client.28706 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.566+0000 7f8532ffd640 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b872ed80 0x5632b8b28140 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e559af61b0af5413 server_cookie=687279294518b43b in_seq=0 out_seq=0 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.567+0000 7f8512ffd640 1 -- 172.21.15.19:0/432547003 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5632b8a85b80 con 0x5632b872ed80 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.567+0000 7f8512ffd640 10 client.28706 handle_command_reply: tid=0 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.567+0000 7f8540834740 1 client.28706 shutdown 2024-05-17T09:45:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.567+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 msgr2=0x5632b8b22070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.567+0000 7f8540834740 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 0x5632b8b22070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f852c0319e0 tx=0x7f852c041000 comp rx=0 tx=0).stop 2024-05-17T09:45:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.568+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 shutdown_connections 2024-05-17T09:45:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.568+0000 7f8540834740 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b872ed80 0x5632b8b28140 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8512ffd640 0 client.28706 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87701b0 0x5632b8b22070 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b874f4d0 0x5632b8b1b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 1 --2- 172.21.15.19:0/432547003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b87472f0 0x5632b8b1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 >> 172.21.15.19:0/432547003 conn(0x5632b8764300 msgr2=0x5632b8b0bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 shutdown_connections 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 1 -- 172.21.15.19:0/432547003 wait complete. 2024-05-17T09:45:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.569+0000 7f8540834740 20 client.28706 trim_cache size 0 max 16384 2024-05-17T09:45:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.601+0000 7f8540834740 1 Processor -- start 2024-05-17T09:45:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.601+0000 7f8540834740 1 -- start start 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.601+0000 7f8540834740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b872ed80 0x5632b8b0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8540834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8540834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f85327fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b872ed80 0x5632b8b0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f85327fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b21f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36598/0 (socket says 172.21.15.19:36598) 2024-05-17T09:45:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b872ed80 0x5632b8b0c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55270/0 (socket says 172.21.15.19:55270) 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f85327fc640 1 -- 172.21.15.19:0/2169512068 learned_addr learned my addr 172.21.15.19:0/2169512068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ad110 con 0x5632b872ed80 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85341aa9e0 con 0x5632b87472f0 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f85341a8830 con 0x5632b874f4d0 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f85337fe640 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f85337fe640 1 -- 172.21.15.19:0/2169512068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 msgr2=0x5632b8b21f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.602+0000 7f85337fe640 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b21f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85337fe640 1 -- 172.21.15.19:0/2169512068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b872ed80 msgr2=0x5632b8b0c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85337fe640 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b872ed80 0x5632b8b0c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85337fe640 1 -- 172.21.15.19:0/2169512068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b8b1bd10 con 0x5632b874f4d0 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85327fc640 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b21f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85337fe640 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b0c880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f852c009bf0 tx=0x7f852c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96a241ae92c12c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85197fa640 1 -- 172.21.15.19:0/2169512068 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c061070 con 0x5632b874f4d0 2024-05-17T09:45:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85197fa640 1 -- 172.21.15.19:0/2169512068 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f852c036040 con 0x5632b874f4d0 2024-05-17T09:45:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.603+0000 7f85197fa640 1 -- 172.21.15.19:0/2169512068 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c06b080 con 0x5632b874f4d0 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.604+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 msgr2=0x5632b8b0c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b0c880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f852c009bf0 tx=0x7f852c0095c0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 shutdown_connections 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b0c880 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b87472f0 0x5632b8b21f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 --2- 172.21.15.19:0/2169512068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b872ed80 0x5632b8b0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 >> 172.21.15.19:0/2169512068 conn(0x5632b8764300 msgr2=0x5632b8b14920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 shutdown_connections 2024-05-17T09:45:03.605 INFO:tasks.workunit.client.0.smithi019.stdout: 10 76968 6.20 MB/sec execute 540 sec latency 80.994 ms 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 -- 172.21.15.19:0/2169512068 wait complete. 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.605+0000 7f8540834740 1 -- start start 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f8540834740 1 Processor -- start 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f8540834740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b872ed80 0x5632b8b1f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f8540834740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87472f0 0x5632b8b1f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f8540834740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b20890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f85327fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87472f0 0x5632b8b1f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f8540834740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85341a8ea0 con 0x5632b87472f0 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f85327fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87472f0 0x5632b8b1f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55278/0 (socket says 172.21.15.19:55278) 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f85327fc640 1 -- 172.21.15.19:0/3513159115 learned_addr learned my addr 172.21.15.19:0/3513159115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f8532ffd640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b872ed80 0x5632b8b1f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.606+0000 7f85337fe640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b20890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85341aa310 con 0x5632b872ed80 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f85341a9570 con 0x5632b874f4d0 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8540834740 10 client.0 _ll_register_callbacks cb 0x7f852c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8532ffd640 1 -- 172.21.15.19:0/3513159115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 msgr2=0x5632b8b20890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8532ffd640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b20890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8532ffd640 1 -- 172.21.15.19:0/3513159115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87472f0 msgr2=0x5632b8b1f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8532ffd640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87472f0 0x5632b8b1f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8532ffd640 1 -- 172.21.15.19:0/3513159115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b8b20e50 con 0x5632b872ed80 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8532ffd640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b872ed80 0x5632b8b1f470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85240e3d70 tx=0x7f85240ec110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86133f4c9b2d243 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.607+0000 7f8512ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f8512ffd640 1 -- 172.21.15.19:0/3513159115 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85240ad190 con 0x5632b872ed80 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f8512ffd640 1 -- 172.21.15.19:0/3513159115 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f85240d00b0 con 0x5632b872ed80 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852c036040 con 0x5632b872ed80 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f85337fe640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b20890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f8512ffd640 1 -- 172.21.15.19:0/3513159115 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85240acfb0 con 0x5632b872ed80 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f8540834740 10 client.28712 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.608+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5632b8b22090 con 0x5632b872ed80 2024-05-17T09:45:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8512ffd640 1 -- 172.21.15.19:0/3513159115 <== mon.2 v2:172.21.15.69:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f85240adaa0 con 0x5632b872ed80 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8540834740 10 client.28712 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8540834740 10 client.28712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8540834740 20 client.28712 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8540834740 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b87701b0 0x5632b8b92cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8540834740 4 client.28712 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f852c036040 con 0x5632b87701b0 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.609+0000 7f85327fc640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b87701b0 0x5632b8b92cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.610+0000 7f8512ffd640 10 client.28712 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.610+0000 7f85327fc640 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b87701b0 0x5632b8b92cb0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a323bb8c7175830 server_cookie=6f762b71b0655de2 in_seq=0 out_seq=0 2024-05-17T09:45:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.724+0000 7f8512ffd640 1 -- 172.21.15.19:0/3513159115 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f852c036040 con 0x5632b87701b0 2024-05-17T09:45:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.724+0000 7f8512ffd640 10 client.28712 handle_command_reply: tid=0 2024-05-17T09:45:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.724+0000 7f8540834740 1 client.28712 shutdown 2024-05-17T09:45:03.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b872ed80 msgr2=0x5632b8b1f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b872ed80 0x5632b8b1f470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85240e3d70 tx=0x7f85240ec110 comp rx=0 tx=0).stop 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 shutdown_connections 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5632b87701b0 0x5632b8b92cb0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8512ffd640 0 client.28712 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5632b874f4d0 0x5632b8b20890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632b87472f0 0x5632b8b1f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 --2- 172.21.15.19:0/3513159115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632b872ed80 0x5632b8b1f470 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 >> 172.21.15.19:0/3513159115 conn(0x5632b8764300 msgr2=0x5632b8b152f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:03.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 shutdown_connections 2024-05-17T09:45:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.725+0000 7f8540834740 1 -- 172.21.15.19:0/3513159115 wait complete. 2024-05-17T09:45:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.726+0000 7f8540834740 20 client.28712 trim_cache size 0 max 16384 2024-05-17T09:45:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:45:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:45:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f851c0b28a0 msgr2=0x7f851c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f851c0b28a0 0x7f851c0b4db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f852c04a000 tx=0x7f852c030b00 comp rx=0 tx=0).stop 2024-05-17T09:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534102b40 msgr2=0x7f853406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534102b40 0x7f853406b120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8534102410 tx=0x7f852400ed10 comp rx=0 tx=0).stop 2024-05-17T09:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 shutdown_connections 2024-05-17T09:45:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f851c0b28a0 0x7f851c0b4db0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8534108a80 0x7f85341a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8534103d80 0x7f853419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 --2- 172.21.15.19:0/460811839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8534102b40 0x7f853406b120 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.728+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 >> 172.21.15.19:0/460811839 conn(0x7f85340fdd30 msgr2=0x7f8534104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.729+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 shutdown_connections 2024-05-17T09:45:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:03.729+0000 7f853997d640 1 -- 172.21.15.19:0/460811839 wait complete. 2024-05-17T09:45:04.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 77170 6.21 MB/sec execute 541 sec latency 156.272 ms 2024-05-17T09:45:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:04 smithi069 ceph-mon[35081]: pgmap v1197: 129 pgs: 129 active+clean; 515 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-05-17T09:45:04.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:04 smithi019 ceph-mon[31620]: pgmap v1197: 129 pgs: 129 active+clean; 515 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-05-17T09:45:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:04 smithi079 ceph-mon[35502]: pgmap v1197: 129 pgs: 129 active+clean; 515 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-05-17T09:45:05.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 77322 6.21 MB/sec execute 542 sec latency 85.509 ms 2024-05-17T09:45:06.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 77474 6.22 MB/sec execute 543 sec latency 87.987 ms 2024-05-17T09:45:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:06 smithi069 ceph-mon[35081]: pgmap v1198: 129 pgs: 129 active+clean; 515 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-05-17T09:45:06.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:06 smithi019 ceph-mon[31620]: pgmap v1198: 129 pgs: 129 active+clean; 515 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-05-17T09:45:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:06 smithi079 ceph-mon[35502]: pgmap v1198: 129 pgs: 129 active+clean; 515 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-05-17T09:45:07.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 77645 6.23 MB/sec execute 544 sec latency 93.669 ms 2024-05-17T09:45:07.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:07 smithi069 ceph-mon[35081]: pgmap v1199: 129 pgs: 129 active+clean; 516 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-05-17T09:45:07.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:07 smithi019 ceph-mon[31620]: pgmap v1199: 129 pgs: 129 active+clean; 516 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-05-17T09:45:08.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:07 smithi079 ceph-mon[35502]: pgmap v1199: 129 pgs: 129 active+clean; 516 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-05-17T09:45:08.603 INFO:tasks.workunit.client.0.smithi019.stdout: 10 77784 6.24 MB/sec execute 545 sec latency 100.317 ms 2024-05-17T09:45:09.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 77921 6.24 MB/sec execute 546 sec latency 67.362 ms 2024-05-17T09:45:10.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78065 6.25 MB/sec execute 547 sec latency 85.015 ms 2024-05-17T09:45:10.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:10 smithi079 ceph-mon[35502]: pgmap v1200: 129 pgs: 129 active+clean; 520 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 3.6 MiB/s wr, 258 op/s 2024-05-17T09:45:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:10 smithi069 ceph-mon[35081]: pgmap v1200: 129 pgs: 129 active+clean; 520 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 3.6 MiB/s wr, 258 op/s 2024-05-17T09:45:10.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:10 smithi019 ceph-mon[31620]: pgmap v1200: 129 pgs: 129 active+clean; 520 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 3.6 MiB/s wr, 258 op/s 2024-05-17T09:45:11.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78200 6.24 MB/sec execute 548 sec latency 73.096 ms 2024-05-17T09:45:12.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78353 6.24 MB/sec execute 549 sec latency 122.114 ms 2024-05-17T09:45:12.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:12 smithi079 ceph-mon[35502]: pgmap v1201: 129 pgs: 129 active+clean; 521 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 4.0 MiB/s wr, 234 op/s 2024-05-17T09:45:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:12 smithi069 ceph-mon[35081]: pgmap v1201: 129 pgs: 129 active+clean; 521 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 4.0 MiB/s wr, 234 op/s 2024-05-17T09:45:12.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:12 smithi019 ceph-mon[31620]: pgmap v1201: 129 pgs: 129 active+clean; 521 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 4.0 MiB/s wr, 234 op/s 2024-05-17T09:45:13.604 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78495 6.24 MB/sec execute 550 sec latency 102.942 ms 2024-05-17T09:45:14.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78627 6.24 MB/sec execute 551 sec latency 89.998 ms 2024-05-17T09:45:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:14 smithi079 ceph-mon[35502]: pgmap v1202: 129 pgs: 129 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 263 op/s 2024-05-17T09:45:14.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:14 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:45:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:14 smithi069 ceph-mon[35081]: pgmap v1202: 129 pgs: 129 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 263 op/s 2024-05-17T09:45:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:14 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:45:14.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:14 smithi019 ceph-mon[31620]: pgmap v1202: 129 pgs: 129 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 263 op/s 2024-05-17T09:45:14.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:14 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:45:15.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78744 6.24 MB/sec execute 552 sec latency 159.169 ms 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:15 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:15 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:15.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:15 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T09:45:16.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 78879 6.24 MB/sec execute 553 sec latency 89.561 ms 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 3.8 MiB/s wr, 219 op/s 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:45:16.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:45:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/3731385765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 msgr2=0x7f5cdc10aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/3731385765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc10aea0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009ca0 tx=0x7f5cd402a8c0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/3731385765 shutdown_connections 2024-05-17T09:45:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/3731385765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc10aea0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/3731385765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 0x7f5cdc104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/3731385765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cdc102b40 0x7f5cdc103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/3731385765 >> 172.21.15.19:0/3731385765 conn(0x7f5cdc0fdd30 msgr2=0x7f5cdc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/3731385765 shutdown_connections 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.886+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/3731385765 wait complete. 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 Processor -- start 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 -- start start 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cdc102b40 0x7f5cdc199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 0x7f5cdc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc10e570 con 0x7f5cdc108a80 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc10e3f0 con 0x7f5cdc102b40 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce3f7d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc10e6f0 con 0x7f5cdc103d80 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce251a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce1518640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 0x7f5cdc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce251a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc1a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40374/0 (socket says 172.21.15.19:40374) 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.887+0000 7f5ce251a640 1 -- 172.21.15.19:0/229996321 learned_addr learned my addr 172.21.15.19:0/229996321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cdc102b40 0x7f5cdc199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1518640 1 -- 172.21.15.19:0/229996321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cdc102b40 msgr2=0x7f5cdc199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1518640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cdc102b40 0x7f5cdc199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1518640 1 -- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 msgr2=0x7f5cdc1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1518640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc1a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1518640 1 -- 172.21.15.19:0/229996321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc1a0af0 con 0x7f5cdc103d80 2024-05-17T09:45:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce1518640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 0x7f5cdc1995a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00e9f0 tx=0x7f5ccc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5cd2ffd640 1 -- 172.21.15.19:0/229996321 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00ccc0 con 0x7f5cdc103d80 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc1a0de0 con 0x7f5cdc103d80 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc1a1250 con 0x7f5cdc103d80 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5cd2ffd640 1 -- 172.21.15.19:0/229996321 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ccc0044b0 con 0x7f5cdc103d80 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.888+0000 7f5cd2ffd640 1 -- 172.21.15.19:0/229996321 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc01c070 con 0x7f5cdc103d80 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5cd2ffd640 1 -- 172.21.15.19:0/229996321 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ccc005020 con 0x7f5cdc103d80 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 Processor -- start 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 -- start start 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a2a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a2ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096664d0 0x559d09a31a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d0999ac30 con 0x559d0965e2f0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559d0998a7d0 con 0x559d096871b0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ceae33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559d0998e810 con 0x559d096664d0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ce1d19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a2ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ce1d19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a2ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40388/0 (socket says 172.21.15.19:40388) 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1801353466 learned_addr learned my addr 172.21.15.19:0/1801353466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.891+0000 7f5ce251a640 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a2a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096664d0 0x559d09a31a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1801353466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096664d0 msgr2=0x559d09a31a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096664d0 0x559d09a31a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1801353466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 msgr2=0x559d09a2a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a2a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1801353466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0969f210 con 0x559d0965e2f0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a2ab30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800cc00 tx=0x7f5cd800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15266bfb465d408c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5cd2ffd640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc80b28a0 0x7f5cc80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5cc7fff640 1 -- 172.21.15.19:0/1801353466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004220 con 0x559d0965e2f0 2024-05-17T09:45:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5ce251a640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc80b28a0 0x7f5cc80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5cd2ffd640 1 -- 172.21.15.19:0/229996321 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5ccc0e2600 con 0x7f5cdc103d80 2024-05-17T09:45:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5cc7fff640 1 -- 172.21.15.19:0/1801353466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cd80043c0 con 0x559d0965e2f0 2024-05-17T09:45:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.892+0000 7f5cc7fff640 1 -- 172.21.15.19:0/1801353466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004c30 con 0x559d0965e2f0 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 -- 172.21.15.19:0/1801353466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 msgr2=0x559d09a2ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a2ab30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800cc00 tx=0x7f5cd800e720 comp rx=0 tx=0).stop 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ce251a640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc80b28a0 0x7f5cc80b4db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40319e0 tx=0x7f5cd40099a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 -- 172.21.15.19:0/1801353466 shutdown_connections 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096664d0 0x559d09a31a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a2ab30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1801353466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a2a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 -- 172.21.15.19:0/1801353466 >> 172.21.15.19:0/1801353466 conn(0x559d0967b300 msgr2=0x559d09a23d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 -- 172.21.15.19:0/1801353466 shutdown_connections 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.893+0000 7f5ceae33740 1 -- 172.21.15.19:0/1801353466 wait complete. 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 -- start start 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 Processor -- start 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096664d0 0x559d09a343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d09670790 con 0x559d0965e2f0 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559d09946890 con 0x559d096664d0 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559d09a32480 con 0x559d096871b0 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ceae33740 10 client.0 _ll_register_callbacks cb 0x559d09949d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a33e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40394/0 (socket says 172.21.15.19:40394) 2024-05-17T09:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 -- 172.21.15.19:0/3811423941 learned_addr learned my addr 172.21.15.19:0/3811423941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce251a640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1518640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096664d0 0x559d09a343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 -- 172.21.15.19:0/3811423941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 msgr2=0x559d09a3b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 -- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096664d0 msgr2=0x559d09a343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096664d0 0x559d09a343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.894+0000 7f5ce1d19640 1 -- 172.21.15.19:0/3811423941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0976b620 con 0x559d0965e2f0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a33e90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800f010 tx=0x7f5cd801cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a827e6df06d3fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5cc57fa640 1 -- 172.21.15.19:0/3811423941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd802a060 con 0x559d0965e2f0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd80043c0 con 0x559d0965e2f0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5cc57fa640 1 -- 172.21.15.19:0/3811423941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5cd801ccd0 con 0x559d0965e2f0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5cc57fa640 1 -- 172.21.15.19:0/3811423941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5cd8031050 con 0x559d0965e2f0 2024-05-17T09:45:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5ceae33740 10 client.19233 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559d09a3e260 con 0x559d0965e2f0 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5cc57fa640 1 -- 172.21.15.19:0/3811423941 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cd80043c0 con 0x559d0965e2f0 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.895+0000 7f5cc57fa640 1 -- 172.21.15.19:0/3811423941 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5cd801a9a0 con 0x559d0965e2f0 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ceae33740 10 client.19233 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ceae33740 10 client.19233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ceae33740 20 client.19233 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ceae33740 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d09645d80 0x559d09a416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ceae33740 4 client.19233 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0999cb80 con 0x559d09645d80 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ce1518640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d09645d80 0x559d09a416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5ce1518640 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d09645d80 0x559d09a416d0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bbae66947f00c19 server_cookie=4481575efd1a2b54 in_seq=0 out_seq=0 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.896+0000 7f5cc57fa640 10 client.19233 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5cc57fa640 1 -- 172.21.15.19:0/3811423941 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559d0999cb80 con 0x559d09645d80 2024-05-17T09:45:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5cc57fa640 10 client.19233 handle_command_reply: tid=0 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 client.19233 shutdown 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 msgr2=0x559d09a33e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a33e90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800f010 tx=0x7f5cd801cae0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 shutdown_connections 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d09645d80 0x559d09a416d0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5cc57fa640 0 client.19233 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096664d0 0x559d09a343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 --2- 172.21.15.19:0/3811423941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d0965e2f0 0x559d09a33e90 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 >> 172.21.15.19:0/3811423941 conn(0x559d0967b300 msgr2=0x559d09a26c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 shutdown_connections 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 1 -- 172.21.15.19:0/3811423941 wait complete. 2024-05-17T09:45:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.897+0000 7f5ceae33740 20 client.19233 trim_cache size 0 max 16384 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 Processor -- start 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 -- start start 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d0965e2f0 0x559d09a2d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1acfa0 con 0x559d096664d0 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1aa870 con 0x559d0965e2f0 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ceae33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1a86c0 con 0x559d096871b0 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ce1518640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.913+0000 7f5ce1518640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a3b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40396/0 (socket says 172.21.15.19:40396) 2024-05-17T09:45:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 -- 172.21.15.19:0/1983849669 learned_addr learned my addr 172.21.15.19:0/1983849669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d0965e2f0 0x559d09a2d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce251a640 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 -- 172.21.15.19:0/1983849669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 msgr2=0x559d09a3b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 -- 172.21.15.19:0/1983849669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d0965e2f0 msgr2=0x559d09a2d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d0965e2f0 0x559d09a2d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 -- 172.21.15.19:0/1983849669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d09a2a160 con 0x559d096664d0 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d0965e2f0 0x559d09a2d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a3b250 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0a9570 tx=0x7f5ccc0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25736426dfd1ef8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5cc77fe640 1 -- 172.21.15.19:0/1983849669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a8fe0 con 0x559d096664d0 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5cc77fe640 1 -- 172.21.15.19:0/1983849669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ccc0a9380 con 0x559d096664d0 2024-05-17T09:45:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.914+0000 7f5cc77fe640 1 -- 172.21.15.19:0/1983849669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0cc0b0 con 0x559d096664d0 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 -- 172.21.15.19:0/1983849669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 msgr2=0x559d09a3b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a3b250 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0a9570 tx=0x7f5ccc0cc040 comp rx=0 tx=0).stop 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 -- 172.21.15.19:0/1983849669 shutdown_connections 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d096871b0 0x559d09a3b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a3b250 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1983849669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d0965e2f0 0x559d09a2d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 -- 172.21.15.19:0/1983849669 >> 172.21.15.19:0/1983849669 conn(0x559d0967b300 msgr2=0x559d09a27560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 -- 172.21.15.19:0/1983849669 shutdown_connections 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 -- 172.21.15.19:0/1983849669 wait complete. 2024-05-17T09:45:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.915+0000 7f5ceae33740 1 -- start start 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 Processor -- start 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 0x559d09a38e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a39360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a3a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1a8d30 con 0x559d096664d0 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1aa1a0 con 0x559d096871b0 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1a9400 con 0x559d0965e2f0 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ceae33740 10 client.0 _ll_register_callbacks cb 0x7f5ccc0e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1518640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a39360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 0x559d09a38e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 0x559d09a38e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:60122/0 (socket says 172.21.15.19:60122) 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1761031891 learned_addr learned my addr 172.21.15.19:0/1761031891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1518640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a39360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40400/0 (socket says 172.21.15.19:40400) 2024-05-17T09:45:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce251a640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a3a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 msgr2=0x559d09a3a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a3a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1761031891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 msgr2=0x559d09a39360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a39360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1d19640 1 -- 172.21.15.19:0/1761031891 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d09a3a850 con 0x559d0965e2f0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.916+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a39360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5ce1d19640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 0x559d09a38e20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005b70 tx=0x7f5cd8010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350fa7f0d9779864 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5cc57fa640 1 -- 172.21.15.19:0/1761031891 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd802a070 con 0x559d0965e2f0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ccc0a9380 con 0x559d0965e2f0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5cc57fa640 1 -- 172.21.15.19:0/1761031891 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5cd801c0c0 con 0x559d0965e2f0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5cc57fa640 1 -- 172.21.15.19:0/1761031891 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5cd801c2c0 con 0x559d0965e2f0 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5ceae33740 10 client.28702 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.917+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559d09a3bae0 con 0x559d0965e2f0 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5cc57fa640 1 -- 172.21.15.19:0/1761031891 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ccc0a9380 con 0x559d0965e2f0 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5cc57fa640 1 -- 172.21.15.19:0/1761031891 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5cd801a460 con 0x559d0965e2f0 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5ceae33740 10 client.28702 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5ceae33740 10 client.28702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5ceae33740 20 client.28702 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d0961b1c0 0x559d09aac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5ceae33740 4 client.28702 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.918+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d09996360 con 0x559d0961b1c0 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.919+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d0961b1c0 0x559d09aac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.919+0000 7f5ce1518640 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d0961b1c0 0x559d09aac0c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a530d757be52bc6d server_cookie=c2728c28d2d89526 in_seq=0 out_seq=0 2024-05-17T09:45:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:16.919+0000 7f5cc57fa640 10 client.28702 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 3.8 MiB/s wr, 219 op/s 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:45:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:45:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi079 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: Unable to set osd_memory_target on smithi069 to 184707891: error parsing value: Value '184707891' is below minimum 939524096 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 3.8 MiB/s wr, 219 op/s 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:45:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.065+0000 7f5cc57fa640 1 -- 172.21.15.19:0/1761031891 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559d09996360 con 0x559d0961b1c0 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.065+0000 7f5cc57fa640 10 client.28702 handle_command_reply: tid=0 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.065+0000 7f5ceae33740 1 client.28702 shutdown 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 msgr2=0x559d09a38e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 0x559d09a38e20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005b70 tx=0x7f5cd8010660 comp rx=0 tx=0).stop 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 shutdown_connections 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x559d0961b1c0 0x559d09aac0c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5cc57fa640 0 client.28702 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d096871b0 0x559d09a3a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d096664d0 0x559d09a39360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 --2- 172.21.15.19:0/1761031891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0965e2f0 0x559d09a38e20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 >> 172.21.15.19:0/1761031891 conn(0x559d0967b300 msgr2=0x559d09a22d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 shutdown_connections 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 1 -- 172.21.15.19:0/1761031891 wait complete. 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.066+0000 7f5ceae33740 20 client.28702 trim_cache size 0 max 16384 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:45:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc80b28a0 msgr2=0x7f5cc80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc80b28a0 0x7f5cc80b4db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40319e0 tx=0x7f5cd40099a0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 msgr2=0x7f5cdc1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 0x7f5cdc1995a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00e9f0 tx=0x7f5ccc00eeb0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 shutdown_connections 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f5cc80b28a0 0x7f5cc80b4db0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cdc108a80 0x7f5cdc1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cdc103d80 0x7f5cdc1995a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 --2- 172.21.15.19:0/229996321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cdc102b40 0x7f5cdc199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 >> 172.21.15.19:0/229996321 conn(0x7f5cdc0fdd30 msgr2=0x7f5cdc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 shutdown_connections 2024-05-17T09:45:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:17.067+0000 7f5ce3f7d640 1 -- 172.21.15.19:0/229996321 wait complete. 2024-05-17T09:45:17.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79043 6.26 MB/sec execute 554 sec latency 177.457 ms 2024-05-17T09:45:18.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79195 6.26 MB/sec execute 555 sec latency 85.442 ms 2024-05-17T09:45:18.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:18 smithi079 ceph-mon[35502]: pgmap v1204: 129 pgs: 129 active+clean; 527 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-17T09:45:18.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:18 smithi069 ceph-mon[35081]: pgmap v1204: 129 pgs: 129 active+clean; 527 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-17T09:45:18.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:18 smithi019 ceph-mon[31620]: pgmap v1204: 129 pgs: 129 active+clean; 527 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-05-17T09:45:19.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79344 6.26 MB/sec execute 556 sec latency 101.857 ms 2024-05-17T09:45:20.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79465 6.25 MB/sec execute 557 sec latency 104.805 ms 2024-05-17T09:45:20.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:20 smithi079 ceph-mon[35502]: pgmap v1205: 129 pgs: 129 active+clean; 520 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-05-17T09:45:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:20 smithi069 ceph-mon[35081]: pgmap v1205: 129 pgs: 129 active+clean; 520 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-05-17T09:45:20.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:20 smithi019 ceph-mon[31620]: pgmap v1205: 129 pgs: 129 active+clean; 520 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-05-17T09:45:21.607 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79621 6.26 MB/sec execute 558 sec latency 86.608 ms 2024-05-17T09:45:22.608 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79783 6.26 MB/sec execute 559 sec latency 79.237 ms 2024-05-17T09:45:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:22 smithi069 ceph-mon[35081]: pgmap v1206: 129 pgs: 129 active+clean; 517 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-05-17T09:45:22.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:22 smithi019 ceph-mon[31620]: pgmap v1206: 129 pgs: 129 active+clean; 517 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-05-17T09:45:23.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:22 smithi079 ceph-mon[35502]: pgmap v1206: 129 pgs: 129 active+clean; 517 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-05-17T09:45:23.609 INFO:tasks.workunit.client.0.smithi019.stdout: 10 79918 6.26 MB/sec execute 560 sec latency 78.642 ms 2024-05-17T09:45:24.609 INFO:tasks.workunit.client.0.smithi019.stdout: 10 80057 6.26 MB/sec execute 561 sec latency 87.446 ms 2024-05-17T09:45:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:24 smithi069 ceph-mon[35081]: pgmap v1207: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-17T09:45:24.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:24 smithi019 ceph-mon[31620]: pgmap v1207: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-17T09:45:25.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:24 smithi079 ceph-mon[35502]: pgmap v1207: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-17T09:45:25.611 INFO:tasks.workunit.client.0.smithi019.stdout: 10 80253 6.28 MB/sec execute 562 sec latency 77.419 ms 2024-05-17T09:45:26.611 INFO:tasks.workunit.client.0.smithi019.stdout: 10 80437 6.28 MB/sec execute 563 sec latency 70.635 ms 2024-05-17T09:45:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:26 smithi069 ceph-mon[35081]: pgmap v1208: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-05-17T09:45:26.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:26 smithi019 ceph-mon[31620]: pgmap v1208: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-05-17T09:45:27.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:26 smithi079 ceph-mon[35502]: pgmap v1208: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-05-17T09:45:27.611 INFO:tasks.workunit.client.0.smithi019.stdout: 10 80625 6.29 MB/sec execute 564 sec latency 109.422 ms 2024-05-17T09:45:28.611 INFO:tasks.workunit.client.0.smithi019.stdout: 10 80740 6.29 MB/sec execute 565 sec latency 73.983 ms 2024-05-17T09:45:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:28 smithi069 ceph-mon[35081]: pgmap v1209: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 284 op/s 2024-05-17T09:45:28.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:28 smithi019 ceph-mon[31620]: pgmap v1209: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 284 op/s 2024-05-17T09:45:29.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:28 smithi079 ceph-mon[35502]: pgmap v1209: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 284 op/s 2024-05-17T09:45:29.611 INFO:tasks.workunit.client.0.smithi019.stdout: 10 80877 6.29 MB/sec execute 566 sec latency 76.491 ms 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 -- 172.21.15.19:0/1159179354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124102b40 msgr2=0x7f8124103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1159179354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124102b40 0x7f8124103840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8118009900 tx=0x7f811802e800 comp rx=0 tx=0).stop 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 -- 172.21.15.19:0/1159179354 shutdown_connections 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1159179354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124108a80 0x7f812410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1159179354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8124103d80 0x7f8124104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1159179354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124102b40 0x7f8124103840 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.243+0000 7f8128b0d640 1 -- 172.21.15.19:0/1159179354 >> 172.21.15.19:0/1159179354 conn(0x7f81240fdd30 msgr2=0x7f81241001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.244+0000 7f8128b0d640 1 -- 172.21.15.19:0/1159179354 shutdown_connections 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.244+0000 7f8128b0d640 1 -- 172.21.15.19:0/1159179354 wait complete. 2024-05-17T09:45:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 Processor -- start 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 -- start start 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8124102b40 0x7f8124113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124103d80 0x7f8124113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124108a80 0x7f8124112060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f812410c2a0 con 0x7f8124102b40 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f812410c120 con 0x7f8124103d80 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8128b0d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f812410c420 con 0x7f8124108a80 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.245+0000 7f8121d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124103d80 0x7f8124113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.246+0000 7f8121d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124103d80 0x7f8124113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47744/0 (socket says 172.21.15.19:47744) 2024-05-17T09:45:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.246+0000 7f8121d9b640 1 -- 172.21.15.19:0/1085327834 learned_addr learned my addr 172.21.15.19:0/1085327834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.246+0000 7f8122d9d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124108a80 0x7f8124112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.246+0000 7f812259c640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8124102b40 0x7f8124113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.248+0000 7f8122d9d640 1 -- 172.21.15.19:0/1085327834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124103d80 msgr2=0x7f8124113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.248+0000 7f8122d9d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124103d80 0x7f8124113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.248+0000 7f8122d9d640 1 -- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8124102b40 msgr2=0x7f8124113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.248+0000 7f8122d9d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8124102b40 0x7f8124113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.248+0000 7f8122d9d640 1 -- 172.21.15.19:0/1085327834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81241128d0 con 0x7f8124108a80 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.248+0000 7f8122d9d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124108a80 0x7f8124112060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f811c009fd0 tx=0x7f811c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.249+0000 7f81137fe640 1 -- 172.21.15.19:0/1085327834 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c00ce20 con 0x7f8124108a80 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.249+0000 7f81137fe640 1 -- 172.21.15.19:0/1085327834 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f811c005020 con 0x7f8124108a80 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.249+0000 7f81137fe640 1 -- 172.21.15.19:0/1085327834 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c010690 con 0x7f8124108a80 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.249+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8124112b60 con 0x7f8124108a80 2024-05-17T09:45:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.249+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241b14d0 con 0x7f8124108a80 2024-05-17T09:45:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.253+0000 7f812f9c0740 1 Processor -- start 2024-05-17T09:45:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.254+0000 7f81137fe640 1 -- 172.21.15.19:0/1085327834 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f811c004a40 con 0x7f8124108a80 2024-05-17T09:45:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.254+0000 7f812f9c0740 1 -- start start 2024-05-17T09:45:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 0x55ecdbd243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd24920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812f9c0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbc96c30 con 0x55ecdb95a2f0 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812f9c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbc867d0 con 0x55ecdb9831b0 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812f9c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbc8a810 con 0x55ecdb9624d0 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 0x55ecdbd243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 0x55ecdbd243a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47754/0 (socket says 172.21.15.19:47754) 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 -- 172.21.15.19:0/574939915 learned_addr learned my addr 172.21.15.19:0/574939915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f812259c640 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd24920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 -- 172.21.15.19:0/574939915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9624d0 msgr2=0x55ecdbd2b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.255+0000 7f8121d9b640 1 -- 172.21.15.19:0/574939915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb95a2f0 msgr2=0x55ecdbd24920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f8121d9b640 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd24920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f8121d9b640 1 -- 172.21.15.19:0/574939915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdb99b210 con 0x55ecdb9831b0 2024-05-17T09:45:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f812259c640 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd24920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:45:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f8121d9b640 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 0x55ecdbd243a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f811400d330 tx=0x7f8114005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd7c1805662d255b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f81137fe640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f810c0b2ac0 0x7f810c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f8110ff9640 1 -- 172.21.15.19:0/574939915 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114009cc0 con 0x55ecdb9831b0 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f8110ff9640 1 -- 172.21.15.19:0/574939915 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81140054b0 con 0x55ecdb9831b0 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f81137fe640 1 -- 172.21.15.19:0/1085327834 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f811c0e29c0 con 0x7f8124108a80 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.256+0000 7f8110ff9640 1 -- 172.21.15.19:0/574939915 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114004af0 con 0x55ecdb9831b0 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812259c640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f810c0b2ac0 0x7f810c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 -- 172.21.15.19:0/574939915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 msgr2=0x55ecdbd243a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 0x55ecdbd243a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f811400d330 tx=0x7f8114005f90 comp rx=0 tx=0).stop 2024-05-17T09:45:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812259c640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f810c0b2ac0 0x7f810c0b4fd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f81180057c0 tx=0x7f8118000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 -- 172.21.15.19:0/574939915 shutdown_connections 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd24920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 --2- 172.21.15.19:0/574939915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9831b0 0x55ecdbd243a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 -- 172.21.15.19:0/574939915 >> 172.21.15.19:0/574939915 conn(0x55ecdb977300 msgr2=0x55ecdbd1fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.257+0000 7f812f9c0740 1 -- 172.21.15.19:0/574939915 shutdown_connections 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 -- 172.21.15.19:0/574939915 wait complete. 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 -- start start 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 Processor -- start 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd2dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 0x55ecdbd350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ecdb96c790 con 0x55ecdb9831b0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbc42890 con 0x55ecdb9624d0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.258+0000 7f812f9c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbd2c260 con 0x55ecdb95a2f0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812f9c0740 10 client.0 _ll_register_callbacks cb 0x55ecdbc45d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f8122d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 0x55ecdbd350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd2dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd2dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:33222/0 (socket says 172.21.15.19:33222) 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 -- 172.21.15.19:0/917239159 learned_addr learned my addr 172.21.15.19:0/917239159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f8122d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 0x55ecdbd350c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46424/0 (socket says 172.21.15.19:46424) 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f8121d9b640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 -- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9624d0 msgr2=0x55ecdbd2e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 -- 172.21.15.19:0/917239159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 msgr2=0x55ecdbd350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 0x55ecdbd350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 -- 172.21.15.19:0/917239159 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdba67620 con 0x55ecdb95a2f0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f8122d9d640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 0x55ecdbd350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f812259c640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd2dc20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8118009900 tx=0x7f811802e800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d58aecf5d16dcca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.259+0000 7f8109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f8109ffb640 1 -- 172.21.15.19:0/917239159 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180366f0 con 0x55ecdb95a2f0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f8109ffb640 1 -- 172.21.15.19:0/917239159 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8118036890 con 0x55ecdb95a2f0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81140054b0 con 0x55ecdb95a2f0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f8109ffb640 1 -- 172.21.15.19:0/917239159 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8118059050 con 0x55ecdb95a2f0 2024-05-17T09:45:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 10 client.28714 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ecdbd38060 con 0x55ecdb95a2f0 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f8109ffb640 1 -- 172.21.15.19:0/917239159 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81140054b0 con 0x55ecdb95a2f0 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f8109ffb640 1 -- 172.21.15.19:0/917239159 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8118035070 con 0x55ecdb95a2f0 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 10 client.28714 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 10 client.28714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 20 client.28714 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb941d80 0x55ecdbd3bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 4 client.28714 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.260+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecdbc98b80 con 0x55ecdb941d80 2024-05-17T09:45:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.261+0000 7f8121d9b640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb941d80 0x55ecdbd3bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.261+0000 7f8109ffb640 10 client.28714 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.262+0000 7f8121d9b640 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb941d80 0x55ecdbd3bca0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a8df7b8b2f3455 server_cookie=a0bad4079083d2ba in_seq=0 out_seq=0 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.262+0000 7f8109ffb640 1 -- 172.21.15.19:0/917239159 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ecdbc98b80 con 0x55ecdb941d80 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.262+0000 7f8109ffb640 10 client.28714 handle_command_reply: tid=0 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.262+0000 7f812f9c0740 1 client.28714 shutdown 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 msgr2=0x55ecdbd2dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd2dc20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8118009900 tx=0x7f811802e800 comp rx=0 tx=0).stop 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 shutdown_connections 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb941d80 0x55ecdbd3bca0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9831b0 0x55ecdbd350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb9624d0 0x55ecdbd2e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f8109ffb640 0 client.28714 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 --2- 172.21.15.19:0/917239159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd2dc20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 >> 172.21.15.19:0/917239159 conn(0x55ecdb977300 msgr2=0x55ecdbd22c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 shutdown_connections 2024-05-17T09:45:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 1 -- 172.21.15.19:0/917239159 wait complete. 2024-05-17T09:45:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.263+0000 7f812f9c0740 20 client.28714 trim_cache size 0 max 16384 2024-05-17T09:45:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 Processor -- start 2024-05-17T09:45:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 -- start start 2024-05-17T09:45:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81241be0f0 con 0x55ecdb9624d0 2024-05-17T09:45:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81241bb9c0 con 0x55ecdb95a2f0 2024-05-17T09:45:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812f9c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f81241b9810 con 0x55ecdb9831b0 2024-05-17T09:45:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f812259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.285+0000 7f8121d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f8122d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47760/0 (socket says 172.21.15.19:47760) 2024-05-17T09:45:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 -- 172.21.15.19:0/951823003 learned_addr learned my addr 172.21.15.19:0/951823003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 -- 172.21.15.19:0/951823003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 msgr2=0x55ecdbd35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd35050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 -- 172.21.15.19:0/951823003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 msgr2=0x55ecdbd419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 -- 172.21.15.19:0/951823003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdbd275b0 con 0x55ecdb95a2f0 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f8122d9d640 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f8121d9b640 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f812259c640 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd414a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81180096c0 tx=0x7f8118009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21aa667a6db2b318 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f810bfff640 1 -- 172.21.15.19:0/951823003 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811804b660 con 0x55ecdb95a2f0 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f810bfff640 1 -- 172.21.15.19:0/951823003 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8118062070 con 0x55ecdb95a2f0 2024-05-17T09:45:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.286+0000 7f810bfff640 1 -- 172.21.15.19:0/951823003 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811803fe60 con 0x55ecdb95a2f0 2024-05-17T09:45:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.287+0000 7f812f9c0740 1 -- 172.21.15.19:0/951823003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 msgr2=0x55ecdbd414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.287+0000 7f812f9c0740 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd414a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81180096c0 tx=0x7f8118009890 comp rx=0 tx=0).stop 2024-05-17T09:45:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- 172.21.15.19:0/951823003 shutdown_connections 2024-05-17T09:45:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 --2- 172.21.15.19:0/951823003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd414a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- 172.21.15.19:0/951823003 >> 172.21.15.19:0/951823003 conn(0x55ecdb977300 msgr2=0x55ecdbd277e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- 172.21.15.19:0/951823003 shutdown_connections 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- 172.21.15.19:0/951823003 wait complete. 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- start start 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 Processor -- start 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd34ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81241b9e80 con 0x55ecdb9624d0 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81241bb2f0 con 0x55ecdb95a2f0 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.288+0000 7f812f9c0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f81241ba550 con 0x55ecdb9831b0 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812f9c0740 10 client.0 _ll_register_callbacks cb 0x7f8118009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd34220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:47772/0 (socket says 172.21.15.19:47772) 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 -- 172.21.15.19:0/164137486 learned_addr learned my addr 172.21.15.19:0/164137486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f8122d9d640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd34ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f8121d9b640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 -- 172.21.15.19:0/164137486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 msgr2=0x55ecdbd34ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd34ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 -- 172.21.15.19:0/164137486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 msgr2=0x55ecdbd34760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd34760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 -- 172.21.15.19:0/164137486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdbda2390 con 0x55ecdb95a2f0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f812259c640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd34220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81180092a0 tx=0x7f811804b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5e209f69ec177 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.289+0000 7f8109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f8109ffb640 1 -- 172.21.15.19:0/164137486 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811806a070 con 0x55ecdb95a2f0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f8109ffb640 1 -- 172.21.15.19:0/164137486 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f811803f600 con 0x55ecdb95a2f0 2024-05-17T09:45:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118062070 con 0x55ecdb95a2f0 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f8109ffb640 1 -- 172.21.15.19:0/164137486 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8118076050 con 0x55ecdb95a2f0 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f812f9c0740 10 client.28745 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ecdbda3620 con 0x55ecdb95a2f0 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f8109ffb640 1 -- 172.21.15.19:0/164137486 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8118062070 con 0x55ecdb95a2f0 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.290+0000 7f8109ffb640 1 -- 172.21.15.19:0/164137486 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8118035040 con 0x55ecdb95a2f0 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f812f9c0740 10 client.28745 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f812f9c0740 10 client.28745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f812f9c0740 20 client.28745 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f812f9c0740 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb9171c0 0x55ecdbda5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f812f9c0740 4 client.28745 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecdbc92360 con 0x55ecdb9171c0 2024-05-17T09:45:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.291+0000 7f8121d9b640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb9171c0 0x55ecdbda5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.292+0000 7f8109ffb640 10 client.28745 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.292+0000 7f8121d9b640 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb9171c0 0x55ecdbda5f60 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7b4e57f4be6b8d server_cookie=e280bae8d8debb2d in_seq=0 out_seq=0 2024-05-17T09:45:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.400+0000 7f8109ffb640 1 -- 172.21.15.19:0/164137486 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ecdbc92360 con 0x55ecdb9171c0 2024-05-17T09:45:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.400+0000 7f8109ffb640 10 client.28745 handle_command_reply: tid=0 2024-05-17T09:45:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.400+0000 7f812f9c0740 1 client.28745 shutdown 2024-05-17T09:45:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.400+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 msgr2=0x55ecdbd34220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.400+0000 7f812f9c0740 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd34220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81180092a0 tx=0x7f811804b710 comp rx=0 tx=0).stop 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 shutdown_connections 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55ecdb9171c0 0x55ecdbda5f60 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ecdb9831b0 0x55ecdbd34ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f8109ffb640 0 client.28745 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ecdb9624d0 0x55ecdbd34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 --2- 172.21.15.19:0/164137486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ecdb95a2f0 0x55ecdbd34220 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 >> 172.21.15.19:0/164137486 conn(0x55ecdb977300 msgr2=0x55ecdbd2df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 shutdown_connections 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 1 -- 172.21.15.19:0/164137486 wait complete. 2024-05-17T09:45:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.401+0000 7f812f9c0740 20 client.28745 trim_cache size 0 max 16384 2024-05-17T09:45:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:45:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:45:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:45:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f810c0b2ac0 msgr2=0x7f810c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f810c0b2ac0 0x7f810c0b4fd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f81180057c0 tx=0x7f8118000bc0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124108a80 msgr2=0x7f8124112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124108a80 0x7f8124112060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f811c009fd0 tx=0x7f811c00ef10 comp rx=0 tx=0).stop 2024-05-17T09:45:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 shutdown_connections 2024-05-17T09:45:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8124108a80 0x7f8124112060 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8124103d80 0x7f8124113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f810c0b2ac0 0x7f810c0b4fd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 --2- 172.21.15.19:0/1085327834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8124102b40 0x7f8124113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 >> 172.21.15.19:0/1085327834 conn(0x7f81240fdd30 msgr2=0x7f8124104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 shutdown_connections 2024-05-17T09:45:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:30.402+0000 7f8128b0d640 1 -- 172.21.15.19:0/1085327834 wait complete. 2024-05-17T09:45:30.612 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81051 6.31 MB/sec execute 567 sec latency 99.739 ms 2024-05-17T09:45:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:30 smithi069 ceph-mon[35081]: pgmap v1210: 129 pgs: 129 active+clean; 528 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.8 MiB/s wr, 280 op/s 2024-05-17T09:45:30.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:30 smithi019 ceph-mon[31620]: pgmap v1210: 129 pgs: 129 active+clean; 528 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.8 MiB/s wr, 280 op/s 2024-05-17T09:45:31.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:30 smithi079 ceph-mon[35502]: pgmap v1210: 129 pgs: 129 active+clean; 528 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.8 MiB/s wr, 280 op/s 2024-05-17T09:45:31.612 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81205 6.31 MB/sec execute 568 sec latency 72.919 ms 2024-05-17T09:45:32.612 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81371 6.31 MB/sec execute 569 sec latency 74.305 ms 2024-05-17T09:45:32.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:32 smithi079 ceph-mon[35502]: pgmap v1211: 129 pgs: 129 active+clean; 529 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-05-17T09:45:32.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:32 smithi069 ceph-mon[35081]: pgmap v1211: 129 pgs: 129 active+clean; 529 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-05-17T09:45:32.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:32 smithi019 ceph-mon[31620]: pgmap v1211: 129 pgs: 129 active+clean; 529 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-05-17T09:45:33.612 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81516 6.31 MB/sec execute 570 sec latency 74.384 ms 2024-05-17T09:45:34.612 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81653 6.31 MB/sec execute 571 sec latency 73.381 ms 2024-05-17T09:45:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:34 smithi069 ceph-mon[35081]: pgmap v1212: 129 pgs: 129 active+clean; 533 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 4.2 MiB/s wr, 282 op/s 2024-05-17T09:45:34.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:34 smithi019 ceph-mon[31620]: pgmap v1212: 129 pgs: 129 active+clean; 533 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 4.2 MiB/s wr, 282 op/s 2024-05-17T09:45:35.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:34 smithi079 ceph-mon[35502]: pgmap v1212: 129 pgs: 129 active+clean; 533 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 4.2 MiB/s wr, 282 op/s 2024-05-17T09:45:35.612 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81815 6.31 MB/sec execute 572 sec latency 96.558 ms 2024-05-17T09:45:36.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 81970 6.32 MB/sec execute 573 sec latency 103.258 ms 2024-05-17T09:45:36.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:36 smithi069 ceph-mon[35081]: pgmap v1213: 129 pgs: 129 active+clean; 533 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-05-17T09:45:36.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:36 smithi019 ceph-mon[31620]: pgmap v1213: 129 pgs: 129 active+clean; 533 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-05-17T09:45:37.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:36 smithi079 ceph-mon[35502]: pgmap v1213: 129 pgs: 129 active+clean; 533 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-05-17T09:45:37.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82130 6.33 MB/sec execute 574 sec latency 101.248 ms 2024-05-17T09:45:38.252 INFO:tasks.quiescer.fs.[cephfs]:Total duration (47.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-17T09:45:38.253 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.95 2024-05-17T09:45:38.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:45:38.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82286 6.33 MB/sec execute 575 sec latency 121.737 ms 2024-05-17T09:45:38.641 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:45:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:38 smithi069 ceph-mon[35081]: pgmap v1214: 129 pgs: 129 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 290 op/s 2024-05-17T09:45:38.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:38 smithi019 ceph-mon[31620]: pgmap v1214: 129 pgs: 129 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 290 op/s 2024-05-17T09:45:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:38 smithi079 ceph-mon[35502]: pgmap v1214: 129 pgs: 129 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 290 op/s 2024-05-17T09:45:39.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82449 6.35 MB/sec execute 576 sec latency 87.333 ms 2024-05-17T09:45:40.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82568 6.34 MB/sec execute 577 sec latency 104.877 ms 2024-05-17T09:45:40.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:40 smithi019 ceph-mon[31620]: pgmap v1215: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-17T09:45:40.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:40 smithi069 ceph-mon[35081]: pgmap v1215: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-17T09:45:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/649365351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb8103300 msgr2=0x7f6fb810f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/649365351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb8103300 0x7f6fb810f880 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b0a0 tx=0x7f6fb000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:45:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/649365351 shutdown_connections 2024-05-17T09:45:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/649365351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb81089e0 0x7f6fb8108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/649365351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb8103300 0x7f6fb810f880 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/649365351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/649365351 >> 172.21.15.19:0/649365351 conn(0x7f6fb80fe6e0 msgr2=0x7f6fb8100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/649365351 shutdown_connections 2024-05-17T09:45:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.992+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/649365351 wait complete. 2024-05-17T09:45:40.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 Processor -- start 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 -- start start 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8103300 0x7f6fb819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb81089e0 0x7f6fb819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81143f0 con 0x7f6fb81089e0 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8114270 con 0x7f6fb81029b0 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbf0b4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8114570 con 0x7f6fb8103300 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbce50700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbce50700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34786/0 (socket says 172.21.15.19:34786) 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbce50700 1 -- 172.21.15.19:0/1045006851 learned_addr learned my addr 172.21.15.19:0/1045006851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.993+0000 7f6fbd651700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb81089e0 0x7f6fb819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fb7fff700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8103300 0x7f6fb819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbce50700 1 -- 172.21.15.19:0/1045006851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8103300 msgr2=0x7f6fb819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbce50700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8103300 0x7f6fb819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbce50700 1 -- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb81089e0 msgr2=0x7f6fb819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbce50700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb81089e0 0x7f6fb819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbce50700 1 -- 172.21.15.19:0/1045006851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb819fcb0 con 0x7f6fb81029b0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbce50700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb819ade0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e970 tx=0x7f6fac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac004a30 con 0x7f6fb81029b0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb819ff40 con 0x7f6fb81029b0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fac00cc70 con 0x7f6fb81029b0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb8110f00 con 0x7f6fb81029b0 2024-05-17T09:45:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.994+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac005520 con 0x7f6fb81029b0 2024-05-17T09:45:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.996+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f9c005330 con 0x7f6fb81029b0 2024-05-17T09:45:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.997+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6fac004bd0 con 0x7f6fb81029b0 2024-05-17T09:45:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.998+0000 7f6fb5ffb700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6fa80b2900 0x7f6fa80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.998+0000 7f6fb7fff700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6fa80b2900 0x7f6fa80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.999+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f6fac0e2550 con 0x7f6fb81029b0 2024-05-17T09:45:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:40.999+0000 7f6fb7fff700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6fa80b2900 0x7f6fa80b4e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009680 tx=0x7f6fa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.003+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6fac00ce20 con 0x7f6fb81029b0 2024-05-17T09:45:41.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:40 smithi079 ceph-mon[35502]: pgmap v1215: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-17T09:45:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.438+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f9c005620 con 0x7f6fb81029b0 2024-05-17T09:45:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.439+0000 7f6fb5ffb700 1 -- 172.21.15.19:0/1045006851 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f6fac0a9ec0 con 0x7f6fb81029b0 2024-05-17T09:45:41.438 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:45:41.438 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:45:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.442+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6fa80b2900 msgr2=0x7f6fa80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.442+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6fa80b2900 0x7f6fa80b4e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009680 tx=0x7f6fa4009260 comp rx=0 tx=0).stop 2024-05-17T09:45:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.442+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 msgr2=0x7f6fb819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.442+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb819ade0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e970 tx=0x7f6fac00ec80 comp rx=0 tx=0).stop 2024-05-17T09:45:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 shutdown_connections 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb81089e0 0x7f6fb819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fb81029b0 0x7f6fb819ade0 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e970 tx=0x7f6fac00ec80 comp rx=0 tx=0).stop 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f6fa80b2900 0x7f6fa80b4e10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 --2- 172.21.15.19:0/1045006851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8103300 0x7f6fb819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 >> 172.21.15.19:0/1045006851 conn(0x7f6fb80fe6e0 msgr2=0x7f6fb8107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.443+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 shutdown_connections 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:41.444+0000 7f6fbf0b4700 1 -- 172.21.15.19:0/1045006851 wait complete. 2024-05-17T09:45:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:45:41.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82698 6.33 MB/sec execute 578 sec latency 82.943 ms 2024-05-17T09:45:41.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:41 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1045006851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:45:41.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:41 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1045006851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:45:42.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:41 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1045006851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:45:42.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82833 6.33 MB/sec execute 579 sec latency 168.444 ms 2024-05-17T09:45:42.866 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:42 smithi019 ceph-mon[31620]: pgmap v1216: 129 pgs: 129 active+clean; 524 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-05-17T09:45:42.909 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0' 2024-05-17T09:45:42.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0 2024-05-17T09:45:42.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:42 smithi069 ceph-mon[35081]: pgmap v1216: 129 pgs: 129 active+clean; 524 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-05-17T09:45:43.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:42 smithi079 ceph-mon[35502]: pgmap v1216: 129 pgs: 129 active+clean; 524 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-05-17T09:45:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:45:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 -- 172.21.15.19:0/3155872717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 msgr2=0x7fcdd81041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 --2- 172.21.15.19:0/3155872717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 0x7fcdd81041a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a3f0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 -- 172.21.15.19:0/3155872717 shutdown_connections 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 --2- 172.21.15.19:0/3155872717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8108a90 0x7fcdd810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 --2- 172.21.15.19:0/3155872717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 0x7fcdd81041a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 --2- 172.21.15.19:0/3155872717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8102b50 0x7fcdd8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 -- 172.21.15.19:0/3155872717 >> 172.21.15.19:0/3155872717 conn(0x7fcdd80fdd40 msgr2=0x7fcdd81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 -- 172.21.15.19:0/3155872717 shutdown_connections 2024-05-17T09:45:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.586+0000 7fcde03a9640 1 -- 172.21.15.19:0/3155872717 wait complete. 2024-05-17T09:45:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 Processor -- start 2024-05-17T09:45:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 -- start start 2024-05-17T09:45:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 0x7fcdd81a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 0x7fcdd81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8108a90 0x7fcdd81a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd810d460 con 0x7fcdd8108a90 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd810d2e0 con 0x7fcdd8103d90 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.587+0000 7fcde03a9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd810d5e0 con 0x7fcdd8102b50 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 0x7fcdd81a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 0x7fcdd81a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35220/0 (socket says 172.21.15.19:35220) 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 -- 172.21.15.19:0/2389888657 learned_addr learned my addr 172.21.15.19:0/2389888657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcddd944640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 0x7fcdd81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde946640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8108a90 0x7fcdd81a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 -- 172.21.15.19:0/2389888657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 msgr2=0x7fcdd81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 0x7fcdd81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 -- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8108a90 msgr2=0x7fcdd81a7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8108a90 0x7fcdd81a7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde145640 1 -- 172.21.15.19:0/2389888657 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd81a82f0 con 0x7fcdd8102b50 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.588+0000 7fcdde946640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8108a90 0x7fcdd81a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.589+0000 7fcdde145640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 0x7fcdd81a07b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000ed00 tx=0x7fcdd000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.589+0000 7fcdd77fe640 1 -- 172.21.15.19:0/2389888657 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0010630 con 0x7fcdd8102b50 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.590+0000 7fcdd77fe640 1 -- 172.21.15.19:0/2389888657 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd0010c60 con 0x7fcdd8102b50 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.590+0000 7fcdd77fe640 1 -- 172.21.15.19:0/2389888657 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0010630 con 0x7fcdd8102b50 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.590+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd81a85e0 con 0x7fcdd8102b50 2024-05-17T09:45:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.590+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd81a8b00 con 0x7fcdd8102b50 2024-05-17T09:45:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 Processor -- start 2024-05-17T09:45:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 -- start start 2024-05-17T09:45:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcdd77fe640 1 -- 172.21.15.19:0/2389888657 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcdd001f020 con 0x7fcdd8102b50 2024-05-17T09:45:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 0x55617b62d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b62d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 0x55617b634820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55617b59fc30 con 0x55617b2632f0 2024-05-17T09:45:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55617b58f7d0 con 0x55617b28c1b0 2024-05-17T09:45:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.594+0000 7fcde725d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55617b593810 con 0x55617b26b4d0 2024-05-17T09:45:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcddd944640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 0x55617b634820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 0x55617b62d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 0x55617b62d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34802/0 (socket says 172.21.15.19:34802) 2024-05-17T09:45:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcddd944640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 0x55617b634820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35232/0 (socket says 172.21.15.19:35232) 2024-05-17T09:45:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 -- 172.21.15.19:0/699313009 learned_addr learned my addr 172.21.15.19:0/699313009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde145640 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b62d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 -- 172.21.15.19:0/699313009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 msgr2=0x55617b634820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 0x55617b634820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 -- 172.21.15.19:0/699313009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 msgr2=0x55617b62d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b62d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.595+0000 7fcdde946640 1 -- 172.21.15.19:0/699313009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617b2a4210 con 0x55617b28c1b0 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdde145640 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b62d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdd77fe640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc00b2ae0 0x7fcdc00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcddd944640 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 0x55617b634820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdde145640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc00b2ae0 0x7fcdc00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdde946640 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 0x55617b62d3a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009bd0 tx=0x7fcdc800e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1045513c2cd4b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdd77fe640 1 -- 172.21.15.19:0/2389888657 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcdd00e2630 con 0x7fcdd8102b50 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdd4bfe640 1 -- 172.21.15.19:0/699313009 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8016070 con 0x55617b28c1b0 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdd4bfe640 1 -- 172.21.15.19:0/699313009 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdc8011900 con 0x55617b28c1b0 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.596+0000 7fcdde145640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc00b2ae0 0x7fcdc00b4ff0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55617b62e780 tx=0x7fcdd00e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.597+0000 7fcdd4bfe640 1 -- 172.21.15.19:0/699313009 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80106e0 con 0x55617b28c1b0 2024-05-17T09:45:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.597+0000 7fcde725d740 1 -- 172.21.15.19:0/699313009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 msgr2=0x55617b62d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.597+0000 7fcde725d740 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 0x55617b62d3a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009bd0 tx=0x7fcdc800e6c0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 -- 172.21.15.19:0/699313009 shutdown_connections 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b26b4d0 0x55617b634820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b62d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 --2- 172.21.15.19:0/699313009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b28c1b0 0x55617b62d3a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 -- 172.21.15.19:0/699313009 >> 172.21.15.19:0/699313009 conn(0x55617b280300 msgr2=0x55617b628d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 -- 172.21.15.19:0/699313009 shutdown_connections 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 -- 172.21.15.19:0/699313009 wait complete. 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 -- start start 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.598+0000 7fcde725d740 1 Processor -- start 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b636d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b637290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55617b275790 con 0x55617b28c1b0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55617b54b890 con 0x55617b26b4d0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55617b635270 con 0x55617b2632f0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcde725d740 10 client.0 _ll_register_callbacks cb 0x55617b54ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b637290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcdde946640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b636d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b637290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:34814/0 (socket says 172.21.15.19:34814) 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 -- 172.21.15.19:0/1508814629 learned_addr learned my addr 172.21.15.19:0/1508814629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcdde946640 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 -- 172.21.15.19:0/1508814629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 msgr2=0x55617b636d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b636d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 -- 172.21.15.19:0/1508814629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 msgr2=0x55617b63e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.599+0000 7fcddd944640 1 -- 172.21.15.19:0/1508814629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617b370620 con 0x55617b26b4d0 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcddd944640 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b637290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02a9e0 tx=0x7fcdcc004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a50c3c3cf87fa6e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcdbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcdbdffb640 1 -- 172.21.15.19:0/1508814629 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035070 con 0x55617b26b4d0 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc8011900 con 0x55617b26b4d0 2024-05-17T09:45:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcdbdffb640 1 -- 172.21.15.19:0/1508814629 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcdcc003ac0 con 0x55617b26b4d0 2024-05-17T09:45:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcdbdffb640 1 -- 172.21.15.19:0/1508814629 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc04d050 con 0x55617b26b4d0 2024-05-17T09:45:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcde725d740 10 client.28763 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55617b6411e0 con 0x55617b26b4d0 2024-05-17T09:45:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.600+0000 7fcdbdffb640 1 -- 172.21.15.19:0/1508814629 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdc8011900 con 0x55617b26b4d0 2024-05-17T09:45:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcdbdffb640 1 -- 172.21.15.19:0/1508814629 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcdcc03b430 con 0x55617b26b4d0 2024-05-17T09:45:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcde725d740 10 client.28763 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcde725d740 10 client.28763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcde725d740 20 client.28763 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcde725d740 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b24ad80 0x55617b6448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcde725d740 4 client.28763 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617b5a1b80 con 0x55617b24ad80 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcdde946640 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b24ad80 0x55617b6448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.601+0000 7fcdbdffb640 10 client.28763 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.602+0000 7fcdde946640 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b24ad80 0x55617b6448e0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3557470694939564 server_cookie=87fdedd5c01d5740 in_seq=0 out_seq=0 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.602+0000 7fcdbdffb640 1 -- 172.21.15.19:0/1508814629 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55617b5a1b80 con 0x55617b24ad80 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.602+0000 7fcdbdffb640 10 client.28763 handle_command_reply: tid=0 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.602+0000 7fcde725d740 1 client.28763 shutdown 2024-05-17T09:45:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 msgr2=0x55617b637290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b637290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02a9e0 tx=0x7fcdcc004510 comp rx=0 tx=0).stop 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 shutdown_connections 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b24ad80 0x55617b6448e0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b637290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcdbdffb640 0 client.28763 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 --2- 172.21.15.19:0/1508814629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b636d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 >> 172.21.15.19:0/1508814629 conn(0x55617b280300 msgr2=0x55617b62bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 shutdown_connections 2024-05-17T09:45:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 1 -- 172.21.15.19:0/1508814629 wait complete. 2024-05-17T09:45:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.603+0000 7fcde725d740 20 client.28763 trim_cache size 0 max 16384 2024-05-17T09:45:43.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 82970 6.33 MB/sec execute 580 sec latency 106.194 ms 2024-05-17T09:45:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.629+0000 7fcde725d740 1 Processor -- start 2024-05-17T09:45:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.631+0000 7fcde725d740 1 -- start start 2024-05-17T09:45:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.632+0000 7fcde725d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b64a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.632+0000 7fcde725d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b64a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.632+0000 7fcde725d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.632+0000 7fcde725d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b5950 con 0x55617b28c1b0 2024-05-17T09:45:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.632+0000 7fcde725d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b3220 con 0x55617b26b4d0 2024-05-17T09:45:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.632+0000 7fcde725d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b1070 con 0x55617b2632f0 2024-05-17T09:45:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b64a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b64a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:35250/0 (socket says 172.21.15.19:35250) 2024-05-17T09:45:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 -- 172.21.15.19:0/2887275863 learned_addr learned my addr 172.21.15.19:0/2887275863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 -- 172.21.15.19:0/2887275863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 msgr2=0x55617b64a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b64a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 -- 172.21.15.19:0/2887275863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 msgr2=0x55617b63e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.633+0000 7fcdde946640 1 -- 172.21.15.19:0/2887275863 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617b637610 con 0x55617b2632f0 2024-05-17T09:45:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.634+0000 7fcdde946640 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b64a0c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009750 tx=0x7fcdc8005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d13c728bea1a2795 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.635+0000 7fcdbffff640 1 -- 172.21.15.19:0/2887275863 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8016040 con 0x55617b2632f0 2024-05-17T09:45:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.635+0000 7fcdbffff640 1 -- 172.21.15.19:0/2887275863 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdc802daf0 con 0x55617b2632f0 2024-05-17T09:45:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.635+0000 7fcdbffff640 1 -- 172.21.15.19:0/2887275863 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8025040 con 0x55617b2632f0 2024-05-17T09:45:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 -- 172.21.15.19:0/2887275863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 msgr2=0x55617b64a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b64a0c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009750 tx=0x7fcdc8005e50 comp rx=0 tx=0).stop 2024-05-17T09:45:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 -- 172.21.15.19:0/2887275863 shutdown_connections 2024-05-17T09:45:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b28c1b0 0x55617b63e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b64a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 --2- 172.21.15.19:0/2887275863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b2632f0 0x55617b64a0c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.637+0000 7fcde725d740 1 -- 172.21.15.19:0/2887275863 >> 172.21.15.19:0/2887275863 conn(0x55617b280300 msgr2=0x55617b630940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.638+0000 7fcde725d740 1 -- 172.21.15.19:0/2887275863 shutdown_connections 2024-05-17T09:45:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.638+0000 7fcde725d740 1 -- 172.21.15.19:0/2887275863 wait complete. 2024-05-17T09:45:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.639+0000 7fcde725d740 1 -- start start 2024-05-17T09:45:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.639+0000 7fcde725d740 1 Processor -- start 2024-05-17T09:45:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.639+0000 7fcde725d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b63bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.639+0000 7fcde725d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b63c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcde725d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b28c1b0 0x55617b63d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcde725d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b16e0 con 0x55617b2632f0 2024-05-17T09:45:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcde725d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b2b50 con 0x55617b26b4d0 2024-05-17T09:45:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcde725d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81b1db0 con 0x55617b28c1b0 2024-05-17T09:45:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcde725d740 10 client.0 _ll_register_callbacks cb 0x7fcdcc0041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcdde946640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b63bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcdde946640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b63bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47734/0 (socket says 172.21.15.19:47734) 2024-05-17T09:45:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcdde946640 1 -- 172.21.15.19:0/2516860277 learned_addr learned my addr 172.21.15.19:0/2516860277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcdde946640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b28c1b0 0x55617b63d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.640+0000 7fcddd944640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b63c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.642+0000 7fcddd944640 1 -- 172.21.15.19:0/2516860277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b28c1b0 msgr2=0x55617b63d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.642+0000 7fcddd944640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b28c1b0 0x55617b63d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.642+0000 7fcddd944640 1 -- 172.21.15.19:0/2516860277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 msgr2=0x55617b63bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.642+0000 7fcddd944640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b63bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.642+0000 7fcddd944640 1 -- 172.21.15.19:0/2516860277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617b63d720 con 0x55617b26b4d0 2024-05-17T09:45:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.642+0000 7fcddd944640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b63c230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55617b634a20 tx=0x7fcdcc03bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec37c7bcd36c73c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcdbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcdbdffb640 1 -- 172.21.15.19:0/2516860277 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035070 con 0x55617b26b4d0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcdbdffb640 1 -- 172.21.15.19:0/2516860277 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcdcc003ac0 con 0x55617b26b4d0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc802daf0 con 0x55617b26b4d0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcdbdffb640 1 -- 172.21.15.19:0/2516860277 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc005df0 con 0x55617b26b4d0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcde725d740 10 client.28766 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.643+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55617b63e9b0 con 0x55617b26b4d0 2024-05-17T09:45:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.644+0000 7fcdbdffb640 1 -- 172.21.15.19:0/2516860277 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdcc03b640 con 0x55617b26b4d0 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.645+0000 7fcdbdffb640 1 -- 172.21.15.19:0/2516860277 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcdcc031500 con 0x55617b26b4d0 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.646+0000 7fcde725d740 10 client.28766 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.646+0000 7fcde725d740 10 client.28766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.646+0000 7fcde725d740 20 client.28766 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.646+0000 7fcde725d740 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b2201c0 0x55617b6c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.646+0000 7fcde725d740 4 client.28766 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.646+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcdc802daf0 con 0x55617b2201c0 2024-05-17T09:45:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.647+0000 7fcdde946640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b2201c0 0x55617b6c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.647+0000 7fcdbdffb640 10 client.28766 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.647+0000 7fcdde946640 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b2201c0 0x55617b6c0390 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99e31c1bb33c5bb server_cookie=2eed0000d78336b9 in_seq=0 out_seq=0 2024-05-17T09:45:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcdbdffb640 1 -- 172.21.15.19:0/2516860277 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fcdc802daf0 con 0x55617b2201c0 2024-05-17T09:45:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcdbdffb640 10 client.28766 handle_command_reply: tid=0 2024-05-17T09:45:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 client.28766 shutdown 2024-05-17T09:45:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 msgr2=0x55617b63c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b63c230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55617b634a20 tx=0x7fcdcc03bd70 comp rx=0 tx=0).stop 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 shutdown_connections 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55617b2201c0 0x55617b6c0390 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55617b28c1b0 0x55617b63d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55617b26b4d0 0x55617b63c230 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 --2- 172.21.15.19:0/2516860277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55617b2632f0 0x55617b63bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 >> 172.21.15.19:0/2516860277 conn(0x55617b280300 msgr2=0x55617b6312e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.794+0000 7fcdbdffb640 0 client.28766 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.795+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 shutdown_connections 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.795+0000 7fcde725d740 1 -- 172.21.15.19:0/2516860277 wait complete. 2024-05-17T09:45:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.795+0000 7fcde725d740 20 client.28766 trim_cache size 0 max 16384 2024-05-17T09:45:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:45:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:45:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.796+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc00b2ae0 msgr2=0x7fcdc00b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.796+0000 7fcde03a9640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc00b2ae0 0x7fcdc00b4ff0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55617b62e780 tx=0x7fcdd00e8550 comp rx=0 tx=0).stop 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.796+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 msgr2=0x7fcdd81a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.796+0000 7fcde03a9640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 0x7fcdd81a07b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000ed00 tx=0x7fcdd000c6a0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.797+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 shutdown_connections 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.797+0000 7fcde03a9640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fcdc00b2ae0 0x7fcdc00b4ff0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.797+0000 7fcde03a9640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd8108a90 0x7fcdd81a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.797+0000 7fcde03a9640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdd8103d90 0x7fcdd81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.797+0000 7fcde03a9640 1 --2- 172.21.15.19:0/2389888657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdd8102b50 0x7fcdd81a07b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.797+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 >> 172.21.15.19:0/2389888657 conn(0x7fcdd80fdd40 msgr2=0x7fcdd8104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.798+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 shutdown_connections 2024-05-17T09:45:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:43.798+0000 7fcde03a9640 1 -- 172.21.15.19:0/2389888657 wait complete. 2024-05-17T09:45:44.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83068 6.32 MB/sec execute 581 sec latency 162.851 ms 2024-05-17T09:45:44.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:44 smithi019 ceph-mon[31620]: pgmap v1217: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-17T09:45:44.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:44 smithi069 ceph-mon[35081]: pgmap v1217: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-17T09:45:45.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:44 smithi079 ceph-mon[35502]: pgmap v1217: 129 pgs: 129 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-17T09:45:45.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83206 6.32 MB/sec execute 582 sec latency 81.015 ms 2024-05-17T09:45:46.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83313 6.32 MB/sec execute 583 sec latency 168.519 ms 2024-05-17T09:45:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:46 smithi069 ceph-mon[35081]: pgmap v1218: 129 pgs: 129 active+clean; 526 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-05-17T09:45:46.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:46 smithi019 ceph-mon[31620]: pgmap v1218: 129 pgs: 129 active+clean; 526 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-05-17T09:45:47.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:46 smithi079 ceph-mon[35502]: pgmap v1218: 129 pgs: 129 active+clean; 526 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-05-17T09:45:47.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83448 6.33 MB/sec execute 584 sec latency 142.205 ms 2024-05-17T09:45:48.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83594 6.34 MB/sec execute 585 sec latency 171.477 ms 2024-05-17T09:45:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:48 smithi069 ceph-mon[35081]: pgmap v1219: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-05-17T09:45:48.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:48 smithi019 ceph-mon[31620]: pgmap v1219: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-05-17T09:45:49.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:48 smithi079 ceph-mon[35502]: pgmap v1219: 129 pgs: 129 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-05-17T09:45:49.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83760 6.34 MB/sec execute 586 sec latency 154.585 ms 2024-05-17T09:45:50.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 83940 6.35 MB/sec execute 587 sec latency 171.179 ms 2024-05-17T09:45:50.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:50 smithi019 ceph-mon[31620]: pgmap v1220: 129 pgs: 129 active+clean; 524 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-17T09:45:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:50 smithi069 ceph-mon[35081]: pgmap v1220: 129 pgs: 129 active+clean; 524 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-17T09:45:51.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:50 smithi079 ceph-mon[35502]: pgmap v1220: 129 pgs: 129 active+clean; 524 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-17T09:45:51.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84040 6.35 MB/sec execute 588 sec latency 175.444 ms 2024-05-17T09:45:52.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84139 6.35 MB/sec execute 589 sec latency 161.887 ms 2024-05-17T09:45:52.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:52 smithi019 ceph-mon[31620]: pgmap v1221: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.4 MiB/s wr, 240 op/s 2024-05-17T09:45:52.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:52 smithi069 ceph-mon[35081]: pgmap v1221: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.4 MiB/s wr, 240 op/s 2024-05-17T09:45:53.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:52 smithi079 ceph-mon[35502]: pgmap v1221: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.4 MiB/s wr, 240 op/s 2024-05-17T09:45:53.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.562+0000 7f19c7b92700 1 -- 172.21.15.19:0/683135797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 msgr2=0x7f19c01038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.562+0000 7f19c7b92700 1 --2- 172.21.15.19:0/683135797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01038c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f19b80098f0 tx=0x7f19b8009c00 comp rx=0 tx=0).stop 2024-05-17T09:45:53.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 -- 172.21.15.19:0/683135797 shutdown_connections 2024-05-17T09:45:53.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 --2- 172.21.15.19:0/683135797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19c0108730 0x7f19c0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 --2- 172.21.15.19:0/683135797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01038c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 --2- 172.21.15.19:0/683135797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c0107de0 0x7f19c01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 -- 172.21.15.19:0/683135797 >> 172.21.15.19:0/683135797 conn(0x7f19c00fede0 msgr2=0x7f19c0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 -- 172.21.15.19:0/683135797 shutdown_connections 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.563+0000 7f19c7b92700 1 -- 172.21.15.19:0/683135797 wait complete. 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.564+0000 7f19c7b92700 1 Processor -- start 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.564+0000 7f19c7b92700 1 -- start start 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c7b92700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c7b92700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c0107de0 0x7f19c01a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c7b92700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19c0108730 0x7f19c019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c7b92700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19c0113260 con 0x7f19c0107de0 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c7b92700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01130e0 con 0x7f19c01034b0 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c7b92700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01133e0 con 0x7f19c0108730 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:35980/0 (socket says 172.21.15.19:35980) 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 -- 172.21.15.19:0/3042278462 learned_addr learned my addr 172.21.15.19:0/3042278462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c512d700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c0107de0 0x7f19c01a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c612f700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19c0108730 0x7f19c019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 -- 172.21.15.19:0/3042278462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19c0108730 msgr2=0x7f19c019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19c0108730 0x7f19c019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 -- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c0107de0 msgr2=0x7f19c01a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c0107de0 0x7f19c01a31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.565+0000 7f19c592e700 1 -- 172.21.15.19:0/3042278462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c019d610 con 0x7f19c01034b0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.566+0000 7f19c592e700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01a2cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f19b8005950 tx=0x7f19b800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.566+0000 7f19beffd700 1 -- 172.21.15.19:0/3042278462 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b800b790 con 0x7f19c01034b0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.566+0000 7f19beffd700 1 -- 172.21.15.19:0/3042278462 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19b80054d0 con 0x7f19c01034b0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.566+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c019d8a0 con 0x7f19c01034b0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.566+0000 7f19beffd700 1 -- 172.21.15.19:0/3042278462 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8003ab0 con 0x7f19c01034b0 2024-05-17T09:45:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.566+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c01a9ad0 con 0x7f19c01034b0 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.570+0000 7f19beffd700 1 -- 172.21.15.19:0/3042278462 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19b800b930 con 0x7f19c01034b0 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.572+0000 7f19beffd700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19a80b2880 0x7f19a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.572+0000 7f19c512d700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19a80b2880 0x7f19a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.572+0000 7f19beffd700 1 -- 172.21.15.19:0/3042278462 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f19b80e34f0 con 0x7f19c01034b0 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.572+0000 7f19d65a7b80 1 Processor -- start 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.572+0000 7f19d65a7b80 1 -- start start 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19c512d700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19a80b2880 0x7f19a80b4d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f19c0107910 tx=0x7f19b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19c592e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19c592e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d19f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54950/0 (socket says 172.21.15.19:54950) 2024-05-17T09:45:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19c592e700 1 -- 172.21.15.19:0/1022352935 learned_addr learned my addr 172.21.15.19:0/1022352935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565037c41f10 con 0x565037c805d0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19c512d700 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.573+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565037c31ab0 con 0x565037d140c0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c612f700 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565037c35af0 con 0x565037b94fe0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c592e700 1 -- 172.21.15.19:0/1022352935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 msgr2=0x565037d1a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c592e700 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d1a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c592e700 1 -- 172.21.15.19:0/1022352935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 msgr2=0x565037d14590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c592e700 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d14590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c592e700 1 -- 172.21.15.19:0/1022352935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565037bdb150 con 0x565037c805d0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.574+0000 7f19c592e700 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d19f30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f19b80aa4f0 tx=0x7f19b800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37ca926ac9a5ded server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.575+0000 7f19a7fff700 1 -- 172.21.15.19:0/1022352935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80aaef0 con 0x565037c805d0 2024-05-17T09:45:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.575+0000 7f19a7fff700 1 -- 172.21.15.19:0/1022352935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19b80aa7f0 con 0x565037c805d0 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.575+0000 7f19a7fff700 1 -- 172.21.15.19:0/1022352935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80e31f0 con 0x565037c805d0 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 msgr2=0x565037d19f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d19f30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f19b80aa4f0 tx=0x7f19b800fa70 comp rx=0 tx=0).stop 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 shutdown_connections 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d19f30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1022352935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.576+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 >> 172.21.15.19:0/1022352935 conn(0x565037bf5360 msgr2=0x565037d0ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.577+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 shutdown_connections 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.577+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1022352935 wait complete. 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.577+0000 7f19d65a7b80 1 -- start start 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.577+0000 7f19d65a7b80 1 Processor -- start 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d23680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037d140c0 0x565037d1da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d23680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565037b38140 con 0x565037d140c0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d23680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:57162/0 (socket says 172.21.15.19:57162) 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 -- 172.21.15.19:0/2725668611 learned_addr learned my addr 172.21.15.19:0/2725668611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c612f700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037d140c0 0x565037d1da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565037b99f40 con 0x565037c805d0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565037d14de0 con 0x565037b94fe0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c512d700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 10 client.0 _ll_register_callbacks cb 0x565037bccaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 -- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 msgr2=0x565037d23bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d23bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 -- 172.21.15.19:0/2725668611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037d140c0 msgr2=0x565037d1da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037d140c0 0x565037d1da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19c592e700 1 -- 172.21.15.19:0/2725668611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565037b81a70 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.578+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b80aa7f0 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19a57fa700 10 client.28756 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19c592e700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d23680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f19b80092a0 tx=0x7f19b80aa030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65be3bd3419bf499 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19a57fa700 1 -- 172.21.15.19:0/2725668611 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80c4060 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19a57fa700 1 -- 172.21.15.19:0/2725668611 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f19b80a6a80 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19a57fa700 1 -- 172.21.15.19:0/2725668611 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19b80aaef0 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 10 client.28756 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=39}) -- 0x565037beb900 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19a57fa700 1 -- 172.21.15.19:0/2725668611 <== mon.1 v2:172.21.15.79:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f19b8070080 con 0x565037b94fe0 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 10 client.28756 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 10 client.28756 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:45:53.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 20 client.28756 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d2d4a0 0x565037d2f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 4 client.28756 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.579+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565037ce0070 con 0x565037d2d4a0 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.580+0000 7f19c512d700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d2d4a0 0x565037d2f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.580+0000 7f19a57fa700 10 client.28756 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.580+0000 7f19c512d700 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d2d4a0 0x565037d2f8c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5be22a4b4b2e840 server_cookie=27c96fe6947df83d in_seq=0 out_seq=0 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.581+0000 7f19a57fa700 1 -- 172.21.15.19:0/2725668611 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565037ce0070 con 0x565037d2d4a0 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.581+0000 7f19a57fa700 10 client.28756 handle_command_reply: tid=0 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.581+0000 7f19d65a7b80 1 client.28756 shutdown 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 msgr2=0x565037d23680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d23680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f19b80092a0 tx=0x7f19b80aa030 comp rx=0 tx=0).stop 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 shutdown_connections 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d23680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d2d4a0 0x565037d2f8c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037d140c0 0x565037d1da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19a57fa700 0 client.28756 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/2725668611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 >> 172.21.15.19:0/2725668611 conn(0x565037bf5360 msgr2=0x565037d130b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 shutdown_connections 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 1 -- 172.21.15.19:0/2725668611 wait complete. 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.582+0000 7f19d65a7b80 20 client.28756 trim_cache size 0 max 16384 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.609+0000 7f19d65a7b80 1 Processor -- start 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.609+0000 7f19d65a7b80 1 -- start start 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 0x565037d0eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d17f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037d140c0 0x565037d08c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19d65a7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b4d90 con 0x565037b94fe0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19d65a7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b2620 con 0x565037c805d0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19d65a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b0470 con 0x565037d140c0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 0x565037d0eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 0x565037d0eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54982/0 (socket says 172.21.15.19:54982) 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 -- 172.21.15.19:0/3698624914 learned_addr learned my addr 172.21.15.19:0/3698624914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c612f700 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037d140c0 0x565037d08c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c512d700 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d17f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 -- 172.21.15.19:0/3698624914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037d140c0 msgr2=0x565037d08c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037d140c0 0x565037d08c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 -- 172.21.15.19:0/3698624914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 msgr2=0x565037d17f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d17f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.610+0000 7f19c592e700 1 -- 172.21.15.19:0/3698624914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565037d23d30 con 0x565037b94fe0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.611+0000 7f19c592e700 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 0x565037d0eeb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f19b80e0c90 tx=0x7f19b8024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12da9db3d1e50706 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.611+0000 7f19a77fe700 1 -- 172.21.15.19:0/3698624914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80aebf0 con 0x565037b94fe0 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.611+0000 7f19a77fe700 1 -- 172.21.15.19:0/3698624914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19b80aa7f0 con 0x565037b94fe0 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.611+0000 7f19a77fe700 1 -- 172.21.15.19:0/3698624914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8003c50 con 0x565037b94fe0 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 -- 172.21.15.19:0/3698624914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 msgr2=0x565037d0eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 0x565037d0eeb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f19b80e0c90 tx=0x7f19b8024000 comp rx=0 tx=0).stop 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 -- 172.21.15.19:0/3698624914 shutdown_connections 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037d140c0 0x565037d08c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037b94fe0 0x565037d0eeb0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/3698624914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037c805d0 0x565037d17f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 -- 172.21.15.19:0/3698624914 >> 172.21.15.19:0/3698624914 conn(0x565037bf5360 msgr2=0x565037d17840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 -- 172.21.15.19:0/3698624914 shutdown_connections 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.612+0000 7f19d65a7b80 1 -- 172.21.15.19:0/3698624914 wait complete. 2024-05-17T09:45:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19d65a7b80 1 -- start start 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19d65a7b80 1 Processor -- start 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d8c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19d65a7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19c512d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d8c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19c592e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19c512d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d8c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54988/0 (socket says 172.21.15.19:54988) 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.613+0000 7f19c592e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d8bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:57182/0 (socket says 172.21.15.19:57182) 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c612f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c612f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d96d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:36020/0 (socket says 172.21.15.19:36020) 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 -- 172.21.15.19:0/1571661490 learned_addr learned my addr 172.21.15.19:0/1571661490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b0ae0 con 0x565037c805d0 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b1f50 con 0x565037d140c0 2024-05-17T09:45:53.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b11b0 con 0x565037b94fe0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19d65a7b80 10 client.0 _ll_register_callbacks cb 0x565037bccaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 -- 172.21.15.19:0/1571661490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 msgr2=0x565037d8bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d8bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 -- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 msgr2=0x565037d96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d96d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565037d97260 con 0x565037c805d0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c592e700 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d8bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19c512d700 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d8c3e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f19c0107bc0 tx=0x7f19b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa825047f8fd40c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.614+0000 7f19a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19a57fa700 1 -- 172.21.15.19:0/1571661490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b402a400 con 0x565037c805d0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19a57fa700 1 -- 172.21.15.19:0/1571661490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f19b402a5a0 con 0x565037c805d0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b80aa7f0 con 0x565037c805d0 2024-05-17T09:45:53.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19a57fa700 1 -- 172.21.15.19:0/1571661490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19b402a7a0 con 0x565037c805d0 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19d65a7b80 10 client.19293 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565037d98470 con 0x565037c805d0 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19a57fa700 1 -- 172.21.15.19:0/1571661490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19b80aa7f0 con 0x565037c805d0 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.615+0000 7f19a57fa700 1 -- 172.21.15.19:0/1571661490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f19b4005020 con 0x565037c805d0 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19d65a7b80 10 client.19293 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19d65a7b80 10 client.19293 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19d65a7b80 20 client.19293 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d98b70 0x565037d9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19d65a7b80 4 client.19293 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}] 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}) -- 0x565037c43e60 con 0x565037d98b70 2024-05-17T09:45:53.417 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 94, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "28d5df25": { 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 94, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 143.9, 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:45:53.418 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19c592e700 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d98b70 0x565037d9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:53.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19a57fa700 10 client.19293 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.616+0000 7f19c592e700 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d98b70 0x565037d9af90 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1972befc12e68499 server_cookie=bd7c10dbc2804326 in_seq=0 out_seq=0 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19a57fa700 1 -- 172.21.15.19:0/1571661490 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565037c43e60 con 0x565037d98b70 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19a57fa700 10 client.19293 handle_command_reply: tid=0 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19d65a7b80 1 client.19293 shutdown 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 msgr2=0x565037d8c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d8c3e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f19c0107bc0 tx=0x7f19b40047c0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 shutdown_connections 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.992+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565037b94fe0 0x565037d8bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x565037d98b70 0x565037d9af90 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19a57fa700 0 client.19293 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565037d140c0 0x565037d96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 1 --2- 172.21.15.19:0/1571661490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565037c805d0 0x565037d8c3e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 >> 172.21.15.19:0/1571661490 conn(0x565037bf5360 msgr2=0x565037d18a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 shutdown_connections 2024-05-17T09:45:53.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 1 -- 172.21.15.19:0/1571661490 wait complete. 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.993+0000 7f19d65a7b80 20 client.19293 trim_cache size 0 max 16384 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19a80b2880 msgr2=0x7f19a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19a80b2880 0x7f19a80b4d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f19c0107910 tx=0x7f19b4009260 comp rx=0 tx=0).stop 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 msgr2=0x7f19c01a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01a2cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f19b8005950 tx=0x7f19b800fa00 comp rx=0 tx=0).stop 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 shutdown_connections 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f19a80b2880 0x7f19a80b4d90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19c0108730 0x7f19c019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c01034b0 0x7f19c01a2cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 --2- 172.21.15.19:0/3042278462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c0107de0 0x7f19c01a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 >> 172.21.15.19:0/3042278462 conn(0x7f19c00fede0 msgr2=0x7f19c0104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 shutdown_connections 2024-05-17T09:45:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:45:52.994+0000 7f19c7b92700 1 -- 172.21.15.19:0/3042278462 wait complete. 2024-05-17T09:45:53.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.34 MB/sec execute 590 sec latency 999.103 ms 2024-05-17T09:45:54.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.33 MB/sec execute 591 sec latency 1999.214 ms 2024-05-17T09:45:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:54 smithi069 ceph-mon[35081]: pgmap v1222: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-05-17T09:45:54.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:54 smithi019 ceph-mon[31620]: pgmap v1222: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-05-17T09:45:55.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:54 smithi079 ceph-mon[35502]: pgmap v1222: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-05-17T09:45:55.613 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.32 MB/sec execute 592 sec latency 2999.357 ms 2024-05-17T09:45:56.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.30 MB/sec execute 593 sec latency 3999.414 ms 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 -- 172.21.15.19:0/3080615646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103d80 msgr2=0x7f62e0104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3080615646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103d80 0x7f62e0104190 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009900 tx=0x7f62d002a410 comp rx=0 tx=0).stop 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 -- 172.21.15.19:0/3080615646 shutdown_connections 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3080615646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62e0108a80 0x7f62e010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3080615646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103d80 0x7f62e0104190 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3080615646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62e0102b40 0x7f62e0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 -- 172.21.15.19:0/3080615646 >> 172.21.15.19:0/3080615646 conn(0x7f62e00fdd30 msgr2=0x7f62e01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 -- 172.21.15.19:0/3080615646 shutdown_connections 2024-05-17T09:45:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.929+0000 7f62e7f72640 1 -- 172.21.15.19:0/3080615646 wait complete. 2024-05-17T09:45:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 Processor -- start 2024-05-17T09:45:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 -- start start 2024-05-17T09:45:56.931 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:56 smithi069 ceph-mon[35081]: pgmap v1223: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-05-17T09:45:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62e0102b40 0x7f62e006b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62e0103d80 0x7f62e019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 0x7f62e01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62e010e520 con 0x7f62e0108a80 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62e010e3a0 con 0x7f62e0103d80 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e7f72640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62e010e6a0 con 0x7f62e0102b40 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e650f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 0x7f62e01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e650f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 0x7f62e01a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41058/0 (socket says 172.21.15.19:41058) 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.930+0000 7f62e650f640 1 -- 172.21.15.19:0/3624150440 learned_addr learned my addr 172.21.15.19:0/3624150440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e550d640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62e0103d80 0x7f62e019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62e0102b40 0x7f62e006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e650f640 1 -- 172.21.15.19:0/3624150440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62e0102b40 msgr2=0x7f62e006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e650f640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62e0102b40 0x7f62e006b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e650f640 1 -- 172.21.15.19:0/3624150440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62e0103d80 msgr2=0x7f62e019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e650f640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62e0103d80 0x7f62e019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e650f640 1 -- 172.21.15.19:0/3624150440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e01a1d00 con 0x7f62e0108a80 2024-05-17T09:45:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e650f640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 0x7f62e01a1530 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f62d800eaa0 tx=0x7f62d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62d6ffd640 1 -- 172.21.15.19:0/3624150440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d800cdb0 con 0x7f62e0108a80 2024-05-17T09:45:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62d6ffd640 1 -- 172.21.15.19:0/3624150440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62d8005020 con 0x7f62e0108a80 2024-05-17T09:45:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e01a1ff0 con 0x7f62e0108a80 2024-05-17T09:45:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62d6ffd640 1 -- 172.21.15.19:0/3624150440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d80116d0 con 0x7f62e0108a80 2024-05-17T09:45:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.931+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01a24b0 con 0x7f62e0108a80 2024-05-17T09:45:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.934+0000 7f62eee23740 1 Processor -- start 2024-05-17T09:45:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.934+0000 7f62d6ffd640 1 -- 172.21.15.19:0/3624150440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f62d80051d0 con 0x7f62e0108a80 2024-05-17T09:45:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.934+0000 7f62eee23740 1 -- start start 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62eee23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131ece7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62eee23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 0x55e131eced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62eee23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b084d0 0x55e131ed5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62eee23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e131e3cc30 con 0x55e131b002f0 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62eee23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e131e2c7d0 con 0x55e131b291b0 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62eee23740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e131e30810 con 0x55e131b084d0 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 0x55e131eced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 0x55e131eced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41070/0 (socket says 172.21.15.19:41070) 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1008156882 learned_addr learned my addr 172.21.15.19:0/1008156882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e550d640 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131ece7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e550d640 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b084d0 0x55e131ed5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62d6ffd640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62cc0b2920 0x7f62cc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1008156882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b084d0 msgr2=0x55e131ed5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b084d0 0x55e131ed5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1008156882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 msgr2=0x55e131ece7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131ece7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1008156882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e131b41210 con 0x55e131b002f0 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e550d640 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b084d0 0x55e131ed5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e650f640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62cc0b2920 0x7f62cc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62d6ffd640 1 -- 172.21.15.19:0/3624150440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62d80e25c0 con 0x7f62e0108a80 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.935+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 0x55e131eced30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b750 tx=0x7f62dc00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d655f507f4008e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.936+0000 7f62cbfff640 1 -- 172.21.15.19:0/1008156882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00be60 con 0x55e131b002f0 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.936+0000 7f62cbfff640 1 -- 172.21.15.19:0/1008156882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62dc015070 con 0x55e131b002f0 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.936+0000 7f62cbfff640 1 -- 172.21.15.19:0/1008156882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc010660 con 0x55e131b002f0 2024-05-17T09:45:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- 172.21.15.19:0/1008156882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 msgr2=0x55e131eced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62e650f640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62cc0b2920 0x7f62cc0b4e30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f62d80a6b50 tx=0x7f62d80cc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 0x55e131eced30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b750 tx=0x7f62dc00bc10 comp rx=0 tx=0).stop 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- 172.21.15.19:0/1008156882 shutdown_connections 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b084d0 0x55e131ed5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b002f0 0x55e131eced30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- 172.21.15.19:0/1008156882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131ece7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- 172.21.15.19:0/1008156882 >> 172.21.15.19:0/1008156882 conn(0x55e131b1d300 msgr2=0x55e131ec5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- 172.21.15.19:0/1008156882 shutdown_connections 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- 172.21.15.19:0/1008156882 wait complete. 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- start start 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 Processor -- start 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131ed80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131ed8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131edf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e131b12790 con 0x55e131b084d0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e131de8890 con 0x55e131b291b0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.937+0000 7f62eee23740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e131ed6680 con 0x55e131b002f0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62eee23740 10 client.0 _ll_register_callbacks cb 0x55e131debd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131ed8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131ed8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41084/0 (socket says 172.21.15.19:41084) 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1865569488 learned_addr learned my addr 172.21.15.19:0/1865569488 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e550d640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131edf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e550d640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131ed80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1865569488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 msgr2=0x55e131ed80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131ed80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 msgr2=0x55e131edf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131edf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 -- 172.21.15.19:0/1865569488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e131c0d620 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e550d640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131ed80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131ed8630 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005c90 tx=0x7f62dc003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5394c8ff4c13cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62c97fa640 1 -- 172.21.15.19:0/1865569488 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00be60 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.938+0000 7f62c97fa640 1 -- 172.21.15.19:0/1865569488 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62dc011650 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62dc015070 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62c97fa640 1 -- 172.21.15.19:0/1865569488 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62dc011850 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 10 client.19305 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e131ee24c0 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62c97fa640 1 -- 172.21.15.19:0/1865569488 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62dc015070 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62c97fa640 1 -- 172.21.15.19:0/1865569488 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f62dc026420 con 0x55e131b084d0 2024-05-17T09:45:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 10 client.19305 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 10 client.19305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 20 client.19305 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131ae7d80 0x55e131ee59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 4 client.19305 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.939+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e131e3eb80 con 0x55e131ae7d80 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.940+0000 7f62e550d640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131ae7d80 0x55e131ee59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.940+0000 7f62c97fa640 10 client.19305 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.940+0000 7f62e550d640 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131ae7d80 0x55e131ee59f0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121579c8b78721a8 server_cookie=d8cdc1c275a3f899 in_seq=0 out_seq=0 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62c97fa640 1 -- 172.21.15.19:0/1865569488 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e131e3eb80 con 0x55e131ae7d80 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62c97fa640 10 client.19305 handle_command_reply: tid=0 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 client.19305 shutdown 2024-05-17T09:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 msgr2=0x55e131ed8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131ed8630 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005c90 tx=0x7f62dc003e00 comp rx=0 tx=0).stop 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 shutdown_connections 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131ae7d80 0x55e131ee59f0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b291b0 0x55e131edf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131ed8630 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 --2- 172.21.15.19:0/1865569488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131ed80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62c97fa640 0 client.19305 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 >> 172.21.15.19:0/1865569488 conn(0x55e131b1d300 msgr2=0x55e131ec8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:56.949 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:56 smithi019 ceph-mon[31620]: pgmap v1223: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-05-17T09:45:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 shutdown_connections 2024-05-17T09:45:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.941+0000 7f62eee23740 1 -- 172.21.15.19:0/1865569488 wait complete. 2024-05-17T09:45:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.942+0000 7f62eee23740 20 client.19305 trim_cache size 0 max 16384 2024-05-17T09:45:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 Processor -- start 2024-05-17T09:45:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 -- start start 2024-05-17T09:45:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131ae7d80 0x55e131ed2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b002f0 0x55e131edf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131eea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01ad110 con 0x55e131b084d0 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01aa9e0 con 0x55e131b002f0 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62eee23740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01a8830 con 0x55e131ae7d80 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62e550d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131eea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62e550d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131eea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41094/0 (socket says 172.21.15.19:41094) 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.957+0000 7f62e550d640 1 -- 172.21.15.19:0/1028620323 learned_addr learned my addr 172.21.15.19:0/1028620323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131ae7d80 0x55e131ed2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b002f0 0x55e131edf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 -- 172.21.15.19:0/1028620323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131ae7d80 msgr2=0x55e131ed2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131ae7d80 0x55e131ed2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 -- 172.21.15.19:0/1028620323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b002f0 msgr2=0x55e131edf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b002f0 0x55e131edf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 -- 172.21.15.19:0/1028620323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e131ed1a80 con 0x55e131b084d0 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62e550d640 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131eea150 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f62d0031840 tx=0x7f62d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527f995f2462e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62cb7fe640 1 -- 172.21.15.19:0/1028620323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0035040 con 0x55e131b084d0 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62cb7fe640 1 -- 172.21.15.19:0/1028620323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62d004d430 con 0x55e131b084d0 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.958+0000 7f62cb7fe640 1 -- 172.21.15.19:0/1028620323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0003850 con 0x55e131b084d0 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 -- 172.21.15.19:0/1028620323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 msgr2=0x55e131eea150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131eea150 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f62d0031840 tx=0x7f62d0009260 comp rx=0 tx=0).stop 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 -- 172.21.15.19:0/1028620323 shutdown_connections 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131eea150 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131b002f0 0x55e131edf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 --2- 172.21.15.19:0/1028620323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131ae7d80 0x55e131ed2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 -- 172.21.15.19:0/1028620323 >> 172.21.15.19:0/1028620323 conn(0x55e131b1d300 msgr2=0x55e131ec9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 -- 172.21.15.19:0/1028620323 shutdown_connections 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 -- 172.21.15.19:0/1028620323 wait complete. 2024-05-17T09:45:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 -- start start 2024-05-17T09:45:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.959+0000 7f62eee23740 1 Processor -- start 2024-05-17T09:45:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131ae7d80 0x55e131ede630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131edeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131f4c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01a8ea0 con 0x55e131b084d0 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01aa310 con 0x55e131ae7d80 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01a9570 con 0x55e131b002f0 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62eee23740 10 client.0 _ll_register_callbacks cb 0x7f62dc005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131f4c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131f4c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41104/0 (socket says 172.21.15.19:41104) 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 -- 172.21.15.19:0/2288280208 learned_addr learned my addr 172.21.15.19:0/2288280208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131edeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131ae7d80 0x55e131ede630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 -- 172.21.15.19:0/2288280208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 msgr2=0x55e131edeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131edeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 -- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131ae7d80 msgr2=0x55e131ede630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131ae7d80 0x55e131ede630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 -- 172.21.15.19:0/2288280208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e131f4c670 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62e550d640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131f4c0b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005ab0 tx=0x7f62d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56c3eb555675f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.960+0000 7f62c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62c97fa640 1 -- 172.21.15.19:0/2288280208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0035040 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62c97fa640 1 -- 172.21.15.19:0/2288280208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f62d00043f0 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d004d430 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62c97fa640 1 -- 172.21.15.19:0/2288280208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62d004d5e0 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 10 client.19311 fetch_fsmap learned FSMap version 39 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e131f4d910 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62c97fa640 1 -- 172.21.15.19:0/2288280208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62d004d430 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62c97fa640 1 -- 172.21.15.19:0/2288280208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f62d0003b10 con 0x55e131b084d0 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 10 client.19311 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 10 client.19311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:45:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 20 client.19311 populate_metadata read hostname 'smithi019' 2024-05-17T09:45:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131b291b0 0x55e131f50280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:45:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 4 client.19311 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:45:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.961+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e131e38360 con 0x55e131b291b0 2024-05-17T09:45:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.962+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131b291b0 0x55e131f50280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:45:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.962+0000 7f62e5d0e640 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131b291b0 0x55e131f50280 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e628aa4456e5c1e0 server_cookie=3e09385fa6842eaf in_seq=0 out_seq=0 2024-05-17T09:45:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:56.962+0000 7f62c97fa640 10 client.19311 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:57.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:56 smithi079 ceph-mon[35502]: pgmap v1223: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-05-17T09:45:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62c97fa640 1 -- 172.21.15.19:0/2288280208 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e131e38360 con 0x55e131b291b0 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62c97fa640 10 client.19311 handle_command_reply: tid=0 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 client.19311 shutdown 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 msgr2=0x55e131f4c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131f4c0b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005ab0 tx=0x7f62d0005fd0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 shutdown_connections 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55e131b291b0 0x55e131f50280 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131b084d0 0x55e131f4c0b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62c97fa640 0 client.19311 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e131b002f0 0x55e131edeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 --2- 172.21.15.19:0/2288280208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e131ae7d80 0x55e131ede630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 >> 172.21.15.19:0/2288280208 conn(0x55e131b1d300 msgr2=0x55e131ec92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 shutdown_connections 2024-05-17T09:45:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 1 -- 172.21.15.19:0/2288280208 wait complete. 2024-05-17T09:45:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.028+0000 7f62eee23740 20 client.19311 trim_cache size 0 max 16384 2024-05-17T09:45:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:45:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:45:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62cc0b2920 msgr2=0x7f62cc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62cc0b2920 0x7f62cc0b4e30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f62d80a6b50 tx=0x7f62d80cc040 comp rx=0 tx=0).stop 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 msgr2=0x7f62e01a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 0x7f62e01a1530 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f62d800eaa0 tx=0x7f62d800ef60 comp rx=0 tx=0).stop 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 shutdown_connections 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f62cc0b2920 0x7f62cc0b4e30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0108a80 0x7f62e01a1530 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62e0103d80 0x7f62e019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 --2- 172.21.15.19:0/3624150440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62e0102b40 0x7f62e006b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:45:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 >> 172.21.15.19:0/3624150440 conn(0x7f62e00fdd30 msgr2=0x7f62e0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:45:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 shutdown_connections 2024-05-17T09:45:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:45:57.029+0000 7f62e7f72640 1 -- 172.21.15.19:0/3624150440 wait complete. 2024-05-17T09:45:57.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.29 MB/sec execute 594 sec latency 4999.516 ms 2024-05-17T09:45:57.682 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '28d5df25', quiesce took 19.4 seconds. Will release after: 4.550000000000001 2024-05-17T09:45:58.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.28 MB/sec execute 595 sec latency 5999.647 ms 2024-05-17T09:45:58.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:45:58 smithi069 ceph-mon[35081]: pgmap v1224: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-05-17T09:45:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:45:58 smithi019 ceph-mon[31620]: pgmap v1224: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-05-17T09:45:59.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:45:58 smithi079 ceph-mon[35502]: pgmap v1224: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-05-17T09:45:59.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.27 MB/sec execute 596 sec latency 6999.714 ms 2024-05-17T09:46:00.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.26 MB/sec execute 597 sec latency 7999.801 ms 2024-05-17T09:46:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:00 smithi069 ceph-mon[35081]: pgmap v1225: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-05-17T09:46:00.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:00 smithi019 ceph-mon[31620]: pgmap v1225: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-05-17T09:46:01.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:00 smithi079 ceph-mon[35502]: pgmap v1225: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-05-17T09:46:01.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.25 MB/sec execute 598 sec latency 8999.907 ms 2024-05-17T09:46:02.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:46:02.502 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:02.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 84152 6.24 MB/sec execute 599 sec latency 9999.945 ms 2024-05-17T09:46:02.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:02 smithi019 ceph-mon[31620]: pgmap v1226: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-05-17T09:46:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:02 smithi069 ceph-mon[35081]: pgmap v1226: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-05-17T09:46:03.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:02 smithi079 ceph-mon[35502]: pgmap v1226: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-05-17T09:46:03.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 cleanup 600 sec 2024-05-17T09:46:04.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/4068141907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38107de0 msgr2=0x7f8f381081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/4068141907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38107de0 0x7f8f381081f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0099c0 tx=0x7f8f2c009cd0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/4068141907 shutdown_connections 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/4068141907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f381034b0 0x7f8f381038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/4068141907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f38108730 0x7f8f38110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/4068141907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38107de0 0x7f8f381081f0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/4068141907 >> 172.21.15.19:0/4068141907 conn(0x7f8f380fedb0 msgr2=0x7f8f381011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/4068141907 shutdown_connections 2024-05-17T09:46:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.154+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/4068141907 wait complete. 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 Processor -- start 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 -- start start 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f381034b0 0x7f8f3819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f38107de0 0x7f8f3819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 0x7f8f3819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f38113260 con 0x7f8f38108730 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381130e0 con 0x7f8f38107de0 2024-05-17T09:46:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.155+0000 7f8f3e8b7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381133e0 con 0x7f8f381034b0 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 0x7f8f3819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 0x7f8f3819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50468/0 (socket says 172.21.15.19:50468) 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 -- 172.21.15.19:0/3855543597 learned_addr learned my addr 172.21.15.19:0/3855543597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f377fe700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f38107de0 0x7f8f3819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f37fff700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f381034b0 0x7f8f3819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 -- 172.21.15.19:0/3855543597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f381034b0 msgr2=0x7f8f3819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f381034b0 0x7f8f3819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 -- 172.21.15.19:0/3855543597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f38107de0 msgr2=0x7f8f3819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f38107de0 0x7f8f3819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 -- 172.21.15.19:0/3855543597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3819fc70 con 0x7f8f38108730 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3ce54700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 0x7f8f3819f5d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8f30007ee0 tx=0x7f8f3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3000dbd0 con 0x7f8f38108730 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f3000dd70 con 0x7f8f38108730 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3819ff60 con 0x7f8f38108730 2024-05-17T09:46:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f300058d0 con 0x7f8f38108730 2024-05-17T09:46:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.156+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f381a7880 con 0x7f8f38108730 2024-05-17T09:46:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.158+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f380512b0 con 0x7f8f38108730 2024-05-17T09:46:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.159+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f3001a020 con 0x7f8f38108730 2024-05-17T09:46:04.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.160+0000 7f8f357fa700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f200b2900 0x7f8f200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:04.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.161+0000 7f8f37fff700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f200b2900 0x7f8f200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:04.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.161+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8f300e23b0 con 0x7f8f38108730 2024-05-17T09:46:04.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.161+0000 7f8f37fff700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f200b2900 0x7f8f200b4e10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:04.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.164+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f30010070 con 0x7f8f38108730 2024-05-17T09:46:04.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.426+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f3802e5e0 con 0x7f8f38108730 2024-05-17T09:46:04.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.427+0000 7f8f357fa700 1 -- 172.21.15.19:0/3855543597 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11853 (secure 0 0 0) 0x7f8f300a9d20 con 0x7f8f38108730 2024-05-17T09:46:04.427 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:46:04.427 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:37:27.654566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f200b2900 msgr2=0x7f8f200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f200b2900 0x7f8f200b4e10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28008040 comp rx=0 tx=0).stop 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 msgr2=0x7f8f3819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 0x7f8f3819f5d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8f30007ee0 tx=0x7f8f3000d3b0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 shutdown_connections 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8f200b2900 0x7f8f200b4e10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f381034b0 0x7f8f3819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f38108730 0x7f8f3819f5d0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 --2- 172.21.15.19:0/3855543597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f38107de0 0x7f8f3819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 >> 172.21.15.19:0/3855543597 conn(0x7f8f380fedb0 msgr2=0x7f8f3806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 shutdown_connections 2024-05-17T09:46:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:04.430+0000 7f8f3e8b7700 1 -- 172.21.15.19:0/3855543597 wait complete. 2024-05-17T09:46:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-05-17T09:46:04.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 cleanup 601 sec 2024-05-17T09:46:04.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:04 smithi019 ceph-mon[31620]: pgmap v1227: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-17T09:46:04.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:04 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/3855543597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:04 smithi069 ceph-mon[35081]: pgmap v1227: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-17T09:46:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:04 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/3855543597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:04 smithi079 ceph-mon[35502]: pgmap v1227: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-05-17T09:46:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:04 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/3855543597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:05.212 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 28d5df25 --release --await' 2024-05-17T09:46:05.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 28d5df25 --release --await 2024-05-17T09:46:05.466 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:05.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 cleanup 602 sec 2024-05-17T09:46:06.615 INFO:tasks.workunit.client.0.smithi019.stdout: 10 cleanup 603 sec 2024-05-17T09:46:06.911 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:06 smithi019 ceph-mon[31620]: pgmap v1228: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 635 KiB/s wr, 70 op/s 2024-05-17T09:46:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:06 smithi069 ceph-mon[35081]: pgmap v1228: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 635 KiB/s wr, 70 op/s 2024-05-17T09:46:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:06 smithi079 ceph-mon[35502]: pgmap v1228: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 635 KiB/s wr, 70 op/s 2024-05-17T09:46:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 -- 172.21.15.19:0/3375360418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107e00 msgr2=0x7fba78108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 --2- 172.21.15.19:0/3375360418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107e00 0x7fba78108210 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fba680099c0 tx=0x7fba68009cd0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 -- 172.21.15.19:0/3375360418 shutdown_connections 2024-05-17T09:46:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 --2- 172.21.15.19:0/3375360418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba781034d0 0x7fba781038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 --2- 172.21.15.19:0/3375360418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba78108750 0x7fba781100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 --2- 172.21.15.19:0/3375360418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107e00 0x7fba78108210 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 -- 172.21.15.19:0/3375360418 >> 172.21.15.19:0/3375360418 conn(0x7fba780fef40 msgr2=0x7fba78101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 -- 172.21.15.19:0/3375360418 shutdown_connections 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.277+0000 7fba7f2de700 1 -- 172.21.15.19:0/3375360418 wait complete. 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 Processor -- start 2024-05-17T09:46:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 -- start start 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 0x7fba7819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba78107e00 0x7fba7819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba78108750 0x7fba7819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba78113210 con 0x7fba781034d0 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba78113090 con 0x7fba78108750 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.278+0000 7fba7f2de700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fba78113390 con 0x7fba78107e00 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 0x7fba7819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 0x7fba7819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50482/0 (socket says 172.21.15.19:50482) 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 -- 172.21.15.19:0/1360688810 learned_addr learned my addr 172.21.15.19:0/1360688810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d87b700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba78108750 0x7fba7819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7c879700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba78107e00 0x7fba7819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 -- 172.21.15.19:0/1360688810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba78107e00 msgr2=0x7fba7819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba78107e00 0x7fba7819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 -- 172.21.15.19:0/1360688810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba78108750 msgr2=0x7fba7819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba78108750 0x7fba7819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 -- 172.21.15.19:0/1360688810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7819fc20 con 0x7fba781034d0 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7d07a700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 0x7fba7819ad50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d950 tx=0x7fba7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba6e7fc700 1 -- 172.21.15.19:0/1360688810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74016070 con 0x7fba781034d0 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba6e7fc700 1 -- 172.21.15.19:0/1360688810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba7400fd20 con 0x7fba781034d0 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7819ff10 con 0x7fba781034d0 2024-05-17T09:46:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.279+0000 7fba6e7fc700 1 -- 172.21.15.19:0/1360688810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7400e3b0 con 0x7fba781034d0 2024-05-17T09:46:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.280+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba781a7830 con 0x7fba781034d0 2024-05-17T09:46:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.282+0000 7fba6e7fc700 1 -- 172.21.15.19:0/1360688810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba74019070 con 0x7fba781034d0 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.283+0000 7fba8dcf3b80 1 Processor -- start 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.283+0000 7fba6e7fc700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fba640b2880 0x7fba640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.283+0000 7fba8dcf3b80 1 -- start start 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.283+0000 7fba7d87b700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fba640b2880 0x7fba640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba6e7fc700 1 -- 172.21.15.19:0/1360688810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fba740e2130 con 0x7fba781034d0 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f50160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55acb2e74e70 con 0x55acb2dc7f40 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acb2e64a10 con 0x55acb2eb3530 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55acb2e68a50 con 0x55acb2f4fd50 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50486/0 (socket says 172.21.15.19:50486) 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 -- 172.21.15.19:0/1115415772 learned_addr learned my addr 172.21.15.19:0/1115415772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7d87b700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fba640b2880 0x7fba640b4d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fba7000f540 tx=0x7fba7000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7d07a700 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7d87b700 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f50160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 -- 172.21.15.19:0/1115415772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 msgr2=0x55acb2f50160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f50160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 -- 172.21.15.19:0/1115415772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 msgr2=0x55acb2f4b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.284+0000 7fba7c879700 1 -- 172.21.15.19:0/1115415772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb2e0e0b0 con 0x55acb2dc7f40 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.285+0000 7fba7c879700 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4bc70 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fba680099c0 tx=0x7fba68005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c7149f5f1b05a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.285+0000 7fba637fe700 1 -- 172.21.15.19:0/1115415772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68027070 con 0x55acb2dc7f40 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.285+0000 7fba637fe700 1 -- 172.21.15.19:0/1115415772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba680037c0 con 0x55acb2dc7f40 2024-05-17T09:46:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.285+0000 7fba637fe700 1 -- 172.21.15.19:0/1115415772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6800b9f0 con 0x55acb2dc7f40 2024-05-17T09:46:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/1115415772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 msgr2=0x55acb2f4bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4bc70 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fba680099c0 tx=0x7fba68005040 comp rx=0 tx=0).stop 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/1115415772 shutdown_connections 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f50160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/1115415772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4bc70 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/1115415772 >> 172.21.15.19:0/1115415772 conn(0x55acb2e282c0 msgr2=0x55acb2f41cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/1115415772 shutdown_connections 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/1115415772 wait complete. 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 -- start start 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.286+0000 7fba8dcf3b80 1 Processor -- start 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f54fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55acb2d6b0a0 con 0x55acb2dc7f40 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acb2dccea0 con 0x55acb2eb3530 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55acb2f50aa0 con 0x55acb2f4fd50 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba8dcf3b80 10 client.0 _ll_register_callbacks cb 0x55acb2dffa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f54fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f54fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50502/0 (socket says 172.21.15.19:50502) 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 -- 172.21.15.19:0/2684031993 learned_addr learned my addr 172.21.15.19:0/2684031993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7c879700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d87b700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f55a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 -- 172.21.15.19:0/2684031993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 msgr2=0x55acb2f55a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f55a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 -- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 msgr2=0x55acb2f554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7d07a700 1 -- 172.21.15.19:0/2684031993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb2db49d0 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.287+0000 7fba7c879700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba7d07a700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f54fb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fba740a9040 tx=0x7fba740a8f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6004c77fb3799417 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba60ff9700 1 -- 172.21.15.19:0/2684031993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74020070 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba60ff9700 1 -- 172.21.15.19:0/2684031993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fba740b1d10 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba680037c0 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba60ff9700 1 -- 172.21.15.19:0/2684031993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba740ad6f0 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 10 client.19329 fetch_fsmap learned FSMap version 39 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55acb2e1e860 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba60ff9700 1 -- 172.21.15.19:0/2684031993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba680037c0 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba60ff9700 1 -- 172.21.15.19:0/2684031993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fba740dfce0 con 0x55acb2dc7f40 2024-05-17T09:46:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 10 client.19329 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 10 client.19329 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 20 client.19329 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2f64c80 0x55acb2f670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 4 client.19329 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.288+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb2f12fd0 con 0x55acb2f64c80 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.289+0000 7fba7c879700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2f64c80 0x55acb2f670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.289+0000 7fba7c879700 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2f64c80 0x55acb2f670c0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579bc0630679a614 server_cookie=4799f6aafbbc1e79 in_seq=0 out_seq=0 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.289+0000 7fba60ff9700 10 client.19329 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba60ff9700 1 -- 172.21.15.19:0/2684031993 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55acb2f12fd0 con 0x55acb2f64c80 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba60ff9700 10 client.19329 handle_command_reply: tid=0 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 client.19329 shutdown 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 msgr2=0x55acb2f54fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f54fb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fba740a9040 tx=0x7fba740a8f40 comp rx=0 tx=0).stop 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 shutdown_connections 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f55a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2f64c80 0x55acb2f670c0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f54fb0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2684031993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 >> 172.21.15.19:0/2684031993 conn(0x55acb2e282c0 msgr2=0x55acb2f3bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba60ff9700 0 client.19329 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.290+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 shutdown_connections 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.291+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2684031993 wait complete. 2024-05-17T09:46:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.291+0000 7fba8dcf3b80 20 client.19329 trim_cache size 0 max 16384 2024-05-17T09:46:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 Processor -- start 2024-05-17T09:46:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 -- start start 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f60de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f6c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba781b0970 con 0x55acb2dc7f40 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba781ae200 con 0x55acb2eb3530 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.311+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fba781ac050 con 0x55acb2f4fd50 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f46a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50518/0 (socket says 172.21.15.19:50518) 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 -- 172.21.15.19:0/2525546233 learned_addr learned my addr 172.21.15.19:0/2525546233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7c879700 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f60de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d87b700 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f6c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 -- 172.21.15.19:0/2525546233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 msgr2=0x55acb2f6c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f6c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 -- 172.21.15.19:0/2525546233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 msgr2=0x55acb2f60de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f60de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 -- 172.21.15.19:0/2525546233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb2f42d00 con 0x55acb2dc7f40 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba7d07a700 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f46a40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b2f0 tx=0x7fba740a8fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb2c18982b6a698 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba62ffd700 1 -- 172.21.15.19:0/2525546233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74020070 con 0x55acb2dc7f40 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba62ffd700 1 -- 172.21.15.19:0/2525546233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba740ad410 con 0x55acb2dc7f40 2024-05-17T09:46:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.312+0000 7fba62ffd700 1 -- 172.21.15.19:0/2525546233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba740b1f00 con 0x55acb2dc7f40 2024-05-17T09:46:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2525546233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 msgr2=0x55acb2f46a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f46a40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b2f0 tx=0x7fba740a8fb0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2525546233 shutdown_connections 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f6c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f46a40 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/2525546233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f60de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2525546233 >> 172.21.15.19:0/2525546233 conn(0x55acb2e282c0 msgr2=0x55acb2f3c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2525546233 shutdown_connections 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.313+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/2525546233 wait complete. 2024-05-17T09:46:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 -- start start 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 Processor -- start 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f4e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba781ac6c0 con 0x55acb2dc7f40 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba781adb30 con 0x55acb2eb3530 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fba781acd90 con 0x55acb2f4fd50 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba8dcf3b80 10 client.0 _ll_register_callbacks cb 0x55acb2dffa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba7d07a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50520/0 (socket says 172.21.15.19:50520) 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba7c879700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.314+0000 7fba7d07a700 1 -- 172.21.15.19:0/114631221 learned_addr learned my addr 172.21.15.19:0/114631221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d87b700 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f4e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d07a700 1 -- 172.21.15.19:0/114631221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 msgr2=0x55acb2f4e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d07a700 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f4e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d07a700 1 -- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 msgr2=0x55acb2f4de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d07a700 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d07a700 1 -- 172.21.15.19:0/114631221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb2fce7a0 con 0x55acb2dc7f40 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba7d07a700 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4d8d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fba740ad8b0 tx=0x7fba740df8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19305eec088e8fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba60ff9700 1 -- 172.21.15.19:0/114631221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba740c7070 con 0x55acb2dc7f40 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba60ff9700 1 -- 172.21.15.19:0/114631221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fba740ad1c0 con 0x55acb2dc7f40 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba740ad410 con 0x55acb2dc7f40 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba60ff9700 1 -- 172.21.15.19:0/114631221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba7408c050 con 0x55acb2dc7f40 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba8dcf3b80 10 client.19335 fetch_fsmap learned FSMap version 39 2024-05-17T09:46:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.315+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55acb2fce7a0 con 0x55acb2dc7f40 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba60ff9700 1 -- 172.21.15.19:0/114631221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba740ca360 con 0x55acb2dc7f40 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba60ff9700 1 -- 172.21.15.19:0/114631221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fba740dfce0 con 0x55acb2dc7f40 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba8dcf3b80 10 client.19335 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba8dcf3b80 10 client.19335 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba8dcf3b80 20 client.19335 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2fcfbd0 0x55acb2fd1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba8dcf3b80 4 client.19335 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "28d5df25", "release": true, "await": true}] 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "set_id": "28d5df25", "release": true, "await": true}) -- 0x7fba740ad410 con 0x55acb2fcfbd0 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.316+0000 7fba7c879700 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2fcfbd0 0x55acb2fd1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.317+0000 7fba60ff9700 10 client.19335 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.317+0000 7fba7c879700 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2fcfbd0 0x55acb2fd1ff0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf77fcb4df03109f server_cookie=4f038c73a7e70b88 in_seq=0 out_seq=0 2024-05-17T09:46:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba60ff9700 1 -- 172.21.15.19:0/114631221 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fba740ad410 con 0x55acb2fcfbd0 2024-05-17T09:46:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba60ff9700 10 client.19335 handle_command_reply: tid=0 2024-05-17T09:46:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 client.19335 shutdown 2024-05-17T09:46:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 msgr2=0x55acb2f4d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4d8d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fba740ad8b0 tx=0x7fba740df8a0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 shutdown_connections 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55acb2f4fd50 0x55acb2f4e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55acb2fcfbd0 0x55acb2fd1ff0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acb2dc7f40 0x55acb2f4d8d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 --2- 172.21.15.19:0/114631221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acb2eb3530 0x55acb2f4de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 >> 172.21.15.19:0/114631221 conn(0x55acb2e282c0 msgr2=0x55acb2f53790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba60ff9700 0 client.19335 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 shutdown_connections 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 1 -- 172.21.15.19:0/114631221 wait complete. 2024-05-17T09:46:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.340+0000 7fba8dcf3b80 20 client.19335 trim_cache size 0 max 16384 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 39, 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 96, 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "28d5df25": { 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 96, 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:46:07.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 143.9, 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-05-17T09:46:07.342 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:46:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:07.343 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fba640b2880 msgr2=0x7fba640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fba640b2880 0x7fba640b4d90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fba7000f540 tx=0x7fba7000b040 comp rx=0 tx=0).stop 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 msgr2=0x7fba7819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 0x7fba7819ad50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fba7400d950 tx=0x7fba7400dc60 comp rx=0 tx=0).stop 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 shutdown_connections 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba78108750 0x7fba7819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034d0 0x7fba7819ad50 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fba640b2880 0x7fba640b4d90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 --2- 172.21.15.19:0/1360688810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba78107e00 0x7fba7819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 >> 172.21.15.19:0/1360688810 conn(0x7fba780fef40 msgr2=0x7fba78075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 shutdown_connections 2024-05-17T09:46:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:07.341+0000 7fba7f2de700 1 -- 172.21.15.19:0/1360688810 wait complete. 2024-05-17T09:46:07.614 INFO:tasks.workunit.client.0.smithi019.stdout: 10 cleanup 604 sec 2024-05-17T09:46:08.133 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '28d5df25', total seconds elapsed: 29.9 2024-05-17T09:46:08.133 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.95 2024-05-17T09:46:08.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:46:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:08.614 INFO:tasks.workunit.client.0.smithi019.stdout: 2 cleanup 605 sec 2024-05-17T09:46:08.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:08 smithi019 ceph-mon[31620]: pgmap v1229: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 379 KiB/s wr, 48 op/s 2024-05-17T09:46:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:08 smithi069 ceph-mon[35081]: pgmap v1229: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 379 KiB/s wr, 48 op/s 2024-05-17T09:46:09.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:08 smithi079 ceph-mon[35502]: pgmap v1229: 129 pgs: 129 active+clean; 526 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 379 KiB/s wr, 48 op/s 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: 0 cleanup 606 sec 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Operation Count AvgLat MaxLat 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: ---------------------------------------- 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: NTCreateX 119874 16.429 26787.136 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Close 88081 0.214 94.456 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Rename 5081 40.883 20500.845 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Unlink 24144 20.998 26471.395 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Qpathinfo 108649 6.326 376.692 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Qfileinfo 18999 0.394 98.825 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Qfsinfo 19915 1.087 110.202 2024-05-17T09:46:09.115 INFO:tasks.workunit.client.0.smithi019.stdout: Sfileinfo 9745 16.121 26486.767 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: Find 42056 48.429 404.890 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: WriteX 59253 6.895 26475.211 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: ReadX 188550 0.020 17.106 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: LockX 394 2.518 93.917 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: UnlockX 394 2.108 18.850 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: Flush 8389 8.899 343.489 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-17T09:46:09.116 INFO:tasks.workunit.client.0.smithi019.stdout:Throughput 6.24139 MB/sec 10 clients 10 procs max_latency=26787.140 ms 2024-05-17T09:46:09.116 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-17T09:46:09.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-17T09:46:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 -- 172.21.15.19:0/1677290685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24102b70 msgr2=0x7f3f24103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/1677290685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24102b70 0x7f3f24103870 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009900 tx=0x7f3f1802a410 comp rx=0 tx=0).stop 2024-05-17T09:46:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 -- 172.21.15.19:0/1677290685 shutdown_connections 2024-05-17T09:46:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/1677290685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24108ab0 0x7f3f2410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/1677290685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f24103db0 0x7f3f241041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/1677290685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24102b70 0x7f3f24103870 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 -- 172.21.15.19:0/1677290685 >> 172.21.15.19:0/1677290685 conn(0x7f3f240fdd80 msgr2=0x7f3f24100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 -- 172.21.15.19:0/1677290685 shutdown_connections 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 -- 172.21.15.19:0/1677290685 wait complete. 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 Processor -- start 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.183+0000 7f3f2b14d640 1 -- start start 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f2b14d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24102b70 0x7f3f24199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f2b14d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f24103db0 0x7f3f241996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f2b14d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 0x7f3f241a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f2b14d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410e520 con 0x7f3f24108ab0 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 0x7f3f241a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f2b14d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410e3a0 con 0x7f3f24102b70 2024-05-17T09:46:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 0x7f3f241a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50536/0 (socket says 172.21.15.19:50536) 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 -- 172.21.15.19:0/4076138213 learned_addr learned my addr 172.21.15.19:0/4076138213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410e6a0 con 0x7f3f24103db0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f23fff640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f24103db0 0x7f3f241996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24102b70 0x7f3f24199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 -- 172.21.15.19:0/4076138213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f24103db0 msgr2=0x7f3f241996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f24103db0 0x7f3f241996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 -- 172.21.15.19:0/4076138213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24102b70 msgr2=0x7f3f24199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24102b70 0x7f3f24199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 -- 172.21.15.19:0/4076138213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f241a0c30 con 0x7f3f24108ab0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.184+0000 7f3f296ea640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 0x7f3f241a05c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00eb10 tx=0x7f3f1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.185+0000 7f3f21ffb640 1 -- 172.21.15.19:0/4076138213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c004b20 con 0x7f3f24108ab0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.185+0000 7f3f21ffb640 1 -- 172.21.15.19:0/4076138213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f1c004cc0 con 0x7f3f24108ab0 2024-05-17T09:46:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.185+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241a0f20 con 0x7f3f24108ab0 2024-05-17T09:46:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.185+0000 7f3f21ffb640 1 -- 172.21.15.19:0/4076138213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c010690 con 0x7f3f24108ab0 2024-05-17T09:46:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.185+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241a1380 con 0x7f3f24108ab0 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 Processor -- start 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f21ffb640 1 -- 172.21.15.19:0/4076138213 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f1c005020 con 0x7f3f24108ab0 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 -- start start 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 0x55df3a2f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f292f0 0x55df3a2f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f314d0 0x55df3a2fca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df3a265c30 con 0x55df39f521b0 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df3a2557d0 con 0x55df39f292f0 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f32002740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df3a259810 con 0x55df39f314d0 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 0x55df3a2f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 0x55df3a2f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50552/0 (socket says 172.21.15.19:50552) 2024-05-17T09:46:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f23fff640 1 -- 172.21.15.19:0/2633361842 learned_addr learned my addr 172.21.15.19:0/2633361842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.188+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f292f0 0x55df3a2f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f23fff640 1 -- 172.21.15.19:0/2633361842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f314d0 msgr2=0x55df3a2fca30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f21ffb640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f140b2920 0x7f3f140b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f23fff640 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f314d0 0x55df3a2fca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f23fff640 1 -- 172.21.15.19:0/2633361842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f292f0 msgr2=0x55df3a2f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f23fff640 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f292f0 0x55df3a2f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f23fff640 1 -- 172.21.15.19:0/2633361842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df39f6a210 con 0x55df39f521b0 2024-05-17T09:46:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f21ffb640 1 -- 172.21.15.19:0/4076138213 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f1c0e2ac0 con 0x7f3f24108ab0 2024-05-17T09:46:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f23fff640 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 0x55df3a2f55b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009c70 tx=0x7f3f10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3e1cd172b3e7ae server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f296ea640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f140b2920 0x7f3f140b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f0effd640 1 -- 172.21.15.19:0/2633361842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10009e20 con 0x55df39f521b0 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f0effd640 1 -- 172.21.15.19:0/2633361842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f100054b0 con 0x55df39f521b0 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.189+0000 7f3f0effd640 1 -- 172.21.15.19:0/2633361842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10019410 con 0x55df39f521b0 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 -- 172.21.15.19:0/2633361842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 msgr2=0x55df3a2f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f296ea640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f140b2920 0x7f3f140b4e30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0a5db0 tx=0x7f3f1c00cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 0x55df3a2f55b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009c70 tx=0x7f3f10011810 comp rx=0 tx=0).stop 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 -- 172.21.15.19:0/2633361842 shutdown_connections 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f314d0 0x55df3a2fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f292f0 0x55df3a2f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 --2- 172.21.15.19:0/2633361842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f521b0 0x55df3a2f55b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.190+0000 7f3f32002740 1 -- 172.21.15.19:0/2633361842 >> 172.21.15.19:0/2633361842 conn(0x55df39f46300 msgr2=0x55df3a2eed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.191+0000 7f3f32002740 1 -- 172.21.15.19:0/2633361842 shutdown_connections 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.191+0000 7f3f32002740 1 -- 172.21.15.19:0/2633361842 wait complete. 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.191+0000 7f3f32002740 1 -- start start 2024-05-17T09:46:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.191+0000 7f3f32002740 1 Processor -- start 2024-05-17T09:46:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a2fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a2ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a306400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df39f3b790 con 0x55df39f292f0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df3a211890 con 0x55df39f314d0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df3a2fd480 con 0x55df39f521b0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f32002740 10 client.0 _ll_register_callbacks cb 0x55df3a214d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a2fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a2fef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50554/0 (socket says 172.21.15.19:50554) 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 -- 172.21.15.19:0/1747407908 learned_addr learned my addr 172.21.15.19:0/1747407908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a306400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a2ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 -- 172.21.15.19:0/1747407908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 msgr2=0x55df3a306400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a306400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 -- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 msgr2=0x55df3a2ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a2ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.192+0000 7f3f23fff640 1 -- 172.21.15.19:0/1747407908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3a036620 con 0x55df39f292f0 2024-05-17T09:46:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f23fff640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a2fef60 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009ab0 tx=0x7f3f10009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86117ca733d9d256 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f03fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f03fff640 1 -- 172.21.15.19:0/1747407908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1002d060 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f03fff640 1 -- 172.21.15.19:0/1747407908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f10025460 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f100054b0 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f03fff640 1 -- 172.21.15.19:0/1747407908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3f10034050 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f32002740 10 client.19347 fetch_fsmap learned FSMap version 40 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df3a309330 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.193+0000 7f3f03fff640 1 -- 172.21.15.19:0/1747407908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f100054b0 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f03fff640 1 -- 172.21.15.19:0/1747407908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f3f10028570 con 0x55df39f292f0 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f32002740 10 client.19347 fetch_fsmap finished waiting for FSMap version 40 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f32002740 10 client.19347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f32002740 20 client.19347 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f32002740 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39f10d80 0x55df3a30c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f32002740 4 client.19347 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:46:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3a267b80 con 0x55df39f10d80 2024-05-17T09:46:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.194+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39f10d80 0x55df3a30c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.207+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39f10d80 0x55df3a30c7e0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53930dd568979c8a server_cookie=56ec55869015ec23 in_seq=0 out_seq=0 2024-05-17T09:46:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.207+0000 7f3f03fff640 10 client.19347 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f03fff640 1 -- 172.21.15.19:0/1747407908 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55df3a267b80 con 0x55df39f10d80 2024-05-17T09:46:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f03fff640 10 client.19347 handle_command_reply: tid=0 2024-05-17T09:46:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 client.19347 shutdown 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 msgr2=0x55df3a2fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a2fef60 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009ab0 tx=0x7f3f10009660 comp rx=0 tx=0).stop 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 shutdown_connections 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39f10d80 0x55df3a30c7e0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a306400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a2ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 --2- 172.21.15.19:0/1747407908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a2fef60 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 >> 172.21.15.19:0/1747407908 conn(0x55df39f46300 msgr2=0x55df3a2f1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f03fff640 0 client.19347 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 shutdown_connections 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 1 -- 172.21.15.19:0/1747407908 wait complete. 2024-05-17T09:46:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.208+0000 7f3f32002740 20 client.19347 trim_cache size 0 max 16384 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 Processor -- start 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 -- start start 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f292f0 0x55df3a311ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 0x55df3a306390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f521b0 0x55df3a312530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241ad0e0 con 0x55df39f314d0 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241aa980 con 0x55df39f521b0 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f32002740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241a87d0 con 0x55df39f292f0 2024-05-17T09:46:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 0x55df3a306390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f292f0 0x55df3a311ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 0x55df3a306390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50562/0 (socket says 172.21.15.19:50562) 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 -- 172.21.15.19:0/3382036915 learned_addr learned my addr 172.21.15.19:0/3382036915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f23fff640 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f521b0 0x55df3a312530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 -- 172.21.15.19:0/3382036915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f292f0 msgr2=0x55df3a311ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f292f0 0x55df3a311ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 -- 172.21.15.19:0/3382036915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f521b0 msgr2=0x55df3a312530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f521b0 0x55df3a312530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.225+0000 7f3f28ee9640 1 -- 172.21.15.19:0/3382036915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3a2ffb60 con 0x55df39f314d0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.226+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 0x55df3a306390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f3f18031460 tx=0x7f3f18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc05f81b67b29881 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.226+0000 7f3f0e7fc640 1 -- 172.21.15.19:0/3382036915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18035040 con 0x55df39f314d0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.226+0000 7f3f0e7fc640 1 -- 172.21.15.19:0/3382036915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f1804b070 con 0x55df39f314d0 2024-05-17T09:46:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.226+0000 7f3f0e7fc640 1 -- 172.21.15.19:0/3382036915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180466f0 con 0x55df39f314d0 2024-05-17T09:46:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 -- 172.21.15.19:0/3382036915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 msgr2=0x55df3a306390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 0x55df3a306390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f3f18031460 tx=0x7f3f18005fd0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 -- 172.21.15.19:0/3382036915 shutdown_connections 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f521b0 0x55df3a312530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f314d0 0x55df3a306390 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 --2- 172.21.15.19:0/3382036915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f292f0 0x55df3a311ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 -- 172.21.15.19:0/3382036915 >> 172.21.15.19:0/3382036915 conn(0x55df39f46300 msgr2=0x55df3a2f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 -- 172.21.15.19:0/3382036915 shutdown_connections 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 -- 172.21.15.19:0/3382036915 wait complete. 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 -- start start 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.227+0000 7f3f32002740 1 Processor -- start 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a3052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a305800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a372eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241a8e40 con 0x55df39f292f0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241aa2b0 con 0x55df39f314d0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a3052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241a9510 con 0x55df39f521b0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a3052c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50566/0 (socket says 172.21.15.19:50566) 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 -- 172.21.15.19:0/3221206950 learned_addr learned my addr 172.21.15.19:0/3221206950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a305800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f32002740 10 client.0 _ll_register_callbacks cb 0x7f3f10009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a372eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 -- 172.21.15.19:0/3221206950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 msgr2=0x55df3a372eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a372eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 -- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 msgr2=0x55df3a305800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a305800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 -- 172.21.15.19:0/3221206950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3a3733f0 con 0x55df39f292f0 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a305800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:46:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f23fff640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a3052c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3f10014040 tx=0x7f3f10009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3f7f05545b4723 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.228+0000 7f3f03fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f03fff640 1 -- 172.21.15.19:0/3221206950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1002d040 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f1804b070 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f03fff640 1 -- 172.21.15.19:0/3221206950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f10005660 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f03fff640 1 -- 172.21.15.19:0/3221206950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3f10028450 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 10 client.19353 fetch_fsmap learned FSMap version 40 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df3a374540 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f03fff640 1 -- 172.21.15.19:0/3221206950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f1804b070 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f03fff640 1 -- 172.21.15.19:0/3221206950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f3f100035f0 con 0x55df39f292f0 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 10 client.19353 fetch_fsmap finished waiting for FSMap version 40 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 10 client.19353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 20 client.19353 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39ee61c0 0x55df3a376e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 4 client.19353 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T09:46:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df3a261360 con 0x55df39ee61c0 2024-05-17T09:46:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.229+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39ee61c0 0x55df3a376e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.230+0000 7f3f03fff640 10 client.19353 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.230+0000 7f3f28ee9640 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39ee61c0 0x55df3a376e00 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c773ed01f9ded28 server_cookie=a497d353c50986a8 in_seq=0 out_seq=0 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f03fff640 1 -- 172.21.15.19:0/3221206950 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55df3a261360 con 0x55df39ee61c0 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f03fff640 10 client.19353 handle_command_reply: tid=0 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 client.19353 shutdown 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 msgr2=0x55df3a3052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a3052c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3f10014040 tx=0x7f3f10009660 comp rx=0 tx=0).stop 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 shutdown_connections 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x55df39ee61c0 0x55df3a376e00 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df39f521b0 0x55df3a372eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df39f314d0 0x55df3a305800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 --2- 172.21.15.19:0/3221206950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df39f292f0 0x55df3a3052c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f03fff640 0 client.19353 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 >> 172.21.15.19:0/3221206950 conn(0x55df39f46300 msgr2=0x55df3a2ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 shutdown_connections 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.347+0000 7f3f32002740 1 -- 172.21.15.19:0/3221206950 wait complete. 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f32002740 20 client.19353 trim_cache size 0 max 16384 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-17T09:46:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-17T09:46:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f140b2920 msgr2=0x7f3f140b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f140b2920 0x7f3f140b4e30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0a5db0 tx=0x7f3f1c00cf70 comp rx=0 tx=0).stop 2024-05-17T09:46:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 msgr2=0x7f3f241a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 0x7f3f241a05c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00eb10 tx=0x7f3f1c00c2d0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 shutdown_connections 2024-05-17T09:46:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f140b2920 0x7f3f140b4e30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f24108ab0 0x7f3f241a05c0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f24103db0 0x7f3f241996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 --2- 172.21.15.19:0/4076138213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24102b70 0x7f3f24199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.348+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 >> 172.21.15.19:0/4076138213 conn(0x7f3f240fdd80 msgr2=0x7f3f24104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.349+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 shutdown_connections 2024-05-17T09:46:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-17T09:46:10.349+0000 7f3f2b14d640 1 -- 172.21.15.19:0/4076138213 wait complete. 2024-05-17T09:46:10.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 -- 172.21.15.19:0/1209835222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106ec0 msgr2=0x7fbf58113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 --2- 172.21.15.19:0/1209835222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106ec0 0x7fbf58113350 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000b0a0 tx=0x7fbf5000b3b0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 -- 172.21.15.19:0/1209835222 shutdown_connections 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 --2- 172.21.15.19:0/1209835222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106ec0 0x7fbf58113350 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 --2- 172.21.15.19:0/1209835222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5810c5b0 0x7fbf5810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 --2- 172.21.15.19:0/1209835222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106570 0x7fbf58106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 -- 172.21.15.19:0/1209835222 >> 172.21.15.19:0/1209835222 conn(0x7fbf58102070 msgr2=0x7fbf581044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 -- 172.21.15.19:0/1209835222 shutdown_connections 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.467+0000 7fbf5d757700 1 -- 172.21.15.19:0/1209835222 wait complete. 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.468+0000 7fbf5d757700 1 Processor -- start 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 -- start start 2024-05-17T09:46:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106570 0x7fbf58110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106ec0 0x7fbf58110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5810c5b0 0x7fbf581110a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581166b0 con 0x7fbf5810c5b0 2024-05-17T09:46:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58116530 con 0x7fbf58106570 2024-05-17T09:46:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.469+0000 7fbf5d757700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58116830 con 0x7fbf58106ec0 2024-05-17T09:46:10.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.470+0000 7fbf56ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106570 0x7fbf58110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.470+0000 7fbf56ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106570 0x7fbf58110620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.19:55560/0 (socket says 172.21.15.19:55560) 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.470+0000 7fbf56ffd700 1 -- 172.21.15.19:0/404967361 learned_addr learned my addr 172.21.15.19:0/404967361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.470+0000 7fbf567fc700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106ec0 0x7fbf58110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.471+0000 7fbf567fc700 1 -- 172.21.15.19:0/404967361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106570 msgr2=0x7fbf58110620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.471+0000 7fbf567fc700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106570 0x7fbf58110620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.471+0000 7fbf567fc700 1 -- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5810c5b0 msgr2=0x7fbf581110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.471+0000 7fbf567fc700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5810c5b0 0x7fbf581110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.471+0000 7fbf567fc700 1 -- 172.21.15.19:0/404967361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf581a4c60 con 0x7fbf58106ec0 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.471+0000 7fbf567fc700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106ec0 0x7fbf58110b60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf58075b70 tx=0x7fbf4800ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.472+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800cbd0 con 0x7fbf58106ec0 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.472+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf4800cd70 con 0x7fbf58106ec0 2024-05-17T09:46:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.472+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581a4f50 con 0x7fbf58106ec0 2024-05-17T09:46:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.472+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a54e0 con 0x7fbf58106ec0 2024-05-17T09:46:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.473+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48018680 con 0x7fbf58106ec0 2024-05-17T09:46:10.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.476+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf44005330 con 0x7fbf58106ec0 2024-05-17T09:46:10.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.479+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbf48018820 con 0x7fbf58106ec0 2024-05-17T09:46:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.480+0000 7fbf3ffff700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbf400b29b0 0x7fbf400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.480+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbf480e26e0 con 0x7fbf58106ec0 2024-05-17T09:46:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.481+0000 7fbf56ffd700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbf400b29b0 0x7fbf400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.481+0000 7fbf56ffd700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbf400b29b0 0x7fbf400b4ec0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0099b0 tx=0x7fbf4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.487+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf480a9ff0 con 0x7fbf58106ec0 2024-05-17T09:46:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.912+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf44005620 con 0x7fbf58106ec0 2024-05-17T09:46:10.911 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:10 smithi019 ceph-mon[31620]: pgmap v1230: 129 pgs: 129 active+clean; 527 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 308 KiB/s wr, 31 op/s 2024-05-17T09:46:10.912 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:10 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:10.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.913+0000 7fbf3ffff700 1 -- 172.21.15.19:0/404967361 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11854 (secure 0 0 0) 0x7fbf480b2070 con 0x7fbf58106ec0 2024-05-17T09:46:10.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:46:10.913 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:46:09.493896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24487]},"id":1}]} 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbf400b29b0 msgr2=0x7fbf400b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbf400b29b0 0x7fbf400b4ec0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0099b0 tx=0x7fbf4c008040 comp rx=0 tx=0).stop 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106ec0 msgr2=0x7fbf58110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106ec0 0x7fbf58110b60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf58075b70 tx=0x7fbf4800ed50 comp rx=0 tx=0).stop 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 shutdown_connections 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5810c5b0 0x7fbf581110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf58106570 0x7fbf58110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fbf400b29b0 0x7fbf400b4ec0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 --2- 172.21.15.19:0/404967361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf58106ec0 0x7fbf58110b60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.916+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 >> 172.21.15.19:0/404967361 conn(0x7fbf58102070 msgr2=0x7fbf58078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.917+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 shutdown_connections 2024-05-17T09:46:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:10.917+0000 7fbf5d757700 1 -- 172.21.15.19:0/404967361 wait complete. 2024-05-17T09:46:10.917 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 40 2024-05-17T09:46:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:10 smithi069 ceph-mon[35081]: pgmap v1230: 129 pgs: 129 active+clean; 527 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 308 KiB/s wr, 31 op/s 2024-05-17T09:46:10.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:10 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:11.020 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-17T09:46:11.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-17T09:46:11.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:10 smithi079 ceph-mon[35502]: pgmap v1230: 129 pgs: 129 active+clean; 527 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 308 KiB/s wr, 31 op/s 2024-05-17T09:46:11.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:10 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:11.597 DEBUG:teuthology.parallel:result is None 2024-05-17T09:46:11.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-17T09:46:11.636 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-17T09:46:11.636 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-17T09:46:11.647 INFO:tasks.quiescer:joining Quiescers 2024-05-17T09:46:11.647 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-17T09:46:11.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T09:46:11.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0' 2024-05-17T09:46:11.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0 2024-05-17T09:46:11.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:11 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/404967361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:11.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:11 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/404967361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:12.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:11 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/404967361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:12 smithi069 ceph-mon[35081]: pgmap v1231: 129 pgs: 129 active+clean; 441 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 281 KiB/s wr, 82 op/s 2024-05-17T09:46:12.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:12 smithi019 ceph-mon[31620]: pgmap v1231: 129 pgs: 129 active+clean; 441 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 281 KiB/s wr, 82 op/s 2024-05-17T09:46:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:12 smithi079 ceph-mon[35502]: pgmap v1231: 129 pgs: 129 active+clean; 441 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 281 KiB/s wr, 82 op/s 2024-05-17T09:46:13.769 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:13 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:13.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:13 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:14.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:13 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:14.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:14 smithi069 ceph-mon[35081]: pgmap v1232: 129 pgs: 129 active+clean; 365 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 337 KiB/s wr, 165 op/s 2024-05-17T09:46:14.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:14 smithi019 ceph-mon[31620]: pgmap v1232: 129 pgs: 129 active+clean; 365 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 337 KiB/s wr, 165 op/s 2024-05-17T09:46:15.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:14 smithi079 ceph-mon[35502]: pgmap v1232: 129 pgs: 129 active+clean; 365 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 337 KiB/s wr, 165 op/s 2024-05-17T09:46:15.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:15 smithi019 ceph-mon[31620]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:15.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:15 smithi019 ceph-mon[31620]: pgmap v1233: 129 pgs: 129 active+clean; 320 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 346 KiB/s wr, 198 op/s 2024-05-17T09:46:15.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 -- 172.21.15.19:0/3036796995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278108750 msgr2=0x7f02781100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 --2- 172.21.15.19:0/3036796995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278108750 0x7f02781100b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f027000b0d0 tx=0x7f027000b3e0 comp rx=0 tx=0).stop 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 -- 172.21.15.19:0/3036796995 shutdown_connections 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 --2- 172.21.15.19:0/3036796995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02781034d0 0x7f02781038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 --2- 172.21.15.19:0/3036796995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278108750 0x7f02781100b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 --2- 172.21.15.19:0/3036796995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0278107e00 0x7f0278108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 -- 172.21.15.19:0/3036796995 >> 172.21.15.19:0/3036796995 conn(0x7f02780fef20 msgr2=0x7f0278101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 -- 172.21.15.19:0/3036796995 shutdown_connections 2024-05-17T09:46:15.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.816+0000 7f027d5a2700 1 -- 172.21.15.19:0/3036796995 wait complete. 2024-05-17T09:46:15.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.817+0000 7f027d5a2700 1 Processor -- start 2024-05-17T09:46:15.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 -- start start 2024-05-17T09:46:15.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 0x7f027819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:15.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0278107e00 0x7f027819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:15.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0278108750 0x7f027819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:15.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0278113230 con 0x7f02781034d0 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02781130b0 con 0x7f0278107e00 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f027d5a2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f02781133b0 con 0x7f0278108750 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 0x7f027819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 0x7f027819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34830/0 (socket says 172.21.15.19:34830) 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 -- 172.21.15.19:0/1151534786 learned_addr learned my addr 172.21.15.19:0/1151534786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f02777fe700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0278108750 0x7f027819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f026e7fc700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0278107e00 0x7f027819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 -- 172.21.15.19:0/1151534786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0278108750 msgr2=0x7f027819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0278108750 0x7f027819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 -- 172.21.15.19:0/1151534786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0278107e00 msgr2=0x7f027819c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0278107e00 0x7f027819c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.818+0000 7f0276ffd700 1 -- 172.21.15.19:0/1151534786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a6ec0 con 0x7f02781034d0 2024-05-17T09:46:15.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.819+0000 7f0276ffd700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 0x7f027819bb70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f026800d980 tx=0x7f026800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:15.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.819+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268016070 con 0x7f02781034d0 2024-05-17T09:46:15.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.819+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0268004830 con 0x7f02781034d0 2024-05-17T09:46:15.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.819+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680054c0 con 0x7f02781034d0 2024-05-17T09:46:15.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.819+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a71b0 con 0x7f02781034d0 2024-05-17T09:46:15.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.819+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a76f0 con 0x7f02781034d0 2024-05-17T09:46:15.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.820+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02780512b0 con 0x7f02781034d0 2024-05-17T09:46:15.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.822+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0268004220 con 0x7f02781034d0 2024-05-17T09:46:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.823+0000 7f0274ff9700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02640b2880 0x7f02640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.823+0000 7f026e7fc700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02640b2880 0x7f02640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.823+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f02680e2820 con 0x7f02781034d0 2024-05-17T09:46:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.823+0000 7f026e7fc700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02640b2880 0x7f02640b4d90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f02781067d0 tx=0x7f0260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:15.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:15.826+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f026801f020 con 0x7f02781034d0 2024-05-17T09:46:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:15 smithi069 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:15 smithi069 ceph-mon[35081]: pgmap v1233: 129 pgs: 129 active+clean; 320 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 346 KiB/s wr, 198 op/s 2024-05-17T09:46:16.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:15 smithi079 ceph-mon[35502]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T09:46:16.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:15 smithi079 ceph-mon[35502]: pgmap v1233: 129 pgs: 129 active+clean; 320 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 346 KiB/s wr, 198 op/s 2024-05-17T09:46:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.105+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f027802e7a0 con 0x7f02781034d0 2024-05-17T09:46:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.106+0000 7f0274ff9700 1 -- 172.21.15.19:0/1151534786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11857 (secure 0 0 0) 0x7f02680a5020 con 0x7f02781034d0 2024-05-17T09:46:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:46:16.105 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4138238464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4138238464},{"type":"v1","addr":"172.21.15.19:6835","nonce":4138238464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2308476934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2308476934},{"type":"v1","addr":"172.21.15.19:6837","nonce":2308476934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1964664313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1964664313},{"type":"v1","addr":"172.21.15.19:6839","nonce":1964664313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2849498318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2849498318},{"type":"v1","addr":"172.21.15.19:6841","nonce":2849498318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1681609094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1681609094},{"type":"v1","addr":"172.21.15.79:6833","nonce":1681609094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/700922121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":700922121},{"type":"v1","addr":"172.21.15.69:6833","nonce":700922121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T09:15:09.950009+0000","modified":"2024-05-17T09:46:14.494505+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24487,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/437548660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":437548660},{"type":"v1","addr":"172.21.15.69:6835","nonce":437548660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1132484996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1132484996},{"type":"v1","addr":"172.21.15.79:6835","nonce":1132484996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/720193094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":720193094},{"type":"v1","addr":"172.21.15.79:6837","nonce":720193094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":122,"addr":"172.21.15.69:6837/1860504493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1860504493},{"type":"v1","addr":"172.21.15.69:6837","nonce":1860504493}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.79:6839/2520420584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2520420584},{"type":"v1","addr":"172.21.15.79:6839","nonce":2520420584}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.69:6839/814844032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":814844032},{"type":"v1","addr":"172.21.15.69:6839","nonce":814844032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24470,24487,24497]},"id":1}]} 2024-05-17T09:46:16.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02640b2880 msgr2=0x7f02640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02640b2880 0x7f02640b4d90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f02781067d0 tx=0x7f0260009260 comp rx=0 tx=0).stop 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 msgr2=0x7f027819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 0x7f027819bb70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f026800d980 tx=0x7f026800dc90 comp rx=0 tx=0).stop 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 shutdown_connections 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f02640b2880 0x7f02640b4d90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0278108750 0x7f027819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781034d0 0x7f027819bb70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 --2- 172.21.15.19:0/1151534786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0278107e00 0x7f027819c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 >> 172.21.15.19:0/1151534786 conn(0x7f02780fef20 msgr2=0x7f0278107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 shutdown_connections 2024-05-17T09:46:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.108+0000 7f027d5a2700 1 -- 172.21.15.19:0/1151534786 wait complete. 2024-05-17T09:46:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.412+0000 7fc919783700 1 -- 172.21.15.19:0/864161692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 msgr2=0x7fc9141038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.412+0000 7fc919783700 1 --2- 172.21.15.19:0/864161692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc9141038c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc9040099c0 tx=0x7fc904009cd0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 -- 172.21.15.19:0/864161692 shutdown_connections 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 --2- 172.21.15.19:0/864161692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc914108730 0x7fc914110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 --2- 172.21.15.19:0/864161692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc9141038c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 --2- 172.21.15.19:0/864161692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc914107de0 0x7fc9141081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 -- 172.21.15.19:0/864161692 >> 172.21.15.19:0/864161692 conn(0x7fc9140fedb0 msgr2=0x7fc9141011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 -- 172.21.15.19:0/864161692 shutdown_connections 2024-05-17T09:46:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 -- 172.21.15.19:0/864161692 wait complete. 2024-05-17T09:46:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.413+0000 7fc919783700 1 Processor -- start 2024-05-17T09:46:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 -- start start 2024-05-17T09:46:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc91419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc914107de0 0x7fc91419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc914108730 0x7fc91419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc914113210 con 0x7fc9141034b0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc914113090 con 0x7fc914107de0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc919783700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc914113390 con 0x7fc914108730 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc91419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc91419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34840/0 (socket says 172.21.15.19:34840) 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 -- 172.21.15.19:0/2482946033 learned_addr learned my addr 172.21.15.19:0/2482946033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc9137fe700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc914108730 0x7fc91419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc9127fc700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc914107de0 0x7fc91419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 -- 172.21.15.19:0/2482946033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc914108730 msgr2=0x7fc91419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc914108730 0x7fc91419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 -- 172.21.15.19:0/2482946033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc914107de0 msgr2=0x7fc91419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc914107de0 0x7fc91419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.414+0000 7fc912ffd700 1 -- 172.21.15.19:0/2482946033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91419fb50 con 0x7fc9141034b0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.415+0000 7fc912ffd700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc91419ad30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc904006010 tx=0x7fc9040055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:16.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.415+0000 7fc90bfff700 1 -- 172.21.15.19:0/2482946033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904026070 con 0x7fc9141034b0 2024-05-17T09:46:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.415+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91419fde0 con 0x7fc9141034b0 2024-05-17T09:46:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.415+0000 7fc90bfff700 1 -- 172.21.15.19:0/2482946033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc90400f460 con 0x7fc9141034b0 2024-05-17T09:46:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.415+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9141a0310 con 0x7fc9141034b0 2024-05-17T09:46:16.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.415+0000 7fc90bfff700 1 -- 172.21.15.19:0/2482946033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904009e60 con 0x7fc9141034b0 2024-05-17T09:46:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.418+0000 7fc90bfff700 1 -- 172.21.15.19:0/2482946033 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc904029070 con 0x7fc9141034b0 2024-05-17T09:46:16.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.419+0000 7fc90bfff700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9000b2880 0x7fc9000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.419+0000 7fc928198b80 1 Processor -- start 2024-05-17T09:46:16.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.419+0000 7fc9127fc700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9000b2880 0x7fc9000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.419+0000 7fc928198b80 1 -- start start 2024-05-17T09:46:16.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.419+0000 7fc90bfff700 1 -- 172.21.15.19:0/2482946033 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc9040e2fd0 con 0x7fc9141034b0 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc9127fc700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9000b2880 0x7fc9000b4d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc914109a40 tx=0x7fc8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc928198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 0x5627633a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc928198b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562763222fe0 0x5627633a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc928198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc928198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5627632cfe30 con 0x5627633a16c0 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc928198b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5627632bf9d0 con 0x562763222fe0 2024-05-17T09:46:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc928198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5627632c3a10 con 0x5627632a0570 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 0x5627633a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 0x5627633a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.19:46802/0 (socket says 172.21.15.19:46802) 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 -- 172.21.15.19:0/395264757 learned_addr learned my addr 172.21.15.19:0/395264757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc9137fe700 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc9127fc700 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562763222fe0 0x5627633a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 -- 172.21.15.19:0/395264757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562763222fe0 msgr2=0x5627633a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562763222fe0 0x5627633a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 -- 172.21.15.19:0/395264757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 msgr2=0x5627633a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.420+0000 7fc912ffd700 1 -- 172.21.15.19:0/395264757 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276320fa70 con 0x5627632a0570 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.421+0000 7fc912ffd700 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 0x5627633a7600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc9040aa200 tx=0x7fc9040a9c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b75d3b37b65d345 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.421+0000 7fc9097fa700 1 -- 172.21.15.19:0/395264757 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040aa970 con 0x5627632a0570 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.421+0000 7fc9097fa700 1 -- 172.21.15.19:0/395264757 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9040aa270 con 0x5627632a0570 2024-05-17T09:46:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.421+0000 7fc9097fa700 1 -- 172.21.15.19:0/395264757 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040b2bb0 con 0x5627632a0570 2024-05-17T09:46:16.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 -- 172.21.15.19:0/395264757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 msgr2=0x5627633a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 0x5627633a7600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc9040aa200 tx=0x7fc9040a9c40 comp rx=0 tx=0).stop 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 -- 172.21.15.19:0/395264757 shutdown_connections 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627632a0570 0x5627633a7600 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 --2- 172.21.15.19:0/395264757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562763222fe0 0x5627633a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 -- 172.21.15.19:0/395264757 >> 172.21.15.19:0/395264757 conn(0x562763261e20 msgr2=0x56276339c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 -- 172.21.15.19:0/395264757 shutdown_connections 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 -- 172.21.15.19:0/395264757 wait complete. 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.422+0000 7fc928198b80 1 -- start start 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 Processor -- start 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x5627633b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5627631c6140 con 0x5627633a16c0 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562763261960 con 0x5627632a0570 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5627633a2300 con 0x562763222fe0 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc928198b80 10 client.0 _ll_register_callbacks cb 0x56276325aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9127fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9137fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633ab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34862/0 (socket says 172.21.15.19:34862) 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9137fe700 1 -- 172.21.15.19:0/3192990368 learned_addr learned my addr 172.21.15.19:0/3192990368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x5627633b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9127fc700 1 -- 172.21.15.19:0/3192990368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 msgr2=0x5627633b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9127fc700 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x5627633b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9127fc700 1 -- 172.21.15.19:0/3192990368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 msgr2=0x5627633ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.423+0000 7fc9127fc700 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc9127fc700 1 -- 172.21.15.19:0/3192990368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276325e830 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc9127fc700 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b11e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5627633a70c0 tx=0x7fc8fc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfcbfdce73db6ed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc8fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/3192990368 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc02b060 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/3192990368 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc8fc01c900 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9040aa270 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/3192990368 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc8fc032050 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc928198b80 10 client.28868 fetch_fsmap learned FSMap version 42 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5627633b8920 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.424+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/3192990368 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9040aa270 con 0x5627632a0570 2024-05-17T09:46:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/3192990368 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fc8fc025030 con 0x5627632a0570 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc928198b80 10 client.28868 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc928198b80 10 client.28868 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc928198b80 20 client.28868 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc928198b80 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633baf60 0x5627633bd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc928198b80 4 client.28868 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627632d1d80 con 0x5627633baf60 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc912ffd700 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633baf60 0x5627633bd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc912ffd700 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633baf60 0x5627633bd380 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2c50b57e1db8f server_cookie=669b7cfe39e9ccea in_seq=0 out_seq=0 2024-05-17T09:46:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.425+0000 7fc8fa7fc700 10 client.28868 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:16.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/3192990368 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5627632d1d80 con 0x5627633baf60 2024-05-17T09:46:16.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc8fa7fc700 10 client.28868 handle_command_reply: tid=0 2024-05-17T09:46:16.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 client.28868 shutdown 2024-05-17T09:46:16.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 msgr2=0x5627633b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b11e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5627633a70c0 tx=0x7fc8fc0047c0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 shutdown_connections 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x5627633b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633baf60 0x5627633bd380 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627633a16c0 0x5627633ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 --2- 172.21.15.19:0/3192990368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b11e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc8fa7fc700 0 client.28868 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 >> 172.21.15.19:0/3192990368 conn(0x562763261e20 msgr2=0x562763396320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.426+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 shutdown_connections 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.427+0000 7fc928198b80 1 -- 172.21.15.19:0/3192990368 wait complete. 2024-05-17T09:46:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.427+0000 7fc928198b80 20 client.28868 trim_cache size 0 max 16384 2024-05-17T09:46:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.447+0000 7fc928198b80 1 Processor -- start 2024-05-17T09:46:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.447+0000 7fc928198b80 1 -- start start 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc928198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763261e20 0x5627633a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc928198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x562763396a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc928198b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc928198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9141b4d70 con 0x562763261e20 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc928198b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9141b2570 con 0x5627632a0570 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc928198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc9141b03c0 con 0x562763222fe0 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763261e20 0x5627633a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763261e20 0x5627633a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34872/0 (socket says 172.21.15.19:34872) 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc912ffd700 1 -- 172.21.15.19:0/1098355748 learned_addr learned my addr 172.21.15.19:0/1098355748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc9127fc700 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x562763396a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc9137fe700 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc9127fc700 1 -- 172.21.15.19:0/1098355748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 msgr2=0x5627633c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.448+0000 7fc9127fc700 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc9127fc700 1 -- 172.21.15.19:0/1098355748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763261e20 msgr2=0x5627633a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc9127fc700 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763261e20 0x5627633a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc9127fc700 1 -- 172.21.15.19:0/1098355748 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627633b1530 con 0x562763222fe0 2024-05-17T09:46:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc9127fc700 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x562763396a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc0046f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9287e1dbe8075 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc908ff9700 1 -- 172.21.15.19:0/1098355748 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0265f0 con 0x562763222fe0 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc908ff9700 1 -- 172.21.15.19:0/1098355748 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8fc026790 con 0x562763222fe0 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.449+0000 7fc908ff9700 1 -- 172.21.15.19:0/1098355748 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00e890 con 0x562763222fe0 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 -- 172.21.15.19:0/1098355748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 msgr2=0x562763396a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x562763396a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc0046f0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 -- 172.21.15.19:0/1098355748 shutdown_connections 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763261e20 0x5627633a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 --2- 172.21.15.19:0/1098355748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763222fe0 0x562763396a30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 -- 172.21.15.19:0/1098355748 >> 172.21.15.19:0/1098355748 conn(0x5627633ab930 msgr2=0x562763396320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 -- 172.21.15.19:0/1098355748 shutdown_connections 2024-05-17T09:46:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.450+0000 7fc928198b80 1 -- 172.21.15.19:0/1098355748 wait complete. 2024-05-17T09:46:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 -- start start 2024-05-17T09:46:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 Processor -- start 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 0x5627633b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763261e20 0x5627633b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9141b0a30 con 0x562763222fe0 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9141b1ea0 con 0x5627632a0570 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc9141b1100 con 0x562763261e20 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc928198b80 10 client.0 _ll_register_callbacks cb 0x56276325aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.451+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 0x5627633b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 0x5627633b7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34878/0 (socket says 172.21.15.19:34878) 2024-05-17T09:46:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 -- 172.21.15.19:0/212590972 learned_addr learned my addr 172.21.15.19:0/212590972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc9127fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763261e20 0x5627633b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc9137fe700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 -- 172.21.15.19:0/212590972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763261e20 msgr2=0x5627633b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763261e20 0x5627633b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 -- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 msgr2=0x5627633b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc912ffd700 1 -- 172.21.15.19:0/212590972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627633b6780 con 0x562763222fe0 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.452+0000 7fc9127fc700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763261e20 0x5627633b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc9137fe700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc912ffd700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 0x5627633b7680 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc904009320 tx=0x7fc9040a9bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d78fb7b2db0d4c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:16.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc8fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/212590972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040c3030 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/212590972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc9040a6860 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8fc026790 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/212590972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc9040aa270 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc928198b80 10 client.19377 fetch_fsmap learned FSMap version 42 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.453+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5627634269d0 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/212590972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8fc026790 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/212590972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fc904069ce0 con 0x562763222fe0 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc928198b80 10 client.19377 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc928198b80 10 client.19377 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc928198b80 20 client.19377 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:16.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc928198b80 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633a16c0 0x562763428fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:16.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc928198b80 4 client.19377 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}] 2024-05-17T09:46:16.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}) -- 0x5627632cb560 con 0x5627633a16c0 2024-05-17T09:46:16.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.454+0000 7fc9127fc700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633a16c0 0x562763428fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:16.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.455+0000 7fc8fa7fc700 10 client.19377 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:16.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.455+0000 7fc9127fc700 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633a16c0 0x562763428fa0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b399b7a5936b8bc1 server_cookie=caaca6026edea49b in_seq=0 out_seq=0 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.460+0000 7fc8fa7fc700 1 -- 172.21.15.19:0/212590972 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5627632cb560 con 0x5627633a16c0 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.460+0000 7fc8fa7fc700 10 client.19377 handle_command_reply: tid=0 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.460+0000 7fc928198b80 1 client.19377 shutdown 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.460+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 msgr2=0x5627633b7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.460+0000 7fc928198b80 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 0x5627633b7680 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc904009320 tx=0x7fc9040a9bc0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 shutdown_connections 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x5627633a16c0 0x562763428fa0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627632a0570 0x5627633b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562763222fe0 0x5627633b7680 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc8fa7fc700 0 client.19377 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:16.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 --2- 172.21.15.19:0/212590972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562763261e20 0x5627633b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 >> 172.21.15.19:0/212590972 conn(0x5627633ab930 msgr2=0x5627633a06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 shutdown_connections 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 1 -- 172.21.15.19:0/212590972 wait complete. 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc928198b80 20 client.19377 trim_cache size 0 max 16384 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 42, 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 98, 2024-05-17T09:46:16.461 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "6f36c806": { 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 98, 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 143.9, 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-05-17T09:46:16.462 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:16.463 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9000b2880 msgr2=0x7fc9000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc919783700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9000b2880 0x7fc9000b4d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc914109a40 tx=0x7fc8fc009260 comp rx=0 tx=0).stop 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.461+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 msgr2=0x7fc91419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc91419ad30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc904006010 tx=0x7fc9040055f0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 shutdown_connections 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fc9000b2880 0x7fc9000b4d90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc914108730 0x7fc91419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9141034b0 0x7fc91419ad30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 --2- 172.21.15.19:0/2482946033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc914107de0 0x7fc91419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 >> 172.21.15.19:0/2482946033 conn(0x7fc9140fedb0 msgr2=0x7fc91406ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 shutdown_connections 2024-05-17T09:46:16.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:16.462+0000 7fc919783700 1 -- 172.21.15.19:0/2482946033 wait complete. 2024-05-17T09:46:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:46:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:16 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1151534786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:46:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:46:16.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:16 smithi069 ceph-mon[35081]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:46:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:46:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:16 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1151534786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:46:16.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:46:16.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:16 smithi019 ceph-mon[31620]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:46:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T09:46:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:16 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1151534786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T09:46:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T09:46:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T09:46:17.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:16 smithi079 ceph-mon[35502]: from='mgr.14152 172.21.15.19:0/583363376' entity='mgr.x' 2024-05-17T09:46:17.380 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --cancel --all' 2024-05-17T09:46:17.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --cancel --all 2024-05-17T09:46:17.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:17 smithi069 ceph-mon[35081]: pgmap v1234: 129 pgs: 129 active+clean; 318 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 349 KiB/s wr, 204 op/s 2024-05-17T09:46:17.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:17 smithi019 ceph-mon[31620]: pgmap v1234: 129 pgs: 129 active+clean; 318 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 349 KiB/s wr, 204 op/s 2024-05-17T09:46:18.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:17 smithi079 ceph-mon[35502]: pgmap v1234: 129 pgs: 129 active+clean; 318 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 349 KiB/s wr, 204 op/s 2024-05-17T09:46:18.024 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6f36c806', quiesce took 9.9 seconds. Will release after: 14.049999999999999 2024-05-17T09:46:18.108 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.112+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/206867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a241045c0 msgr2=0x7f8a241049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.112+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/206867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a241045c0 0x7f8a241049d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8a200099c0 tx=0x7f8a20009cd0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/206867882 shutdown_connections 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/206867882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24109840 0x7f8a241111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/206867882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a241045c0 0x7f8a241049d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/206867882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a24108ef0 0x7f8a24109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/206867882 >> 172.21.15.19:0/206867882 conn(0x7f8a240ffe90 msgr2=0x7f8a241022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/206867882 shutdown_connections 2024-05-17T09:46:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/206867882 wait complete. 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 Processor -- start 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.113+0000 7f8a2c1c3700 1 -- start start 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2c1c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a241045c0 0x7f8a24076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2c1c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 0x7f8a24077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2c1c3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a24109840 0x7f8a24077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2c1c3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24114320 con 0x7f8a24108ef0 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2c1c3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241141a0 con 0x7f8a24109840 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2c1c3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241144a0 con 0x7f8a241045c0 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 0x7f8a24077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 0x7f8a24077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34902/0 (socket says 172.21.15.19:34902) 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 -- 172.21.15.19:0/2259023100 learned_addr learned my addr 172.21.15.19:0/2259023100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a241045c0 0x7f8a24076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2a760700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a24109840 0x7f8a24077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 -- 172.21.15.19:0/2259023100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a241045c0 msgr2=0x7f8a24076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a241045c0 0x7f8a24076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 -- 172.21.15.19:0/2259023100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a24109840 msgr2=0x7f8a24077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a24109840 0x7f8a24077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.114+0000 7f8a2975e700 1 -- 172.21.15.19:0/2259023100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a241a9070 con 0x7f8a24108ef0 2024-05-17T09:46:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.115+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 0x7f8a24077100 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c8f0 tx=0x7f8a1400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.115+0000 7f8a1affd700 1 -- 172.21.15.19:0/2259023100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a140078d0 con 0x7f8a24108ef0 2024-05-17T09:46:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.115+0000 7f8a1affd700 1 -- 172.21.15.19:0/2259023100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1400f450 con 0x7f8a24108ef0 2024-05-17T09:46:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.115+0000 7f8a1affd700 1 -- 172.21.15.19:0/2259023100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14018660 con 0x7f8a24108ef0 2024-05-17T09:46:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.115+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241a9360 con 0x7f8a24108ef0 2024-05-17T09:46:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.115+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a241a98a0 con 0x7f8a24108ef0 2024-05-17T09:46:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.118+0000 7f8a1affd700 1 -- 172.21.15.19:0/2259023100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8a1400f900 con 0x7f8a24108ef0 2024-05-17T09:46:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.119+0000 7f8a1affd700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a100b2880 0x7f8a100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.119+0000 7f8a3abd8b80 1 Processor -- start 2024-05-17T09:46:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.119+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a100b2880 0x7f8a100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.119+0000 7f8a1affd700 1 -- 172.21.15.19:0/2259023100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8a140e2400 con 0x7f8a24108ef0 2024-05-17T09:46:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.119+0000 7f8a3abd8b80 1 -- start start 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f318e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f3193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f313460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564b9f240de0 con 0x564b9f27f4a0 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564b9f230980 con 0x564b9f312f90 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564b9f2349c0 con 0x564b9f193eb0 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a100b2880 0x7f8a100b4d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8a20000c00 tx=0x7f8a20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f318e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f318e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34914/0 (socket says 172.21.15.19:34914) 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 -- 172.21.15.19:0/4136504640 learned_addr learned my addr 172.21.15.19:0/4136504640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a2975e700 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f3193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a2a760700 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f313460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 -- 172.21.15.19:0/4136504640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 msgr2=0x564b9f3193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f3193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 -- 172.21.15.19:0/4136504640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 msgr2=0x564b9f313460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f313460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 -- 172.21.15.19:0/4136504640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9f1da020 con 0x564b9f27f4a0 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a2975e700 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f3193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.120+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f318e00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f8a20015000 tx=0x7f8a2001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38971fd3b1377b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.121+0000 7f8a0ffff700 1 -- 172.21.15.19:0/4136504640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20005650 con 0x564b9f27f4a0 2024-05-17T09:46:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.121+0000 7f8a0ffff700 1 -- 172.21.15.19:0/4136504640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a20003b10 con 0x564b9f27f4a0 2024-05-17T09:46:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.121+0000 7f8a0ffff700 1 -- 172.21.15.19:0/4136504640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2002f970 con 0x564b9f27f4a0 2024-05-17T09:46:20.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.121+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/4136504640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 msgr2=0x564b9f318e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.121+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f318e00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f8a20015000 tx=0x7f8a2001e000 comp rx=0 tx=0).stop 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/4136504640 shutdown_connections 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f313460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f318e00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/4136504640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f3193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/4136504640 >> 172.21.15.19:0/4136504640 conn(0x564b9f1f4230 msgr2=0x564b9f30dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/4136504640 shutdown_connections 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/4136504640 wait complete. 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 -- start start 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.122+0000 7f8a3abd8b80 1 Processor -- start 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f31caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f31d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f31d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564b9f137010 con 0x564b9f312f90 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564b9f198e10 con 0x564b9f193eb0 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564b9f31fa90 con 0x564b9f27f4a0 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a3abd8b80 10 client.0 _ll_register_callbacks cb 0x564b9f1cb9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f31d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f31d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34924/0 (socket says 172.21.15.19:34924) 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 -- 172.21.15.19:0/2531378086 learned_addr learned my addr 172.21.15.19:0/2531378086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f31d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f31caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 -- 172.21.15.19:0/2531378086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 msgr2=0x564b9f31d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f31d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 -- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 msgr2=0x564b9f31caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f31caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.123+0000 7f8a2a760700 1 -- 172.21.15.19:0/2531378086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9f180940 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a2a760700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f31d570 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0042d0 tx=0x7f8a1c004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e2a0d52bd0cada server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/2531378086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c01a070 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/2531378086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8a1c004db0 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a20003b10 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/2531378086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8a1c025050 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 10 client.19389 fetch_fsmap learned FSMap version 42 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564b9f1ea7d0 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/2531378086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a20003b10 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/2531378086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f8a1c016660 con 0x564b9f312f90 2024-05-17T09:46:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 10 client.19389 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 10 client.19389 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 20 client.19389 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f32c8f0 0x564b9f32ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 4 client.19389 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.124+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b9f2def40 con 0x564b9f32c8f0 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.125+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f32c8f0 0x564b9f32ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.125+0000 7f8a0d7fa700 10 client.19389 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.125+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f32c8f0 0x564b9f32ed80 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dcbbfc9200d21a server_cookie=2466976605219e1f in_seq=0 out_seq=0 2024-05-17T09:46:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/2531378086 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564b9f2def40 con 0x564b9f32c8f0 2024-05-17T09:46:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a0d7fa700 10 client.19389 handle_command_reply: tid=0 2024-05-17T09:46:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 client.19389 shutdown 2024-05-17T09:46:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 msgr2=0x564b9f31d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f31d570 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0042d0 tx=0x7f8a1c004300 comp rx=0 tx=0).stop 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 shutdown_connections 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f32c8f0 0x564b9f32ed80 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f31d570 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f31caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2531378086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f31d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a0d7fa700 0 client.19389 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.126+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 >> 172.21.15.19:0/2531378086 conn(0x564b9f1f4230 msgr2=0x564b9f311f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.127+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 shutdown_connections 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.127+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2531378086 wait complete. 2024-05-17T09:46:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.127+0000 7f8a3abd8b80 20 client.19389 trim_cache size 0 max 16384 2024-05-17T09:46:20.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.147+0000 7f8a3abd8b80 1 Processor -- start 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.147+0000 7f8a3abd8b80 1 -- start start 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.147+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f308560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.147+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f311f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.147+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f3124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241b4d50 con 0x564b9f27f4a0 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241b2550 con 0x564b9f312f90 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241b03a0 con 0x564b9f193eb0 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f311f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f311f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34928/0 (socket says 172.21.15.19:34928) 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 -- 172.21.15.19:0/2998428650 learned_addr learned my addr 172.21.15.19:0/2998428650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2a760700 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f3124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f308560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 -- 172.21.15.19:0/2998428650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 msgr2=0x564b9f308560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f308560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 -- 172.21.15.19:0/2998428650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 msgr2=0x564b9f3124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f3124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 -- 172.21.15.19:0/2998428650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9f31b060 con 0x564b9f27f4a0 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f308560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a2975e700 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f311f80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f8a140a6880 tx=0x7f8a14091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb32fe8fb53dc5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a0f7fe700 1 -- 172.21.15.19:0/2998428650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a140e20b0 con 0x564b9f27f4a0 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a0f7fe700 1 -- 172.21.15.19:0/2998428650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a140910b0 con 0x564b9f27f4a0 2024-05-17T09:46:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.148+0000 7f8a0f7fe700 1 -- 172.21.15.19:0/2998428650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a140b1fe0 con 0x564b9f27f4a0 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2998428650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 msgr2=0x564b9f311f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f311f80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f8a140a6880 tx=0x7f8a14091040 comp rx=0 tx=0).stop 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2998428650 shutdown_connections 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f193eb0 0x564b9f308560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f312f90 0x564b9f3124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/2998428650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f27f4a0 0x564b9f311f80 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2998428650 >> 172.21.15.19:0/2998428650 conn(0x564b9f1f4230 msgr2=0x564b9f316d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2998428650 shutdown_connections 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.149+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/2998428650 wait complete. 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 -- start start 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 Processor -- start 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f325240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f321be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f322120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241b0a10 con 0x564b9f312f90 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241b1e80 con 0x564b9f193eb0 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241b10e0 con 0x564b9f27f4a0 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a3abd8b80 10 client.0 _ll_register_callbacks cb 0x564b9f1cb9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.150+0000 7f8a2a760700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f322120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f322120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34932/0 (socket says 172.21.15.19:34932) 2024-05-17T09:46:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a29f5f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f325240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 -- 172.21.15.19:0/3973725166 learned_addr learned my addr 172.21.15.19:0/3973725166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2975e700 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f321be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 -- 172.21.15.19:0/3973725166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 msgr2=0x564b9f321be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f321be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 -- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 msgr2=0x564b9f325240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f325240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 -- 172.21.15.19:0/3973725166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9f322790 con 0x564b9f312f90 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a2a760700 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f322120 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c016a80 tx=0x7f8a1c005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b659b5d29e18859 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/3973725166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c02d020 con 0x564b9f312f90 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/3973725166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8a1c004a70 con 0x564b9f312f90 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.151+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a140910b0 con 0x564b9f312f90 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/3973725166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8a1c038050 con 0x564b9f312f90 2024-05-17T09:46:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 10 client.19395 fetch_fsmap learned FSMap version 42 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564b9f398250 con 0x564b9f312f90 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/3973725166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a140910b0 con 0x564b9f312f90 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/3973725166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f8a1c007b20 con 0x564b9f312f90 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 10 client.19395 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 10 client.19395 resolve_mds: validated gid 24497 aka daemon mds.b 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 20 client.19395 populate_metadata read hostname 'smithi019' 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f3988d0 0x564b9f39acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 4 client.19395 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 --> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x564b9f242d30 con 0x564b9f3988d0 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.152+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f3988d0 0x564b9f39acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a0d7fa700 10 client.19395 ms_handle_connect on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a29f5f700 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f3988d0 0x564b9f39acf0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14ecdb3540db793 server_cookie=3ed1b7aef99e6952 in_seq=0 out_seq=0 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a0d7fa700 1 -- 172.21.15.19:0/3973725166 <== mds.0 v2:172.21.15.69:6838/814844032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x564b9f242d30 con 0x564b9f3988d0 2024-05-17T09:46:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a0d7fa700 10 client.19395 handle_command_reply: tid=0 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 client.19395 shutdown 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 msgr2=0x564b9f322120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f322120 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c016a80 tx=0x7f8a1c005e90 comp rx=0 tx=0).stop 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 shutdown_connections 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:6838/814844032,v1:172.21.15.69:6839/814844032] conn(0x564b9f3988d0 0x564b9f39acf0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b9f312f90 0x564b9f322120 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564b9f193eb0 0x564b9f325240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 --2- 172.21.15.19:0/3973725166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b9f27f4a0 0x564b9f321be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a0d7fa700 0 client.19395 ms_handle_reset on v2:172.21.15.69:6838/814844032 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 >> 172.21.15.19:0/3973725166 conn(0x564b9f1f4230 msgr2=0x564b9f311f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 shutdown_connections 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.153+0000 7f8a3abd8b80 1 -- 172.21.15.19:0/3973725166 wait complete. 2024-05-17T09:46:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a3abd8b80 20 client.19395 trim_cache size 0 max 16384 2024-05-17T09:46:20.158 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:20.158 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 42, 2024-05-17T09:46:20.158 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24497, 2024-05-17T09:46:20.158 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 99, 2024-05-17T09:46:20.158 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": {} 2024-05-17T09:46:20.158 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a100b2880 msgr2=0x7f8a100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a100b2880 0x7f8a100b4d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8a20000c00 tx=0x7f8a20005fb0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 msgr2=0x7f8a24077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 0x7f8a24077100 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c8f0 tx=0x7f8a1400cc00 comp rx=0 tx=0).stop 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 shutdown_connections 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f8a100b2880 0x7f8a100b4d90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a241045c0 0x7f8a24076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a24109840 0x7f8a24077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 --2- 172.21.15.19:0/2259023100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a24108ef0 0x7f8a24077100 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 >> 172.21.15.19:0/2259023100 conn(0x7f8a240ffe90 msgr2=0x7f8a241022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 shutdown_connections 2024-05-17T09:46:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:20.154+0000 7f8a2c1c3700 1 -- 172.21.15.19:0/2259023100 wait complete. 2024-05-17T09:46:20.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:20 smithi019 ceph-mon[31620]: pgmap v1235: 129 pgs: 129 active+clean; 318 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 347 KiB/s wr, 205 op/s 2024-05-17T09:46:20.881 INFO:tasks.quiescer:done joining Quiescers 2024-05-17T09:46:20.881 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-17T09:46:20.891 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:46:20.900 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-17T09:46:20.909 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-17T09:46:20.910 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a4b16c58a5d7aa60dee491f6f096b7509e2f1269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-17T09:46:20.910 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-17T09:46:20.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T09:46:20.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:20 smithi069 ceph-mon[35081]: pgmap v1235: 129 pgs: 129 active+clean; 318 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 347 KiB/s wr, 205 op/s 2024-05-17T09:46:21.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:20 smithi079 ceph-mon[35502]: pgmap v1235: 129 pgs: 129 active+clean; 318 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 347 KiB/s wr, 205 op/s 2024-05-17T09:46:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 -- 172.21.15.19:0/2422653861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c102b20 msgr2=0x7f020c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 --2- 172.21.15.19:0/2422653861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c102b20 0x7f020c103820 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009900 tx=0x7f01fc02a410 comp rx=0 tx=0).stop 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 -- 172.21.15.19:0/2422653861 shutdown_connections 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 --2- 172.21.15.19:0/2422653861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020c108a60 0x7f020c10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 --2- 172.21.15.19:0/2422653861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c103d60 0x7f020c104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 --2- 172.21.15.19:0/2422653861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c102b20 0x7f020c103820 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 -- 172.21.15.19:0/2422653861 >> 172.21.15.19:0/2422653861 conn(0x7f020c0fdd10 msgr2=0x7f020c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 -- 172.21.15.19:0/2422653861 shutdown_connections 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.064+0000 7f0211ce0640 1 -- 172.21.15.19:0/2422653861 wait complete. 2024-05-17T09:46:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 Processor -- start 2024-05-17T09:46:21.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 -- start start 2024-05-17T09:46:21.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020c102b20 0x7f020c0ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 0x7f020c0fe240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c108a60 0x7f020c0fe780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10e4d0 con 0x7f020c103d60 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10e350 con 0x7f020c102b20 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f0211ce0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10e650 con 0x7f020c108a60 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f020affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 0x7f020c0fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f020affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 0x7f020c0fe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34954/0 (socket says 172.21.15.19:34954) 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f020affd640 1 -- 172.21.15.19:0/1972912537 learned_addr learned my addr 172.21.15.19:0/1972912537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.065+0000 7f020b7fe640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020c102b20 0x7f020c0ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020bfff640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c108a60 0x7f020c0fe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020affd640 1 -- 172.21.15.19:0/1972912537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c108a60 msgr2=0x7f020c0fe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020affd640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c108a60 0x7f020c0fe780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020affd640 1 -- 172.21.15.19:0/1972912537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020c102b20 msgr2=0x7f020c0ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020affd640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020c102b20 0x7f020c0ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020affd640 1 -- 172.21.15.19:0/1972912537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0feeb0 con 0x7f020c103d60 2024-05-17T09:46:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f020affd640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 0x7f020c0fe240 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f020000d770 tx=0x7f020000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200017070 con 0x7f020c103d60 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02000054b0 con 0x7f020c103d60 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c1a4290 con 0x7f020c103d60 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200004c60 con 0x7f020c103d60 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.066+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c1a4750 con 0x7f020c103d60 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.067+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c1a49b0 con 0x7f020c103d60 2024-05-17T09:46:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.069+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0200004e00 con 0x7f020c103d60 2024-05-17T09:46:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.070+0000 7f0208ff9640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f01f40b2920 0x7f01f40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.070+0000 7f020b7fe640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f01f40b2920 0x7f01f40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.070+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f02000e1fd0 con 0x7f020c103d60 2024-05-17T09:46:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.070+0000 7f020b7fe640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f01f40b2920 0x7f01f40b4e30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f01fc005760 tx=0x7f01fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:21.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.075+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02000a9940 con 0x7f020c103d60 2024-05-17T09:46:21.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.271+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f020c1a4c20 con 0x7f020c103d60 2024-05-17T09:46:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.274+0000 7f0208ff9640 1 -- 172.21.15.19:0/1972912537 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f02000a9240 con 0x7f020c103d60 2024-05-17T09:46:21.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:46:21.274 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9c7a48b0-142c-11ef-bc9a-c7b262605968","created":"2024-05-17T09:06:43.021173+0000","modified":"2024-05-17T09:16:48.308262+0000","last_up_change":"2024-05-17T09:14:03.552628+0000","last_in_change":"2024-05-17T09:13:43.314092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T09:10:10.865915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T09:15:01.931223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T09:15:05.954895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T09:15:18.012606+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7de60f57-fe91-4722-8fac-00c122296aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6803","nonce":3945774453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6805","nonce":3945774453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6809","nonce":3945774453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3945774453},{"type":"v1","addr":"172.21.15.19:6807","nonce":3945774453}]},"public_addr":"172.21.15.19:6803/3945774453","cluster_addr":"172.21.15.19:6805/3945774453","heartbeat_back_addr":"172.21.15.19:6809/3945774453","heartbeat_front_addr":"172.21.15.19:6807/3945774453","state":["exists","up"]},{"osd":1,"uuid":"840241e0-3ae2-4033-8bbb-01a3935c3123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6801","nonce":3851861092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6803","nonce":3851861092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6807","nonce":3851861092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3851861092},{"type":"v1","addr":"172.21.15.69:6805","nonce":3851861092}]},"public_addr":"172.21.15.69:6801/3851861092","cluster_addr":"172.21.15.69:6803/3851861092","heartbeat_back_addr":"172.21.15.69:6807/3851861092","heartbeat_front_addr":"172.21.15.69:6805/3851861092","state":["exists","up"]},{"osd":2,"uuid":"afc95a8c-798a-4d1a-abb7-e7a50f8f2c9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6801","nonce":3984657664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6803","nonce":3984657664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6807","nonce":3984657664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":3984657664},{"type":"v1","addr":"172.21.15.79:6805","nonce":3984657664}]},"public_addr":"172.21.15.79:6801/3984657664","cluster_addr":"172.21.15.79:6803/3984657664","heartbeat_back_addr":"172.21.15.79:6807/3984657664","heartbeat_front_addr":"172.21.15.79:6805/3984657664","state":["exists","up"]},{"osd":3,"uuid":"774aea07-241e-4f94-aa94-5e8823cb5b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6811","nonce":494922873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6813","nonce":494922873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6817","nonce":494922873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":494922873},{"type":"v1","addr":"172.21.15.19:6815","nonce":494922873}]},"public_addr":"172.21.15.19:6811/494922873","cluster_addr":"172.21.15.19:6813/494922873","heartbeat_back_addr":"172.21.15.19:6817/494922873","heartbeat_front_addr":"172.21.15.19:6815/494922873","state":["exists","up"]},{"osd":4,"uuid":"8760fee7-c08a-4a4e-bc91-754119cb9d3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6809","nonce":3507791356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6811","nonce":3507791356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6815","nonce":3507791356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3507791356},{"type":"v1","addr":"172.21.15.69:6813","nonce":3507791356}]},"public_addr":"172.21.15.69:6809/3507791356","cluster_addr":"172.21.15.69:6811/3507791356","heartbeat_back_addr":"172.21.15.69:6815/3507791356","heartbeat_front_addr":"172.21.15.69:6813/3507791356","state":["exists","up"]},{"osd":5,"uuid":"7609df91-0a4b-4f8f-9a41-9d644e9a3ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6809","nonce":3720398663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6811","nonce":3720398663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6815","nonce":3720398663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3720398663},{"type":"v1","addr":"172.21.15.79:6813","nonce":3720398663}]},"public_addr":"172.21.15.79:6809/3720398663","cluster_addr":"172.21.15.79:6811/3720398663","heartbeat_back_addr":"172.21.15.79:6815/3720398663","heartbeat_front_addr":"172.21.15.79:6813/3720398663","state":["exists","up"]},{"osd":6,"uuid":"37a6a6d7-3e20-4503-afce-96f03af0e774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6819","nonce":1984910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6821","nonce":1984910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6825","nonce":1984910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1984910928},{"type":"v1","addr":"172.21.15.19:6823","nonce":1984910928}]},"public_addr":"172.21.15.19:6819/1984910928","cluster_addr":"172.21.15.19:6821/1984910928","heartbeat_back_addr":"172.21.15.19:6825/1984910928","heartbeat_front_addr":"172.21.15.19:6823/1984910928","state":["exists","up"]},{"osd":7,"uuid":"829c24b4-70f4-4d39-8126-1766fe7920b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6817","nonce":2368256164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6819","nonce":2368256164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6823","nonce":2368256164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2368256164},{"type":"v1","addr":"172.21.15.69:6821","nonce":2368256164}]},"public_addr":"172.21.15.69:6817/2368256164","cluster_addr":"172.21.15.69:6819/2368256164","heartbeat_back_addr":"172.21.15.69:6823/2368256164","heartbeat_front_addr":"172.21.15.69:6821/2368256164","state":["exists","up"]},{"osd":8,"uuid":"106603bf-b7f4-438c-abc4-b660029e1128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6817","nonce":2695186498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6819","nonce":2695186498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6823","nonce":2695186498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2695186498},{"type":"v1","addr":"172.21.15.79:6821","nonce":2695186498}]},"public_addr":"172.21.15.79:6817/2695186498","cluster_addr":"172.21.15.79:6819/2695186498","heartbeat_back_addr":"172.21.15.79:6823/2695186498","heartbeat_front_addr":"172.21.15.79:6821/2695186498","state":["exists","up"]},{"osd":9,"uuid":"e132748b-b8b9-4b25-b233-d786e59d9330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6827","nonce":2798428087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6829","nonce":2798428087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6833","nonce":2798428087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2798428087},{"type":"v1","addr":"172.21.15.19:6831","nonce":2798428087}]},"public_addr":"172.21.15.19:6827/2798428087","cluster_addr":"172.21.15.19:6829/2798428087","heartbeat_back_addr":"172.21.15.19:6833/2798428087","heartbeat_front_addr":"172.21.15.19:6831/2798428087","state":["exists","up"]},{"osd":10,"uuid":"9a14a50a-2686-4e06-9473-0cf82a95e252","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6825","nonce":3630021153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6827","nonce":3630021153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6831","nonce":3630021153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3630021153},{"type":"v1","addr":"172.21.15.69:6829","nonce":3630021153}]},"public_addr":"172.21.15.69:6825/3630021153","cluster_addr":"172.21.15.69:6827/3630021153","heartbeat_back_addr":"172.21.15.69:6831/3630021153","heartbeat_front_addr":"172.21.15.69:6829/3630021153","state":["exists","up"]},{"osd":11,"uuid":"2a4cbff5-754e-4a24-8ad4-7ed4c3d82b68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6825","nonce":2698518199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6827","nonce":2698518199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6831","nonce":2698518199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2698518199},{"type":"v1","addr":"172.21.15.79:6829","nonce":2698518199}]},"public_addr":"172.21.15.79:6825/2698518199","cluster_addr":"172.21.15.79:6827/2698518199","heartbeat_back_addr":"172.21.15.79:6831/2698518199","heartbeat_front_addr":"172.21.15.79:6829/2698518199","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:20.444064+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:09:44.180077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:08.503935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:34.040879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:10:58.760332+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:23.880935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:11:50.331055+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:16.371826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:12:41.942309+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:09.419199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:13:35.570118+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T09:14:01.894611+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/21803007":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6801/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3426431385":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/4126868109":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3686497433":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6801/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:0/3120865986":"2024-05-18T09:07:46.771034+0000","172.21.15.19:6800/747673746":"2024-05-18T09:07:20.018567+0000","172.21.15.19:6800/2577177111":"2024-05-18T09:07:46.771034+0000","172.21.15.19:0/3378222227":"2024-05-18T09:07:20.018567+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f01f40b2920 msgr2=0x7f01f40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f01f40b2920 0x7f01f40b4e30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f01fc005760 tx=0x7f01fc000bc0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 msgr2=0x7f020c0fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 0x7f020c0fe240 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f020000d770 tx=0x7f020000dc30 comp rx=0 tx=0).stop 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 shutdown_connections 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c108a60 0x7f020c0fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f01f40b2920 0x7f01f40b4e30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f020c103d60 0x7f020c0fe240 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 --2- 172.21.15.19:0/1972912537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020c102b20 0x7f020c0ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 >> 172.21.15.19:0/1972912537 conn(0x7f020c0fdd10 msgr2=0x7f020c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 shutdown_connections 2024-05-17T09:46:21.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.276+0000 7f0211ce0640 1 -- 172.21.15.19:0/1972912537 wait complete. 2024-05-17T09:46:21.290 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T09:10:10.865915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-17T09:15:01.931223+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-17T09:15:05.954895+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-17T09:15:18.012606+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-17T09:46:21.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 -- 172.21.15.19:0/2478652780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c108a30 msgr2=0x7fa28c10ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 --2- 172.21.15.19:0/2478652780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c108a30 0x7fa28c10ae50 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009ca0 tx=0x7fa27c02a8c0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 -- 172.21.15.19:0/2478652780 shutdown_connections 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 --2- 172.21.15.19:0/2478652780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c108a30 0x7fa28c10ae50 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 --2- 172.21.15.19:0/2478652780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa28c103d30 0x7fa28c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 --2- 172.21.15.19:0/2478652780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c102af0 0x7fa28c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 -- 172.21.15.19:0/2478652780 >> 172.21.15.19:0/2478652780 conn(0x7fa28c0fdce0 msgr2=0x7fa28c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 -- 172.21.15.19:0/2478652780 shutdown_connections 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.410+0000 7fa2941b2640 1 -- 172.21.15.19:0/2478652780 wait complete. 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 Processor -- start 2024-05-17T09:46:21.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 -- start start 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 0x7fa28c06ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c103d30 0x7fa28c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa28c108a30 0x7fa28c1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10e500 con 0x7fa28c102af0 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10e380 con 0x7fa28c103d30 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa2941b2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10e680 con 0x7fa28c108a30 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 0x7fa28c06ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 0x7fa28c06ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34976/0 (socket says 172.21.15.19:34976) 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 -- 172.21.15.19:0/1997812986 learned_addr learned my addr 172.21.15.19:0/1997812986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa29274f640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa28c108a30 0x7fa28c1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa29174d640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c103d30 0x7fa28c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 -- 172.21.15.19:0/1997812986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa28c108a30 msgr2=0x7fa28c1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa28c108a30 0x7fa28c1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 -- 172.21.15.19:0/1997812986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c103d30 msgr2=0x7fa28c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c103d30 0x7fa28c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 -- 172.21.15.19:0/1997812986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c1a1c30 con 0x7fa28c102af0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa29174d640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c103d30 0x7fa28c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.411+0000 7fa291f4e640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 0x7fa28c06ab90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e9f0 tx=0x7fa28400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.412+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28400ccc0 con 0x7fa28c102af0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.412+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2840044b0 con 0x7fa28c102af0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.412+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c1a1f20 con 0x7fa28c102af0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.412+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284010430 con 0x7fa28c102af0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.412+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a23e0 con 0x7fa28c102af0 2024-05-17T09:46:21.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.413+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c107840 con 0x7fa28c102af0 2024-05-17T09:46:21.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.414+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa284003f90 con 0x7fa28c102af0 2024-05-17T09:46:21.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.415+0000 7fa28affd640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa2740b2870 0x7fa2740b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.415+0000 7fa29174d640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa2740b2870 0x7fa2740b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.415+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa2840e2b30 con 0x7fa28c102af0 2024-05-17T09:46:21.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.416+0000 7fa29174d640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa2740b2870 0x7fa2740b4d80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa28c06a530 tx=0x7fa280005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:21.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.419+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2840aa4d0 con 0x7fa28c102af0 2024-05-17T09:46:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.612+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa28c02de90 con 0x7fa28c102af0 2024-05-17T09:46:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.612+0000 7fa28affd640 1 -- 172.21.15.19:0/1997812986 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fa2840a9dd0 con 0x7fa28c102af0 2024-05-17T09:46:21.611 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa2740b2870 msgr2=0x7fa2740b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa2740b2870 0x7fa2740b4d80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa28c06a530 tx=0x7fa280005db0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 msgr2=0x7fa28c06ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 0x7fa28c06ab90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e9f0 tx=0x7fa28400eeb0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 shutdown_connections 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa28c108a30 0x7fa28c1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.614+0000 7fa2941b2640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa28c103d30 0x7fa28c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.615+0000 7fa2941b2640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fa2740b2870 0x7fa2740b4d80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.615+0000 7fa2941b2640 1 --2- 172.21.15.19:0/1997812986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa28c102af0 0x7fa28c06ab90 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.615+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 >> 172.21.15.19:0/1997812986 conn(0x7fa28c0fdce0 msgr2=0x7fa28c104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.615+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 shutdown_connections 2024-05-17T09:46:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.615+0000 7fa2941b2640 1 -- 172.21.15.19:0/1997812986 wait complete. 2024-05-17T09:46:21.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:21 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1972912537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:46:21.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-17T09:46:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 -- 172.21.15.19:0/4072834438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8104e40 msgr2=0x7feff8105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 --2- 172.21.15.19:0/4072834438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8104e40 0x7feff8105250 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fefec009900 tx=0x7fefec02a410 comp rx=0 tx=0).stop 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 -- 172.21.15.19:0/4072834438 shutdown_connections 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 --2- 172.21.15.19:0/4072834438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff8109b40 0x7feff810bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 --2- 172.21.15.19:0/4072834438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8104e40 0x7feff8105250 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 --2- 172.21.15.19:0/4072834438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feff8103c00 0x7feff8104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 -- 172.21.15.19:0/4072834438 >> 172.21.15.19:0/4072834438 conn(0x7feff80fedf0 msgr2=0x7feff81012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 -- 172.21.15.19:0/4072834438 shutdown_connections 2024-05-17T09:46:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.746+0000 7feffdc3b640 1 -- 172.21.15.19:0/4072834438 wait complete. 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 Processor -- start 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 -- start start 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 0x7feff819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff8104e40 0x7feff819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feff8109b40 0x7feff81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feff810f5d0 con 0x7feff8103c00 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feff810f450 con 0x7feff8104e40 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feffdc3b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feff810f750 con 0x7feff8109b40 2024-05-17T09:46:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feff77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 0x7feff819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feff77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 0x7feff819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34992/0 (socket says 172.21.15.19:34992) 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.747+0000 7feff77fe640 1 -- 172.21.15.19:0/1609744718 learned_addr learned my addr 172.21.15.19:0/1609744718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff7fff640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feff8109b40 0x7feff81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff6ffd640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff8104e40 0x7feff819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff77fe640 1 -- 172.21.15.19:0/1609744718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feff8109b40 msgr2=0x7feff81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff77fe640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feff8109b40 0x7feff81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff77fe640 1 -- 172.21.15.19:0/1609744718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff8104e40 msgr2=0x7feff819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff77fe640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff8104e40 0x7feff819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff77fe640 1 -- 172.21.15.19:0/1609744718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff81a1db0 con 0x7feff8103c00 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff77fe640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 0x7feff819a270 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fefe800e9f0 tx=0x7fefe800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe800ccc0 con 0x7feff8103c00 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff81a20a0 con 0x7feff8103c00 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefe80035f0 con 0x7feff8103c00 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe800ce60 con 0x7feff8103c00 2024-05-17T09:46:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.748+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81a25f0 con 0x7feff8103c00 2024-05-17T09:46:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.749+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff8108950 con 0x7feff8103c00 2024-05-17T09:46:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.751+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fefe8004000 con 0x7feff8103c00 2024-05-17T09:46:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.751+0000 7feff4ff9640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fefe00b27f0 0x7fefe00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.752+0000 7feff6ffd640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fefe00b27f0 0x7fefe00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.752+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fefe80e28c0 con 0x7feff8103c00 2024-05-17T09:46:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.752+0000 7feff6ffd640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fefe00b27f0 0x7fefe00b4d00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feff819b310 tx=0x7fefec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.756+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fefe80aa230 con 0x7feff8103c00 2024-05-17T09:46:21.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:21 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1972912537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:46:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.947+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feff80638c0 con 0x7feff8103c00 2024-05-17T09:46:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.948+0000 7feff4ff9640 1 -- 172.21.15.19:0/1609744718 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fefe80b22b0 con 0x7feff8103c00 2024-05-17T09:46:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fefe00b27f0 msgr2=0x7fefe00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fefe00b27f0 0x7fefe00b4d00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feff819b310 tx=0x7fefec000bc0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 msgr2=0x7feff819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 0x7feff819a270 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fefe800e9f0 tx=0x7fefe800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 shutdown_connections 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feff8109b40 0x7feff81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff8104e40 0x7feff819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fefe00b27f0 0x7fefe00b4d00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 --2- 172.21.15.19:0/1609744718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feff8103c00 0x7feff819a270 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 >> 172.21.15.19:0/1609744718 conn(0x7feff80fedf0 msgr2=0x7feff8105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 shutdown_connections 2024-05-17T09:46:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:21.950+0000 7feffdc3b640 1 -- 172.21.15.19:0/1609744718 wait complete. 2024-05-17T09:46:21.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-17T09:46:22.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:21 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1972912537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T09:46:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 -- 172.21.15.19:0/1066856421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98103d60 msgr2=0x7efd98104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 --2- 172.21.15.19:0/1066856421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98103d60 0x7efd98104170 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009900 tx=0x7efd8c02a410 comp rx=0 tx=0).stop 2024-05-17T09:46:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 -- 172.21.15.19:0/1066856421 shutdown_connections 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 --2- 172.21.15.19:0/1066856421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd98108a60 0x7efd9810ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 --2- 172.21.15.19:0/1066856421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98103d60 0x7efd98104170 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 --2- 172.21.15.19:0/1066856421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98102b20 0x7efd98103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 -- 172.21.15.19:0/1066856421 >> 172.21.15.19:0/1066856421 conn(0x7efd980fdd50 msgr2=0x7efd981001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 -- 172.21.15.19:0/1066856421 shutdown_connections 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.079+0000 7efd9eb01640 1 -- 172.21.15.19:0/1066856421 wait complete. 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 Processor -- start 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 -- start start 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 0x7efd98075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98103d60 0x7efd9819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd98108a60 0x7efd981a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810e4f0 con 0x7efd98102b20 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810e370 con 0x7efd98108a60 2024-05-17T09:46:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9eb01640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810e670 con 0x7efd98103d60 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 0x7efd98075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 0x7efd98075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35018/0 (socket says 172.21.15.19:35018) 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 -- 172.21.15.19:0/363980859 learned_addr learned my addr 172.21.15.19:0/363980859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9d09e640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd98108a60 0x7efd981a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd97fff640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98103d60 0x7efd9819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 -- 172.21.15.19:0/363980859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98103d60 msgr2=0x7efd9819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98103d60 0x7efd9819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 -- 172.21.15.19:0/363980859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd98108a60 msgr2=0x7efd981a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd98108a60 0x7efd981a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd9c89d640 1 -- 172.21.15.19:0/363980859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd981a1d80 con 0x7efd98102b20 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.080+0000 7efd97fff640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98103d60 0x7efd9819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:46:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.081+0000 7efd9c89d640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 0x7efd98075910 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e9f0 tx=0x7efd8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.081+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800ccc0 con 0x7efd98102b20 2024-05-17T09:46:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.081+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd880035f0 con 0x7efd98102b20 2024-05-17T09:46:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.081+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800ce60 con 0x7efd98102b20 2024-05-17T09:46:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.081+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd981a2070 con 0x7efd98102b20 2024-05-17T09:46:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.081+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd981a25c0 con 0x7efd98102b20 2024-05-17T09:46:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.082+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd98107870 con 0x7efd98102b20 2024-05-17T09:46:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.084+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efd88004000 con 0x7efd98102b20 2024-05-17T09:46:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.084+0000 7efd95ffb640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd800b2870 0x7efd800b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.084+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efd880e2a40 con 0x7efd98102b20 2024-05-17T09:46:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.085+0000 7efd97fff640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd800b2870 0x7efd800b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.085+0000 7efd97fff640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd800b2870 0x7efd800b4d80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd8c006010 tx=0x7efd8c031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.088+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd880aa350 con 0x7efd98102b20 2024-05-17T09:46:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.279+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd9802ddb0 con 0x7efd98102b20 2024-05-17T09:46:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.279+0000 7efd95ffb640 1 -- 172.21.15.19:0/363980859 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7efd880a9c50 con 0x7efd98102b20 2024-05-17T09:46:22.279 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd800b2870 msgr2=0x7efd800b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd800b2870 0x7efd800b4d80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd8c006010 tx=0x7efd8c031420 comp rx=0 tx=0).stop 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 msgr2=0x7efd98075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 0x7efd98075910 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e9f0 tx=0x7efd8800eeb0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 shutdown_connections 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd98108a60 0x7efd981a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd98103d60 0x7efd9819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7efd800b2870 0x7efd800b4d80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 --2- 172.21.15.19:0/363980859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd98102b20 0x7efd98075910 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.281+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 >> 172.21.15.19:0/363980859 conn(0x7efd980fdd50 msgr2=0x7efd98104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.282+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 shutdown_connections 2024-05-17T09:46:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.282+0000 7efd9eb01640 1 -- 172.21.15.19:0/363980859 wait complete. 2024-05-17T09:46:22.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-17T09:46:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 -- 172.21.15.19:0/1446305584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 msgr2=0x7f26101055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 --2- 172.21.15.19:0/1446305584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101055a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2600009900 tx=0x7f260002a410 comp rx=0 tx=0).stop 2024-05-17T09:46:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 -- 172.21.15.19:0/1446305584 shutdown_connections 2024-05-17T09:46:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 --2- 172.21.15.19:0/1446305584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2610102340 0x7f261010bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 --2- 172.21.15.19:0/1446305584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2610101840 0x7f2610101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 --2- 172.21.15.19:0/1446305584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101055a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 -- 172.21.15.19:0/1446305584 >> 172.21.15.19:0/1446305584 conn(0x7f26100fd460 msgr2=0x7f26100ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 -- 172.21.15.19:0/1446305584 shutdown_connections 2024-05-17T09:46:22.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.413+0000 7f26168f2640 1 -- 172.21.15.19:0/1446305584 wait complete. 2024-05-17T09:46:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 Processor -- start 2024-05-17T09:46:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 -- start start 2024-05-17T09:46:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2610101840 0x7f261019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2610102340 0x7f261019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f261010f590 con 0x7f2610105190 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f261010f410 con 0x7f2610101840 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f26168f2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f261010f710 con 0x7f2610102340 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f2614e8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f2614e8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35028/0 (socket says 172.21.15.19:35028) 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f2614e8f640 1 -- 172.21.15.19:0/1077088554 learned_addr learned my addr 172.21.15.19:0/1077088554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f260ffff640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2610101840 0x7f261019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.414+0000 7f2607fff640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2610102340 0x7f261019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f2614e8f640 1 -- 172.21.15.19:0/1077088554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2610102340 msgr2=0x7f261019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f2614e8f640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2610102340 0x7f261019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f2614e8f640 1 -- 172.21.15.19:0/1077088554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2610101840 msgr2=0x7f261019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f2614e8f640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2610101840 0x7f261019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f2614e8f640 1 -- 172.21.15.19:0/1077088554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101a1d90 con 0x7f2610105190 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f2614e8f640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101a1720 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f260800eaa0 tx=0x7f260800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800cdb0 con 0x7f2610105190 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2608005020 con 0x7f2610105190 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26080116d0 con 0x7f2610105190 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101a2080 con 0x7f2610105190 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.415+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a2580 con 0x7f2610105190 2024-05-17T09:46:22.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.416+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610108b20 con 0x7f2610105190 2024-05-17T09:46:22.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.418+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26080051d0 con 0x7f2610105190 2024-05-17T09:46:22.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.419+0000 7f260dffb640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25fc0b2870 0x7f25fc0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.419+0000 7f260ffff640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25fc0b2870 0x7f25fc0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.419+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f26080e2be0 con 0x7f2610105190 2024-05-17T09:46:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.420+0000 7f260ffff640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25fc0b2870 0x7f25fc0b4d80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f26000057c0 tx=0x7f2600005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.422+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2608016030 con 0x7f2610105190 2024-05-17T09:46:22.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.617+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f261002ddb0 con 0x7f2610105190 2024-05-17T09:46:22.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.617+0000 7f260dffb640 1 -- 172.21.15.19:0/1077088554 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f26080aa5b0 con 0x7f2610105190 2024-05-17T09:46:22.617 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.619+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25fc0b2870 msgr2=0x7f25fc0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25fc0b2870 0x7f25fc0b4d80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f26000057c0 tx=0x7f2600005720 comp rx=0 tx=0).stop 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 msgr2=0x7f26101a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101a1720 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f260800eaa0 tx=0x7f260800ef60 comp rx=0 tx=0).stop 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 shutdown_connections 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f25fc0b2870 0x7f25fc0b4d80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2610105190 0x7f26101a1720 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2610102340 0x7f261019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 --2- 172.21.15.19:0/1077088554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2610101840 0x7f261019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 >> 172.21.15.19:0/1077088554 conn(0x7f26100fd460 msgr2=0x7f2610105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 shutdown_connections 2024-05-17T09:46:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.620+0000 7f26168f2640 1 -- 172.21.15.19:0/1077088554 wait complete. 2024-05-17T09:46:22.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 -- 172.21.15.19:0/1184391175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 msgr2=0x7f3f5c104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/1184391175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c104170 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009900 tx=0x7f3f4402a410 comp rx=0 tx=0).stop 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 -- 172.21.15.19:0/1184391175 shutdown_connections 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/1184391175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f5c108a60 0x7f3f5c10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/1184391175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c104170 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/1184391175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f5c102b20 0x7f3f5c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 -- 172.21.15.19:0/1184391175 >> 172.21.15.19:0/1184391175 conn(0x7f3f5c0fdd10 msgr2=0x7f3f5c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 -- 172.21.15.19:0/1184391175 shutdown_connections 2024-05-17T09:46:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.749+0000 7f3f61d9d640 1 -- 172.21.15.19:0/1184391175 wait complete. 2024-05-17T09:46:22.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 Processor -- start 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 -- start start 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f5c102b20 0x7f3f5c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f5c108a60 0x7f3f5c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10f600 con 0x7f3f5c103d60 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10f480 con 0x7f3f5c102b20 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.750+0000 7f3f61d9d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10f780 con 0x7f3f5c108a60 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35044/0 (socket says 172.21.15.19:35044) 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 -- 172.21.15.19:0/2694165538 learned_addr learned my addr 172.21.15.19:0/2694165538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5b7fe640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f5c102b20 0x7f3f5c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5bfff640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f5c108a60 0x7f3f5c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 -- 172.21.15.19:0/2694165538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f5c108a60 msgr2=0x7f3f5c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f5c108a60 0x7f3f5c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 -- 172.21.15.19:0/2694165538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f5c102b20 msgr2=0x7f3f5c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f5c102b20 0x7f3f5c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 -- 172.21.15.19:0/2694165538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c1a1ce0 con 0x7f3f5c103d60 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f5affd640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c19a6e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009230 tx=0x7f3f44005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4403f030 con 0x7f3f5c103d60 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f440043d0 con 0x7f3f5c103d60 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44035070 con 0x7f3f5c103d60 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.751+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c1a1f70 con 0x7f3f5c103d60 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.752+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1a23d0 con 0x7f3f5c103d60 2024-05-17T09:46:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.753+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c107870 con 0x7f3f5c103d60 2024-05-17T09:46:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.755+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f44042030 con 0x7f3f5c103d60 2024-05-17T09:46:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.756+0000 7f3f58ff9640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f480b2920 0x7f3f480b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:46:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.756+0000 7f3f5b7fe640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f480b2920 0x7f3f480b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:46:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.756+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3f44102de0 con 0x7f3f5c103d60 2024-05-17T09:46:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.757+0000 7f3f5b7fe640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f480b2920 0x7f3f480b4e30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c103720 tx=0x7f3f4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:46:22.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:22.760+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f440ca0a0 con 0x7f3f5c103d60 2024-05-17T09:46:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:22 smithi069 ceph-mon[35081]: pgmap v1236: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 307 KiB/s wr, 201 op/s 2024-05-17T09:46:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:22 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1997812986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:46:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:22 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1609744718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T09:46:22.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:22 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/363980859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T09:46:22.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:22 smithi019 ceph-mon[31620]: pgmap v1236: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 307 KiB/s wr, 201 op/s 2024-05-17T09:46:22.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:22 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1997812986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:46:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:22 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1609744718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T09:46:22.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:22 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/363980859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T09:46:23.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.007+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3f5c02ddf0 con 0x7f3f5c103d60 2024-05-17T09:46:23.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.017+0000 7f3f58ff9640 1 -- 172.21.15.19:0/2694165538 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815641 (secure 0 0 0) 0x7f3f440ce3a0 con 0x7f3f5c103d60 2024-05-17T09:46:23.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:22 smithi079 ceph-mon[35502]: pgmap v1236: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 307 KiB/s wr, 201 op/s 2024-05-17T09:46:23.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:22 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1997812986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T09:46:23.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:22 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1609744718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T09:46:23.023 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:22 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/363980859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T09:46:23.027 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-17T09:46:23.027 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 18, 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 14152, 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3017687651 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3017687651 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/3017687651", 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-05-17T09:07:46.771161+0000", 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24109, 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-05-17T09:46:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.034 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.036 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-17T09:46:23.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-17T09:46:23.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.054 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.056 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-17T09:46:23.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.065 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.066 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.067 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-17T09:46:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-17T09:46:23.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-17T09:46:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T09:46:23.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T09:46:23.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-17T09:46:23.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.095 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.096 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T09:46:23.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-17T09:46:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-17T09:46:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-17T09:46:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-17T09:46:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-17T09:46:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-17T09:46:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-17T09:46:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-17T09:46:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-17T09:46:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-17T09:46:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-17T09:46:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T09:46:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T09:46:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T09:46:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T09:46:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T09:46:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-17T09:46:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-17T09:46:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-17T09:46:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-17T09:46:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-17T09:46:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-17T09:46:23.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.211 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.218 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-17T09:46:23.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-17T09:46:23.228 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.229 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.232 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-17T09:46:23.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-17T09:46:23.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T09:46:23.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-17T09:46:23.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.246 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.249 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T09:46:23.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.259 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.262 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.263 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.267 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-17T09:46:23.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.272 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.274 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.277 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T09:46:23.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.279 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.282 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-17T09:46:23.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.290 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.291 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.300 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.303 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.304 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.307 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-17T09:46:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.326 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.335 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.339 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.344 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.357 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T09:46:23.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-17T09:46:23.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.362 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.363 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-17T09:46:23.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-17T09:46:23.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.370 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-17T09:46:23.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.382 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.393 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.397 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-17T09:46:23.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.404 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-17T09:46:23.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.422 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.423 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.431 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.433 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.436 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.442 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.445 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.446 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.457 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.460 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T09:46:23.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.469 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-17T09:46:23.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-17T09:46:23.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-17T09:46:23.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.476 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.484 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.486 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.490 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.492 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-17T09:46:23.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.495 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-17T09:46:23.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T09:46:23.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.503 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24110, 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T09:46:23.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.514 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-17T09:46:23.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-17T09:46:23.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.528 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.529 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T09:46:23.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.537 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.540 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-17T09:46:23.543 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-17T09:46:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-17T09:46:23.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-17T09:46:23.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-17T09:46:23.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-17T09:46:23.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-17T09:46:23.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-17T09:46:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-17T09:46:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-17T09:46:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-17T09:46:23.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-17T09:46:23.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-17T09:46:23.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T09:46:23.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T09:46:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-17T09:46:23.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.586 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.590 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-17T09:46:23.597 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-17T09:46:23.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.613 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.614 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-17T09:46:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.626 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-17T09:46:23.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-17T09:46:23.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.654 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T09:46:23.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T09:46:23.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T09:46:23.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T09:46:23.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T09:46:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T09:46:23.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T09:46:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T09:46:23.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T09:46:23.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T09:46:23.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T09:46:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-17T09:46:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-17T09:46:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.678 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.679 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.680 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.681 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.684 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-17T09:46:23.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-17T09:46:23.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.688 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-17T09:46:23.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.690 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.693 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-17T09:46:23.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-17T09:46:23.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-17T09:46:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-17T09:46:23.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.712 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.713 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.717 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-17T09:46:23.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-17T09:46:23.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-17T09:46:23.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.726 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.732 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.734 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.739 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.745 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.747 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.748 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-17T09:46:23.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.755 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-17T09:46:23.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.765 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.766 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.767 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.770 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.780 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.788 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.791 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.792 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.796 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.803 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.805 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-17T09:46:23.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.821 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-17T09:46:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:23.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.828 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.830 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-17T09:46:23.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-17T09:46:23.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-17T09:46:23.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-17T09:46:23.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.847 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-17T09:46:23.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.853 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.856 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.858 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.867 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.870 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.871 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-17T09:46:23.877 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.878 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.890 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-17T09:46:23.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.899 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-17T09:46:23.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.906 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.920 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.922 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-17T09:46:23.925 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.928 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.931 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T09:46:23.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.953 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.954 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.957 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.961 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.965 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.968 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T09:46:23.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-17T09:46:23.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T09:46:23.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T09:46:23.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:23.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:23.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "restful" 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.989 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.990 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:23.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:23.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-17T09:46:23.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-17T09:46:23.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-17T09:46:23.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-17T09:46:24.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-17T09:46:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-17T09:46:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-17T09:46:24.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-17T09:46:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-17T09:46:24.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.017 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-17T09:46:24.020 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:24.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-17T09:46:24.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-17T09:46:24.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-17T09:46:24.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-17T09:46:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-17T09:46:24.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T09:46:24.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-17T09:46:24.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T09:46:24.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-17T09:46:24.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-17T09:46:24.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:24.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.051 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.052 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-17T09:46:24.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-17T09:46:24.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-17T09:46:24.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-17T09:46:24.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-17T09:46:24.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-17T09:46:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-17T09:46:24.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-17T09:46:24.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.061 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.064 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T09:46:24.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T09:46:24.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-17T09:46:24.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-17T09:46:24.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.075 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-17T09:46:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T09:46:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T09:46:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T09:46:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T09:46:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T09:46:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T09:46:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T09:46:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T09:46:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-17T09:46:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T09:46:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T09:46:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T09:46:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T09:46:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T09:46:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T09:46:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T09:46:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T09:46:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-17T09:46:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-17T09:46:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-17T09:46:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-17T09:46:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.203 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.206 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-17T09:46:24.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.212 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-17T09:46:24.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.216 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.217 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.227 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-17T09:46:24.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.239 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-17T09:46:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.253 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.260 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.269 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.279 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.282 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.286 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.289 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.290 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-17T09:46:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.310 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.315 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.321 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-17T09:46:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T09:46:24.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.325 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.334 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.357 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.361 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.362 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T09:46:24.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.389 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.390 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.391 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.392 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-17T09:46:24.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.399 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.402 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.406 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.413 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.416 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.419 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.422 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T09:46:24.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T09:46:24.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-17T09:46:24.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.445 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.454 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.455 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.464 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-17T09:46:24.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-17T09:46:24.471 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.472 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.475 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-17T09:46:24.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-17T09:46:24.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-17T09:46:24.484 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-17T09:46:24.485 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-17T09:46:24.486 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-17T09:46:24.487 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-17T09:46:24.488 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 3, 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 377361485 2024-05-17T09:46:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1294883711 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 732951443 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-17T09:46:24.491 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-17T09:46:24.492 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3537872081 2024-05-17T09:46:24.492 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.492 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:24.492 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:24.492 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-17T09:46:24.492 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f480b2920 msgr2=0x7f3f480b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f480b2920 0x7f3f480b4e30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c103720 tx=0x7f3f4c005f90 comp rx=0 tx=0).stop 2024-05-17T09:46:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 msgr2=0x7f3f5c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c19a6e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009230 tx=0x7f3f44005e80 comp rx=0 tx=0).stop 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 shutdown_connections 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f5c108a60 0x7f3f5c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7f3f480b2920 0x7f3f480b4e30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f5c103d60 0x7f3f5c19a6e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 --2- 172.21.15.19:0/2694165538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f5c102b20 0x7f3f5c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 >> 172.21.15.19:0/2694165538 conn(0x7f3f5c0fdd10 msgr2=0x7f3f5c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 shutdown_connections 2024-05-17T09:46:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:46:23.023+0000 7f3f61d9d640 1 -- 172.21.15.19:0/2694165538 wait complete. 2024-05-17T09:46:24.549 DEBUG:tasks.check_counter:Getting stats from a 2024-05-17T09:46:24.549 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-17T09:46:24.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:23 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1077088554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T09:46:24.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:23 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/2694165538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T09:46:24.550 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:23 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1077088554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T09:46:24.551 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:23 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/2694165538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T09:46:24.551 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:23 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1077088554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T09:46:24.551 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:23 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/2694165538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T09:46:24.816 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:24.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:24 smithi069 ceph-mon[35081]: pgmap v1237: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 212 KiB/s wr, 141 op/s 2024-05-17T09:46:24.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:24 smithi019 ceph-mon[31620]: pgmap v1237: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 212 KiB/s wr, 141 op/s 2024-05-17T09:46:25.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:24 smithi079 ceph-mon[35502]: pgmap v1237: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 212 KiB/s wr, 141 op/s 2024-05-17T09:46:26.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:26 smithi019 ceph-mon[31620]: pgmap v1238: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 53 op/s 2024-05-17T09:46:26.760 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2912, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001858173, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000043973, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000642661, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000827786, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000275928 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 385, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 304000, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.027147981, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.021184413, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002532318, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 383, 2024-05-17T09:46:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020452589, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000053401 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 304000 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 504, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 501, 2024-05-17T09:46:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 595280, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 311008, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.066406154, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.028165037, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.023908628, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010284656, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 500, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.019044084, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000038088 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 595280, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 311008 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-17T09:46:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148951, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-17T09:46:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-17T09:46:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 507, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 523992, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 507, 2024-05-17T09:46:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 507, 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 523992, 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:26.772 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:26.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:26 smithi069 ceph-mon[35081]: pgmap v1238: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 53 op/s 2024-05-17T09:46:27.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:26 smithi079 ceph-mon[35502]: pgmap v1238: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 53 op/s 2024-05-17T09:46:27.580 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-17T09:46:27.581 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-17T09:46:27.581 DEBUG:tasks.check_counter:Getting stats from d 2024-05-17T09:46:27.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-17T09:46:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:28.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:28 smithi069 ceph-mon[35081]: pgmap v1239: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-05-17T09:46:28.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:28 smithi019 ceph-mon[31620]: pgmap v1239: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-05-17T09:46:29.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:28 smithi079 ceph-mon[35502]: pgmap v1239: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-17T09:46:30.099 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001487675, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000354624, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:30.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 390, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 304880, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.026619088, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.021104647, 2024-05-17T09:46:30.101 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002080508, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 388, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.015807013, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000040739 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 304880 2024-05-17T09:46:30.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 515, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 510, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 600160, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 315456, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.069699628, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.029674095, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.026603502, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010285623, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 510, 2024-05-17T09:46:30.103 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020824260, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000040831 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 600160, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 315456 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-17T09:46:30.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:30.105 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:30.106 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147937, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-17T09:46:30.107 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:30.108 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 514, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 524966, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 514, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 514, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 524966, 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-17T09:46:30.109 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:30.110 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:30.110 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:30.110 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:30.110 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:30.110 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:30.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:30 smithi019 ceph-mon[31620]: pgmap v1240: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-17T09:46:30.833 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-17T09:46:30.833 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-17T09:46:30.833 DEBUG:tasks.check_counter:Getting stats from g 2024-05-17T09:46:30.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-17T09:46:30.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:30 smithi069 ceph-mon[35081]: pgmap v1240: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-17T09:46:31.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:30 smithi079 ceph-mon[35502]: pgmap v1240: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-17T09:46:31.091 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:32.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:32 smithi019 ceph-mon[31620]: pgmap v1241: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-17T09:46:32.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:32 smithi079 ceph-mon[35502]: pgmap v1241: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 506, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 503, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 595824, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 312288, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.069172345, 2024-05-17T09:46:32.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.029250401, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.023259146, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010643842, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 502, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.022538407, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000044897 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.852 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 595824, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 312288 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 7, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 389, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3392, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 304800, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.027275942, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.021001209, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002734802, 2024-05-17T09:46:32.853 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 388, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.019404497, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000050011 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 304800 2024-05-17T09:46:32.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000373475, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000276372, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:32.855 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:32.856 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:32.857 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:32.858 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147960, 2024-05-17T09:46:32.859 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:32.860 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 509, 2024-05-17T09:46:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 524252, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 509, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 509, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 524252, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:32.862 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:32.863 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:32.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:32 smithi069 ceph-mon[35081]: pgmap v1241: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-17T09:46:33.565 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-17T09:46:33.565 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-17T09:46:33.565 DEBUG:tasks.check_counter:Getting stats from j 2024-05-17T09:46:33.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-17T09:46:33.823 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:46:34.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:34 smithi069 ceph-mon[35081]: pgmap v1242: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-17T09:46:34.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:34 smithi019 ceph-mon[31620]: pgmap v1242: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-17T09:46:35.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:34 smithi079 ceph-mon[35502]: pgmap v1242: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001301438, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T09:46:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000258073, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 392, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 305232, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.027108829, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.021194948, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002304559, 2024-05-17T09:46:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 390, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.017068530, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000043765 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 305232 2024-05-17T09:46:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 517, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 512, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 600704, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 316736, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.068268058, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.028764537, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.025349428, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010569808, 2024-05-17T09:46:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 512, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.019986505, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000039036 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 600704, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 316736 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-17T09:46:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147977, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 29, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-17T09:46:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 516, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 525226, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 516, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 516, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 525226, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-17T09:46:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-17T09:46:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-17T09:46:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-17T09:46:35.658 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-17T09:46:36.388 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-17T09:46:36.388 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-17T09:46:36.389 DEBUG:tasks.check_counter:Getting stats from b 2024-05-17T09:46:36.389 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-17T09:46:36.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:36 smithi019 ceph-mon[31620]: pgmap v1243: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-17T09:46:36.709 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:46:36.830 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:36 smithi069 ceph-mon[35081]: pgmap v1243: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-17T09:46:37.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:36 smithi079 ceph-mon[35502]: pgmap v1243: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-17T09:46:38.619 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-17T09:46:38.619 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:38.619 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 84616, 2024-05-17T09:46:38.619 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 95417, 2024-05-17T09:46:38.619 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 20334858, 2024-05-17T09:46:38.620 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 110444565, 2024-05-17T09:46:38.620 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 398, 2024-05-17T09:46:38.620 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-17T09:46:38.620 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 6.926313942, 2024-05-17T09:46:38.620 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 3.058580227, 2024-05-17T09:46:38.620 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 3.252708083, 2024-05-17T09:46:38.621 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.951011406, 2024-05-17T09:46:38.621 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:38.621 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 95415, 2024-05-17T09:46:38.621 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 9.155219065, 2024-05-17T09:46:38.621 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000095951 2024-05-17T09:46:38.622 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.622 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:38.622 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 125745, 2024-05-17T09:46:38.622 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.020315246, 2024-05-17T09:46:38.622 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000161 2024-05-17T09:46:38.622 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1098410, 2024-05-17T09:46:38.623 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1143343, 2024-05-17T09:46:38.624 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 324454550, 2024-05-17T09:46:38.624 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 727966810, 2024-05-17T09:46:38.624 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 13, 2024-05-17T09:46:38.624 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2024-05-17T09:46:38.624 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 49.981125801, 2024-05-17T09:46:38.625 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 20.205487973, 2024-05-17T09:46:38.625 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 35.746835270, 2024-05-17T09:46:38.625 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 1.071038780, 2024-05-17T09:46:38.625 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:38.625 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1143343, 2024-05-17T09:46:38.625 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 26.968670642, 2024-05-17T09:46:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000023587 2024-05-17T09:46:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2147307, 2024-05-17T09:46:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.290854204, 2024-05-17T09:46:38.626 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000135 2024-05-17T09:46:38.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.627 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 774576, 2024-05-17T09:46:38.627 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 360416 2024-05-17T09:46:38.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.627 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:38.628 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 136409, 2024-05-17T09:46:38.628 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 129926, 2024-05-17T09:46:38.628 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 33535725, 2024-05-17T09:46:38.628 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 71790571, 2024-05-17T09:46:38.628 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 12, 2024-05-17T09:46:38.628 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-05-17T09:46:38.629 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 7.818215681, 2024-05-17T09:46:38.629 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 3.050933579, 2024-05-17T09:46:38.629 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 5.159638492, 2024-05-17T09:46:38.629 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.646814232, 2024-05-17T09:46:38.629 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:38.630 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 129924, 2024-05-17T09:46:38.630 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 9.171824791, 2024-05-17T09:46:38.630 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000070593 2024-05-17T09:46:38.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.630 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:38.630 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 230237, 2024-05-17T09:46:38.631 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.033562017, 2024-05-17T09:46:38.631 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000145 2024-05-17T09:46:38.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.631 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T09:46:38.631 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 861040 2024-05-17T09:46:38.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-17T09:46:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-17T09:46:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-17T09:46:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-17T09:46:38.632 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-17T09:46:38.633 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-17T09:46:38.633 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 80126, 2024-05-17T09:46:38.633 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 62.447223288, 2024-05-17T09:46:38.633 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000779362 2024-05-17T09:46:38.633 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.634 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.634 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-17T09:46:38.634 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-17T09:46:38.634 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-17T09:46:38.634 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 15018, 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.979174729, 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000131786 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 1009249, 2024-05-17T09:46:38.635 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 983860, 2024-05-17T09:46:38.636 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-17T09:46:38.636 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 983860, 2024-05-17T09:46:38.636 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1046.296790761, 2024-05-17T09:46:38.636 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001063461 2024-05-17T09:46:38.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.637 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-17T09:46:38.637 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 25389, 2024-05-17T09:46:38.637 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 5, 2024-05-17T09:46:38.637 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 2, 2024-05-17T09:46:38.637 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 2660, 2024-05-17T09:46:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-05-17T09:46:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-17T09:46:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 154, 2024-05-17T09:46:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 11, 2024-05-17T09:46:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 132, 2024-05-17T09:46:38.638 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 11, 2024-05-17T09:46:38.639 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 137, 2024-05-17T09:46:38.639 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 1271, 2024-05-17T09:46:38.639 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 2, 2024-05-17T09:46:38.639 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 2, 2024-05-17T09:46:38.639 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 4, 2024-05-17T09:46:38.639 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 1961715, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 1950099, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 10196, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 41, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 6, 2024-05-17T09:46:38.640 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 278, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 31, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 960, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 31, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 151, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 170, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 4, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 234, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2024-05-17T09:46:38.641 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 170, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 81, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 7, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 74, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 89, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 32, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 46763, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 32679, 2024-05-17T09:46:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 59533, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 9863, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 606, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 28150 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-17T09:46:38.643 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 18, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 9, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 6, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 44, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 94, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 5, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 7715, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 7710, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-17T09:46:38.644 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 25, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 2980, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 7, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 34, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-05-17T09:46:38.645 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 76879, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 76691, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 76691, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 188, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 510, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 183, 2024-05-17T09:46:38.646 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 181, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 181, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 1, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 1, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 167736508, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 167933013, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 41084, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 166.854839371, 2024-05-17T09:46:38.647 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004061309 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 31, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 8471, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 8440, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 30, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 166, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 136, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 148, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 13350, 2024-05-17T09:46:38.648 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 13202, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 2, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 8395, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 8393, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 145232, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 231824 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 1047155, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 25059, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 1009249, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 716, 2024-05-17T09:46:38.649 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 50304, 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 8198, 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 128.824094064, 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.015714088 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 924315, 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 497.185330667, 2024-05-17T09:46:38.650 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000537895 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-17T09:46:38.651 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 26188, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 218.341891516, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.008337478 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.652 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.653 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-17T09:46:38.654 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.654 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 119, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.417241938, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003506234 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.655 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.657 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-17T09:46:38.657 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4939, 2024-05-17T09:46:38.657 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 5.779590907, 2024-05-17T09:46:38.657 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001170194 2024-05-17T09:46:38.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.658 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-17T09:46:38.658 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 10689, 2024-05-17T09:46:38.658 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 53.082128340, 2024-05-17T09:46:38.658 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004966051 2024-05-17T09:46:38.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1453, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 17.511186248, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.012051745 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.659 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 64, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.631186843, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.009862294 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.660 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-17T09:46:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-17T09:46:38.661 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 5, 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 52.959669824, 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 10.591933964 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 220, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.175718357, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000798719 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 8, 2024-05-17T09:46:38.663 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.023559675, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002944959 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 7662, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 71.365192382, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.009314172 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-17T09:46:38.664 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 3, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 2, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-05-17T09:46:38.665 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 62460, 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 62460, 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 1630, 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 240, 2024-05-17T09:46:38.666 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 240, 2024-05-17T09:46:38.667 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:38.667 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:38.667 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:38.667 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:38.667 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:38.668 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:38.668 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:38.668 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:38.668 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:38.668 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:38.668 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:38.669 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:38.670 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:38.671 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-17T09:46:38.671 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:38.671 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:38.671 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:38.671 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:38.672 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 302631, 2024-05-17T09:46:38.672 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 98, 2024-05-17T09:46:38.672 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:38.672 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:38.672 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-17T09:46:38.672 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-17T09:46:38.673 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:38.673 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:38.673 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:38.673 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:38.673 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 8984, 2024-05-17T09:46:38.674 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 172, 2024-05-17T09:46:38.674 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:38.674 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:38.674 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:38.674 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 226136, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2278, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 69530, 2024-05-17T09:46:38.675 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 174106323, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 37, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 69530, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 69530, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 768.962674318, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.011059437 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 69530, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 85132 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 69530, 2024-05-17T09:46:38.676 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 1304, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 68226, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 2529, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 5575, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 1120, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 42197, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 1012, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-05-17T09:46:38.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 15961, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 184, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 8535, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-17T09:46:38.678 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-17T09:46:38.679 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 89, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 11, 2024-05-17T09:46:38.680 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 12, 2024-05-17T09:46:38.681 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2024-05-17T09:46:38.681 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-05-17T09:46:38.681 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-17T09:46:38.681 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 6660, 2024-05-17T09:46:38.681 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 14, 2024-05-17T09:46:38.681 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 1345 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 2284 2024-05-17T09:46:38.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 17, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 15, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 15372, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 7710, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.683 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 1319431, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 259537111, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 1319431, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1319431, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 259537111, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.684 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:38.685 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:38.686 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:38.686 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 69530, 2024-05-17T09:46:38.686 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 174946241, 2024-05-17T09:46:38.686 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 69530, 2024-05-17T09:46:38.686 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 174946241, 2024-05-17T09:46:38.687 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:38.687 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.687 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.687 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.687 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.687 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.688 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-17T09:46:38.688 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:38.688 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-17T09:46:38.688 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:38.688 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:38.689 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:38.689 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:38.689 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:38.689 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 69530, 2024-05-17T09:46:38.689 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 69530, 2024-05-17T09:46:38.689 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 69530, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 69530, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-17T09:46:38.690 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 7710, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 840006, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 7710, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1113, 2024-05-17T09:46:38.691 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 840006, 2024-05-17T09:46:38.692 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:38.692 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.692 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.692 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.692 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.692 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x55ca2be5eb60": { 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 76879, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 163738709, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 76879, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 41084, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 163738709, 2024-05-17T09:46:38.693 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:38.694 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:38.694 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:38.694 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:38.694 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.694 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:38.694 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-17T09:46:38.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:38 smithi069 ceph-mon[35081]: pgmap v1244: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:46:38.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:38 smithi019 ceph-mon[31620]: pgmap v1244: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:46:39.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:38 smithi079 ceph-mon[35502]: pgmap v1244: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:46:39.390 INFO:tasks.check_counter:Daemon mds.b mds.exported=31 2024-05-17T09:46:39.390 INFO:tasks.check_counter:Daemon mds.b mds.imported=31 2024-05-17T09:46:39.390 DEBUG:tasks.check_counter:Getting stats from e 2024-05-17T09:46:39.390 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-17T09:46:39.639 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:46:40.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:40 smithi069 ceph-mon[35081]: pgmap v1245: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-17T09:46:40.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:40 smithi019 ceph-mon[31620]: pgmap v1245: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-17T09:46:41.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:40 smithi079 ceph-mon[35502]: pgmap v1245: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1355, 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1357, 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 50680513, 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 430954, 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 8, 2024-05-17T09:46:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.139803448, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.037277147, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.060057518, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.032006093, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1355, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.041867124, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000030898 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2156, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2545, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 71505836, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1519569, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 7, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.227194958, 2024-05-17T09:46:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.059791209, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.090326732, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.059640460, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2543, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.076763772, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000030186 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 834592 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3338, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 3316, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 35101752, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1207366, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.340824236, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.106025537, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.112616838, 2024-05-17T09:46:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.097750271, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3316, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.117041079, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000035295 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 689824, 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 317920 2024-05-17T09:46:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 10700, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.263543064, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000024630 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-17T09:46:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 0, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 13133, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 13133, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-05-17T09:46:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 13133, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 1, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-17T09:46:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T09:46:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-05-17T09:46:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-17T09:46:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-17T09:46:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 91718, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-05-17T09:46:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 117, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 9, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 159042825, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 159042226, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 91718 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 13, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 13184, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 13171, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 11, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 88, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 77, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 13133, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 13803, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 670, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2024-05-17T09:46:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 137624, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 215440 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 0, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 0, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 0, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-17T09:46:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-17T09:46:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-17T09:46:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 2, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 1, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-05-17T09:46:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 2898, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 226424, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 79, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 8984, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 172, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 9289224, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 52660, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 6306, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 21, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 3, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 6306, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6306, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 4.130906658, 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000655075 2024-05-17T09:46:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6306, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 6307 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 6306, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 6306, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 0, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 3531, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 0, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 2773, 2024-05-17T09:46:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 1, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-17T09:46:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-17T09:46:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 89, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 2, 2024-05-17T09:46:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 9, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 10, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 1, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 0, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 2, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 0 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 0, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 6845, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 156641465, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 6845, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6845, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 156641465, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 6306, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 154849775, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6306, 2024-05-17T09:46:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 154849775, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 6306, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 6306, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6306, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 6306, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x55eff51d69c0": { 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:41.597 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-17T09:46:42.512 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-17T09:46:42.512 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-17T09:46:42.512 DEBUG:tasks.check_counter:Getting stats from h 2024-05-17T09:46:42.512 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-17T09:46:42.755 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:46:42.877 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:42 smithi069 ceph-mon[35081]: pgmap v1246: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T09:46:42.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:42 smithi019 ceph-mon[31620]: pgmap v1246: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T09:46:43.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:42 smithi079 ceph-mon[35502]: pgmap v1246: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-17T09:46:44.455 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 515626, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 588493, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 156788643, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 390819142, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 31, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 27.091110060, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 11.205612951, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 19.129040760, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.410604582, 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:44.456 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 588491, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 12.948728522, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000022003 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1047484, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.167423795, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000159 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 703424, 2024-05-17T09:46:44.457 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 342320 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 91831, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 81419, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 24360826, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 93727889, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 9, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 6.039686808, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 2.602577679, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 3.242429365, 2024-05-17T09:46:44.458 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.743905165, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 81419, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 2.743049723, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000033690 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 124773, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.021241367, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000170 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 834304 2024-05-17T09:46:44.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 36043, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 31249, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 9145177, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 67391105, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 8, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 2.459170383, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.828124639, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.906502000, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.749804396, 2024-05-17T09:46:44.460 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 31247, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.688792993, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000054046 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 23281, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.004450782, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000191 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:44.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 47652, 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 51.512296969, 2024-05-17T09:46:44.462 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001081010 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 25488, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 3.027358230, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000118775 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 448821, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 448431, 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-05-17T09:46:44.463 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 448431, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 994.117326714, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002216879 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 390, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 11, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 15, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 2, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 2, 2024-05-17T09:46:44.464 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 11, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 11, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 140, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 4, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 874013, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 843776, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 388, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 36, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 6, 2024-05-17T09:46:44.465 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 17, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 39, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 17, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 404, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 12, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 4, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 234, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 12, 2024-05-17T09:46:44.466 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 12, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 10, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 31402, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 50342, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 53286, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 17052, 2024-05-17T09:46:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 842, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 47920 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 6, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 39, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 24, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 61, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 19, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 13123, 2024-05-17T09:46:44.468 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 13123, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 25, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 4325, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 17, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-05-17T09:46:44.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 91718, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 91718, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 117, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 117, 2024-05-17T09:46:44.470 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 9, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 159042825, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 24659, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 116.561045635, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004726916 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.471 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 20, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 13207, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 13187, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 19, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 104, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 85, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 15, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 16312, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 16297, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 13147, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 13147, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 146584, 2024-05-17T09:46:44.472 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223632 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 488699, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 51, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 448821, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 1025, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 9096, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 12816, 2024-05-17T09:46:44.473 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 384.827956984, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.030027150 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 386243, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 191.829323847, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000496654 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.474 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6679, 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 10.985829162, 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001644831 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.475 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.477 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.478 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 8650, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 8.287130769, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000958049 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 17521, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 59.492381841, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003395490 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-05-17T09:46:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2948, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 21.559804208, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.007313366 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.481 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 30, 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.209248074, 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.006974935 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.482 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 11, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 51.943574273, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 4.722143115 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 440, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.270482636, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000614733 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.484 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 13093, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 264.711594920, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.020217795 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-05-17T09:46:44.485 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 2, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 1, 2024-05-17T09:46:44.486 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 48701, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 48701, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 1631, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:44.487 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:44.488 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:44.489 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 279751, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 93, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 8984, 2024-05-17T09:46:44.490 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 172, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 81344, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 362, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-05-17T09:46:44.491 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 55979, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 157310874, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 33, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 55979, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 55979, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 294.715390845, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.005264749 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-05-17T09:46:44.492 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 55979, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 57686 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 55979, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 1560, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 54419, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 1548, 2024-05-17T09:46:44.493 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 26207, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 925, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 26264, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 12, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-05-17T09:46:44.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-05-17T09:46:44.495 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 89, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 2, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 12, 2024-05-17T09:46:44.496 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 1827, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 873 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-05-17T09:46:44.497 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 3309 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 17, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 16, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 26216, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 13123, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:44.498 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 643496, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 132347696, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 643496, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 643496, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 132347696, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 55979, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 158741041, 2024-05-17T09:46:44.500 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 55979, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 158741041, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-05-17T09:46:44.501 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 55979, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 55979, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 55979, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 55979, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.502 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 13123, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1430167, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 13123, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1548, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1430167, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.503 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x5587bdaf49c0": { 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 91718, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 154848521, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 91718, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:44.504 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 24659, 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 154848521, 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:44.505 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-17T09:46:44.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:44 smithi069 ceph-mon[35081]: pgmap v1247: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:46:44.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:44 smithi019 ceph-mon[31620]: pgmap v1247: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:46:45.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:44 smithi079 ceph-mon[35502]: pgmap v1247: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-17T09:46:45.387 INFO:tasks.check_counter:Daemon mds.h mds.exported=17 2024-05-17T09:46:45.387 INFO:tasks.check_counter:Daemon mds.h mds.imported=17 2024-05-17T09:46:45.387 DEBUG:tasks.check_counter:Getting stats from k 2024-05-17T09:46:45.387 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-17T09:46:45.630 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T09:46:46.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:46 smithi069 ceph-mon[35081]: pgmap v1248: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:46:46.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:46 smithi019 ceph-mon[31620]: pgmap v1248: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:46:47.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:46 smithi079 ceph-mon[35502]: pgmap v1248: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:46:47.566 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001085644, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000233120, 2024-05-17T09:46:47.567 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 395, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 305760, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2024-05-17T09:46:47.569 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.021446719, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.016059931, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001781291, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 393, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.020735909, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000052763 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.570 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 305760 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 522, 2024-05-17T09:46:47.571 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 517, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 602064, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 319936, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.069593991, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.027574898, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.026769237, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.011386436, 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:47.572 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 517, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.021938027, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000042433 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 602064, 2024-05-17T09:46:47.573 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 319936 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:47.574 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:47.575 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:47.576 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:47.577 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 147937, 2024-05-17T09:46:47.578 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 28, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:47.579 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-05-17T09:46:47.580 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-05-17T09:46:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 521, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 525876, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 521, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 521, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 525876, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-05-17T09:46:47.583 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-05-17T09:46:48.250 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-17T09:46:48.250 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-17T09:46:48.250 DEBUG:tasks.check_counter:Getting stats from c 2024-05-17T09:46:48.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-17T09:46:48.549 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:46:48.880 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:48 smithi079 ceph-mon[35502]: pgmap v1249: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:46:48.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:48 smithi069 ceph-mon[35081]: pgmap v1249: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:46:48.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:48 smithi019 ceph-mon[31620]: pgmap v1249: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-17T09:46:50.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:50 smithi079 ceph-mon[35502]: pgmap v1250: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 3891, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 3891, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 68248559, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1236524, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.362850709, 2024-05-17T09:46:50.772 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.096790554, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.126836202, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.110100276, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3890, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.126117287, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000032420 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.773 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2217, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2197, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 59993203, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 851942, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 6, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 5, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.218174155, 2024-05-17T09:46:50.774 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.066016519, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.088797383, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.050046701, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2196, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.070867748, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000032271 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.775 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 687344, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 318272 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 1157, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1546, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 38038467, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1291501, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 5, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.131791536, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.045394142, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.044039038, 2024-05-17T09:46:50.776 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.031076003, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1544, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.053191570, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000034450 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-17T09:46:50.777 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 924528 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 11474, 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.382482037, 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000033334 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.779 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 7775, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 5341, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 2434, 2024-05-17T09:46:50.780 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 146, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 2, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-05-17T09:46:50.781 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T09:46:50.782 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T09:46:50.783 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-17T09:46:50.784 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-05-17T09:46:50.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 76879, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-17T09:46:50.786 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 3, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 122, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 167736508, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 167933013, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 167930516, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 76879 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 26, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 8378, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 8352, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 22, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 131, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 109, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 7775, 2024-05-17T09:46:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 9133, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 1358, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 129724, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 215440 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-05-17T09:46:50.789 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.790 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-17T09:46:50.791 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.793 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.794 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-17T09:46:50.794 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.794 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.794 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.795 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.796 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.798 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.799 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.801 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 3, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 2, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2902, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:50.802 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:50.803 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:50.804 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 241807, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 84, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:50.805 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8984, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 172, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 5560352, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 31796, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-17T09:46:50.806 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 6719, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 26, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 4, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 6719, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6719, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 4.205905936, 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000625972 2024-05-17T09:46:50.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6719, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 6720 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 6719, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 6719, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 3644, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 3072, 2024-05-17T09:46:50.808 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 2, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-17T09:46:50.809 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-17T09:46:50.810 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 89, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 11, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 11, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-05-17T09:46:50.811 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.812 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 7261, 2024-05-17T09:46:50.813 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 165596012, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 7261, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 7261, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 165596012, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-17T09:46:50.814 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 6719, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 163738719, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 6719, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 163738719, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 6719, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 6719, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 6719, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 6719, 2024-05-17T09:46:50.816 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:50.817 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x558eecf209c0": { 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-17T09:46:50.818 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:50.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:50.820 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-17T09:46:50.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:50 smithi069 ceph-mon[35081]: pgmap v1250: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:46:50.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:50 smithi019 ceph-mon[31620]: pgmap v1250: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-17T09:46:51.610 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-17T09:46:51.611 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-17T09:46:51.611 DEBUG:tasks.check_counter:Getting stats from f 2024-05-17T09:46:51.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-17T09:46:51.857 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:46:52.679 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:52 smithi069 ceph-mon[35081]: pgmap v1251: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:46:52.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:52 smithi019 ceph-mon[31620]: pgmap v1251: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:46:53.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:52 smithi079 ceph-mon[35502]: pgmap v1251: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 3926, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 3926, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 64063523, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1247651, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.364805799, 2024-05-17T09:46:53.938 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.103034800, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.120735744, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.112897458, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3925, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.128505885, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000032740 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-17T09:46:53.939 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 1492, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1882, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 68191523, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1399884, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 9, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 6, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.170439712, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.045093861, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.072707110, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.039638257, 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:53.940 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1880, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.058053937, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000030879 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 926320 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 1621, 2024-05-17T09:46:53.941 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1601, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 63879210, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 662828, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 6, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.185646666, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.051420779, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.087630934, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.036814710, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1600, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.052613299, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000032883 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.942 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 687456, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 318752 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-17T09:46:53.943 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 11074, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.267447600, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000024150 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.944 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 16968, 2024-05-17T09:46:53.945 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 1968, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 15000, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 14156, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 1, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-17T09:46:53.946 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-17T09:46:53.947 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-05-17T09:46:53.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-17T09:46:53.949 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-17T09:46:53.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 125040, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 132, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 12, 2024-05-17T09:46:53.951 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 31271194, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 197837767, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 197836629, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 125040 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 14, 2024-05-17T09:46:53.952 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 17042, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 17028, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 12, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 76, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 64, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 16968, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 17624, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 656, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 137908, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 215440 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-17T09:46:53.953 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.958 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-17T09:46:53.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.958 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-17T09:46:53.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.960 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.961 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.962 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.963 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-17T09:46:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-17T09:46:53.965 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2910, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:53.966 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:53.967 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-17T09:46:53.968 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 241807, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 84, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:53.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8984, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 172, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 11992658, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 68101, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-17T09:46:53.970 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 6493, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 21, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 3, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 6493, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6493, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 4.654895856, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000716909 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6493, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 6494 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.971 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 6493, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 6493, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 3552, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 2939, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-17T09:46:53.972 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 1, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-17T09:46:53.973 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-17T09:46:53.974 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 89, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 11, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 12, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 1, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.975 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-05-17T09:46:53.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 7035, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 195470373, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 7035, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 7035, 2024-05-17T09:46:53.977 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 195470373, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:53.978 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 6493, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 193646458, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 6493, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 193646458, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:53.979 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 6493, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 6493, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 6493, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 6493, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:53.981 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x555e0633e9c0": { 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:53.982 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:53.983 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-17T09:46:54.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:54 smithi079 ceph-mon[35502]: pgmap v1252: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:46:54.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:54 smithi069 ceph-mon[35081]: pgmap v1252: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:46:54.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:54 smithi019 ceph-mon[31620]: pgmap v1252: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-17T09:46:54.981 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-17T09:46:54.981 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-17T09:46:54.981 DEBUG:tasks.check_counter:Getting stats from i 2024-05-17T09:46:54.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-17T09:46:55.228 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:46:56.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:56 smithi079 ceph-mon[35502]: pgmap v1253: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T09:46:56.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:56 smithi069 ceph-mon[35081]: pgmap v1253: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T09:46:56.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:56 smithi019 ceph-mon[31620]: pgmap v1253: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 31390, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 31394, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 7934729, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 74486172, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 27, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 6, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 2.759578433, 2024-05-17T09:46:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 1.028961961, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.562374250, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.992217511, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 31389, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.683981136, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000053648 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 16, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000002578, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000161 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 137254, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 149302, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 34372938, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 84577220, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 9, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 7.341481559, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 3.002425960, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 4.446873540, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.554012141, 2024-05-17T09:46:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 149304, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 3.168818715, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000021223 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 249563, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.033596152, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000134 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 839504 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 576904, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 663421, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 175484906, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 503806584, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 12, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 6, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 30.573532614, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 12.178472600, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 21.476859389, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 1.248263525, 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 663420, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 12.061967781, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000018181 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1139291, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.161939376, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000142 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 705968, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 326656 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-17T09:46:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 77873, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 50.627907020, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000650134 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-17T09:46:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 32213, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 2.634828709, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000081793 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 479090, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 479082, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 479082, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 939.869267873, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001961812 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 8, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 13, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 28, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 9, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 8, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 11, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 15, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 90, 2024-05-17T09:46:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 4, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 4, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 4, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 948442, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 882097, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 6, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 37, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 144, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 14, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 112, 2024-05-17T09:46:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 14, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 556, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 68, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 68, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T09:46:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 68, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 1134, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 35488, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 57157, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 62492, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 21143, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 1010, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 60875 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-17T09:46:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 6, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 43, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 31, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 52, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 44, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 1, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 16949, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 16948, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-17T09:46:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 25, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 4147, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 14, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 125040, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 27671, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 24581, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 100459, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 3090, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 159, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 31, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 27, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 132, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 9, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 4, 2024-05-17T09:46:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 31271194, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 197837767, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 0, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 39234, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 129.228313133, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.003293783 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 0 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 24, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 17005, 2024-05-17T09:46:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 16981, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 23, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 80, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 57, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 28, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 23967, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 23939, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 4, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 17011, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 17007, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 155584, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 223632 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 552518, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 474, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 479090, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 1231, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 17331, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 16664, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 342.910237494, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.020577906 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 392468, 2024-05-17T09:46:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 151.956176320, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000387181 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 15037, 2024-05-17T09:46:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 10.565043325, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000702603 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-17T09:46:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 11562, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 8.052099880, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000696427 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 22420, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 62.039673131, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002767157 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3434, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 68.637744714, 2024-05-17T09:46:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.019987695 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-17T09:46:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 30, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.205282644, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.006842754 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 8, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 76.374711598, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 9.546838949 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 540, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.224373934, 2024-05-17T09:46:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000415507 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-17T09:46:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 16919, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 218.903924833, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.012938348 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-17T09:46:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 49272, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 49272, 2024-05-17T09:46:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 1644, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 287712, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 95, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8984, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 172, 2024-05-17T09:46:57.119 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 107255, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 724, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-17T09:46:57.120 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 79343, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 196702446, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 33, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 79343, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 79343, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 333.152574956, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.004198890 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 79343, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 81420 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 79343, 2024-05-17T09:46:57.121 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 2025, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 77318, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 2, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 21, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 1957, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 41191, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 1099, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 33931, 2024-05-17T09:46:57.122 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 68, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 42, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-17T09:46:57.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-17T09:46:57.124 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 89, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 12, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 12, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 2096, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 0, 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 1013 2024-05-17T09:46:57.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 1, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 3307 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 16, 2024-05-17T09:46:57.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 16, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 33867, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 16948, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 745544, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 150660968, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 745544, 2024-05-17T09:46:57.127 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 745544, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 150660968, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:57.128 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 79343, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 198549546, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 79343, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 198549546, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 79343, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 79343, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 79343, 2024-05-17T09:46:57.130 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 79343, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 16948, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1847100, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 16948, 2024-05-17T09:46:57.131 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 1957, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1847100, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x55eaf41aea90": { 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:57.132 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 125040, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 193643463, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 125040, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 39234, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 193643463, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:57.133 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-17T09:46:57.880 INFO:tasks.check_counter:Daemon mds.i mds.exported=14 2024-05-17T09:46:57.880 INFO:tasks.check_counter:Daemon mds.i mds.imported=14 2024-05-17T09:46:57.881 DEBUG:tasks.check_counter:Getting stats from l 2024-05-17T09:46:57.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-17T09:46:58.122 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T09:46:58.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:46:58 smithi079 ceph-mon[35502]: pgmap v1254: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-17T09:46:58.930 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:46:58 smithi069 ceph-mon[35081]: pgmap v1254: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-17T09:46:58.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:46:58 smithi019 ceph-mon[31620]: pgmap v1254: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-17T09:46:59.974 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-17T09:46:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T09:46:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 0, 2024-05-17T09:46:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2, 2024-05-17T09:46:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 0, 2024-05-17T09:46:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 0, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 0, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.001167234, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.000249333, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 7, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 400, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 3392, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 306736, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-05-17T09:46:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.021668688, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.015976293, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.001850026, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 399, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.022281328, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000055842 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 306736 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 521, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 518, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 599968, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 321408, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 2, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-05-17T09:46:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.072313337, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.028266252, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.027557349, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.012262287, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 517, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.021981365, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000042517 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 599968, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 321408 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 0, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T09:46:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T09:46:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T09:46:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 148704, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 29, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-17T09:46:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 928, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 2, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T09:46:59.984 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 524, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 526266, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 524, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 524, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 526266, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-17T09:46:59.985 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-17T09:46:59.986 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-17T09:46:59.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:59.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-17T09:46:59.986 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-17T09:47:00.705 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:00 smithi079 ceph-mon[35502]: pgmap v1255: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-17T09:47:00.733 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-17T09:47:00.733 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-17T09:47:00.733 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-17T09:47:00.743 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-17T09:47:00.743 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:47:00.744 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T09:47:00.777 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:47:00.777 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T09:47:00.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T09:47:00.906 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:00 smithi019 ceph-mon[31620]: pgmap v1255: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-17T09:47:00.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:00 smithi069 ceph-mon[35081]: pgmap v1255: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-17T09:47:01.143 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:47:02.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:02 smithi019 ceph-mon[31620]: pgmap v1256: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:47:02.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 -- 172.21.15.19:0/3502290150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c109870 msgr2=0x7fb07c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 --2- 172.21.15.19:0/3502290150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c109870 0x7fb07c1111d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b0d0 tx=0x7fb07400b3e0 comp rx=0 tx=0).stop 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 -- 172.21.15.19:0/3502290150 shutdown_connections 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 --2- 172.21.15.19:0/3502290150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c104640 0x7fb07c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 --2- 172.21.15.19:0/3502290150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c109870 0x7fb07c1111d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 --2- 172.21.15.19:0/3502290150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c108f20 0x7fb07c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 -- 172.21.15.19:0/3502290150 >> 172.21.15.19:0/3502290150 conn(0x7fb07c079ef0 msgr2=0x7fb07c07a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 -- 172.21.15.19:0/3502290150 shutdown_connections 2024-05-17T09:47:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.903+0000 7fb08401d700 1 -- 172.21.15.19:0/3502290150 wait complete. 2024-05-17T09:47:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 Processor -- start 2024-05-17T09:47:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 -- start start 2024-05-17T09:47:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 0x7fb07c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c108f20 0x7fb07c1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c109870 0x7fb07c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c114350 con 0x7fb07c104640 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1141d0 con 0x7fb07c109870 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.904+0000 7fb08401d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1144d0 con 0x7fb07c108f20 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 0x7fb07c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 0x7fb07c1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47874/0 (socket says 172.21.15.19:47874) 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 -- 172.21.15.19:0/4232501598 learned_addr learned my addr 172.21.15.19:0/4232501598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb0825ba700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c109870 0x7fb07c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb0815b8700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c108f20 0x7fb07c1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 -- 172.21.15.19:0/4232501598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c108f20 msgr2=0x7fb07c1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c108f20 0x7fb07c1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 -- 172.21.15.19:0/4232501598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c109870 msgr2=0x7fb07c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c109870 0x7fb07c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 -- 172.21.15.19:0/4232501598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c19b260 con 0x7fb07c104640 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb0815b8700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c108f20 0x7fb07c1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb0825ba700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c109870 0x7fb07c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb081db9700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 0x7fb07c1a09a0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb07800d950 tx=0x7fb07800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078016070 con 0x7fb07c104640 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.905+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c19b550 con 0x7fb07c104640 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.906+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c1a7750 con 0x7fb07c104640 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.906+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb078009dd0 con 0x7fb07c104640 2024-05-17T09:47:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.906+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07800f3c0 con 0x7fb07c104640 2024-05-17T09:47:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.907+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb060005330 con 0x7fb07c104640 2024-05-17T09:47:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.908+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb078019070 con 0x7fb07c104640 2024-05-17T09:47:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.909+0000 7fb072ffd700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0680b2900 0x7fb0680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.909+0000 7fb0815b8700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0680b2900 0x7fb0680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.909+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb0780e2460 con 0x7fb07c104640 2024-05-17T09:47:02.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.910+0000 7fb0815b8700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0680b2900 0x7fb0680b4e10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005e10 tx=0x7fb06c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:47:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:02.914+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0780a9dd0 con 0x7fb07c104640 2024-05-17T09:47:02.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:02 smithi069 ceph-mon[35081]: pgmap v1256: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:47:03.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:02 smithi079 ceph-mon[35502]: pgmap v1256: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-17T09:47:03.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.142+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb060005620 con 0x7fb07c104640 2024-05-17T09:47:03.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.142+0000 7fb072ffd700 1 -- 172.21.15.19:0/4232501598 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+588 (secure 0 0 0) 0x7fb0780a96d0 con 0x7fb07c104640 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/21803007 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3426431385 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4126868109 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3686497433 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3120865986 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3378222227 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0680b2900 msgr2=0x7fb0680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0680b2900 0x7fb0680b4e10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005e10 tx=0x7fb06c005d60 comp rx=0 tx=0).stop 2024-05-17T09:47:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 msgr2=0x7fb07c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 0x7fb07c1a09a0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb07800d950 tx=0x7fb07800dc60 comp rx=0 tx=0).stop 2024-05-17T09:47:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 shutdown_connections 2024-05-17T09:47:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb07c109870 0x7fb07c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07c104640 0x7fb07c1a09a0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7fb0680b2900 0x7fb0680b4e10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 --2- 172.21.15.19:0/4232501598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb07c108f20 0x7fb07c1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 >> 172.21.15.19:0/4232501598 conn(0x7fb07c079ef0 msgr2=0x7fb07c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 shutdown_connections 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:03.145+0000 7fb08401d700 1 -- 172.21.15.19:0/4232501598 wait complete. 2024-05-17T09:47:03.331 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-17T09:47:03.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:03 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/4232501598' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:47:03.892 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-17T09:47:03.892 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T09:47:03.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:03 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/4232501598' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:47:03.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a shell --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T09:47:04.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:03 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/4232501598' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:47:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T09:47:04.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:04 smithi069 ceph-mon[35081]: pgmap v1257: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-05-17T09:47:04.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:04 smithi019 ceph-mon[31620]: pgmap v1257: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-05-17T09:47:05.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:04 smithi079 ceph-mon[35502]: pgmap v1257: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-05-17T09:47:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 -- 172.21.15.19:0/329983853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff754105470 msgr2=0x7ff754105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 --2- 172.21.15.19:0/329983853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff754105470 0x7ff754105880 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff744009990 tx=0x7ff744009ca0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 -- 172.21.15.19:0/329983853 shutdown_connections 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 --2- 172.21.15.19:0/329983853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff75410e8e0 0x7ff754110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 --2- 172.21.15.19:0/329983853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff754105470 0x7ff754105880 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 --2- 172.21.15.19:0/329983853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff754105dc0 0x7ff75410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 -- 172.21.15.19:0/329983853 >> 172.21.15.19:0/329983853 conn(0x7ff7540fef20 msgr2=0x7ff754101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 -- 172.21.15.19:0/329983853 shutdown_connections 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 -- 172.21.15.19:0/329983853 wait complete. 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.071+0000 7ff75a88c700 1 Processor -- start 2024-05-17T09:47:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 -- start start 2024-05-17T09:47:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff754105470 0x7ff7541a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff754105dc0 0x7ff7541a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 0x7ff75419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff754114300 con 0x7ff75410e8e0 2024-05-17T09:47:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff754114180 con 0x7ff754105dc0 2024-05-17T09:47:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff75a88c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff754114480 con 0x7ff754105470 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff758e29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 0x7ff75419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff758e29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 0x7ff75419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38402/0 (socket says 172.21.15.19:38402) 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff758e29700 1 -- 172.21.15.19:0/1738641767 learned_addr learned my addr 172.21.15.19:0/1738641767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff7537fe700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff754105dc0 0x7ff7541a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.072+0000 7ff753fff700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff754105470 0x7ff7541a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff758e29700 1 -- 172.21.15.19:0/1738641767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff754105470 msgr2=0x7ff7541a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff758e29700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff754105470 0x7ff7541a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff758e29700 1 -- 172.21.15.19:0/1738641767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff754105dc0 msgr2=0x7ff7541a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff758e29700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff754105dc0 0x7ff7541a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff758e29700 1 -- 172.21.15.19:0/1738641767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75419b1e0 con 0x7ff75410e8e0 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff758e29700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 0x7ff75419ab00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff74c007ee0 tx=0x7ff74c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:47:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004bb0 con 0x7ff75410e8e0 2024-05-17T09:47:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75419b4d0 con 0x7ff75410e8e0 2024-05-17T09:47:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff74c004d50 con 0x7ff75410e8e0 2024-05-17T09:47:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0058b0 con 0x7ff75410e8e0 2024-05-17T09:47:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.073+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a7770 con 0x7ff75410e8e0 2024-05-17T09:47:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.074+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7540512b0 con 0x7ff75410e8e0 2024-05-17T09:47:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.076+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff74c01a020 con 0x7ff75410e8e0 2024-05-17T09:47:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.077+0000 7ff7517fa700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff73c0b2880 0x7ff73c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T09:47:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.077+0000 7ff753fff700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff73c0b2880 0x7ff73c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T09:47:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.077+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff74c0e1ba0 con 0x7ff75410e8e0 2024-05-17T09:47:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.077+0000 7ff753fff700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff73c0b2880 0x7ff73c0b4d90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b5c0 tx=0x7ff744005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T09:47:06.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.082+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff74c010070 con 0x7ff75410e8e0 2024-05-17T09:47:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.311+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff75402e5e0 con 0x7ff75410e8e0 2024-05-17T09:47:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.311+0000 7ff7517fa700 1 -- 172.21.15.19:0/1738641767 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+588 (secure 0 0 0) 0x7ff74c0a9510 con 0x7ff75410e8e0 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/21803007 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3426431385 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4126868109 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3686497433 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3120865986 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/747673746 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2577177111 2024-05-18T09:07:46.771034+0000 2024-05-17T09:47:06.311 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3378222227 2024-05-18T09:07:20.018567+0000 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.313+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff73c0b2880 msgr2=0x7ff73c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.313+0000 7ff75a88c700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff73c0b2880 0x7ff73c0b4d90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b5c0 tx=0x7ff744005960 comp rx=0 tx=0).stop 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.313+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 msgr2=0x7ff75419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 0x7ff75419ab00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff74c007ee0 tx=0x7ff74c00d3b0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 shutdown_connections 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:6800/3017687651,v1:172.21.15.19:6801/3017687651] conn(0x7ff73c0b2880 0x7ff73c0b4d90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff754105470 0x7ff7541a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff75410e8e0 0x7ff75419ab00 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 --2- 172.21.15.19:0/1738641767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff754105dc0 0x7ff7541a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T09:47:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 >> 172.21.15.19:0/1738641767 conn(0x7ff7540fef20 msgr2=0x7ff7540ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T09:47:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 shutdown_connections 2024-05-17T09:47:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-17T09:47:06.314+0000 7ff75a88c700 1 -- 172.21.15.19:0/1738641767 wait complete. 2024-05-17T09:47:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-17T09:47:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:06 smithi069 ceph-mon[35081]: pgmap v1258: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T09:47:06.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:06 smithi069 ceph-mon[35081]: from='client.? 172.21.15.19:0/1738641767' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:47:06.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:06 smithi019 ceph-mon[31620]: pgmap v1258: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T09:47:06.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:06 smithi019 ceph-mon[31620]: from='client.? 172.21.15.19:0/1738641767' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:47:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:06 smithi079 ceph-mon[35502]: pgmap v1258: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T09:47:07.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:06 smithi079 ceph-mon[35502]: from='client.? 172.21.15.19:0/1738641767' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T09:47:07.040 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi019.front.sepia.ceph.com... 2024-05-17T09:47:07.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:47:07.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T09:47:07.087 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T09:47:07.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:ceph-fuse[88770]: fuse finished with error 0 and tester_r 0 2024-05-17T09:47:08.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:08 smithi069 ceph-mon[35081]: pgmap v1259: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-17T09:47:08.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:08 smithi019 ceph-mon[31620]: pgmap v1259: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-17T09:47:09.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:08 smithi079 ceph-mon[35502]: pgmap v1259: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-17T09:47:09.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:09 smithi069 ceph-mon[35081]: pgmap v1260: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-17T09:47:09.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:09 smithi019 ceph-mon[31620]: pgmap v1260: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-17T09:47:10.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:09 smithi079 ceph-mon[35502]: pgmap v1260: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-17T09:47:12.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:12 smithi069 ceph-mon[35081]: pgmap v1261: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T09:47:12.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:12 smithi019 ceph-mon[31620]: pgmap v1261: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T09:47:13.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:12 smithi079 ceph-mon[35502]: pgmap v1261: 129 pgs: 129 active+clean; 317 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-17T09:47:13.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T09:47:13.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:47:13.189 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T09:47:13.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:47:13.230 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-17T09:47:13.290 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T09:47:13.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:47:13.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T09:47:13.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T09:47:13.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-17T09:47:13.468 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-17T09:47:13.479 INFO:tasks.cephadm:Teardown begin 2024-05-17T09:47:13.479 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a4b16c58a5d7aa60dee491f6f096b7509e2f1269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_a4b16c58a5d7aa60dee491f6f096b7509e2f1269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-17T09:47:13.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:47:13.527 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:47:13.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:47:13.625 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-17T09:47:13.625 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T09:47:13.652 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T09:47:13.671 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T09:47:13.698 INFO:tasks.cephadm:Stopping all daemons... 2024-05-17T09:47:13.698 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-17T09:47:13.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a 2024-05-17T09:47:14.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:13 smithi019 systemd[1]: Stopping Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:14.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:47:14.014+0000 7f497b7d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:14.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a[31616]: 2024-05-17T09:47:14.014+0000 7f497b7d2700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-17T09:47:14.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:14 smithi019 podman[133146]: 2024-05-17 09:47:14.098769222 +0000 UTC m=+0.193659529 container died 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-17T09:47:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:14 smithi019 podman[133146]: 2024-05-17 09:47:14.632725875 +0000 UTC m=+0.727616223 container cleanup 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD) 2024-05-17T09:47:14.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:14 smithi019 bash[133146]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a 2024-05-17T09:47:15.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-17T09:47:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:14 smithi019 podman[133159]: 2024-05-17 09:47:14.876171903 +0000 UTC m=+0.773299226 container remove 05794409a601c86e219c01cb8e7487db26cea4d171dd6038590de768689b5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:47:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:15 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-17T09:47:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:15 smithi019 systemd[1]: Stopped Ceph mon.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:47:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 17 09:47:15 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.a.service: Consumed 40.936s CPU time. 2024-05-17T09:47:15.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:15.180 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-17T09:47:15.180 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-17T09:47:15.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.b 2024-05-17T09:47:15.564 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:15 smithi069 systemd[1]: Stopping Ceph mon.b for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-b[35077]: 2024-05-17T09:47:15.564+0000 7faf85876700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-b[35077]: 2024-05-17T09:47:15.564+0000 7faf85876700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-17T09:47:15.929 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:15 smithi069 podman[75681]: 2024-05-17 09:47:15.69806707 +0000 UTC m=+0.281611514 container died c59ff213553a90f2b8a83b03979a5cf493021b0432333eba4d416931d608512a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-b, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T09:47:16.370 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:16 smithi069 podman[75681]: 2024-05-17 09:47:16.087437862 +0000 UTC m=+0.670982306 container cleanup c59ff213553a90f2b8a83b03979a5cf493021b0432333eba4d416931d608512a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True) 2024-05-17T09:47:16.371 INFO:journalctl@ceph.mon.b.smithi069.stdout:May 17 09:47:16 smithi069 bash[75681]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-b 2024-05-17T09:47:16.512 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-17T09:47:16.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:16.558 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-17T09:47:16.558 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-17T09:47:16.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.c 2024-05-17T09:47:16.968 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:16 smithi079 systemd[1]: Stopping Ceph mon.c for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:17.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-c[35485]: 2024-05-17T09:47:16.968+0000 7f609eeb8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:17.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-c[35485]: 2024-05-17T09:47:16.968+0000 7f609eeb8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-17T09:47:17.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:17 smithi079 podman[77997]: 2024-05-17 09:47:17.132196811 +0000 UTC m=+0.299851352 container died feca069ebc75a314feb7e321a8c8258304c605c0d33b7d01a325671e6edd81e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:17.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:17 smithi079 podman[77997]: 2024-05-17 09:47:17.528594405 +0000 UTC m=+0.696248947 container cleanup feca069ebc75a314feb7e321a8c8258304c605c0d33b7d01a325671e6edd81e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:47:17.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 17 09:47:17 smithi079 bash[77997]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mon-c 2024-05-17T09:47:17.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-17T09:47:17.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:17.941 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-17T09:47:17.941 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-17T09:47:17.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x 2024-05-17T09:47:18.308 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:47:17 smithi019 systemd[1]: Stopping Ceph mgr.x for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:18.596 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:47:18 smithi019 podman[133267]: 2024-05-17 09:47:18.310105938 +0000 UTC m=+0.158408020 container died 02efc6d99c29d95596618fa1b8efc5bf8c3e7cc973370a854ce6894cb96b9a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True) 2024-05-17T09:47:18.941 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:47:18 smithi019 podman[133267]: 2024-05-17 09:47:18.831941779 +0000 UTC m=+0.680243857 container cleanup 02efc6d99c29d95596618fa1b8efc5bf8c3e7cc973370a854ce6894cb96b9a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3790c0feaaca36c33bb1b31470733325d7d91e6a, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T09:47:18.941 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 17 09:47:18 smithi019 bash[133267]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-x 2024-05-17T09:47:19.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-17T09:47:19.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:19.342 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-17T09:47:19.342 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-17T09:47:19.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.y 2024-05-17T09:47:19.679 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:47:19 smithi069 systemd[1]: Stopping Ceph mgr.y for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:20.010 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:47:19 smithi069 podman[75803]: 2024-05-17 09:47:19.69833076 +0000 UTC m=+0.162163717 container died 843efbb5ae808383f8d27c3da12c9bf64bd81956a04b308f4a706ce3817939f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-17T09:47:20.326 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:47:20 smithi069 podman[75803]: 2024-05-17 09:47:20.118448708 +0000 UTC m=+0.582281667 container cleanup 843efbb5ae808383f8d27c3da12c9bf64bd81956a04b308f4a706ce3817939f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-05-17T09:47:20.326 INFO:journalctl@ceph.mgr.y.smithi069.stdout:May 17 09:47:20 smithi069 bash[75803]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-y 2024-05-17T09:47:20.542 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-17T09:47:20.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:20.576 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-17T09:47:20.577 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-17T09:47:20.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.z 2024-05-17T09:47:20.938 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:47:20 smithi079 systemd[1]: Stopping Ceph mgr.z for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:21.251 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:47:20 smithi079 podman[78119]: 2024-05-17 09:47:20.94018591 +0000 UTC m=+0.160757162 container died 76c49bc365b0714021e241adb878f7bc5239511e9a776464acbd591d917830e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:21.521 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:47:21 smithi079 podman[78119]: 2024-05-17 09:47:21.335625584 +0000 UTC m=+0.556196828 container cleanup 76c49bc365b0714021e241adb878f7bc5239511e9a776464acbd591d917830e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-17T09:47:21.521 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 17 09:47:21 smithi079 bash[78119]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mgr-z 2024-05-17T09:47:21.769 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-17T09:47:21.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:21.803 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-17T09:47:21.803 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-17T09:47:21.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.0 2024-05-17T09:47:22.126 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:21 smithi019 systemd[1]: Stopping Ceph osd.0 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:22.408 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0[43698]: 2024-05-17T09:47:22.126+0000 7f5534773700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:22.408 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0[43698]: 2024-05-17T09:47:22.127+0000 7f5534773700 -1 osd.0 89 *** Got signal Terminated *** 2024-05-17T09:47:22.409 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0[43698]: 2024-05-17T09:47:22.127+0000 7f5534773700 -1 osd.0 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:47:27.786 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:27 smithi019 podman[133388]: 2024-05-17 09:47:27.482214432 +0000 UTC m=+5.469405672 container died 30ddb289d49ed7aa2c8901e159334269f776a1b2704d033b073c9093da98b3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-17T09:47:28.162 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:27 smithi019 podman[133388]: 2024-05-17 09:47:27.886700548 +0000 UTC m=+5.873891785 container cleanup 30ddb289d49ed7aa2c8901e159334269f776a1b2704d033b073c9093da98b3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-17T09:47:28.162 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:27 smithi019 bash[133388]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0 2024-05-17T09:47:28.440 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:28 smithi019 podman[133419]: 2024-05-17 09:47:28.163746492 +0000 UTC m=+0.679668021 container remove 30ddb289d49ed7aa2c8901e159334269f776a1b2704d033b073c9093da98b3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-17T09:47:28.755 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:28 smithi019 podman[133507]: 2024-05-17 09:47:28.495590701 +0000 UTC m=+0.018600073 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:47:28.755 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:28 smithi019 podman[133507]: 2024-05-17 09:47:28.639864395 +0000 UTC m=+0.162873754 container create 85f47bd68c0a432e268efd23aebf6926098f4d7cdbc61a31eb64dce53e275e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:29.191 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:28 smithi019 podman[133507]: 2024-05-17 09:47:28.892471752 +0000 UTC m=+0.415481112 container init 85f47bd68c0a432e268efd23aebf6926098f4d7cdbc61a31eb64dce53e275e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-17T09:47:29.191 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:28 smithi019 podman[133507]: 2024-05-17 09:47:28.895904987 +0000 UTC m=+0.418914360 container start 85f47bd68c0a432e268efd23aebf6926098f4d7cdbc61a31eb64dce53e275e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:47:29.191 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:28 smithi019 podman[133507]: 2024-05-17 09:47:28.984379901 +0000 UTC m=+0.507389260 container attach 85f47bd68c0a432e268efd23aebf6926098f4d7cdbc61a31eb64dce53e275e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T09:47:29.628 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 17 09:47:29 smithi019 podman[133507]: 2024-05-17 09:47:29.257808612 +0000 UTC m=+0.780817972 container died 85f47bd68c0a432e268efd23aebf6926098f4d7cdbc61a31eb64dce53e275e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-05-17T09:47:29.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-17T09:47:30.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:30.028 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-17T09:47:30.028 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-17T09:47:30.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.1 2024-05-17T09:47:30.346 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:30 smithi069 systemd[1]: Stopping Ceph osd.1 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:30.679 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1[39654]: 2024-05-17T09:47:30.347+0000 7f0d5e855700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:30.679 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1[39654]: 2024-05-17T09:47:30.347+0000 7f0d5e855700 -1 osd.1 89 *** Got signal Terminated *** 2024-05-17T09:47:30.679 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1[39654]: 2024-05-17T09:47:30.347+0000 7f0d5e855700 -1 osd.1 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:47:36.255 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:35 smithi069 podman[75928]: 2024-05-17 09:47:35.935663033 +0000 UTC m=+5.706310116 container died 07e7050c4530f4fee1a52d774e99da74e19a13d383394a71aad39f4f57f7c524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-17T09:47:36.549 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:36 smithi069 podman[75928]: 2024-05-17 09:47:36.37381269 +0000 UTC m=+6.144459773 container cleanup 07e7050c4530f4fee1a52d774e99da74e19a13d383394a71aad39f4f57f7c524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-17T09:47:36.549 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:36 smithi069 bash[75928]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1 2024-05-17T09:47:36.929 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:36 smithi069 podman[75944]: 2024-05-17 09:47:36.650886503 +0000 UTC m=+0.709138520 container remove 07e7050c4530f4fee1a52d774e99da74e19a13d383394a71aad39f4f57f7c524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:47:37.429 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:37 smithi069 podman[76043]: 2024-05-17 09:47:37.064859663 +0000 UTC m=+0.019511404 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:47:37.429 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:37 smithi069 podman[76043]: 2024-05-17 09:47:37.193473695 +0000 UTC m=+0.148125434 container create 99ccafb2b9eb0ef9760adc906b582a35fa22872538e78cb1cfec4f059171e24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-17T09:47:37.840 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:37 smithi069 podman[76043]: 2024-05-17 09:47:37.489186353 +0000 UTC m=+0.443838099 container init 99ccafb2b9eb0ef9760adc906b582a35fa22872538e78cb1cfec4f059171e24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T09:47:37.840 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:37 smithi069 podman[76043]: 2024-05-17 09:47:37.492966114 +0000 UTC m=+0.447617845 container start 99ccafb2b9eb0ef9760adc906b582a35fa22872538e78cb1cfec4f059171e24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T09:47:37.840 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:37 smithi069 podman[76043]: 2024-05-17 09:47:37.580412692 +0000 UTC m=+0.535064442 container attach 99ccafb2b9eb0ef9760adc906b582a35fa22872538e78cb1cfec4f059171e24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T09:47:38.179 INFO:journalctl@ceph.osd.1.smithi069.stdout:May 17 09:47:37 smithi069 podman[76043]: 2024-05-17 09:47:37.841996025 +0000 UTC m=+0.796647757 container died 99ccafb2b9eb0ef9760adc906b582a35fa22872538e78cb1cfec4f059171e24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-17T09:47:38.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-17T09:47:38.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:38.527 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-17T09:47:38.527 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-17T09:47:38.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.2 2024-05-17T09:47:38.863 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:38 smithi079 systemd[1]: Stopping Ceph osd.2 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:39.134 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2[40637]: 2024-05-17T09:47:38.863+0000 7f8bca8b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:39.135 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2[40637]: 2024-05-17T09:47:38.863+0000 7f8bca8b5700 -1 osd.2 89 *** Got signal Terminated *** 2024-05-17T09:47:39.135 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2[40637]: 2024-05-17T09:47:38.863+0000 7f8bca8b5700 -1 osd.2 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:47:44.707 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:44 smithi079 podman[78243]: 2024-05-17 09:47:44.399503843 +0000 UTC m=+5.666351087 container died 5b3221df2e581192030916adf69d1f5ee8b1f4fac521fad9d59f71666d30f6e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-17T09:47:45.021 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:44 smithi079 podman[78243]: 2024-05-17 09:47:44.81489217 +0000 UTC m=+6.081739409 container cleanup 5b3221df2e581192030916adf69d1f5ee8b1f4fac521fad9d59f71666d30f6e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:47:45.021 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:44 smithi079 bash[78243]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2 2024-05-17T09:47:45.469 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:45 smithi079 podman[78272]: 2024-05-17 09:47:45.042834944 +0000 UTC m=+0.641698364 container remove 5b3221df2e581192030916adf69d1f5ee8b1f4fac521fad9d59f71666d30f6e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:45.728 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:45 smithi079 podman[78363]: 2024-05-17 09:47:45.373997973 +0000 UTC m=+0.017554239 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:47:45.728 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:45 smithi079 podman[78363]: 2024-05-17 09:47:45.501549368 +0000 UTC m=+0.145105657 container create 231466f655c5bdcf3920a598853bd6b8f8aebbc72f0f0e7a3316f9257c7f31b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD) 2024-05-17T09:47:45.728 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:45 smithi079 podman[78363]: 2024-05-17 09:47:45.729554457 +0000 UTC m=+0.373110715 container init 231466f655c5bdcf3920a598853bd6b8f8aebbc72f0f0e7a3316f9257c7f31b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-17T09:47:46.021 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:45 smithi079 podman[78363]: 2024-05-17 09:47:45.732939873 +0000 UTC m=+0.376496126 container start 231466f655c5bdcf3920a598853bd6b8f8aebbc72f0f0e7a3316f9257c7f31b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:46.021 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:45 smithi079 podman[78363]: 2024-05-17 09:47:45.871953722 +0000 UTC m=+0.515509981 container attach 231466f655c5bdcf3920a598853bd6b8f8aebbc72f0f0e7a3316f9257c7f31b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:47:46.465 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 17 09:47:46 smithi079 podman[78363]: 2024-05-17 09:47:46.209391776 +0000 UTC m=+0.852948029 container died 231466f655c5bdcf3920a598853bd6b8f8aebbc72f0f0e7a3316f9257c7f31b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-17T09:47:46.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-17T09:47:46.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:46.839 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-17T09:47:46.839 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-17T09:47:46.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.3 2024-05-17T09:47:47.148 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:46 smithi019 systemd[1]: Stopping Ceph osd.3 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:47.441 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:47 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3[50171]: 2024-05-17T09:47:47.149+0000 7fac79aef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:47.441 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:47 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3[50171]: 2024-05-17T09:47:47.149+0000 7fac79aef700 -1 osd.3 89 *** Got signal Terminated *** 2024-05-17T09:47:47.441 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:47 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3[50171]: 2024-05-17T09:47:47.149+0000 7fac79aef700 -1 osd.3 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:47:48.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:47.875+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:49.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:48.893+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:49.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:48.709+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:49.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:49.541+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:50.023 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:49.942+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:50.023 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:49.707+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:50.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:50 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:50.504+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:50.990 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:50.676+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:50.991 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:50.580+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:51.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:50 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:50.842+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:51.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:50.992+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:51.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:51 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:51.549+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:52.017 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:51.559+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:52.017 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:51.628+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:52.091 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:51 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3[50171]: 2024-05-17T09:47:51.923+0000 7fac760a5700 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.821586+0000 front 2024-05-17T09:47:26.821624+0000 (oldest deadline 2024-05-17T09:47:51.521789+0000) 2024-05-17T09:47:52.092 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:51 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:51.809+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:52.270 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:52.018+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:52.691 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:52 smithi019 podman[133612]: 2024-05-17 09:47:52.637166245 +0000 UTC m=+5.594321376 container died a5ee99d07711131f0d3f501f405905092610ff05b618686dfb27cec69652a0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-17T09:47:52.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:52 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:52.553+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:52.980 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:52.538+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:52.981 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:52.584+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:53.148 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:52 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:52.763+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:53.270 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:52.982+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:53.417 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:53 smithi019 podman[133612]: 2024-05-17 09:47:53.150346905 +0000 UTC m=+6.107502013 container cleanup a5ee99d07711131f0d3f501f405905092610ff05b618686dfb27cec69652a0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:53.417 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:53 smithi019 bash[133612]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3 2024-05-17T09:47:53.417 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:53 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:53.286+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:53.690 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:53 smithi019 podman[133637]: 2024-05-17 09:47:53.419178412 +0000 UTC m=+0.784293555 container remove a5ee99d07711131f0d3f501f405905092610ff05b618686dfb27cec69652a0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:53.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:53 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:53.527+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:53.942 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:53.582+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:53.942 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:53.600+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:54.086 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:53 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:53.788+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:54.086 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:53 smithi019 podman[133728]: 2024-05-17 09:47:53.827756138 +0000 UTC m=+0.017695392 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:47:54.087 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:53 smithi019 podman[133728]: 2024-05-17 09:47:53.970209054 +0000 UTC m=+0.160148330 container create 0ce1708ea7dfcdffc21fb05d86d3ccf3898cece61c422c87ddf6a39f13c045b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-17T09:47:54.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:53.944+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:54.441 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:54 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:54.239+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:54.441 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:54 smithi019 podman[133728]: 2024-05-17 09:47:54.230963457 +0000 UTC m=+0.420902714 container init 0ce1708ea7dfcdffc21fb05d86d3ccf3898cece61c422c87ddf6a39f13c045b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:47:54.441 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:54 smithi019 podman[133728]: 2024-05-17 09:47:54.234506256 +0000 UTC m=+0.424445513 container start 0ce1708ea7dfcdffc21fb05d86d3ccf3898cece61c422c87ddf6a39f13c045b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:47:54.441 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:54 smithi019 podman[133728]: 2024-05-17 09:47:54.322633676 +0000 UTC m=+0.512572934 container attach 0ce1708ea7dfcdffc21fb05d86d3ccf3898cece61c422c87ddf6a39f13c045b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-17T09:47:54.764 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 17 09:47:54 smithi019 podman[133728]: 2024-05-17 09:47:54.54821306 +0000 UTC m=+0.738152322 container died 0ce1708ea7dfcdffc21fb05d86d3ccf3898cece61c422c87ddf6a39f13c045b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:47:54.904 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:54.574+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:54.905 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:54.608+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:54.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:54 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:54.511+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:55.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:54 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:54.764+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:55.270 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:54.906+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:55.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-17T09:47:55.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:47:55.392 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-17T09:47:55.392 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-17T09:47:55.392 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.4 2024-05-17T09:47:55.679 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:55 smithi069 systemd[1]: Stopping Ceph osd.4 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:47:55.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:55 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:55.479+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:55.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:55 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:55.279+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:55.933 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:55.534+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:55.933 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:55.572+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:56.179 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:55 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:47:55.729+0000 7f9418887700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:47:56.179 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:55 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:47:55.729+0000 7f9418887700 -1 osd.4 89 *** Got signal Terminated *** 2024-05-17T09:47:56.179 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:55 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:47:55.729+0000 7f9418887700 -1 osd.4 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:47:56.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:55 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:55.738+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:56.264 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:55.934+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:56.521 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:56.488+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:56.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:56 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:56.327+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:56.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:56 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:56.456+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:56.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:56 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:56.456+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:47:56.950 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:56.608+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:57.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:56 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:56.756+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:57.270 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:56.952+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:57.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:47:57 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:47:57.111+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:47:57.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:57 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:57.335+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:57.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:57 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:57.503+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:57.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:57 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:57.648+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:57.929 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:57 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:47:57.470+0000 7f9414e3d700 -1 osd.4 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.727201+0000 front 2024-05-17T09:47:32.727156+0000 (oldest deadline 2024-05-17T09:47:56.827311+0000) 2024-05-17T09:47:57.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:57 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:57.448+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:57.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:57 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:57.448+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:47:58.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:57 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:57.718+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:58.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:57 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:57.718+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:47:58.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:57 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:57.984+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:58.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:47:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:47:58.108+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:47:58.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:58 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:58.379+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:58.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:58 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:58.379+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:47:58.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:58.466+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:58.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:58 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:58.680+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:58.929 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:47:58.517+0000 7f9414e3d700 -1 osd.4 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.727201+0000 front 2024-05-17T09:47:32.727156+0000 (oldest deadline 2024-05-17T09:47:56.827311+0000) 2024-05-17T09:47:58.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:58.471+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:58.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:58 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:58.471+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:47:59.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:58 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:58.722+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:47:59.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:58 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:58.723+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:47:59.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:47:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:47:59.133+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:47:59.438 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:59.031+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:47:59.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:59 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:59.347+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:47:59.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:47:59 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:47:59.347+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:47:59.692 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:59.440+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:47:59.692 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:47:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:47:59.440+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:47:59.929 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:47:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:47:59.508+0000 7f9414e3d700 -1 osd.4 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.727201+0000 front 2024-05-17T09:47:32.727156+0000 (oldest deadline 2024-05-17T09:47:56.827311+0000) 2024-05-17T09:47:59.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:59.459+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:47:59.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:47:59 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:47:59.459+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:47:59.992 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:59.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:47:59.992 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:47:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:47:59.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:00.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:59 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:59.736+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:00.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:47:59 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:47:59.736+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:00.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:47:59 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:47:59.994+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:00.430 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:00.095+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:00.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:00.370+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:00.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:00.370+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:00.741 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:00.473+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:00.741 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:00.473+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:00.929 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4[46097]: 2024-05-17T09:48:00.476+0000 7f9414e3d700 -1 osd.4 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.727201+0000 front 2024-05-17T09:47:32.727156+0000 (oldest deadline 2024-05-17T09:47:56.827311+0000) 2024-05-17T09:48:00.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:00.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:00.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:00 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:00.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:01.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:00.957+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:01.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:00.957+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:01.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:00.743+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:01.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:00 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:00.743+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:01.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:00.738+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:01.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:00.738+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:01.412 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:01.077+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:01.412 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:01 smithi069 podman[76150]: 2024-05-17 09:48:01.134556448 +0000 UTC m=+5.536252962 container died 05d855392ef1681d9a04ab9eb423dd5b9d0dda88b40e5b8227fb3030558b5a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-17T09:48:01.679 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:01 smithi069 podman[76150]: 2024-05-17 09:48:01.539192519 +0000 UTC m=+5.940889030 container cleanup 05d855392ef1681d9a04ab9eb423dd5b9d0dda88b40e5b8227fb3030558b5a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-17T09:48:01.679 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:01 smithi069 bash[76150]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4 2024-05-17T09:48:01.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:01.484+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:01.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:01 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:01.484+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:01.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:01 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:01.362+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:01.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:01 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:01.362+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:01.753 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:01.498+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:01.753 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:01.498+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:02.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:01.908+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:02.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:01.908+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:02.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:01.755+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:02.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:01 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:01.755+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:02.147 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:02.078+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:02.148 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:01 smithi069 podman[76165]: 2024-05-17 09:48:01.832542091 +0000 UTC m=+0.691431333 container remove 05d855392ef1681d9a04ab9eb423dd5b9d0dda88b40e5b8227fb3030558b5a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T09:48:02.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:01 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:01.740+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:02.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:01 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:01.740+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:02.409 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:02 smithi069 podman[76251]: 2024-05-17 09:48:02.052433584 +0000 UTC m=+0.017669273 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:02.410 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:02 smithi069 podman[76251]: 2024-05-17 09:48:02.175120204 +0000 UTC m=+0.140355928 container create 001f42f8fdf574884f9e9975dbea17ba86f4494e9607f8783dadec856aee0da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-17T09:48:02.410 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:02 smithi069 podman[76251]: 2024-05-17 09:48:02.411242847 +0000 UTC m=+0.376478538 container init 001f42f8fdf574884f9e9975dbea17ba86f4494e9607f8783dadec856aee0da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-17T09:48:02.679 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:02 smithi069 podman[76251]: 2024-05-17 09:48:02.414824531 +0000 UTC m=+0.380060210 container start 001f42f8fdf574884f9e9975dbea17ba86f4494e9607f8783dadec856aee0da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS) 2024-05-17T09:48:02.679 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:02 smithi069 podman[76251]: 2024-05-17 09:48:02.502715455 +0000 UTC m=+0.467951155 container attach 001f42f8fdf574884f9e9975dbea17ba86f4494e9607f8783dadec856aee0da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-17T09:48:02.680 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:02.478+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:02.680 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:02 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:02.478+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:02.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:02 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:02.382+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:02.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:02 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:02.382+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:02.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:02.509+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:02.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:02.509+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:02.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:02.731+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:02.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:02.731+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:03.021 INFO:journalctl@ceph.osd.4.smithi069.stdout:May 17 09:48:02 smithi069 podman[76251]: 2024-05-17 09:48:02.721925893 +0000 UTC m=+0.687161573 container died 001f42f8fdf574884f9e9975dbea17ba86f4494e9607f8783dadec856aee0da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-17T09:48:03.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:02 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:02.718+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:03.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:02 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:02.718+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:03.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:02.900+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:03.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:02 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:02.900+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:03.346 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:03.051+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:03.371 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-17T09:48:03.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:03.411 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-17T09:48:03.411 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-17T09:48:03.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.5 2024-05-17T09:48:03.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:03.469+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:03.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:03 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:03.469+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:03.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:03 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:03.410+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:03.690 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:03 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:03.410+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:03.732 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:03 smithi079 systemd[1]: Stopping Ceph osd.5 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:03.732 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:03.506+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:03.732 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:03.506+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:04.021 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:03.746+0000 7f818e5f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:04.021 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:03.746+0000 7f818e5f5700 -1 osd.5 89 *** Got signal Terminated *** 2024-05-17T09:48:04.021 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:03.746+0000 7f818e5f5700 -1 osd.5 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:04.022 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:03.897+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:04.022 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:03.897+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:04.023 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:03.733+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:04.023 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:03 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:03.733+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:04.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:03 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:03.755+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:04.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:03 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:03.755+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:04.424 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:04.016+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:04.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:04.425+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:04.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:04 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:04.425+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:04.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:04 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:04.437+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:04.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:04 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:04.437+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:04.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:04 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:04.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:04.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:04 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:04.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:04.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:04 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:04.518+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:04.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:04 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:04.518+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:05.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:04 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:04.723+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:05.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:04 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:04.723+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:05.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:04 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:04.897+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:05.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:04 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:04.897+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:05.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:05 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:05.046+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:05.683 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:05 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:05.408+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:05.683 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:05 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:05.408+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:05.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:05.713+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:05.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:05.713+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:05.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:05.491+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:05.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:05.491+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:05.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:05.491+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.245351+0000 front 2024-05-17T09:47:39.245319+0000 (oldest deadline 2024-05-17T09:48:05.145478+0000) 2024-05-17T09:48:05.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:05 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:05.445+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:05.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:05 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:05.445+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:05.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:05 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:05.684+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:05.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:05 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:05.684+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:06.270 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:05.942+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:06.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:05 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:05.942+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:06.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:06 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:06.044+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:06.690 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:06 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:06.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:06.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:06 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:06.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:06.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:06 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:06.434+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:06.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:06 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:06.434+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:06.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:06.471+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:06.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:06.471+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:06.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:06.471+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.245351+0000 front 2024-05-17T09:47:39.245319+0000 (oldest deadline 2024-05-17T09:48:05.145478+0000) 2024-05-17T09:48:06.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:06.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:06.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:06.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:06.772 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:06.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:06.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:06 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:06.477+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:06.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:06 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:06.477+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:07.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:06.976+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:07.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:06 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:06.976+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:07.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:06 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:06.995+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:07.673 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:07 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:07.424+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:07.673 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:07 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:07.424+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:07.712 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:07 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:07.505+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:07.712 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:07 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:07.505+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:07.713 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:07 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:07.505+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:07.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:07.703+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:07.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:07.703+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:07.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:07.703+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:07.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:07.474+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:07.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:07.474+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:07.771 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:07.474+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.245351+0000 front 2024-05-17T09:47:39.245319+0000 (oldest deadline 2024-05-17T09:48:05.145478+0000) 2024-05-17T09:48:07.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:07 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:07.675+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:07.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:07 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:07.675+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:08.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:07.992+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:08.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:07.992+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:08.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:07 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:07.992+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:08.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:08 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:08.045+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:08.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:08 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:08.045+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:08.553 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:08.467+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.042545+0000 front 2024-05-17T09:47:26.042435+0000 (oldest deadline 2024-05-17T09:47:50.142518+0000) 2024-05-17T09:48:08.553 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:08.467+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.545316+0000 front 2024-05-17T09:47:34.545029+0000 (oldest deadline 2024-05-17T09:47:59.244976+0000) 2024-05-17T09:48:08.553 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5[47245]: 2024-05-17T09:48:08.467+0000 7f8186ba3700 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.245351+0000 front 2024-05-17T09:47:39.245319+0000 (oldest deadline 2024-05-17T09:48:05.145478+0000) 2024-05-17T09:48:08.688 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:08.404+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:08.689 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:08.404+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:08.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:08 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:08.516+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:08.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:08 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:08.516+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:08.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:08 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:08.516+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:08.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:08.690+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:08.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:08.690+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:08.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:08.690+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:08.966 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:08.704+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:08.966 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:08.704+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:08.966 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:08.704+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:09.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:08.966+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:09.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:08.966+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:09.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:08 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:08.966+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:09.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:09 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:09.042+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:09.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:09 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:09.042+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:09.649 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:09 smithi079 podman[78470]: 2024-05-17 09:48:09.349861442 +0000 UTC m=+5.732124806 container died 9c70af9ae649365eca09835c2fcf8684552c75812d863211dfd76fff2a5d2460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-17T09:48:09.671 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:09 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:09.387+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:09.671 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:09 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:09.387+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:09.671 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:09 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:09.387+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:09.671 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:09 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:09.671+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:09.671 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:09 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:09.671+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:09.671 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:09 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:09.671+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:09.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:09 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:09.543+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:09.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:09 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:09.543+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:09.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:09 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:09.543+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:09.947 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:09 smithi079 podman[78470]: 2024-05-17 09:48:09.746507896 +0000 UTC m=+6.128771259 container cleanup 9c70af9ae649365eca09835c2fcf8684552c75812d863211dfd76fff2a5d2460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True) 2024-05-17T09:48:09.947 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:09 smithi079 bash[78470]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5 2024-05-17T09:48:09.947 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:09.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:09.947 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:09.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:09.947 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:09.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:10.271 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78495]: 2024-05-17 09:48:10.007233025 +0000 UTC m=+0.652576016 container remove 9c70af9ae649365eca09835c2fcf8684552c75812d863211dfd76fff2a5d2460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-17T09:48:10.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:09.947+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:10.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:09.947+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:10.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:09 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:09.947+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:10.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:10.018+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:10.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:10.018+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:10.651 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78588]: 2024-05-17 09:48:10.31657119 +0000 UTC m=+0.109785243 container create 9e580b65bf3cac50b41bbbc44ea7df404152ec41ad5865bdbbd841a7a8d7fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:48:10.651 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78588]: 2024-05-17 09:48:10.223961194 +0000 UTC m=+0.017175257 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:10.651 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78588]: 2024-05-17 09:48:10.552937181 +0000 UTC m=+0.346151228 container init 9e580b65bf3cac50b41bbbc44ea7df404152ec41ad5865bdbbd841a7a8d7fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:48:10.652 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78588]: 2024-05-17 09:48:10.55625432 +0000 UTC m=+0.349468381 container start 9e580b65bf3cac50b41bbbc44ea7df404152ec41ad5865bdbbd841a7a8d7fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-17T09:48:10.667 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:10.359+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:10.722 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:10.359+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:10.722 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:10.359+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:10.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:10.546+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:10.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:10.546+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:10.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:10.546+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:10.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:10.668+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:10.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:10.668+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:10.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:10 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:10.668+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:10.970 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:10 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:10.724+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:10.970 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:10 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:10.724+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:10.970 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:10 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:10.724+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:10.970 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78588]: 2024-05-17 09:48:10.65280514 +0000 UTC m=+0.446019217 container attach 9e580b65bf3cac50b41bbbc44ea7df404152ec41ad5865bdbbd841a7a8d7fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-17T09:48:10.970 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 17 09:48:10 smithi079 podman[78588]: 2024-05-17 09:48:10.84958303 +0000 UTC m=+0.642797084 container died 9e580b65bf3cac50b41bbbc44ea7df404152ec41ad5865bdbbd841a7a8d7fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-17T09:48:11.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:10 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:10.968+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:11.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:10 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:10.968+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:11.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:10 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:10.968+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:11.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:10.990+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:11.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:10 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:10.990+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:11.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-17T09:48:11.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:11.518 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-17T09:48:11.518 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-17T09:48:11.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.6 2024-05-17T09:48:11.663 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:11.381+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:11.663 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:11.381+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:11.663 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:11.381+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:11.663 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 systemd[1]: Stopping Ceph osd.6 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:11.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:11 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:11.507+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:11.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:11 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:11.507+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:11.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:11 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:11.507+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:11.940 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:11.837+0000 7f179ebb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:11.940 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:11.837+0000 7f179ebb7700 -1 osd.6 89 *** Got signal Terminated *** 2024-05-17T09:48:11.941 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:11.837+0000 7f179ebb7700 -1 osd.6 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:11.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:11.663+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:11.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:11.663+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:11.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:11.663+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:11.956 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:11 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:11.703+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:11.956 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:11 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:11.703+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:11.956 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:11 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:11.703+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:12.270 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:11 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:11.956+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:12.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:11 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:11.956+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:12.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:11 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:11.956+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:12.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:11 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:11.962+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:12.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:11 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:11.962+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:12.616 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:12.349+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:12.616 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:12.349+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:12.616 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:12.349+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:12.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:12 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:12.543+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:12.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:12 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:12.543+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:12.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:12 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:12.543+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:12.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:12.618+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:12.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:12.618+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:12.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:12 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:12.618+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:13.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:12 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:12.917+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:13.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:12 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:12.917+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:13.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:12 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:12.917+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:13.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:12 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:12.724+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:13.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:12 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:12.724+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:13.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:12 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:12.724+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:13.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:12 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:12.993+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:13.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:12 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:12.993+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:13.651 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:13.328+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:13.651 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:13.328+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:13.651 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:13.328+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:13.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:13 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:13.519+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:13.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:13 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:13.519+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:13.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:13 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:13.519+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:13.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:13.652+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:13.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:13.652+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:13.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:13 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:13.652+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:14.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:13 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:13.933+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:14.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:13 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:13.933+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:14.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:13 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:13.933+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:14.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:13 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:13.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:14.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:13 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:13.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:14.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:13 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:13.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:14.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:13 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:13.959+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:14.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:13 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:13.959+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:14.367+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:14.367+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:14.367+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:14.367+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:49.174015+0000 front 2024-05-17T09:47:49.173951+0000 (oldest deadline 2024-05-17T09:48:13.874132+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:14.612+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:14.612+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:14.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:14 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:14.612+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:14.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:14 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:14.508+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:14.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:14 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:14.508+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:14.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:14 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:14.508+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:14.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:14 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:14.508+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:15.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:14 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:14.928+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:15.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:14 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:14.928+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:15.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:14 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:14.928+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:15.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:14 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:14.695+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:15.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:14 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:14.695+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:15.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:14 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:14.695+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:15.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:14 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:14.940+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:15.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:14 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:14.940+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:15.398+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:15.398+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:15.398+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:15.398+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:49.174015+0000 front 2024-05-17T09:47:49.173951+0000 (oldest deadline 2024-05-17T09:48:13.874132+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:15.645+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:15.645+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:15.645+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:15.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:15 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:15.645+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:15.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:15.477+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:15.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:15.477+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:15.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:15.477+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:15.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:15.477+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:16.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:15 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:15.884+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:16.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:15 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:15.884+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:16.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:15 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:15.884+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:16.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:15 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:15.658+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:16.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:15 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:15.658+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:16.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:15 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:15.658+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:16.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:15.940+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:16.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:15.940+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:16.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:15 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:15.940+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:16.437+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:26.870561+0000 front 2024-05-17T09:47:26.870597+0000 (oldest deadline 2024-05-17T09:47:52.770775+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:16.437+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.771666+0000 front 2024-05-17T09:47:32.771516+0000 (oldest deadline 2024-05-17T09:47:57.471580+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:16.437+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.772768+0000 front 2024-05-17T09:47:42.772818+0000 (oldest deadline 2024-05-17T09:48:08.672868+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6[57144]: 2024-05-17T09:48:16.437+0000 7f179b16d700 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:49.174015+0000 front 2024-05-17T09:47:49.173951+0000 (oldest deadline 2024-05-17T09:48:13.874132+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:16.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:16.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:16.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:16.691 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:16 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:16.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:16.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:16.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:16.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:16.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:16.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:16.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:16.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:16.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:17.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:16.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:17.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:16.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:17.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:16.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:17.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:16.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:17.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:16.619+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:17.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:16.619+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:17.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:16.619+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:17.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:16 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:16.619+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:17.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:16.953+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:17.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:16.953+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:17.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:16.953+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:17.545 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:17 smithi019 podman[133834]: 2024-05-17 09:48:17.259614415 +0000 UTC m=+5.535397429 container died 5dd2c9ec430f423403d965bc44f79bbabfad6c947d4f690fcffc631887ca2f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-17T09:48:17.796 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:17 smithi019 podman[133834]: 2024-05-17 09:48:17.780159084 +0000 UTC m=+6.055942114 container cleanup 5dd2c9ec430f423403d965bc44f79bbabfad6c947d4f690fcffc631887ca2f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-17T09:48:17.797 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:17 smithi019 bash[133834]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6 2024-05-17T09:48:17.797 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:17.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:17.797 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:17.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:17.797 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:17.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:17.797 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:17 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:17.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:17.918 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:17.539+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:17.918 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:17.539+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:17.918 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:17.539+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:17.918 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:17.539+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:18.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:17.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:18.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:17.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:18.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:17.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:18.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:17.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:18.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:17.639+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:18.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:17.639+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:18.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:17.639+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:18.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:17 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:17.639+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:18.168 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:17.919+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:18.168 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:17.919+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:18.168 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:17 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:17.919+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:18.440 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:18 smithi019 podman[133850]: 2024-05-17 09:48:18.015078331 +0000 UTC m=+0.755578624 container remove 5dd2c9ec430f423403d965bc44f79bbabfad6c947d4f690fcffc631887ca2f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T09:48:18.697 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:18.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:18.697 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:18.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:18.697 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:18.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:18.697 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:18 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:18.656+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:18.698 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:18 smithi019 podman[133938]: 2024-05-17 09:48:18.350621698 +0000 UTC m=+0.017591050 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:18.698 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:18 smithi019 podman[133938]: 2024-05-17 09:48:18.468211067 +0000 UTC m=+0.135180409 container create 8bfad0ae30df6ff9364dd5432e394ed3f7eae42817640589ef3ea994612660b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux ) 2024-05-17T09:48:18.698 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:18 smithi019 podman[133938]: 2024-05-17 09:48:18.695902825 +0000 UTC m=+0.362872155 container init 8bfad0ae30df6ff9364dd5432e394ed3f7eae42817640589ef3ea994612660b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-17T09:48:18.698 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:18 smithi019 podman[133938]: 2024-05-17 09:48:18.69929267 +0000 UTC m=+0.366262016 container start 8bfad0ae30df6ff9364dd5432e394ed3f7eae42817640589ef3ea994612660b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T09:48:18.885 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:18.553+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:18.885 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:18.553+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:18.885 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:18.553+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:18.885 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:18.553+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:18.997 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:18.841+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:18.997 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:18.841+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:18.997 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:18.841+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:18.997 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:18.841+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:18.997 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:18.655+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:18.998 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:18.655+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:18.998 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:18.655+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:18.998 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:18 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:18.655+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:19.007 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:18 smithi019 podman[133938]: 2024-05-17 09:48:18.804230437 +0000 UTC m=+0.471199788 container attach 8bfad0ae30df6ff9364dd5432e394ed3f7eae42817640589ef3ea994612660b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:48:19.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:18.886+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:19.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:18.886+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:19.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:18 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:18.886+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:19.440 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 17 09:48:19 smithi019 podman[133938]: 2024-05-17 09:48:19.009903061 +0000 UTC m=+0.676872400 container died 8bfad0ae30df6ff9364dd5432e394ed3f7eae42817640589ef3ea994612660b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-05-17T09:48:19.723 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:19.674+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:19.723 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:19.674+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:19.723 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:19.674+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:19.723 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:19 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:19.674+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:19.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-17T09:48:19.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:19.787 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-17T09:48:19.787 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-17T09:48:19.788 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.7 2024-05-17T09:48:19.815 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:19.526+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:19.816 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:19.526+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:19.816 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:19.526+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:19.816 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:19.526+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:20.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:19.802+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:20.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:19.802+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:20.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:19.802+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:20.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:19.802+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:20.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:19.688+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:20.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:19.688+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:20.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:19.688+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:20.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:19 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:19.688+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:20.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:19 smithi069 systemd[1]: Stopping Ceph osd.7 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:20.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:19.925+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:20.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:19.925+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:20.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:19 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:19.925+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:20.679 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:20.326+0000 7f829cc07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:20.679 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:20.326+0000 7f829cc07700 -1 osd.7 89 *** Got signal Terminated *** 2024-05-17T09:48:20.679 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:20.326+0000 7f829cc07700 -1 osd.7 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:20.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:20.567+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:20.680 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:20.567+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:20.680 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:20.567+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:20.680 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:20.567+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:20.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:20 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:20.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:20.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:20 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:20.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:20.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:20 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:20.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:20.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:20 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:20.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:21.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:20.771+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:20.771+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:20.771+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:20.771+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:20.710+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:20.710+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:20.710+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:21.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:20 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:20.710+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:21.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:20.965+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:21.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:20.965+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:21.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:20 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:20.965+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:21.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:21.586+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:21.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:21.586+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:21.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:21.586+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:21.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:21.586+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:21.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:21.586+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:21.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:21 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:21.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:21.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:21 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:21.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:21.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:21 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:21.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:21.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:21 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:21.681+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:22.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:21.809+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:22.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:21.809+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:22.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:21.809+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:22.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:21.809+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:22.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:21.745+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:22.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:21.745+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:22.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:21.745+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:22.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:21 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:21.745+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:22.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:21.956+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:22.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:21.956+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:22.429 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:21 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:21.956+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:22.906 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:22.545+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:22.906 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:22.545+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:22.906 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:22.545+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:22.906 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:22.545+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:22.906 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:22.545+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:22.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:22.648+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:22.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:22.648+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:22.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:22.648+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:22.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:22 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:22.648+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:23.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:22.906+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:23.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:22.906+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:23.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:22.906+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:23.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:22.906+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:58.394837+0000 front 2024-05-17T09:47:58.394804+0000 (oldest deadline 2024-05-17T09:48:22.495037+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:22.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:22.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:22.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:22.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:22.800+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:22.800+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:22.800+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:23.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:22.800+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:23.272 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:22 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:22.800+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:23.888 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:23.498+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:23.889 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:23.498+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:23.889 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:23.498+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:23.889 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:23.498+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:23.889 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:23.498+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:23.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:23 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:23.635+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:23.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:23 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:23.635+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:23.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:23 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:23.635+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:23.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:23 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:23.635+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:23.799+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:23.799+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:23.799+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:23.799+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:23.799+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:23.742+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:23.742+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:24.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:23.742+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:24.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:23.742+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:24.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:23 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:23.742+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:24.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:23.889+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:24.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:23.889+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:24.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:23.889+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:24.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:23 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:23.889+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:58.394837+0000 front 2024-05-17T09:47:58.394804+0000 (oldest deadline 2024-05-17T09:48:22.495037+0000) 2024-05-17T09:48:24.904 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:24.495+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:24.904 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:24.495+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:24.904 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:24.495+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:24.904 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:24.495+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:24.904 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:24.495+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:24.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:24.615+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:24.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:24.615+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:24.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:24.615+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:24.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:24 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:24.615+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:24.825+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:24.825+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:24.825+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:24.825+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:24.825+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:24.717+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:25.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:24.717+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:25.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:24.717+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:25.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:24.717+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:25.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:24 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:24.717+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:25.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:24.905+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.090485+0000 front 2024-05-17T09:47:32.090606+0000 (oldest deadline 2024-05-17T09:47:56.790611+0000) 2024-05-17T09:48:25.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:24.905+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.792338+0000 front 2024-05-17T09:47:43.792244+0000 (oldest deadline 2024-05-17T09:48:07.892406+0000) 2024-05-17T09:48:25.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:24.905+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793758+0000 front 2024-05-17T09:47:50.793921+0000 (oldest deadline 2024-05-17T09:48:15.493792+0000) 2024-05-17T09:48:25.179 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:24 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7[52628]: 2024-05-17T09:48:24.905+0000 7f82991bd700 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:58.394837+0000 front 2024-05-17T09:47:58.394804+0000 (oldest deadline 2024-05-17T09:48:22.495037+0000) 2024-05-17T09:48:25.790 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:25 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:25.485+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:25.790 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:25 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:25.485+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:25.790 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:25 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:25.485+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:25.790 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:25 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:25.485+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:25.791 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:25 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:25.485+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:25.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:25.574+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:25.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:25.574+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:25.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:25.574+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:25.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:25 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:25.574+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:26.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:25.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:26.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:25.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:26.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:25.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:26.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:25.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:26.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:25.856+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:26.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:25.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:26.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:25.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:26.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:25.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:26.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:25.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:26.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:25 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:25.698+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:26.084 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:25 smithi069 podman[76370]: 2024-05-17 09:48:25.791364418 +0000 UTC m=+5.803330089 container died 22712f785ab1feceb81cf35551d76bdb5513ee3b5021af1ce9b2a30bd220f62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-17T09:48:26.417 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:26 smithi069 podman[76370]: 2024-05-17 09:48:26.193526466 +0000 UTC m=+6.205492137 container cleanup 22712f785ab1feceb81cf35551d76bdb5513ee3b5021af1ce9b2a30bd220f62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-17T09:48:26.418 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:26 smithi069 bash[76370]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7 2024-05-17T09:48:26.679 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:26 smithi069 podman[76385]: 2024-05-17 09:48:26.419709891 +0000 UTC m=+0.625452016 container remove 22712f785ab1feceb81cf35551d76bdb5513ee3b5021af1ce9b2a30bd220f62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:48:26.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:26 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:26.481+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:26.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:26 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:26.481+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:26.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:26 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:26.481+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:26.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:26 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:26.481+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:26.679 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:26 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:26.481+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:26.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:26 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:26.587+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:26.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:26 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:26.587+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:26.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:26 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:26.587+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:26.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:26 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:26.587+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:26.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:26 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:26.587+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:26.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:26.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:26.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:26.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:26.847+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:26.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:27.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:26.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:27.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:26.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:27.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:26.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:27.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:26 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:26.689+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:27.063 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:26 smithi069 podman[76472]: 2024-05-17 09:48:26.660148003 +0000 UTC m=+0.017455143 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:27.063 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:26 smithi069 podman[76472]: 2024-05-17 09:48:26.787431722 +0000 UTC m=+0.144738860 container create 3718dab3e37ab82d5fde75575ecee3f9783b6c5c6c46ba75f93352bb7102469d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T09:48:27.063 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:27 smithi069 podman[76472]: 2024-05-17 09:48:27.065221446 +0000 UTC m=+0.422528592 container init 3718dab3e37ab82d5fde75575ecee3f9783b6c5c6c46ba75f93352bb7102469d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-17T09:48:27.357 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:27 smithi069 podman[76472]: 2024-05-17 09:48:27.068783054 +0000 UTC m=+0.426090203 container start 3718dab3e37ab82d5fde75575ecee3f9783b6c5c6c46ba75f93352bb7102469d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-17T09:48:27.357 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:27 smithi069 podman[76472]: 2024-05-17 09:48:27.165188744 +0000 UTC m=+0.522495894 container attach 3718dab3e37ab82d5fde75575ecee3f9783b6c5c6c46ba75f93352bb7102469d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:48:27.631 INFO:journalctl@ceph.osd.7.smithi069.stdout:May 17 09:48:27 smithi069 podman[76472]: 2024-05-17 09:48:27.359735635 +0000 UTC m=+0.717042781 container died 3718dab3e37ab82d5fde75575ecee3f9783b6c5c6c46ba75f93352bb7102469d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:48:27.632 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:27.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:27.632 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:27.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:27.632 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:27.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:27.632 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:27.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:27.632 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:27.503+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:27.936 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-17T09:48:27.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:27 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:27.636+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:27.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:27 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:27.636+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:27.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:27 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:27.636+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:27.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:27 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:27.636+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:27.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:27 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:27.636+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:28.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:28.015 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-17T09:48:28.016 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-17T09:48:28.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.8 2024-05-17T09:48:28.020 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:27.854+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:27.854+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:27.854+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:27.854+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:27.854+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:27.680+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:27.680+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:27.680+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:27.680+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:28.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:27 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:27.680+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:28.331 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 systemd[1]: Stopping Ceph osd.8 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:28.655 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.331+0000 7f8ea2f4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:28.656 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.331+0000 7f8ea2f4b700 -1 osd.8 89 *** Got signal Terminated *** 2024-05-17T09:48:28.656 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.331+0000 7f8ea2f4b700 -1 osd.8 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:28.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:28 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:28.544+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:28.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:28 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:28.544+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:28.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:28 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:28.544+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:28.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:28 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:28.544+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:28.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:28 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:28.544+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:28.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:28 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:28.649+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:28.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:28 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:28.649+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:28.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:28 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:28.649+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:28.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:28 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:28.649+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:28.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:28 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:28.649+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:29.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.848+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:29.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.848+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:29.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.848+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:29.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.848+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:29.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:28.848+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:29.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:28.657+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:29.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:28.657+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:29.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:28.657+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:29.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:28.657+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:29.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:28 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:28.657+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:29.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:29 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:29.499+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:29.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:29 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:29.499+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:29.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:29 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:29.499+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:29.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:29 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:29.499+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:29.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:29 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:29.499+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:29.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:29 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:29.499+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:30.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:29.851+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:30.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:29.851+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:30.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:29.851+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:30.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:29.851+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:30.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:29.851+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:30.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:29.616+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:30.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:29.616+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:30.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:29.616+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:30.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:29.616+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:30.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:29 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:29.616+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:30.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:29 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:29.697+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:30.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:29 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:29.697+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:30.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:29 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:29.697+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:30.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:29 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:29.697+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:30.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:29 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:29.697+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:30.854 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:30.604+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:30.854 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:30.604+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:30.854 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:30.604+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:30.855 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:30.604+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:30.855 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:30.604+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:30.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:30.491+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:30.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:30.491+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:30.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:30.491+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:30.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:30.491+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:30.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:30.491+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:30.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:30 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:30.491+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:30.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:30 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:30.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:30.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:30 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:30.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:30.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:30 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:30.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:30.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:30 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:30.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:30.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:30 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:30.687+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:31.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:30.855+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:31.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:30.855+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:31.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:30.855+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:31.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:30.855+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:31.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:30 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:30.855+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:31.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:31.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:31.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:31.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:31.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:31.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:31.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:31.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:31.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:31.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:31.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:31.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:32.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:31.860+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:32.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:31.860+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:32.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:31.860+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:32.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:31.860+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:32.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:31.860+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:32.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:31.647+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:32.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:31.647+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:32.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:31.647+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:32.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:31.647+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:32.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:31.647+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:32.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:31 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:31.647+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:32.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:31 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:31.705+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:32.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:31 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:31.705+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:32.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:31 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:31.705+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:32.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:31 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:31.705+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:32.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:31 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:31.705+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:32.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:32 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:32.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:32.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:32 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:32.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:32.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:32 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:32.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:32.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:32 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:32.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:32.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:32 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:32.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:32.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:32 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:32.502+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:32.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:32 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:32.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:32.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:32 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:32.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:32.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:32 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:32.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:32.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:32 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:32.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:32.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:32 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:32.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:33.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:32.882+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.526115+0000 front 2024-05-17T09:47:25.526067+0000 (oldest deadline 2024-05-17T09:47:47.826167+0000) 2024-05-17T09:48:33.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:32.882+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.828664+0000 front 2024-05-17T09:47:34.828472+0000 (oldest deadline 2024-05-17T09:48:00.128528+0000) 2024-05-17T09:48:33.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:32.882+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:43.629862+0000 front 2024-05-17T09:47:43.629888+0000 (oldest deadline 2024-05-17T09:48:07.630806+0000) 2024-05-17T09:48:33.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:32.882+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.032611+0000 front 2024-05-17T09:47:51.032528+0000 (oldest deadline 2024-05-17T09:48:16.332585+0000) 2024-05-17T09:48:33.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8[53729]: 2024-05-17T09:48:32.882+0000 7f8e9f501700 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:56.333037+0000 front 2024-05-17T09:47:56.333062+0000 (oldest deadline 2024-05-17T09:48:22.233005+0000) 2024-05-17T09:48:33.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:32.666+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:33.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:32.666+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:33.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:32.666+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:33.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:32.666+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:33.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:32.666+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:33.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:32 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:32.666+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:33.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:33 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:33.482+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:33.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:33 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:33.482+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:33.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:33 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:33.482+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:33.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:33 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:33.482+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:33.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:33 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:33.482+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:33.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:33 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:33.482+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:33 smithi079 podman[78698]: 2024-05-17 09:48:33.81905376 +0000 UTC m=+5.606729788 container died 7e4ec2b5b519f0a416f712a85ff37f9419f0add03c2b1efba6f230902f34de3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:33.668+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:33.668+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:33.668+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:33.668+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:33.668+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:34.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:33 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:33.668+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:34.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:33 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:33.696+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:34.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:33 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:33.696+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:34.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:33 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:33.696+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:34.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:33 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:33.696+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:34.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:33 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:33.696+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:34.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:33 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:33.696+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:34.381 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:34 smithi079 podman[78698]: 2024-05-17 09:48:34.157122133 +0000 UTC m=+5.944798154 container cleanup 7e4ec2b5b519f0a416f712a85ff37f9419f0add03c2b1efba6f230902f34de3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2) 2024-05-17T09:48:34.382 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:34 smithi079 bash[78698]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8 2024-05-17T09:48:34.669 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:34 smithi079 podman[78715]: 2024-05-17 09:48:34.383624552 +0000 UTC m=+0.563941745 container remove 7e4ec2b5b519f0a416f712a85ff37f9419f0add03c2b1efba6f230902f34de3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-17T09:48:34.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:34 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:34.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:34.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:34 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:34.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:34.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:34 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:34.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:34.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:34 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:34.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:34.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:34 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:34.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:34.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:34 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:34.480+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:34.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:34.731+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:34.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:34.731+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:34.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:34.731+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:34.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:34.731+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:34.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:34.731+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:34.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:34 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:34.731+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:34.954 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:34 smithi079 podman[78802]: 2024-05-17 09:48:34.725912059 +0000 UTC m=+0.017077086 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:34.954 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:34 smithi079 podman[78802]: 2024-05-17 09:48:34.859661358 +0000 UTC m=+0.150826380 container create a78b750cf48bb1438b8f93e534e7362734aa73866f062f2d6ecda7d024c79929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T09:48:34.955 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:34 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:34.671+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:34.955 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:34 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:34.671+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:34.955 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:34 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:34.671+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:34.955 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:34 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:34.671+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:34.955 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:34 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:34.671+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:34.955 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:34 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:34.671+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:35.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:35 smithi079 podman[78802]: 2024-05-17 09:48:35.045987736 +0000 UTC m=+0.337152763 container init a78b750cf48bb1438b8f93e534e7362734aa73866f062f2d6ecda7d024c79929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:48:35.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:35 smithi079 podman[78802]: 2024-05-17 09:48:35.049354465 +0000 UTC m=+0.340519500 container start a78b750cf48bb1438b8f93e534e7362734aa73866f062f2d6ecda7d024c79929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T09:48:35.271 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:35 smithi079 podman[78802]: 2024-05-17 09:48:35.121083861 +0000 UTC m=+0.412248903 container attach a78b750cf48bb1438b8f93e534e7362734aa73866f062f2d6ecda7d024c79929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-17T09:48:35.538 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 17 09:48:35 smithi079 podman[78802]: 2024-05-17 09:48:35.346915964 +0000 UTC m=+0.638080992 container died a78b750cf48bb1438b8f93e534e7362734aa73866f062f2d6ecda7d024c79929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T09:48:35.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:35 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:35.452+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:35.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:35 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:35.452+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:35.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:35 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:35.452+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:35.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:35 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:35.452+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:35.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:35 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:35.452+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:35.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:35 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:35.452+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:35.957 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:35 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:35.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:35.957 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:35 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:35.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:35.957 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:35 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:35.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:35.957 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:35 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:35.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:35.957 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:35 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:35.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:35.957 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:35 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:35.686+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:35.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-17T09:48:36.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:36.059 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-17T09:48:36.059 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-17T09:48:36.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.9 2024-05-17T09:48:36.088 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:35 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:35.694+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:36.088 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:35 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:35.694+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:36.088 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:35 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:35.694+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:36.088 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:35 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:35.694+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:36.088 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:35 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:35.694+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:36.088 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:35 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:35.694+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:36.418 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 systemd[1]: Stopping Ceph osd.9 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:36.683 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.420+0000 7fc9e4809700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.420+0000 7fc9e4809700 -1 osd.9 89 *** Got signal Terminated *** 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.420+0000 7fc9e4809700 -1 osd.9 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:36.684 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:36 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:36.685+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:36.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:36.718+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:36.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:36.718+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:36.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:36.718+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:36.772 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:36.718+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:36.772 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:36.718+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:36.772 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:36.718+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:36.830 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:36.450+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:36.831 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:36.450+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:36.831 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:36.450+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:36.831 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:36.450+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:36.831 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:36.450+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:36.831 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:36 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:36.450+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:37.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:37 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:37.466+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:37.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:37 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:37.466+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:37.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:37 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:37.466+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:37.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:37 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:37.466+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:37.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:37 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:37.466+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:37.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:37 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:37.466+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:38.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:37 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:37.696+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:38.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:37 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:37.696+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:38.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:37 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:37.696+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:38.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:37 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:37.696+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:38.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:37 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:37.696+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:38.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:37 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:37.696+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:38.190 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:37 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:37.704+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:38.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:37 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:37.704+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:38.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:37 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:37.704+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:38.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:37 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:37.704+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:38.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:37 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:37.704+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:38.191 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:37 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:37.704+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:38.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:38 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:38.494+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:38.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:38 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:38.494+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:38.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:38 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:38.494+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:38.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:38 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:38.494+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:38.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:38 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:38.494+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:38.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:38 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:38.494+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:38.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:38 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:38.682+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:38.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:38 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:38.682+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:38.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:38 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:38.682+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:38.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:38 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:38.682+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:38.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:38 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:38.682+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:38.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:38 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:38.682+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:39.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:38.692+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:39.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:38.692+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:39.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:38.692+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:39.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:38.692+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:39.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:38.692+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:39.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:38 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:38.692+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:39.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:39.522+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:39.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:39.522+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:39.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:39.522+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:39.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:39.522+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:39.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:39.522+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:39.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:39 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:39.522+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:39.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:39.642+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:39.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:39.642+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:39.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:39.642+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:39.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:39.642+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:39.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:39.642+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:39.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:39 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:39.642+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:40.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:39 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:39.694+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:40.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:39 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:39.694+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:40.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:39 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:39.694+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:40.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:39 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:39.694+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:40.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:39 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:39.694+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:40.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:39 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:39.694+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:40.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:40 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:40.551+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:40.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:40 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:40.551+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:40.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:40 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:40.551+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:40.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:40 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:40.551+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:40.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:40 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:40.551+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:40.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:40 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:40.551+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:40.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:40.631+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:24.958688+0000 front 2024-05-17T09:47:24.958716+0000 (oldest deadline 2024-05-17T09:47:50.258945+0000) 2024-05-17T09:48:40.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:40.631+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.560339+0000 front 2024-05-17T09:47:32.560265+0000 (oldest deadline 2024-05-17T09:47:57.260361+0000) 2024-05-17T09:48:40.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:40.631+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:42.562024+0000 front 2024-05-17T09:47:42.561734+0000 (oldest deadline 2024-05-17T09:48:07.861742+0000) 2024-05-17T09:48:40.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:40.631+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.962826+0000 front 2024-05-17T09:47:51.962868+0000 (oldest deadline 2024-05-17T09:48:14.863048+0000) 2024-05-17T09:48:40.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:40.631+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:48:00.665000+0000 front 2024-05-17T09:48:00.665199+0000 (oldest deadline 2024-05-17T09:48:26.564840+0000) 2024-05-17T09:48:40.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:40 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9[64034]: 2024-05-17T09:48:40.631+0000 7fc9dcdb7700 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.065962+0000 front 2024-05-17T09:48:07.066083+0000 (oldest deadline 2024-05-17T09:48:32.965755+0000) 2024-05-17T09:48:41.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:40 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:40.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:41.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:40 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:40.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:41.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:40 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:40.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:41.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:40 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:40.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:41.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:40 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:40.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:41.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:40 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:40.693+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:41.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:41 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:41.576+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:41.948 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:41 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:41.576+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:41.949 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:41 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:41.576+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:41.949 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:41 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:41.576+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:41.949 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:41 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:41.576+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:41.949 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:41 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:41.576+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:42.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:41.697+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:42.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:41.697+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:42.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:41.697+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:42.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:41.697+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:42.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:41.697+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:42.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:41.697+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:42.310 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:42 smithi019 podman[134057]: 2024-05-17 09:48:42.033761162 +0000 UTC m=+5.762434443 container died 48370521cbe268e62f595cb601e9c7e5b5d4776d8f7094a3d916b193b0bac2d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-17T09:48:42.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:42.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:42.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:42.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:42.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:42.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:42.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:42 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:42.595+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:42.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:42 smithi019 podman[134057]: 2024-05-17 09:48:42.645017257 +0000 UTC m=+6.373690522 container cleanup 48370521cbe268e62f595cb601e9c7e5b5d4776d8f7094a3d916b193b0bac2d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:48:42.940 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:42 smithi019 bash[134057]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9 2024-05-17T09:48:42.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:42 smithi019 podman[134071]: 2024-05-17 09:48:42.863178838 +0000 UTC m=+0.822514778 container remove 48370521cbe268e62f595cb601e9c7e5b5d4776d8f7094a3d916b193b0bac2d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-17T09:48:43.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:43.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:43.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:43.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:43.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:43.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:43.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:42 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:42.721+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:43.515 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:43 smithi019 podman[134158]: 2024-05-17 09:48:43.108212136 +0000 UTC m=+0.017886270 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:43.515 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:43 smithi019 podman[134158]: 2024-05-17 09:48:43.214013859 +0000 UTC m=+0.123688038 container create 43581410bb9a177f2aada2b1d957a205e10c3ec22f428d558072a41785802601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:48:43.515 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:43 smithi019 podman[134158]: 2024-05-17 09:48:43.441643299 +0000 UTC m=+0.351317435 container init 43581410bb9a177f2aada2b1d957a205e10c3ec22f428d558072a41785802601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-05-17T09:48:43.515 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:43 smithi019 podman[134158]: 2024-05-17 09:48:43.445100507 +0000 UTC m=+0.354774648 container start 43581410bb9a177f2aada2b1d957a205e10c3ec22f428d558072a41785802601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:48:43.515 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:43 smithi019 podman[134158]: 2024-05-17 09:48:43.516532307 +0000 UTC m=+0.426206457 container attach 43581410bb9a177f2aada2b1d957a205e10c3ec22f428d558072a41785802601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T09:48:43.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:43.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:43.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:43.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:43.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:43.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:43.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:43 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:43.587+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:43.941 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 17 09:48:43 smithi019 podman[134158]: 2024-05-17 09:48:43.753395221 +0000 UTC m=+0.663069355 container died 43581410bb9a177f2aada2b1d957a205e10c3ec22f428d558072a41785802601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:44.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:43 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:43.771+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:44.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-17T09:48:44.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:44.443 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-17T09:48:44.443 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-17T09:48:44.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.10 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 systemd[1]: Stopping Ceph osd.10 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:44.779 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:44.780 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.605+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:44.780 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.779+0000 7fa2b3c7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:44.780 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.779+0000 7fa2b3c7a700 -1 osd.10 89 *** Got signal Terminated *** 2024-05-17T09:48:44.780 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:44 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:44.779+0000 7fa2b3c7a700 -1 osd.10 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:45.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:44 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:44.819+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:45.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:45.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:45.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:45.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:45.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:45.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:45.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:45 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:45.614+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:46.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:45 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:45.817+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:46.880 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:46 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:46.624+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:47.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:47.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:47.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:47.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:47.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:47.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:47.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:46 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:46.797+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:47.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:47.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:47.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:47.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:47.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:47.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:47.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:47 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:47.661+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:48.085 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:47.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:49.179 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:48 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:48.698+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:49.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:48 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:48.796+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:49.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:25.029993+0000 front 2024-05-17T09:47:25.030011+0000 (oldest deadline 2024-05-17T09:47:48.530144+0000) 2024-05-17T09:48:49.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:32.031453+0000 front 2024-05-17T09:47:32.031317+0000 (oldest deadline 2024-05-17T09:47:56.131435+0000) 2024-05-17T09:48:49.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:41.432711+0000 front 2024-05-17T09:47:41.432750+0000 (oldest deadline 2024-05-17T09:48:06.732580+0000) 2024-05-17T09:48:49.929 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:51.433631+0000 front 2024-05-17T09:47:51.433719+0000 (oldest deadline 2024-05-17T09:48:14.333662+0000) 2024-05-17T09:48:49.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.534861+0000 front 2024-05-17T09:47:59.534840+0000 (oldest deadline 2024-05-17T09:48:21.235213+0000) 2024-05-17T09:48:49.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:06.536358+0000 front 2024-05-17T09:48:06.536337+0000 (oldest deadline 2024-05-17T09:48:29.436077+0000) 2024-05-17T09:48:49.930 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:49 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10[59161]: 2024-05-17T09:48:49.651+0000 7fa2b0230700 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:11.137206+0000 front 2024-05-17T09:48:11.137324+0000 (oldest deadline 2024-05-17T09:48:42.337677+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:50.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:49 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:49.810+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:50.429 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:50 smithi069 podman[76590]: 2024-05-17 09:48:50.173379335 +0000 UTC m=+5.522460558 container died ece9aa6fc9a41ca5cd33a78cf79b372088c99bbe84c851cd56308068a9b83622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:48:50.804 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:50 smithi069 podman[76590]: 2024-05-17 09:48:50.529856502 +0000 UTC m=+5.878937727 container cleanup ece9aa6fc9a41ca5cd33a78cf79b372088c99bbe84c851cd56308068a9b83622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T09:48:50.804 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:50 smithi069 bash[76590]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10 2024-05-17T09:48:51.156 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:50 smithi069 podman[76607]: 2024-05-17 09:48:50.806385972 +0000 UTC m=+0.629555664 container remove ece9aa6fc9a41ca5cd33a78cf79b372088c99bbe84c851cd56308068a9b83622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:51.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:50.786+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:51.429 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:51 smithi069 podman[76694]: 2024-05-17 09:48:51.062000786 +0000 UTC m=+0.025865851 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:51.429 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:51 smithi069 podman[76694]: 2024-05-17 09:48:51.165615592 +0000 UTC m=+0.129480644 container create fbc769e46bc8fc26615f1171c040cc4ad3fa7c4b2e22b6d6acc758889568d3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-17T09:48:51.732 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:51 smithi069 podman[76694]: 2024-05-17 09:48:51.434386772 +0000 UTC m=+0.398251829 container init fbc769e46bc8fc26615f1171c040cc4ad3fa7c4b2e22b6d6acc758889568d3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-17T09:48:51.732 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:51 smithi069 podman[76694]: 2024-05-17 09:48:51.4376139 +0000 UTC m=+0.401478952 container start fbc769e46bc8fc26615f1171c040cc4ad3fa7c4b2e22b6d6acc758889568d3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-17T09:48:51.732 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:51 smithi069 podman[76694]: 2024-05-17 09:48:51.501348326 +0000 UTC m=+0.465213380 container attach fbc769e46bc8fc26615f1171c040cc4ad3fa7c4b2e22b6d6acc758889568d3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-17T09:48:52.169 INFO:journalctl@ceph.osd.10.smithi069.stdout:May 17 09:48:51 smithi069 podman[76694]: 2024-05-17 09:48:51.734515716 +0000 UTC m=+0.698380774 container died fbc769e46bc8fc26615f1171c040cc4ad3fa7c4b2e22b6d6acc758889568d3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-17T09:48:52.193 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-17T09:48:52.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:48:52.231 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-17T09:48:52.231 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-17T09:48:52.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.11 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:52.259 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:51 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:51.822+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-17T09:48:22.600416+0000 front 2024-05-17T09:48:22.600360+0000 (oldest deadline 2024-05-17T09:48:50.800803+0000) 2024-05-17T09:48:52.521 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 systemd[1]: Stopping Ceph osd.11 for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:48:53.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.652+0000 7f8e9426b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:48:53.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.652+0000 7f8e9426b700 -1 osd.11 89 *** Got signal Terminated *** 2024-05-17T09:48:53.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.652+0000 7f8e9426b700 -1 osd.11 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T09:48:53.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:53.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:53.021 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:53.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:53.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:53.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:53.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:53.022 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:52 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:52.844+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-17T09:48:22.600416+0000 front 2024-05-17T09:48:22.600360+0000 (oldest deadline 2024-05-17T09:48:50.800803+0000) 2024-05-17T09:48:54.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:54.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:54.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:54.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:54.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:54.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:54.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:54.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:53 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:53.873+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-17T09:48:22.600416+0000 front 2024-05-17T09:48:22.600360+0000 (oldest deadline 2024-05-17T09:48:50.800803+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:55.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:54 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:54.899+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-17T09:48:22.600416+0000 front 2024-05-17T09:48:22.600360+0000 (oldest deadline 2024-05-17T09:48:50.800803+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:56.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-17T09:48:22.600416+0000 front 2024-05-17T09:48:22.600360+0000 (oldest deadline 2024-05-17T09:48:50.800803+0000) 2024-05-17T09:48:56.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:55 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:55.872+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6820 osd.8 since back 2024-05-17T09:48:30.801183+0000 front 2024-05-17T09:48:30.801209+0000 (oldest deadline 2024-05-17T09:48:54.901179+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-17T09:47:23.889121+0000 front 2024-05-17T09:47:23.888922+0000 (oldest deadline 2024-05-17T09:47:47.989140+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2024-05-17T09:47:34.991663+0000 front 2024-05-17T09:47:34.991436+0000 (oldest deadline 2024-05-17T09:47:59.691321+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-17T09:47:39.691676+0000 front 2024-05-17T09:47:39.691651+0000 (oldest deadline 2024-05-17T09:48:06.092479+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-17T09:47:50.793514+0000 front 2024-05-17T09:47:50.793372+0000 (oldest deadline 2024-05-17T09:48:16.093351+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2024-05-17T09:47:59.495077+0000 front 2024-05-17T09:47:59.494972+0000 (oldest deadline 2024-05-17T09:48:22.994906+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-17T09:48:07.997597+0000 front 2024-05-17T09:48:07.997544+0000 (oldest deadline 2024-05-17T09:48:31.497640+0000) 2024-05-17T09:48:57.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-17T09:48:16.699579+0000 front 2024-05-17T09:48:16.699702+0000 (oldest deadline 2024-05-17T09:48:42.599597+0000) 2024-05-17T09:48:57.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2024-05-17T09:48:22.600416+0000 front 2024-05-17T09:48:22.600360+0000 (oldest deadline 2024-05-17T09:48:50.800803+0000) 2024-05-17T09:48:57.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:56 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11[60453]: 2024-05-17T09:48:56.884+0000 7f8e8c819700 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.79:6820 osd.8 since back 2024-05-17T09:48:30.801183+0000 front 2024-05-17T09:48:30.801209+0000 (oldest deadline 2024-05-17T09:48:54.901179+0000) 2024-05-17T09:48:58.271 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:58 smithi079 podman[78919]: 2024-05-17 09:48:58.003395544 +0000 UTC m=+5.573272475 container died 53f06deaa3f9dc936b46682f0619411d0f7e3b1633b6b9aff6eb4e9854be1015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:48:58.635 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:58 smithi079 podman[78919]: 2024-05-17 09:48:58.402270249 +0000 UTC m=+5.972147179 container cleanup 53f06deaa3f9dc936b46682f0619411d0f7e3b1633b6b9aff6eb4e9854be1015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-17T09:48:58.635 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:58 smithi079 bash[78919]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11 2024-05-17T09:48:58.954 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:58 smithi079 podman[78936]: 2024-05-17 09:48:58.63710852 +0000 UTC m=+0.633625586 container remove 53f06deaa3f9dc936b46682f0619411d0f7e3b1633b6b9aff6eb4e9854be1015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-17T09:48:59.211 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:58 smithi079 podman[79021]: 2024-05-17 09:48:58.858795381 +0000 UTC m=+0.016897866 image pull 1a138b128acac2828b1e94c183fa53d90daf3f8e1eae7972965c89a5182818a9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6 2024-05-17T09:48:59.211 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:58 smithi079 podman[79021]: 2024-05-17 09:48:58.990605621 +0000 UTC m=+0.148708113 container create b67e00d16ab1591313862b653db58474c2c9c9ce248d3ae0f73cbda20b10d26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True) 2024-05-17T09:48:59.211 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:59 smithi079 podman[79021]: 2024-05-17 09:48:59.209657216 +0000 UTC m=+0.367759702 container init b67e00d16ab1591313862b653db58474c2c9c9ce248d3ae0f73cbda20b10d26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-17T09:48:59.503 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:59 smithi079 podman[79021]: 2024-05-17 09:48:59.212858501 +0000 UTC m=+0.370960987 container start b67e00d16ab1591313862b653db58474c2c9c9ce248d3ae0f73cbda20b10d26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-17T09:48:59.504 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:59 smithi079 podman[79021]: 2024-05-17 09:48:59.309436161 +0000 UTC m=+0.467538663 container attach b67e00d16ab1591313862b653db58474c2c9c9ce248d3ae0f73cbda20b10d26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:48:59.771 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 17 09:48:59 smithi079 podman[79021]: 2024-05-17 09:48:59.505964099 +0000 UTC m=+0.664066590 container died b67e00d16ab1591313862b653db58474c2c9c9ce248d3ae0f73cbda20b10d26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-17T09:49:00.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-17T09:49:00.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:00.161 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-17T09:49:00.161 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-17T09:49:00.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.a 2024-05-17T09:49:00.441 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:00 smithi019 systemd[1]: Stopping Ceph mds.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:00.941 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a[71288]: 2024-05-17T09:49:00.479+0000 7f32a4695700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:00.941 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:00 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a[71288]: 2024-05-17T09:49:00.479+0000 7f32a4695700 -1 mds.a *** got signal Terminated *** 2024-05-17T09:49:02.190 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:01 smithi019 podman[134265]: 2024-05-17 09:49:01.943460433 +0000 UTC m=+1.567895275 container died 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:49:02.596 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:02 smithi019 podman[134265]: 2024-05-17 09:49:02.363934089 +0000 UTC m=+1.988368922 container cleanup 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-17T09:49:02.596 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:02 smithi019 bash[134265]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a 2024-05-17T09:49:02.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-17T09:49:02.856 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:02 smithi019 podman[134287]: 2024-05-17 09:49:02.598651065 +0000 UTC m=+0.651140867 container remove 7b974af6d09555679085516d2e4b69ac01532bd6b42a69ceaeed1f6db3846d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-17T09:49:02.856 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:02 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-17T09:49:02.856 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:02 smithi019 systemd[1]: Stopped Ceph mds.a for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:49:02.856 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 17 09:49:02 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.a.service: Consumed 1.320s CPU time. 2024-05-17T09:49:02.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:02.870 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-17T09:49:02.871 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-17T09:49:02.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.d 2024-05-17T09:49:03.190 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 17 09:49:02 smithi019 systemd[1]: Stopping Ceph mds.d for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:03.691 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 17 09:49:03 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-d[70036]: 2024-05-17T09:49:03.209+0000 7f5f762c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:03.691 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 17 09:49:03 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-d[70036]: 2024-05-17T09:49:03.209+0000 7f5f762c5700 -1 mds.d *** got signal Terminated *** 2024-05-17T09:49:07.475 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 17 09:49:07 smithi019 podman[134398]: 2024-05-17 09:49:07.219122414 +0000 UTC m=+4.119699841 container died 9b7cbf1ff599b0cfe66d2e1342ec64adf8f909acc36958a96f2ef9d828113199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:49:07.895 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 17 09:49:07 smithi019 podman[134398]: 2024-05-17 09:49:07.60985096 +0000 UTC m=+4.510428429 container cleanup 9b7cbf1ff599b0cfe66d2e1342ec64adf8f909acc36958a96f2ef9d828113199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T09:49:07.895 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 17 09:49:07 smithi019 bash[134398]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-d 2024-05-17T09:49:08.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-17T09:49:08.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:08.066 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-17T09:49:08.066 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-17T09:49:08.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.g 2024-05-17T09:49:08.412 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:08 smithi019 systemd[1]: Stopping Ceph mds.g for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:08.690 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g[70844]: 2024-05-17T09:49:08.413+0000 7fd6ba8cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:08.690 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:08 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g[70844]: 2024-05-17T09:49:08.413+0000 7fd6ba8cf700 -1 mds.g *** got signal Terminated *** 2024-05-17T09:49:10.152 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:09 smithi019 podman[134519]: 2024-05-17 09:49:09.881321501 +0000 UTC m=+1.577274297 container died 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:49:10.440 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:10 smithi019 podman[134519]: 2024-05-17 09:49:10.339556832 +0000 UTC m=+2.035509627 container cleanup 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-17T09:49:10.441 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:10 smithi019 bash[134519]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g 2024-05-17T09:49:10.773 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-17T09:49:10.797 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:10 smithi019 podman[134531]: 2024-05-17 09:49:10.549484156 +0000 UTC m=+0.648790729 container remove 9153c36085c58017c49f0ff5081b2ec7a42704c24ccf374ceb01943290bc8d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-17T09:49:10.798 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:10 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-17T09:49:10.798 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:10 smithi019 systemd[1]: Stopped Ceph mds.g for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:49:10.798 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 17 09:49:10 smithi019 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.g.service: Consumed 1.310s CPU time. 2024-05-17T09:49:10.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:10.808 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-17T09:49:10.808 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-17T09:49:10.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.j 2024-05-17T09:49:11.142 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:49:10 smithi019 systemd[1]: Stopping Ceph mds.j for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:11.440 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:49:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j[69592]: 2024-05-17T09:49:11.143+0000 7f1b44763700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:11.441 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:49:11 smithi019 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j[69592]: 2024-05-17T09:49:11.143+0000 7f1b44763700 -1 mds.j *** got signal Terminated *** 2024-05-17T09:49:15.434 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:49:15 smithi019 podman[134640]: 2024-05-17 09:49:15.154526591 +0000 UTC m=+4.110640449 container died 8c824afc6b5e6067a8fa4348b4ef65721223b573b0ac13ccfc4a3ae2da5483db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:49:15.690 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:49:15 smithi019 podman[134640]: 2024-05-17 09:49:15.627062215 +0000 UTC m=+4.583176067 container cleanup 8c824afc6b5e6067a8fa4348b4ef65721223b573b0ac13ccfc4a3ae2da5483db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:49:15.690 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 17 09:49:15 smithi019 bash[134640]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-j 2024-05-17T09:49:16.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-17T09:49:16.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:16.089 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-17T09:49:16.090 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-17T09:49:16.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.b 2024-05-17T09:49:16.396 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:16 smithi069 systemd[1]: Stopping Ceph mds.b for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:16.656 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b[64746]: 2024-05-17T09:49:16.396+0000 7f9278622700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:16.656 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:16 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b[64746]: 2024-05-17T09:49:16.396+0000 7f9278622700 -1 mds.b *** got signal Terminated *** 2024-05-17T09:49:21.679 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:21 smithi069 podman[76799]: 2024-05-17 09:49:21.376446289 +0000 UTC m=+5.088753134 container died b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:49:22.154 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:21 smithi069 podman[76799]: 2024-05-17 09:49:21.913517387 +0000 UTC m=+5.625824253 container cleanup b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-05-17T09:49:22.155 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:21 smithi069 bash[76799]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b 2024-05-17T09:49:22.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-17T09:49:22.410 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:22 smithi069 podman[76825]: 2024-05-17 09:49:22.156530855 +0000 UTC m=+0.781604442 container remove b5bad18d1aebf2e4dd96c601bbcafc83dd337caa3848d807fbe1611218fbb9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-17T09:49:22.411 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:22 smithi069 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.b.service: Deactivated successfully. 2024-05-17T09:49:22.411 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:22 smithi069 systemd[1]: Stopped Ceph mds.b for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:49:22.411 INFO:journalctl@ceph.mds.b.smithi069.stdout:May 17 09:49:22 smithi069 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.b.service: Consumed 16min 36.378s CPU time. 2024-05-17T09:49:22.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:22.424 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-17T09:49:22.424 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-17T09:49:22.424 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.e 2024-05-17T09:49:22.679 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:49:22 smithi069 systemd[1]: Stopping Ceph mds.e for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:23.179 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:49:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e[64141]: 2024-05-17T09:49:22.767+0000 7fc1cc022700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:23.179 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:49:22 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e[64141]: 2024-05-17T09:49:22.767+0000 7fc1cc022700 -1 mds.e *** got signal Terminated *** 2024-05-17T09:49:27.066 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:49:26 smithi069 podman[76931]: 2024-05-17 09:49:26.805923085 +0000 UTC m=+4.158175631 container died d569de8c19bf0f1bbcfcf4bfee4329348c0de2383d3cce7871cd6e5ba20a7e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:49:27.429 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:49:27 smithi069 podman[76931]: 2024-05-17 09:49:27.167472469 +0000 UTC m=+4.519724994 container cleanup d569de8c19bf0f1bbcfcf4bfee4329348c0de2383d3cce7871cd6e5ba20a7e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T09:49:27.429 INFO:journalctl@ceph.mds.e.smithi069.stdout:May 17 09:49:27 smithi069 bash[76931]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-e 2024-05-17T09:49:27.598 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-17T09:49:27.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:27.632 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-17T09:49:27.632 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-17T09:49:27.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.h 2024-05-17T09:49:28.179 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:49:27 smithi069 systemd[1]: Stopping Ceph mds.h for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:28.179 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:49:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h[64444]: 2024-05-17T09:49:27.937+0000 7f9b058ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:28.179 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:49:27 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h[64444]: 2024-05-17T09:49:27.937+0000 7f9b058ad700 -1 mds.h *** got signal Terminated *** 2024-05-17T09:49:30.794 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:49:30 smithi069 podman[77051]: 2024-05-17 09:49:30.528377034 +0000 UTC m=+2.676021261 container died fcc5974a4e124246e336b67980ec2d6fc6c136c45a8cdcdcd1ce91a9923f7ce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:49:31.145 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:49:30 smithi069 podman[77051]: 2024-05-17 09:49:30.896093561 +0000 UTC m=+3.043737789 container cleanup fcc5974a4e124246e336b67980ec2d6fc6c136c45a8cdcdcd1ce91a9923f7ce2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD) 2024-05-17T09:49:31.145 INFO:journalctl@ceph.mds.h.smithi069.stdout:May 17 09:49:30 smithi069 bash[77051]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-h 2024-05-17T09:49:31.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-17T09:49:31.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:31.306 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-17T09:49:31.306 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-17T09:49:31.306 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.k 2024-05-17T09:49:31.428 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:31 smithi069 systemd[1]: Stopping Ceph mds.k for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:31.929 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k[63825]: 2024-05-17T09:49:31.615+0000 7f88e2d8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:31.929 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:31 smithi069 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k[63825]: 2024-05-17T09:49:31.615+0000 7f88e2d8a700 -1 mds.k *** got signal Terminated *** 2024-05-17T09:49:35.530 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:35 smithi069 podman[77173]: 2024-05-17 09:49:35.282589483 +0000 UTC m=+3.760375036 container died 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T09:49:35.531 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:35 smithi069 podman[77173]: 2024-05-17 09:49:35.532491401 +0000 UTC m=+4.010276952 container cleanup 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-17T09:49:35.531 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:35 smithi069 bash[77173]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k 2024-05-17T09:49:35.928 INFO:journalctl@ceph.mds.k.smithi069.stdout:May 17 09:49:35 smithi069 podman[77187]: 2024-05-17 09:49:35.710859912 +0000 UTC m=+0.423510302 container remove 113eecb51671560970a468e4c868d38f7610ad54bd9bdd2ae9a4375101f4c459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f) 2024-05-17T09:49:35.956 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-17T09:49:35.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:35.988 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-17T09:49:35.988 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-17T09:49:35.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.c 2024-05-17T09:49:36.270 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:36 smithi079 systemd[1]: Stopping Ceph mds.c for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:37.021 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c[65765]: 2024-05-17T09:49:36.576+0000 7f0bc9c9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:37.021 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:36 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c[65765]: 2024-05-17T09:49:36.576+0000 7f0bc9c9a700 -1 mds.c *** got signal Terminated *** 2024-05-17T09:49:40.982 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:40 smithi079 podman[79128]: 2024-05-17 09:49:40.65575479 +0000 UTC m=+4.475849517 container died 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:49:40.982 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:40 smithi079 podman[79128]: 2024-05-17 09:49:40.983883898 +0000 UTC m=+4.803978610 container cleanup 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T09:49:41.271 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:40 smithi079 bash[79128]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c 2024-05-17T09:49:41.271 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 17 09:49:41 smithi079 podman[79152]: 2024-05-17 09:49:41.1936426 +0000 UTC m=+0.536292349 container remove 5fb9fd294d804ab2c4dc2f5c3f897f933ea215461157c623a05593f1d16370bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-17T09:49:41.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-17T09:49:41.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:41.452 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-17T09:49:41.452 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-17T09:49:41.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.f 2024-05-17T09:49:41.770 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:41 smithi079 systemd[1]: Stopping Ceph mds.f for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:42.270 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f[65463]: 2024-05-17T09:49:41.799+0000 7f9c6c92f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:42.271 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:41 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f[65463]: 2024-05-17T09:49:41.799+0000 7f9c6c92f700 -1 mds.f *** got signal Terminated *** 2024-05-17T09:49:46.271 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 podman[79262]: 2024-05-17 09:49:46.01992496 +0000 UTC m=+4.346237863 container died 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T09:49:46.670 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 podman[79262]: 2024-05-17 09:49:46.429159189 +0000 UTC m=+4.755472115 container cleanup 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-17T09:49:46.670 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 bash[79262]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f 2024-05-17T09:49:46.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-17T09:49:46.941 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 podman[79274]: 2024-05-17 09:49:46.67199762 +0000 UTC m=+0.647532002 container remove 75d43ba0bff50059e2982cd39932d076ab47be5b825addf8a2974095be46fc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T09:49:46.941 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-17T09:49:46.941 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 systemd[1]: Stopped Ceph mds.f for 9c7a48b0-142c-11ef-bc9a-c7b262605968. 2024-05-17T09:49:46.941 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 17 09:49:46 smithi079 systemd[1]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.f.service: Consumed 21.901s CPU time. 2024-05-17T09:49:46.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:46.954 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-17T09:49:46.954 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-17T09:49:46.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.i 2024-05-17T09:49:47.270 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:47 smithi079 systemd[1]: Stopping Ceph mds.i for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:47.533 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i[66069]: 2024-05-17T09:49:47.282+0000 7fedbd7f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:47.534 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:47 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i[66069]: 2024-05-17T09:49:47.282+0000 7fedbd7f6700 -1 mds.i *** got signal Terminated *** 2024-05-17T09:49:49.864 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:49 smithi079 podman[79384]: 2024-05-17 09:49:49.560018966 +0000 UTC m=+2.387301250 container died 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T09:49:49.864 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:49 smithi079 podman[79384]: 2024-05-17 09:49:49.866112779 +0000 UTC m=+2.693395068 container cleanup 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-17T09:49:50.270 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:49 smithi079 bash[79384]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i 2024-05-17T09:49:50.271 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 17 09:49:50 smithi079 podman[79399]: 2024-05-17 09:49:50.100610502 +0000 UTC m=+0.537432934 container remove 8ed450be6521577a5d7e45b1f3a46e40128e3f1d62dc742e3b0d4b93454fa8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T09:49:50.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-17T09:49:50.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:50.386 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-17T09:49:50.387 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-17T09:49:50.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.l 2024-05-17T09:49:50.684 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:50 smithi079 systemd[1]: Stopping Ceph mds.l for 9c7a48b0-142c-11ef-bc9a-c7b262605968... 2024-05-17T09:49:51.021 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l[65149]: 2024-05-17T09:49:50.685+0000 7f2ac0eec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T09:49:51.021 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:50 smithi079 ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l[65149]: 2024-05-17T09:49:50.685+0000 7f2ac0eec700 -1 mds.l *** got signal Terminated *** 2024-05-17T09:49:52.842 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:52 smithi079 podman[79507]: 2024-05-17 09:49:52.539875717 +0000 UTC m=+1.935102232 container died 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-05-17T09:49:52.842 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:52 smithi079 podman[79507]: 2024-05-17 09:49:52.844477026 +0000 UTC m=+2.239703540 container cleanup 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-17T09:49:53.180 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:52 smithi079 bash[79507]: ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l 2024-05-17T09:49:53.180 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 17 09:49:53 smithi079 podman[79519]: 2024-05-17 09:49:53.078963707 +0000 UTC m=+0.537517034 container remove 04e9b62046addbdd22c93cc03771da01a3e5bc16438ab0dc1e401cf9cebbd98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e57a6d14f5341124a8070ccb8c1ba97e4bc16b687d40c128d3602ff6fb6017d6, name=ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968-mds-l, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012301-debug-3790c0f, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-17T09:49:53.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9c7a48b0-142c-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-17T09:49:53.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T09:49:53.226 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-17T09:49:53.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T09:49:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:49:56.317 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T09:49:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:49:59.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T09:49:59.252 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T09:50:01.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:50:01.742 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:50:01.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T09:50:01.804 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-17T09:50:01.806 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi019/crash 2024-05-17T09:50:01.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T09:50:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T09:50:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-05-17T09:50:01.849 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi069/crash 2024-05-17T09:50:01.850 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T09:50:01.887 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T09:50:01.887 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-05-17T09:50:01.889 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi079/crash 2024-05-17T09:50:01.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T09:50:01.927 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T09:50:01.928 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now 2024-05-17T09:50:01.929 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-17T09:50:01.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-17T09:50:02.006 INFO:teuthology.orchestra.run.smithi019.stdout:2024-05-17T09:22:39.031709+0000 mds.h (mds.2) 2 : cluster [WRN] Scrub error on inode 0x100000004bf (/volumes/qa/sv_0/b1c9b9fa-b203-4512-bd57-a87511b595df/client.0/tmp/clients/client0/~dmtmp/WORD/HANDOUT.DOC) see mds.h log and `damage ls` output for details 2024-05-17T09:50:02.006 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-17T09:50:02.006 INFO:tasks.cephadm:Compressing logs... 2024-05-17T09:50:02.006 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T09:50:02.009 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T09:50:02.012 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T09:50:02.039 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T09:50:02.040 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T09:50:02.040 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T09:50:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T09:50:02.044 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-17T09:50:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T09:50:02.046 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-17T09:50:02.046 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.f.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-17T09:50:02.053 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-17T09:50:02.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.c.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-17T09:50:02.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T09:50:02.054 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-17T09:50:02.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.i.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-17T09:50:02.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T09:50:02.054 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-17T09:50:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-17T09:50:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T09:50:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T09:50:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-17T09:50:02.056 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T09:50:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-17T09:50:02.068 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.l.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T09:50:02.150 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-17T09:50:02.151 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.b.log: /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.k.log: 84.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T09:50:02.160 INFO:teuthology.orchestra.run.smithi069.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T09:50:02.166 INFO:teuthology.orchestra.run.smithi019.stderr: 83.9% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T09:50:02.174 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T09:50:02.300 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-17T09:50:02.335 INFO:teuthology.orchestra.run.smithi019.stderr: 94.7% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-17T09:50:02.353 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-17T09:50:02.353 INFO:teuthology.orchestra.run.smithi079.stderr: 91.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T09:50:02.365 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-17T09:50:02.368 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T09:50:02.381 INFO:teuthology.orchestra.run.smithi079.stderr: 93.6% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-17T09:50:02.382 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T09:50:02.421 INFO:teuthology.orchestra.run.smithi019.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T09:50:02.425 INFO:teuthology.orchestra.run.smithi069.stderr: 93.5% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-17T09:50:02.458 INFO:teuthology.orchestra.run.smithi069.stderr: 90.5% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T09:50:02.489 INFO:teuthology.orchestra.run.smithi079.stderr: 94.6% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-17T09:50:02.528 INFO:teuthology.orchestra.run.smithi069.stderr: 87.3% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T09:50:02.538 INFO:teuthology.orchestra.run.smithi019.stderr: 94.7% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-17T09:50:02.587 INFO:teuthology.orchestra.run.smithi079.stderr: 84.0% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T09:50:02.594 INFO:teuthology.orchestra.run.smithi079.stderr: 90.5% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T09:50:02.621 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-17T09:50:02.755 INFO:teuthology.orchestra.run.smithi079.stderr: 87.3% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T09:50:06.534 INFO:teuthology.orchestra.run.smithi079.stderr: 91.7% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-17T09:50:14.013 INFO:teuthology.orchestra.run.smithi069.stderr: 91.7% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-17T09:50:19.677 INFO:teuthology.orchestra.run.smithi079.stderr: 94.9% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-17T09:50:38.887 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-17T09:50:44.540 INFO:teuthology.orchestra.run.smithi079.stderr: 95.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-17T09:51:18.238 INFO:teuthology.orchestra.run.smithi079.stderr: 91.2% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-17T09:51:25.873 INFO:teuthology.orchestra.run.smithi079.stderr: 91.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-17T09:51:45.853 INFO:teuthology.orchestra.run.smithi079.stderr: 91.0% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-17T09:52:09.625 INFO:teuthology.orchestra.run.smithi079.stderr: 91.3% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-17T09:53:08.763 INFO:teuthology.orchestra.run.smithi069.stderr: 94.9% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-17T09:53:20.995 INFO:teuthology.orchestra.run.smithi069.stderr: 91.2% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-17T09:54:06.312 INFO:teuthology.orchestra.run.smithi019.stderr: 91.5% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-17T09:54:12.976 INFO:teuthology.orchestra.run.smithi079.stderr: 94.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-17T09:54:12.978 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-05-17T09:54:12.978 INFO:teuthology.orchestra.run.smithi079.stderr:real 4m10.952s 2024-05-17T09:54:12.978 INFO:teuthology.orchestra.run.smithi079.stderr:user 4m9.523s 2024-05-17T09:54:12.978 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m9.202s 2024-05-17T09:54:14.830 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-17T09:54:35.738 INFO:teuthology.orchestra.run.smithi069.stderr: 91.6% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-17T09:54:59.107 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-17T09:55:24.226 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-17T09:56:51.657 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-17T09:57:06.969 INFO:teuthology.orchestra.run.smithi069.stderr: 91.2% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-17T10:00:27.505 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-17T10:00:27.506 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-05-17T10:00:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:real 10m25.483s 2024-05-17T10:00:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:user 6m37.931s 2024-05-17T10:00:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m16.702s 2024-05-17T10:09:49.169 INFO:teuthology.orchestra.run.smithi069.stderr: 94.1% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-17T10:12:15.168 INFO:teuthology.orchestra.run.smithi069.stderr: 95.2% -- replaced with /var/log/ceph/9c7a48b0-142c-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-17T10:12:15.242 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-05-17T10:12:15.242 INFO:teuthology.orchestra.run.smithi069.stderr:real 22m13.216s 2024-05-17T10:12:15.242 INFO:teuthology.orchestra.run.smithi069.stderr:user 7m44.416s 2024-05-17T10:12:15.242 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m24.098s 2024-05-17T10:12:15.242 INFO:tasks.cephadm:Archiving logs... 2024-05-17T10:12:15.242 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi019/log 2024-05-17T10:12:15.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T10:13:17.980 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi069/log 2024-05-17T10:13:17.993 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T10:14:21.834 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi079/log 2024-05-17T10:14:21.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T10:15:01.118 INFO:tasks.cephadm:Removing cluster... 2024-05-17T10:15:01.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --force 2024-05-17T10:15:01.434 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T10:15:01.819 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --force 2024-05-17T10:15:03.422 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T10:15:04.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9c7a48b0-142c-11ef-bc9a-c7b262605968 --force 2024-05-17T10:15:04.377 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 9c7a48b0-142c-11ef-bc9a-c7b262605968 2024-05-17T10:15:04.950 INFO:tasks.cephadm:Removing cephadm ... 2024-05-17T10:15:04.950 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T10:15:04.970 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T10:15:05.007 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T10:15:05.024 INFO:tasks.cephadm:Teardown complete 2024-05-17T10:15:05.025 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-17T10:15:05.039 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a4b16c58a5d7aa60dee491f6f096b7509e2f1269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_a4b16c58a5d7aa60dee491f6f096b7509e2f1269/teuthology/task/install/__init__.py", line 630, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_a4b16c58a5d7aa60dee491f6f096b7509e2f1269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3790c0feaaca36c33bb1b31470733325d7d91e6a/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-17T10:15:05.039 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-17T10:15:05.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T10:15:05.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T10:15:05.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T10:15:05.163 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-17T10:15:05.174 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-17T10:15:05.174 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T10:15:05.177 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T10:15:05.180 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T10:15:05.193 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-17T10:15:05.197 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-05-17T10:15:05.197 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-05-17T10:15:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T10:15:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-05-17T10:15:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +1167us[+1161us] +/- 61ms 2024-05-17T10:15:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1160us[-1166us] +/- 57ms 2024-05-17T10:15:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +63us[ +57us] +/- 91ms 2024-05-17T10:15:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T10:15:05.229 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T10:15:05.229 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-17T10:15:05.229 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +1194us[+1200us] +/- 61ms 2024-05-17T10:15:05.229 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1142us[-1136us] +/- 58ms 2024-05-17T10:15:05.229 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +121us[ +127us] +/- 91ms 2024-05-17T10:15:05.229 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T10:15:05.275 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T10:15:05.276 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-05-17T10:15:05.276 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1179us[+1123us] +/- 61ms 2024-05-17T10:15:05.276 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1129us[-1185us] +/- 58ms 2024-05-17T10:15:05.276 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +76us[ +20us] +/- 91ms 2024-05-17T10:15:05.276 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T10:15:05.292 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-17T10:15:05.303 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-17T10:15:05.304 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-17T10:15:05.315 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T10:15:05.476 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T10:15:05.622 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T10:15:05.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T10:15:05.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T10:15:05.789 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T10:15:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T10:15:05.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T10:15:05.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T10:15:05.851 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-17T10:15:05.862 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-17T10:15:05.873 INFO:teuthology.task.internal:Duration was 4577.414712 seconds 2024-05-17T10:15:05.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-17T10:15:05.883 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-17T10:15:05.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T10:15:05.886 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T10:15:05.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T10:15:05.946 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T10:15:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T10:15:05.949 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T10:15:06.320 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-17T10:15:06.320 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-05-17T10:15:06.320 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T10:15:06.349 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-05-17T10:15:06.349 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T10:15:06.378 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-05-17T10:15:06.378 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T10:15:06.407 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-17T10:15:06.407 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T10:15:06.410 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T10:15:06.421 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T10:15:06.474 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-17T10:15:06.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T10:15:06.483 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T10:15:06.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T10:15:07.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-17T10:15:07.748 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-17T10:15:07.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T10:15:07.777 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T10:15:07.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T10:15:07.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-17T10:15:07.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T10:15:07.856 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T10:15:07.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T10:15:07.886 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-05-17T10:15:07.898 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-05-17T10:15:07.990 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-05-17T10:15:08.032 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T10:15:08.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T10:15:08.047 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T10:15:08.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T10:15:08.063 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T10:15:08.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T10:15:08.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-17T10:15:08.092 INFO:teuthology.task.internal:Transferring archived files... 2024-05-17T10:15:08.094 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi019 2024-05-17T10:15:08.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T10:15:08.161 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi069 2024-05-17T10:15:08.161 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T10:15:08.200 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866/remote/smithi079 2024-05-17T10:15:08.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T10:15:08.241 INFO:teuthology.task.internal:Removing archive directory... 2024-05-17T10:15:08.242 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T10:15:08.244 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T10:15:08.247 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T10:15:08.291 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-17T10:15:08.305 INFO:teuthology.task.internal:Not uploading archives. 2024-05-17T10:15:08.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-17T10:15:08.316 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-17T10:15:08.317 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T10:15:08.319 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T10:15:08.322 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T10:15:08.332 INFO:teuthology.orchestra.run.smithi019.stdout: 266040 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 10:15 /home/ubuntu/cephtest 2024-05-17T10:15:08.336 INFO:teuthology.orchestra.run.smithi069.stdout: 265566 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 10:15 /home/ubuntu/cephtest 2024-05-17T10:15:08.347 INFO:teuthology.orchestra.run.smithi079.stdout: 265585 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 10:15 /home/ubuntu/cephtest 2024-05-17T10:15:08.348 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-17T10:15:08.360 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-17T10:15:08.372 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-17T10:15:08.372 INFO:teuthology.nuke:Checking targets against current locks 2024-05-17T10:15:08.393 DEBUG:teuthology.nuke:shortname: smithi019 2024-05-17T10:15:08.394 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T10:15:08.415 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 08:50:38.049332', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIhOaq4rZKfGooxW8/3rousAoL4Nx9brZTjPH3S8SeabhcwVQUyLSppm52Y9qCqWXc1jTECiGcHXN6A87ELWkFY='} 2024-05-17T10:15:08.418 DEBUG:teuthology.nuke:shortname: smithi069 2024-05-17T10:15:08.418 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T10:15:08.438 DEBUG:teuthology.nuke:shortname: smithi079 2024-05-17T10:15:08.438 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T10:15:08.442 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 08:50:38.051065', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAD8qQdkjGC9r8PuczunJVRByWDvrlvBQ40xz2G9I0eJqzvVr1G3C2tYsAAtPywIrd70bvZR5rDYMkylIvE7cPc='} 2024-05-17T10:15:08.463 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_04:08:01-fs-wip-pdonnell-testing-20240517.012301-debug-distro-default-smithi/7710866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 08:50:38.047351', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfyPwM0Uj4udMx3HtbAudco+vYaRaVVWR08BcMYHu1zgRtKZm6XkCXqkkyhxZPjRBynkUyBxUe6L1jefh1yawo='} 2024-05-17T10:15:08.472 INFO:teuthology.orchestra.console.smithi019:Power off 2024-05-17T10:15:08.472 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-17T10:15:08.498 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-05-17T10:15:08.498 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:08.512 INFO:teuthology.orchestra.console.smithi069:Power off 2024-05-17T10:15:08.513 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-17T10:15:08.524 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-05-17T10:15:08.535 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-05-17T10:15:08.535 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:08.547 INFO:teuthology.orchestra.console.smithi079:Power off 2024-05-17T10:15:08.548 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-17T10:15:08.558 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-05-17T10:15:08.570 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2024-05-17T10:15:08.570 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:08.593 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-05-17T10:15:12.526 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:12.560 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:12.594 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:12.637 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-05-17T10:15:12.671 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-05-17T10:15:12.705 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-05-17T10:15:16.639 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:16.672 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:16.706 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:16.751 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-05-17T10:15:16.785 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-05-17T10:15:16.818 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-05-17T10:15:16.853 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-05-17T10:15:20.786 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:20.819 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-17T10:15:20.897 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-05-17T10:15:20.931 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2024-05-17T10:15:20.998 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-05-17T10:15:21.032 INFO:teuthology.orchestra.console.smithi079:Power off completed 2024-05-17T10:15:21.137 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4577.414712190628 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-05-17T10:15:21.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T10:15:21.223 INFO:teuthology.run:FAIL